TY - GEN

T1 - An experiment in automatic indexing using the HASSET thesaurus

AU - El-Haj, M.

AU - Balkan, L.

AU - Barbalet, S.

AU - Bell, L.

AU - Shepherdson, J.

PY - 2013/9/17

Y1 - 2013/9/17

N2 - In this paper we present the tools, techniques and evaluation results of an automatic indexing experiment we conducted on the UK Data Archive/UK Data Service data-related document collection, as part of the Jisc-funded SKOS-HASSET project. We examined the quality of an automatic indexer based on a controlled vocabulary called the Humanities and Social Science Electronic Thesaurus (HASSET). We used the Keyphrase Extraction Algorithm (KEA), a text mining and a machine learning tool. KEA builds a classifier model using training documents with known keywords which is then applied to help assign keywords to new documents. We performed extensive manual and automatic evaluation on the results using recall, precision and F1 scores. The quality of the KEA indexing was measured a) automatically by the degree of overlap between the automated indexing decisions and those originally made by the human indexer and b) manually by comparing KEA's output with the source text. This paper explains how and why we applied the chosen technical solutions, and how we intend to take forward any lessons learned from this work in the future.

AB - In this paper we present the tools, techniques and evaluation results of an automatic indexing experiment we conducted on the UK Data Archive/UK Data Service data-related document collection, as part of the Jisc-funded SKOS-HASSET project. We examined the quality of an automatic indexer based on a controlled vocabulary called the Humanities and Social Science Electronic Thesaurus (HASSET). We used the Keyphrase Extraction Algorithm (KEA), a text mining and a machine learning tool. KEA builds a classifier model using training documents with known keywords which is then applied to help assign keywords to new documents. We performed extensive manual and automatic evaluation on the results using recall, precision and F1 scores. The quality of the KEA indexing was measured a) automatically by the degree of overlap between the automated indexing decisions and those originally made by the human indexer and b) manually by comparing KEA's output with the source text. This paper explains how and why we applied the chosen technical solutions, and how we intend to take forward any lessons learned from this work in the future.

KW - document handling

KW - indexing

KW - learning (artificial intelligence)

KW - thesauri

KW - HASSET

KW - HASSET thesaurus

KW - Jisc-funded SKOS-HASSET project

KW - KEA

KW - UK data archive-UK data service data related document collection

KW - automated indexing decisions

KW - automatic indexer

KW - automatic indexing

KW - controlled vocabulary

KW - human indexer

KW - humanities and social science electronic thesaurus

KW - keyphrase extraction algorithm

KW - machine learning tool

KW - text mining

KW - Gold

KW - Indexing

KW - Machine assisted indexing

KW - Manuals

KW - Standards

KW - Thesauri

KW - Training

U2 - 10.1109/CEEC.2013.6659437

DO - 10.1109/CEEC.2013.6659437

M3 - Conference contribution/Paper

SP - 13

EP - 18

BT - Computer Science and Electronic Engineering Conference (CEEC), 2013 5th

PB - IEEE

ER -