Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
}
TY - GEN
T1 - Tagging the Bard
T2 - Corpus Linguistics Conference (CL2007)
AU - Rayson, Paul
AU - Archer, Dawn
AU - Baron, Alistair
AU - Culpeper, Jonathan
AU - Smith, Nicholas
PY - 2007
Y1 - 2007
N2 - In this paper we focus on automatic part-of-speech (POS) annotation, in the context of historical English texts. Techniques that were originally developed for modern English have been applied to numerous other languages over recent years. Despite this diversification, it is still almost invariably the case that the texts being analysed are from contemporary rather than historical sources. Although there is some recognition among historical linguists of the advantages of annotation for the retrieval of lexical, grammatical and other linguistic phenomena, the implementation of such forms of annotation by automatic methods is problematic. For example, changes in grammar over time will lead to a mismatch between probabilistic language models derived from, say, Present-day English and Middle English. Similarly, variability and changes in spelling can cause problems for POS taggers with fixed lexicons and rulebases. To determine the extent of the problem, and develop possible solutions, we decided to evaluate the accuracy of existing POS taggers, trained on modern English, when they are applied to Early Modern English (EModE) datasets. We focus here on the CLAWS POS tagger, a hybrid rule-based and statistical tool for English, and use as experimental data the Shakespeare First Folio and the Lampeter Corpus. First, using a manually post-edited test set, we evaluate the accuracy of CLAWS when no modifications are made either to its grammatical model or to its lexicon. We then compare this output with CLAWS' performance when using a pre-processor that detects spelling variants and matches them to modern equivalents. This experiment highlights (i) the extent to which the handling of orthographic variants is sufficient for the tagging accuracy of EModE data to approximate to the levels attained on modernday text(s), and (ii) in turn, whether revisions to the lexical resources and language models of POS taggers need to be made.
AB - In this paper we focus on automatic part-of-speech (POS) annotation, in the context of historical English texts. Techniques that were originally developed for modern English have been applied to numerous other languages over recent years. Despite this diversification, it is still almost invariably the case that the texts being analysed are from contemporary rather than historical sources. Although there is some recognition among historical linguists of the advantages of annotation for the retrieval of lexical, grammatical and other linguistic phenomena, the implementation of such forms of annotation by automatic methods is problematic. For example, changes in grammar over time will lead to a mismatch between probabilistic language models derived from, say, Present-day English and Middle English. Similarly, variability and changes in spelling can cause problems for POS taggers with fixed lexicons and rulebases. To determine the extent of the problem, and develop possible solutions, we decided to evaluate the accuracy of existing POS taggers, trained on modern English, when they are applied to Early Modern English (EModE) datasets. We focus here on the CLAWS POS tagger, a hybrid rule-based and statistical tool for English, and use as experimental data the Shakespeare First Folio and the Lampeter Corpus. First, using a manually post-edited test set, we evaluate the accuracy of CLAWS when no modifications are made either to its grammatical model or to its lexicon. We then compare this output with CLAWS' performance when using a pre-processor that detects spelling variants and matches them to modern equivalents. This experiment highlights (i) the extent to which the handling of orthographic variants is sufficient for the tagging accuracy of EModE data to approximate to the levels attained on modernday text(s), and (ii) in turn, whether revisions to the lexical resources and language models of POS taggers need to be made.
M3 - Conference contribution/Paper
BT - Proceedings of the Corpus Linguistics Conference
A2 - Davies, Matthew
A2 - Rayson, Paul
A2 - Hunston, Susan
A2 - Danielsson, Pernilla
PB - UCREL
Y2 - 27 July 2007 through 30 July 2007
ER -