Research output: Contribution to Journal/Magazine › Journal article › peer-review
Research output: Contribution to Journal/Magazine › Journal article › peer-review
}
TY - JOUR
T1 - Part-of-speech ratios in English corpora.
AU - Hardie, Andrew
PY - 2007
Y1 - 2007
N2 - Using part-of-speech (POS) tagged corpora, Hudson (1994) reports that approximately 37% of English tokens are nouns, where 'noun' is a superordinate category including nouns, pronouns and other word-classes. It is argued here that difficulties relating to the boundaries of Hudson's 'noun' category demonstrate that there is no uncontroversial way to derive such a superordinate category from POS tagging. Decisions regarding the boundary of the 'noun' category have small but statistically significant effects on the ratio that emerges for 'nouns' as a whole. Tokenisation and categorisation differences between tagging schemes make it problematic to compare the ratio of 'nouns' across different tagsets. The precise figures for POS ratios are therefore effectively artefacts of the tagset. However, these objections to the use of POS ratios do not apply to their use as a metric of variation for comparing data-sets tagged with the same tagging scheme.
AB - Using part-of-speech (POS) tagged corpora, Hudson (1994) reports that approximately 37% of English tokens are nouns, where 'noun' is a superordinate category including nouns, pronouns and other word-classes. It is argued here that difficulties relating to the boundaries of Hudson's 'noun' category demonstrate that there is no uncontroversial way to derive such a superordinate category from POS tagging. Decisions regarding the boundary of the 'noun' category have small but statistically significant effects on the ratio that emerges for 'nouns' as a whole. Tokenisation and categorisation differences between tagging schemes make it problematic to compare the ratio of 'nouns' across different tagsets. The precise figures for POS ratios are therefore effectively artefacts of the tagset. However, these objections to the use of POS ratios do not apply to their use as a metric of variation for comparing data-sets tagged with the same tagging scheme.
U2 - 10.1075/ijcl.12.1.05har
DO - 10.1075/ijcl.12.1.05har
M3 - Journal article
VL - 12
SP - 55
EP - 81
JO - International Journal of Corpus Linguistics
JF - International Journal of Corpus Linguistics
SN - 1384-6655
IS - 1
ER -