TY - CHAP

T1 - Reliability and Validity of Research with Corpora of Music

AU - Marsden, Alan

PY - 2022/8/18

Y1 - 2022/8/18

N2 - Corpus musicologists seek conclusions which are valid and applicable for demonstrably abstract reasons. This chapter examines several aspects of reliability and validity in corpus musicology: the avoidance of bias, particularly through the deliberate exclusion of expert judgment; issues of the reliability of methods of measurement used and conversion from one form of information to another; variances in interpreted data involving human judgments; errors in corpora; the notions of “random” or “representative” samples; and statistical significance in both hypothesis-testing and exploratory studies. Some research aims to extend or test theory through building models, now frequently employing machine-learning. Particular care is required to separate training and test materials to avoid over-fitting, i.e., building a model which works for the specific data used but not for other music. Even in the absence of machine-learning, continually working with the same or a small number of corpora has similar dangers of drawing conclusions which lack wider applicability. When working with several separate corpora is not possible, techniques such as bootstrapping can be employed to estimate the reliability of conclusions. Even working with corpora which are comprehensive, such as the entire output of a composer, is not problem-free. As corpora become ever more comprehensive and corpus musicology more common, musicology is likely to have to rediscover its links with composition and its concerns with not-yet-existent music.

AB - Corpus musicologists seek conclusions which are valid and applicable for demonstrably abstract reasons. This chapter examines several aspects of reliability and validity in corpus musicology: the avoidance of bias, particularly through the deliberate exclusion of expert judgment; issues of the reliability of methods of measurement used and conversion from one form of information to another; variances in interpreted data involving human judgments; errors in corpora; the notions of “random” or “representative” samples; and statistical significance in both hypothesis-testing and exploratory studies. Some research aims to extend or test theory through building models, now frequently employing machine-learning. Particular care is required to separate training and test materials to avoid over-fitting, i.e., building a model which works for the specific data used but not for other music. Even in the absence of machine-learning, continually working with the same or a small number of corpora has similar dangers of drawing conclusions which lack wider applicability. When working with several separate corpora is not possible, techniques such as bootstrapping can be employed to estimate the reliability of conclusions. Even working with corpora which are comprehensive, such as the entire output of a composer, is not problem-free. As corpora become ever more comprehensive and corpus musicology more common, musicology is likely to have to rediscover its links with composition and its concerns with not-yet-existent music.

U2 - 10.1093/oxfordhb/9780190945442.013.7

DO - 10.1093/oxfordhb/9780190945442.013.7

M3 - Chapter (peer-reviewed)

SN - 9780190945442

SP - C7.P1–C7.S11

BT - The Oxford Handbook of Music and Corpus Studies

A2 - Shanahan, Daniel

A2 - Burgoyne, John Ashley

A2 - Quinn, Ian

PB - Oxford University Press

CY - Oxford

ER -