TY - GEN

T1 - Automated handling of anaphoric ambiguity in requirements: a multi-solution study

AU - Ezzini, Saad

AU - Abualhaija, Sallam

AU - Arora, Chetan

AU - Sabetzadeh, Mehrdad

PY - 2022/7/5

Y1 - 2022/7/5

N2 - Ambiguity is a pervasive issue in natural-language requirements. A common source of ambiguity in requirements is when a pronoun is anaphoric. In requirements engineering, anaphoric ambiguity occurs when a pronoun can plausibly refer to different entities and thus be interpreted differently by different readers. In this paper, we develop an accurate and practical automated approach for handling anaphoric ambiguity in requirements, addressing both ambiguity detection and anaphora interpretation. In view of the multiple competing natural language processing (NLP) and machine learning (ML) technologies that one can utilize, we simultaneously pursue six alternative solutions, empirically assessing each using a col-lection of ˜1,350 industrial requirements. The alternative solution strategies that we consider are natural choices induced by the existing technologies; these choices frequently arise in other automation tasks involving natural-language requirements. A side-by-side em-pirical examination of these choices helps develop insights about the usefulness of different state-of-the-art NLP and ML technologies for addressing requirements engineering problems. For the ambigu-ity detection task, we observe that supervised ML outperforms both a large-scale language model, SpanBERT (a variant of BERT), as well as a solution assembled from off-the-shelf NLP coreference re-solvers. In contrast, for anaphora interpretation, SpanBERT yields the most accurate solution. In our evaluation, (1) the best solution for anaphoric ambiguity detection has an average precision of ˜60% and a recall of 100%, and (2) the best solution for anaphora interpretation (resolution) has an average success rate of ˜98%.

AB - Ambiguity is a pervasive issue in natural-language requirements. A common source of ambiguity in requirements is when a pronoun is anaphoric. In requirements engineering, anaphoric ambiguity occurs when a pronoun can plausibly refer to different entities and thus be interpreted differently by different readers. In this paper, we develop an accurate and practical automated approach for handling anaphoric ambiguity in requirements, addressing both ambiguity detection and anaphora interpretation. In view of the multiple competing natural language processing (NLP) and machine learning (ML) technologies that one can utilize, we simultaneously pursue six alternative solutions, empirically assessing each using a col-lection of ˜1,350 industrial requirements. The alternative solution strategies that we consider are natural choices induced by the existing technologies; these choices frequently arise in other automation tasks involving natural-language requirements. A side-by-side em-pirical examination of these choices helps develop insights about the usefulness of different state-of-the-art NLP and ML technologies for addressing requirements engineering problems. For the ambigu-ity detection task, we observe that supervised ML outperforms both a large-scale language model, SpanBERT (a variant of BERT), as well as a solution assembled from off-the-shelf NLP coreference re-solvers. In contrast, for anaphora interpretation, SpanBERT yields the most accurate solution. In our evaluation, (1) the best solution for anaphoric ambiguity detection has an average precision of ˜60% and a recall of 100%, and (2) the best solution for anaphora interpretation (resolution) has an average success rate of ˜98%.

KW - Ambiguity

KW - BERT

KW - Language Models

KW - Machine Learning (ML)

KW - Natural Language Processing (NLP)

KW - Natural-language Requirements

KW - Requirements Engineering

U2 - 10.1145/3510003.3510157

DO - 10.1145/3510003.3510157

M3 - Conference contribution/Paper

T3 - Proceedings - International Conference on Software Engineering

SP - 187

EP - 199

BT - ICSE

ER -