The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline

Computing and Communications

Electronic data

2024.bea-1.51
Final published version, 218 KB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review

Published

Standard

The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline. / Shardlow, Matthew; Alva-Manchego, Fernando ; Batista-Navarro, Riza Theresa et al.
Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). Kerrville: Association for Computational Linguistics, 2024. p. 571-589.

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review

Harvard

Shardlow, M, Alva-Manchego, F, Batista-Navarro, RT, Bott, S, Calderon Ramirez, S, Cardon, R, François, T, Hayakawa, A, Horbach, A, Hülsing, A, Imperia, JM, Nohej, A, Ide, Y, North, K, Occhipinti, L, Rojas, NP, Raihan, MN, Ranasinghe, T, Salazar, MS, Štajner, S, Zampieri, M & Saggion, H 2024, The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline. in Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). Association for Computational Linguistics, Kerrville, pp. 571-589, The 19th Workshop on Innovative Use of NLP for Building Educational Applications , 20/06/24. <https://aclanthology.org/2024.bea-1.51/>

APA

Shardlow, M., Alva-Manchego, F., Batista-Navarro, R. T., Bott, S., Calderon Ramirez, S., Cardon, R., François, T., Hayakawa, A., Horbach, A., Hülsing, A., Imperia, J. M., Nohej, A., Ide, Y., North, K., Occhipinti, L., Rojas, N. P., Raihan, M. N., Ranasinghe, T., Salazar, M. S., ... Saggion, H. (2024). The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) (pp. 571-589). Association for Computational Linguistics. https://aclanthology.org/2024.bea-1.51/

Vancouver

Shardlow M, Alva-Manchego F, Batista-Navarro RT, Bott S, Calderon Ramirez S, Cardon R et al. The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). Kerrville: Association for Computational Linguistics. 2024. p. 571-589

Author

Shardlow, Matthew ; Alva-Manchego, Fernando ; Batista-Navarro, Riza Theresa et al. / The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline. Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). Kerrville : Association for Computational Linguistics, 2024. pp. 571-589

Bibtex

@inproceedings{3c7bdd3a59bc4d4bbeebe5e89642fafc,

title = "The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline",

abstract = "We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson{\textquoteright}s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.",

author = "Matthew Shardlow and Fernando Alva-Manchego and Batista-Navarro, {Riza Theresa} and Stefan Bott and {Calderon Ramirez}, Saul and R{\'e}mi Cardon and Thomas Fran{\c c}ois and Akio Hayakawa and Andrea Horbach and Anna H{\"u}lsing and Imperia, {Joseph Marvin} and Adam Nohej and Yusuke Ide and Kai North and Laura Occhipinti and Rojas, {Nelson Per{\'e}z} and Raihan, {Md Nishat} and Tharindu Ranasinghe and Salazar, {Martin Solis} and Sanja {\v S}tajner and Marcos Zampieri and Horacio Saggion",

year = "2024",

month = jun,

day = "20",

language = "English",

pages = "571--589",

booktitle = "Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)",

publisher = "Association for Computational Linguistics",

note = "The 19th Workshop on Innovative Use of NLP for Building Educational Applications , (BEA 2024) ; Conference date: 20-06-2024 Through 20-06-2024",

}

RIS

TY - GEN

T1 - The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline

AU - Shardlow, Matthew

AU - Alva-Manchego, Fernando

AU - Batista-Navarro, Riza Theresa

AU - Bott, Stefan

AU - Calderon Ramirez, Saul

AU - Cardon, Rémi

AU - François, Thomas

AU - Hayakawa, Akio

AU - Horbach, Andrea

AU - Hülsing, Anna

AU - Imperia, Joseph Marvin

AU - Nohej, Adam

AU - Ide, Yusuke

AU - North, Kai

AU - Occhipinti, Laura

AU - Rojas, Nelson Peréz

AU - Raihan, Md Nishat

AU - Ranasinghe, Tharindu

AU - Salazar, Martin Solis

AU - Štajner, Sanja

AU - Zampieri, Marcos

AU - Saggion, Horacio

PY - 2024/6/20

Y1 - 2024/6/20

N2 - We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson’s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.

AB - We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson’s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.

M3 - Conference contribution/Paper

SP - 571

EP - 589

BT - Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)

PB - Association for Computational Linguistics

CY - Kerrville

T2 - The 19th Workshop on Innovative Use of NLP for Building Educational Applications

Y2 - 20 June 2024 through 20 June 2024

ER -

Research

Electronic data

Links