Standard
CSEPrompts: A Benchmark of Introductory Computer Science Prompts. / Raihan, Md Nishat; Goswami, Dhiman; Puspo, Sadiya Sayara Chowdhury et al.
Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings. ed. / Annalisa Appice; Hanane Azzag; Mohand-Said Hacid; Allel Hadjali; Zbigniew Ras. Cham: Springer Nature, 2024. p. 45-54 (Lecture Notes in Computer Science; Vol. 14670).
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Harvard
Raihan, MN, Goswami, D, Puspo, SSC, Newman, C
, Ranasinghe, T & Zampieri, M 2024,
CSEPrompts: A Benchmark of Introductory Computer Science Prompts. in A Appice, H Azzag, M-S Hacid, A Hadjali & Z Ras (eds),
Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings. Lecture Notes in Computer Science, vol. 14670, Springer Nature, Cham, pp. 45-54, Foundations of Intelligent Systems: 27th International Symposium, Poitiers, France,
17/07/24.
https://doi.org/10.1007/978-3-031-62700-2APA
Raihan, M. N., Goswami, D., Puspo, S. S. C., Newman, C.
, Ranasinghe, T., & Zampieri, M. (2024).
CSEPrompts: A Benchmark of Introductory Computer Science Prompts. In A. Appice, H. Azzag, M.-S. Hacid, A. Hadjali, & Z. Ras (Eds.),
Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings (pp. 45-54). (Lecture Notes in Computer Science; Vol. 14670). Springer Nature.
https://doi.org/10.1007/978-3-031-62700-2Vancouver
Raihan MN, Goswami D, Puspo SSC, Newman C
, Ranasinghe T, Zampieri M.
CSEPrompts: A Benchmark of Introductory Computer Science Prompts. In Appice A, Azzag H, Hacid MS, Hadjali A, Ras Z, editors, Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings. Cham: Springer Nature. 2024. p. 45-54. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-62700-2
Author
Raihan, Md Nishat ; Goswami, Dhiman ; Puspo, Sadiya Sayara Chowdhury et al. /
CSEPrompts : A Benchmark of Introductory Computer Science Prompts. Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings. editor / Annalisa Appice ; Hanane Azzag ; Mohand-Said Hacid ; Allel Hadjali ; Zbigniew Ras. Cham : Springer Nature, 2024. pp. 45-54 (Lecture Notes in Computer Science).
Bibtex
@inproceedings{23b00f3c444849c38d0d1d144ad5a79e,
title = "CSEPrompts: A Benchmark of Introductory Computer Science Prompts",
abstract = "Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs) that are trained on massive amounts of data and often have trillions of parameters. Commercial applications (e.g., ChatGPT) have made this technology available to the general public, thus making it possible to use LLMs to produce high-quality texts for academic and professional purposes. Schools and universities are aware of the increasing use of AI-generated content by students and they have been researching the impact of this new technology and its potential misuse. Educational programs in Computer Science (CS) and related fields are particularly affected because LLMs are also capable of generating programming code in various programming languages. To help understand the potential impact of publicly available LLMs in CS education, we introduce CSEPrompts (https://github.com/mraihan-gmu/CSEPrompts), a framework with hundreds of programming exercise prompts and multiple-choice questions retrieved from introductory CS and programming courses. We also provide experimental results on CSEPrompts to evaluate the performance of several LLMs with respect to generating Python code and answering basic computer science and programming questions.",
author = "Raihan, {Md Nishat} and Dhiman Goswami and Puspo, {Sadiya Sayara Chowdhury} and Christian Newman and Tharindu Ranasinghe and Marcos Zampieri",
year = "2024",
month = jun,
day = "17",
doi = "10.1007/978-3-031-62700-2",
language = "English",
isbn = "9783031626999",
series = "Lecture Notes in Computer Science",
publisher = "Springer Nature",
pages = "45--54",
editor = "Annalisa Appice and Hanane Azzag and Mohand-Said Hacid and Allel Hadjali and Zbigniew Ras",
booktitle = "Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings",
note = "Foundations of Intelligent Systems: 27th International Symposium ; Conference date: 17-07-2024 Through 19-07-2024",
}
RIS
TY - GEN
T1 - CSEPrompts
T2 - Foundations of Intelligent Systems: 27th International Symposium
AU - Raihan, Md Nishat
AU - Goswami, Dhiman
AU - Puspo, Sadiya Sayara Chowdhury
AU - Newman, Christian
AU - Ranasinghe, Tharindu
AU - Zampieri, Marcos
PY - 2024/6/17
Y1 - 2024/6/17
N2 - Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs) that are trained on massive amounts of data and often have trillions of parameters. Commercial applications (e.g., ChatGPT) have made this technology available to the general public, thus making it possible to use LLMs to produce high-quality texts for academic and professional purposes. Schools and universities are aware of the increasing use of AI-generated content by students and they have been researching the impact of this new technology and its potential misuse. Educational programs in Computer Science (CS) and related fields are particularly affected because LLMs are also capable of generating programming code in various programming languages. To help understand the potential impact of publicly available LLMs in CS education, we introduce CSEPrompts (https://github.com/mraihan-gmu/CSEPrompts), a framework with hundreds of programming exercise prompts and multiple-choice questions retrieved from introductory CS and programming courses. We also provide experimental results on CSEPrompts to evaluate the performance of several LLMs with respect to generating Python code and answering basic computer science and programming questions.
AB - Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs) that are trained on massive amounts of data and often have trillions of parameters. Commercial applications (e.g., ChatGPT) have made this technology available to the general public, thus making it possible to use LLMs to produce high-quality texts for academic and professional purposes. Schools and universities are aware of the increasing use of AI-generated content by students and they have been researching the impact of this new technology and its potential misuse. Educational programs in Computer Science (CS) and related fields are particularly affected because LLMs are also capable of generating programming code in various programming languages. To help understand the potential impact of publicly available LLMs in CS education, we introduce CSEPrompts (https://github.com/mraihan-gmu/CSEPrompts), a framework with hundreds of programming exercise prompts and multiple-choice questions retrieved from introductory CS and programming courses. We also provide experimental results on CSEPrompts to evaluate the performance of several LLMs with respect to generating Python code and answering basic computer science and programming questions.
U2 - 10.1007/978-3-031-62700-2
DO - 10.1007/978-3-031-62700-2
M3 - Conference contribution/Paper
SN - 9783031626999
T3 - Lecture Notes in Computer Science
SP - 45
EP - 54
BT - Foundations of Intelligent Systems: 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, Proceedings
A2 - Appice, Annalisa
A2 - Azzag, Hanane
A2 - Hacid, Mohand-Said
A2 - Hadjali, Allel
A2 - Ras, Zbigniew
PB - Springer Nature
CY - Cham
Y2 - 17 July 2024 through 19 July 2024
ER -
