Accepted author manuscript, 1.37 MB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Accepted author manuscript, 1.59 MB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
}
TY - GEN
T1 - Probing Social Bias in Labor Market Text Generation by ChatGPT
T2 - A Masked Language Model Approach
AU - Ding, Lei
AU - Hu, Yang
AU - Denier, Nicole
AU - Shi, Enze
AU - Zhang, Junxi
AU - Hu, Qirui
AU - Hughes, Karen D.
AU - Kong, Linglong
AU - Jiang, Bei
PY - 2024/12/10
Y1 - 2024/12/10
N2 - As generative large language models (LLMs) such as ChatGPT gain widespread adoption in various domains, their potential to propagate and amplify social biases, particularly in high-stakes areas such as the labor market, has become a pressing concern. AI algorithms are not only widely used in the selection of job applicants, individual job seekers may also make use of generative LLMs to help develop their job application materials. Against this backdrop, this research builds on a novel experimental design to examine social biases within ChatGPT-generated job applications in response to real job advertisements. By simulating the process of job application creation, we examine the language patterns and biases that emerge when the model is prompted with diverse job postings. Notably, we present a novel bias evaluation framework based on Masked Language Models to quantitatively assess social bias based on validated inventories of social cues/words, enabling a systematic analysis of the language used. Our findings show that the increasing adoption of generative AI, not only by employers but also increasingly by individual job seekers, can reinforce and exacerbate gender and social inequalities in the labor market through the use of biased and gendered language.
AB - As generative large language models (LLMs) such as ChatGPT gain widespread adoption in various domains, their potential to propagate and amplify social biases, particularly in high-stakes areas such as the labor market, has become a pressing concern. AI algorithms are not only widely used in the selection of job applicants, individual job seekers may also make use of generative LLMs to help develop their job application materials. Against this backdrop, this research builds on a novel experimental design to examine social biases within ChatGPT-generated job applications in response to real job advertisements. By simulating the process of job application creation, we examine the language patterns and biases that emerge when the model is prompted with diverse job postings. Notably, we present a novel bias evaluation framework based on Masked Language Models to quantitatively assess social bias based on validated inventories of social cues/words, enabling a systematic analysis of the language used. Our findings show that the increasing adoption of generative AI, not only by employers but also increasingly by individual job seekers, can reinforce and exacerbate gender and social inequalities in the labor market through the use of biased and gendered language.
M3 - Conference contribution/Paper
VL - 37
T3 - Advances in Neural Information Processing Systems
BT - Advances in Neural Information Processing Systems 37 (NeurIPS 2024)
A2 - Globerson, A.
A2 - Mackey, L.
A2 - Belgrave, D.
A2 - Fan, A.
A2 - Paquet, U.
A2 - Tomczak, J.
A2 - Zhang, C.
ER -