The Silicone Ceiling: Auditing GPT's Race and Gender Biases in Hiring

    Read original: arXiv:2405.04412 - Published 5/13/2024 by Lena Armstrong, Abbey Liu, Stephen MacNeil, Danae Metaxa
    The Silicone Ceiling: Auditing GPT's Race and Gender Biases in Hiring

    Overview

    • This paper examines the racial and gender biases present in the language model GPT, which is commonly used to assist with hiring and recruitment tasks.
    • The researchers conducted a series of audits to assess how GPT responds to prompts related to job qualifications and applicant characteristics.
    • The findings reveal significant biases, with GPT exhibiting preferences for candidates perceived as white and male, and making negative assessments of candidates from marginalized groups.

    Plain English Explanation

    The paper investigates the biases that can creep into hiring decisions when language models like GPT are used to assist with the process. GPT is a powerful AI system that can generate human-like text. While these models can be helpful tools, they may also reflect and amplify societal biases around race and gender.

    The researchers designed a series of audits to test how GPT responds to prompts related to job qualifications and applicant characteristics. For example, they provided GPT with similar resumes but altered the names to signal different racial or gender identities. They found that GPT often assessed the "white-sounding" candidates more favorably, viewing them as more qualified and hireable. In contrast, GPT tended to make negative judgments about candidates with "non-white" or female-coded names.

    These biases can have serious consequences, unfairly disadvantaging certain groups in the hiring process. As AI systems play an increasingly prominent role in recruitment and employment decisions, it's crucial to understand and mitigate these biases. Failing to address these issues can lead to further entrenching societal inequities.

    Technical Explanation

    The researchers conducted a series of audits to examine the racial and gender biases present in the GPT language model. They generated job-related prompts, such as "Summarize the key qualifications of this applicant," and provided GPT with similar resumes that only differed in the candidate's name (to signal race and gender).

    The team found that GPT exhibited significant biases, often assessing candidates with "white-sounding" names more positively in terms of their qualifications and hireability. Conversely, GPT tended to make more negative judgments about candidates with "non-white" or female-coded names, portraying them as less competent and suitable for the roles.

    These biases were present across a range of job types and seniority levels, indicating that the issues are pervasive within GPT's decision-making processes. The researchers also found that the biases persisted even when they adjusted the prompts to be more explicit about the need for fair and unbiased assessments.

    Critical Analysis

    The paper provides a robust and well-designed study that sheds light on a critical issue in the use of language models like GPT for hiring and recruitment. The researchers acknowledge that their findings are limited to the specific prompts and resume variations they tested, and that further research is needed to fully understand the scope and nuances of the biases.

    Additionally, the paper does not delve into the potential causes of these biases, such as the data used to train the language model or the societal biases reflected in that data. Exploring these underlying factors could yield valuable insights into how to mitigate the problem.

    The paper also raises important questions about the ethical and legal implications of using language models in high-stakes decision-making processes, where biases can have significant, real-world consequences for individuals and communities. Policymakers and industry leaders will need to grapple with these issues as the use of AI in hiring continues to expand.

    Conclusion

    This paper provides a comprehensive audit of the racial and gender biases present in the GPT language model, which is commonly used to assist with hiring and recruitment tasks. The findings reveal that GPT exhibits significant preferences for candidates perceived as white and male, while making negative assessments of candidates from marginalized groups.

    These biases can have serious consequences, unfairly disadvantaging certain individuals in the hiring process and perpetuating societal inequities. As AI systems play an increasingly prominent role in employment decisions, it is crucial that we understand and address these biases to ensure fair and equitable practices. The insights from this paper can inform the development of more ethical and responsible AI systems for use in hiring and beyond.



    This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

    Total Score

    0

    Follow @aimodelsfyi on 𝕏 →

    Related Papers

    The Silicone Ceiling: Auditing GPT's Race and Gender Biases in Hiring
    Total Score

    0

    The Silicone Ceiling: Auditing GPT's Race and Gender Biases in Hiring

    Lena Armstrong, Abbey Liu, Stephen MacNeil, Danae Metaxa

    Large language models (LLMs) are increasingly being introduced in workplace settings, with the goals of improving efficiency and fairness. However, concerns have arisen regarding these models' potential to reflect or exacerbate social biases and stereotypes. This study explores the potential impact of LLMs on hiring practices. To do so, we conduct an algorithm audit of race and gender biases in one commonly-used LLM, OpenAI's GPT-3.5, taking inspiration from the history of traditional offline resume audits. We conduct two studies using names with varied race and gender connotations: resume assessment (Study 1) and resume generation (Study 2). In Study 1, we ask GPT to score resumes with 32 different names (4 names for each combination of the 2 gender and 4 racial groups) and two anonymous options across 10 occupations and 3 evaluation tasks (overall rating, willingness to interview, and hireability). We find that the model reflects some biases based on stereotypes. In Study 2, we prompt GPT to create resumes (10 for each name) for fictitious job candidates. When generating resumes, GPT reveals underlying biases; women's resumes had occupations with less experience, while Asian and Hispanic resumes had immigrant markers, such as non-native English and non-U.S. education and work experiences. Our findings contribute to a growing body of literature on LLM biases, in particular when used in workplace contexts.

    Read more

    5/13/2024

    Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval
    Total Score

    0

    Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

    Kyra Wilson, Aylin Caliskan

    Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the same. However, given the biases which are embedded within LLMs, it is unclear whether they can be used in this scenario without disadvantaging groups based on their protected attributes. In this work, we investigate the possibilities of using LLMs in a resume screening setting via a document retrieval framework that simulates job candidate selection. Using that framework, we then perform a resume audit study to determine whether a selection of Massive Text Embedding (MTE) models are biased in resume screening scenarios. We simulate this for nine occupations, using a collection of over 500 publicly available resumes and 500 job descriptions. We find that the MTEs are biased, significantly favoring White-associated names in 85.1% of cases and female-associated names in only 11.1% of cases, with a minority of cases showing no statistically significant differences. Further analyses show that Black males are disadvantaged in up to 100% of cases, replicating real-world patterns of bias in employment settings, and validate three hypotheses of intersectionality. We also find an impact of document length as well as the corpus frequency of names in the selection of resumes. These findings have implications for widely used AI tools that are automating employment, fairness, and tech policy.

    Read more

    8/22/2024

    💬

    Total Score

    0

    Evaluation of Bias Towards Medical Professionals in Large Language Models

    Xi Chen, Yang Xu, MingKe You, Li Wang, WeiZhi Liu, Jian Li

    This study evaluates whether large language models (LLMs) exhibit biases towards medical professionals. Fictitious candidate resumes were created to control for identity factors while maintaining consistent qualifications. Three LLMs (GPT-4, Claude-3-haiku, and Mistral-Large) were tested using a standardized prompt to evaluate resumes for specific residency programs. Explicit bias was tested by changing gender and race information, while implicit bias was tested by changing names while hiding race and gender. Physician data from the Association of American Medical Colleges was used to compare with real-world demographics. 900,000 resumes were evaluated. All LLMs exhibited significant gender and racial biases across medical specialties. Gender preferences varied, favoring male candidates in surgery and orthopedics, while preferring females in dermatology, family medicine, obstetrics and gynecology, pediatrics, and psychiatry. Claude-3 and Mistral-Large generally favored Asian candidates, while GPT-4 preferred Black and Hispanic candidates in several specialties. Tests revealed strong preferences towards Hispanic females and Asian males in various specialties. Compared to real-world data, LLMs consistently chose higher proportions of female and underrepresented racial candidates than their actual representation in the medical workforce. GPT-4, Claude-3, and Mistral-Large showed significant gender and racial biases when evaluating medical professionals for residency selection. These findings highlight the potential for LLMs to perpetuate biases and compromise healthcare workforce diversity if used without proper bias mitigation strategies.

    Read more

    7/18/2024

    You Gotta be a Doctor, Lin: An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations
    Total Score

    0

    You Gotta be a Doctor, Lin: An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations

    Huy Nghiem, John Prindle, Jieyu Zhao, Hal Daum'e III

    Social science research has shown that candidates with names indicative of certain races or genders often face discrimination in employment practices. Similarly, Large Language Models (LLMs) have demonstrated racial and gender biases in various applications. In this study, we utilize GPT-3.5-Turbo and Llama 3-70B-Instruct to simulate hiring decisions and salary recommendations for candidates with 320 first names that strongly signal their race and gender, across over 750,000 prompts. Our empirical results indicate a preference among these models for hiring candidates with White female-sounding names over other demographic groups across 40 occupations. Additionally, even among candidates with identical qualifications, salary recommendations vary by as much as 5% between different subgroups. A comparison with real-world labor data reveals inconsistent alignment with U.S. labor market characteristics, underscoring the necessity of risk investigation of LLM-powered systems.

    Read more

    10/8/2024