I'm categorizing LLM as a productivity tool: Examining ethics of LLM use in HCI research practices

Introduction

The paper discusses the increasing use of large language models (LLMs) in human-computer interaction (HCI) research and the associated ethical concerns. LLMs are being integrated throughout the research process, from ideation to data analysis and writing. Researchers perceive LLMs as enabling new possibilities for building tools, generating ideas, and simplifying workflows. However, the paper also highlights potential ethical issues, such as harmful outputs, privacy violations, intellectual integrity concerns, and overtrust in LLMs.

While HCI researchers acknowledge these ethical considerations, they often struggle to identify and address them effectively. Reasons include perceived lack of control over the LLM supply chain, lack of established best practices, and competing priorities taking precedence over ethical concerns.

The paper calls for engaging with institutional review boards (IRBs), redesigning informed consent processes, developing tools to interrupt the LLM supply chain, providing learning opportunities on the ethics of LLM use in HCI, and shifting academic incentives to prioritize ethical considerations in research.

Overall, the paper underscores the importance of foregrounding research ethics as LLMs become integrated into HCI research practices.

Related Work

The paper discusses the increasing use of large language models (LLMs) in human-computer interaction (HCI) research for various purposes, such as research ideation, data generation and analysis, system design and development, and more. It highlights the long-standing ethical considerations in HCI research, including responsible conduct in human subjects studies, privacy, informed consent, institutional review boards (IRBs), and mitigating biases. The paper acknowledges the ethical challenges brought by emerging technologies like AI, necessitating the development of new ethical guidelines and bridging the gap between ethics and AI practices.

The paper then discusses the ethical risks and harms associated with LLMs, such as discrimination and exclusion from biased training data, privacy violations, hallucinations leading to misinformation, malicious activities like scams and phishing, anthropomorphization leading to manipulation, and increasing inequality and job displacement. The authors note recent research efforts to assess and mitigate these harms, including privacy-preserving strategies, combating misinformation, prevention measures for misuse, and improving transparency and explainability.

However, the paper identifies a gap in understanding the unique ethical challenges faced by HCI researchers when integrating LLMs into their research projects. It aims to explore these challenges and examine how researchers manage them in practice.

Methods

The paper describes a mixed-method study to examine how HCI researchers apply large language models (LLMs) in their research workflows and their ethical considerations for using LLM-based tools. The study involved conducting a survey with 50 respondents to gather broad perspectives, followed by semi-structured interviews with 16 HCI researchers to investigate their approaches to ethical considerations in more detail.

The survey aimed to identify how HCI researchers use LLMs and any ethical challenges they encountered. It covered questions about their LLM usage, ethical considerations, and demographic information. The survey responses were analyzed using descriptive statistics and qualitative analysis of open-ended questions.

The interviews focused on LLM use across the research workflow, specific ethical considerations, the process of navigating ethical concerns, the role of IRBs and ethical frameworks, and incentives and accountability. Participants were recruited through various channels, and the interviews were conducted online. The qualitative data from the interviews were analyzed using reflexive thematic analysis.

The study employed a rigorous approach to data collection and analysis, involving multiple coders, iterative coding, and regular team discussions to define themes. The findings aimed to provide a holistic view of LLM use practices and ethical considerations among HCI researchers.

Findings

The paper discusses how HCI researchers are using large language models (LLMs) like ChatGPT, GPT-4 API, Bard, and Bing Chat in various stages of their research workflow, including ideation, literature review, study design, data analysis, system building, evaluation, and paper writing. Researchers perceive LLMs as enabling new possibilities in their work.

However, researchers express several ethical concerns regarding LLM use, such as potential harms from biased or harmful outputs, privacy violations from data leaks, ambiguity around intellectual integrity and authorship, overtrust and overreliance on LLMs, and environmental and societal impacts.

The paper outlines four main approaches researchers take to navigate these ethical concerns:

1. Conditional and reactive engagement, where they address ethics based on perceived risk levels or react after issues arise rather than proactively.

2. Limited disclosure practices, with researchers often not formally reporting LLM use to study participants, IRBs, or in publications, likening LLMs to productivity tools.

3. Restricting LLM use, carefully verifying outputs before use, and reflecting through group discussions to mitigate risks.

4. Delaying responsibility by expressing uncertainty over who is accountable for ethical issues and postponing addressing concerns to avoid hindering innovation.

Overall, while aware of potential ethical pitfalls, researchers currently lack clear guidelines and tend to employ ad-hoc strategies that do not directly address underlying ethical challenges.

Discussion

The paper discusses the implications of HCI researchers' approaches to using large language models (LLMs) in their research projects and provides opportunities to better engage with ethical considerations.

Key points:

1. Most researchers did not consider it necessary to report LLM usage to Institutional Review Boards (IRBs), which could lead to challenges in replicating studies and understanding methodological decisions.

2. Researchers acknowledged limited transparency to study participants regarding LLM usage, often rationalizing it as avoiding overwhelming them with technical details.

3. Researchers expressed a lack of control over the functionality and outputs of LLMs, as they are situated downstream in the "LLM supply chain."

4. The paper suggests proactively engaging with IRBs, re-examining informed consent processes, developing tools and methods to interrupt the LLM supply chain, creating learning opportunities on ethics of LLM use, and shifting academic incentives to foreground ethical concerns.

5. It calls for collaborative efforts between researchers, policymakers, and LLM companies to create "living guidelines" for responsible use of LLMs in research.

6. The paper advocates for a cultural shift within academia to demonstrate a commitment to ethical use of LLMs, potentially through changes in recognition and funding criteria.

Limitations and Future Work

The study explores emerging practices around ethics and large language models (LLMs) among human-computer interaction (HCI) researchers. However, it has certain limitations:

1. Sample constraints: The sample was limited due to the exploratory nature of the study and the snowball sampling method used. HCI research encompasses diverse traditions, many of which were not included, highlighting the need for more comprehensive and systematic future studies.

2. Geographical bias: The interview sample primarily consisted of researchers from the USA. Future research should explore ethical challenges and practices related to LLM usage by researchers from other regions.

3. Potential selection bias: The study may have attracted respondents who are conscious of their LLM usage and open to discussing their experiences, overlooking perspectives from researchers with different levels of awareness or willingness to discuss the topic.

The paper suggests that future research should explore how ethical practices with LLMs vary across different research methodologies, domains, settings (industry and academia), to gain a broader perspective on the subject.

Conclusion

The paper explores how human-computer interaction (HCI) researchers have integrated large language models (LLMs) into their research practices and the ethical concerns they have encountered. The authors conducted a survey and interviews to gather empirical data.

The results indicate that while HCI researchers are using LLMs across various stages of their research process and are aware of potential ethical issues, they often face challenges in effectively identifying and navigating those concerns within their own projects.

Based on these findings, the paper discusses potential approaches to support the formation of emerging ethical norms for using LLMs in HCI research. The authors encourage HCI researchers to engage with institutional review boards (IRBs), collaborate with policymakers and generative AI companies to create guidelines for responsible LLM use, and re-examine the informed consent process.

The paper also highlights the need for technological support to interrupt the LLM supply chain and the importance of creating learning opportunities for HCI researchers to understand the ethics of LLM use. Additionally, the authors suggest shifting academic incentives to prioritize ethical concerns.