Divergent Creativity in Humans and Large Language Models

    Read original: arXiv:2405.13012 - Published 5/24/2024 by Antoine Bellemare-Pepin (CoCo Lab, Psychology department, Universit'e de Montr'eal, Montreal, QC, Canada, Music department, Concordia University, Montreal, QC and 37 others

    💬

    Overview

    • Recent advancements in Large Language Models (LLMs) have led to claims that they are approaching human-level creativity.
    • However, there has been a lack of systematic evaluation of LLM creativity compared to human divergent thinking.
    • This research paper aims to bridge this gap by leveraging creativity science to analyze divergent creativity in both state-of-the-art LLMs and a large dataset of human responses.

    Plain English Explanation

    The paper explores the creative capabilities of advanced Large Language Models (LLMs) compared to humans. LLMs are a type of artificial intelligence that can generate human-like text. There has been growing excitement, but also some concern, about whether these models are becoming as creative as humans.

    To investigate this, the researchers used a framework for analyzing divergent creativity - the ability to come up with many different and original ideas. They compared the performance of LLMs to a large dataset of 100,000 human responses on various creative tasks.

    Surprisingly, the results suggest that in certain specific creative activities, such as divergent association (generating related but unique ideas) and creative writing, the LLMs were actually able to outperform humans. This challenges the common assumption that human creativity is inherently superior to what can be achieved artificially.

    However, the research also highlights the need to better understand the distinctive elements of human inventive thought processes compared to what can be generated by machines. This could help guide the development of even more creative AI systems in the future.

    Technical Explanation

    The researchers leveraged recent advances in creativity science to build a framework for in-depth analysis of divergent creativity. This allowed them to quantitatively benchmark the performance of state-of-the-art LLMs against a substantial dataset of 100,000 human responses.

    The key findings were that LLMs can indeed surpass human capabilities in specific creative tasks, such as divergent association and creative writing. The researchers attribute this to the LLMs' ability to rapidly generate a large and diverse set of relevant ideas by drawing upon their broad knowledge base and powerful language generation capabilities.

    However, the research also highlights the need for more granular inquiry into the distinctive elements that constitute human inventive thought processes, compared to the artificial generation of creative outputs. This could inform the development of LLMs with enhanced creative abilities that better emulate the nuances of human creativity.

    Critical Analysis

    The research provides a valuable and rigorous framework for comparing the creative abilities of LLMs and humans. By focusing on divergent creativity, the authors have identified specific areas where LLMs can outperform humans, which challenges the common assumption of human creative superiority.

    That said, the paper acknowledges the need for further research to fully understand the differences between human and machine creativity. The authors note that their analysis is limited to certain types of creative tasks, and there may be other aspects of creativity where humans maintain a clear advantage.

    Additionally, the paper does not delve deeply into potential concerns or risks associated with LLMs surpassing human creativity. There may be ethical implications or unintended consequences that warrant further exploration, such as the homogenization effects of LLMs on human creative expression.

    Overall, this research represents an important step in the ongoing dialogue around the creative capabilities of advanced AI systems. By enhancing creativity in LLMs, we may unlock new possibilities, but it is crucial to also understand the aspects of human memory that contribute to our unique creative abilities.

    Conclusion

    This research paper provides a comprehensive and thought-provoking exploration of the creative capabilities of Large Language Models (LLMs) compared to humans. By leveraging a rigorous framework for analyzing divergent creativity, the authors have found that LLMs can surpass human performance in certain creative tasks, challenging the common assumption of human creative superiority.

    However, the paper also highlights the need for deeper understanding of the distinctive elements that constitute human inventive thought processes, in order to further enhance the creativity of LLMs. This research opens up new paths for the development of more creative AI systems, while also encouraging critical reflection on the nature of human creativity and its interplay with artificial intelligence.



    This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

    Total Score

    0

    Follow @aimodelsfyi on 𝕏 →

    Related Papers

    💬

    Total Score

    0

    Divergent Creativity in Humans and Large Language Models

    Antoine Bellemare-Pepin (CoCo Lab, Psychology department, Universit'e de Montr'eal, Montreal, QC, Canada, Music department, Concordia University, Montreal, QC, Canada), Franc{c}ois Lespinasse (Sociology and Anthropology department, Concordia University, Montreal, QC, Canada), Philipp Tholke (CoCo Lab, Psychology department, Universit'e de Montr'eal, Montreal, QC, Canada), Yann Harel (CoCo Lab, Psychology department, Universit'e de Montr'eal, Montreal, QC, Canada), Kory Mathewson (Mila), Jay A. Olson (Department of Psychology, University of Toronto Mississauga, Mississauga, ON, Canada), Yoshua Bengio (Mila, Department of Computer Science and Operations Research, Universit'e de Montr'eal, Montreal, QC, Canada), Karim Jerbi (CoCo Lab, Psychology department, Universit'e de Montr'eal, Montreal, QC, Canada, UNIQUE Center)

    The recent surge in the capabilities of Large Language Models (LLMs) has led to claims that they are approaching a level of creativity akin to human capabilities. This idea has sparked a blend of excitement and apprehension. However, a critical piece that has been missing in this discourse is a systematic evaluation of LLM creativity, particularly in comparison to human divergent thinking. To bridge this gap, we leverage recent advances in creativity science to build a framework for in-depth analysis of divergent creativity in both state-of-the-art LLMs and a substantial dataset of 100,000 humans. We found evidence suggesting that LLMs can indeed surpass human capabilities in specific creative tasks such as divergent association and creative writing. Our quantitative benchmarking framework opens up new paths for the development of more creative LLMs, but it also encourages more granular inquiries into the distinctive elements that constitute human inventive thought processes, compared to those that can be artificially generated.

    Read more

    5/24/2024

    Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking
    Total Score

    0

    Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

    Harsh Kumar, Jonathan Vincentius, Ewan Jordan, Ashton Anderson

    Large language models are transforming the creative process by offering unprecedented capabilities to algorithmically generate ideas. While these tools can enhance human creativity when people co-create with them, it's unclear how this will impact unassisted human creativity. We conducted two large pre-registered parallel experiments involving 1,100 participants attempting tasks targeting the two core components of creativity, divergent and convergent thinking. We compare the effects of two forms of large language model (LLM) assistance -- a standard LLM providing direct answers and a coach-like LLM offering guidance -- with a control group receiving no AI assistance, and focus particularly on how all groups perform in a final, unassisted stage. Our findings reveal that while LLM assistance can provide short-term boosts in creativity during assisted tasks, it may inadvertently hinder independent creative performance when users work without assistance, raising concerns about the long-term impact on human creativity and cognition.

    Read more

    10/8/2024

    💬

    Total Score

    0

    Characterising the Creative Process in Humans and Large Language Models

    Surabhi S. Nath, Peter Dayan, Claire Stevenson

    Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on textit{products}, with little attention on the creative textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and contrast with behaviour in a Verbal Fluency Task. We use sentence embeddings to identify response categories and compute semantic similarities, which we use to generate jump profiles. Our results corroborate earlier work in humans reporting both persistent (deep search in few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, where both pathways lead to similar creativity scores. LLMs were found to be biased towards either persistent or flexible paths, that varied across tasks. Though LLMs as a population match human profiles, their relationship with creativity is different, where the more flexible models score higher on creativity. Our dataset and scripts are available on href{https://github.com/surabhisnath/Creative_Process}{GitHub}.

    Read more

    6/7/2024

    💬

    Total Score

    1

    On the Creativity of Large Language Models

    Giorgio Franceschelli, Mirco Musolesi

    Large Language Models (LLMs) are revolutionizing several areas of Artificial Intelligence. One of the most remarkable applications is creative writing, e.g., poetry or storytelling: the generated outputs are often of astonishing quality. However, a natural question arises: can LLMs be really considered creative? In this article, we first analyze the development of LLMs under the lens of creativity theories, investigating the key open questions and challenges. In particular, we focus our discussion on the dimensions of value, novelty, and surprise as proposed by Margaret Boden in her work. Then, we consider different classic perspectives, namely product, process, press, and person. We discuss a set of ``easy'' and ``hard'' problems in machine creativity, presenting them in relation to LLMs. Finally, we examine the societal impact of these technologies with a particular focus on the creative industries, analyzing the opportunities offered, the challenges arising from them, and the potential associated risks, from both legal and ethical points of view.

    Read more

    9/19/2024