Are you more imaginative than AI chatbots such as ChatGPT? The average person may not be, at least when it comes to one specific creative task, a new study finds.
Creativity has been thought of as a province unique to human beings, notwithstanding the occasional argument over whether gorillas and other animals can make art. Recently, however, generative AI systems such as Midjourney, DALL-E and Stable Diffusion have produced artwork intriguing enough to spark debate about the differences between human and machine creativity, and their potential impact on the job market.
To explore the limits of machine creativity, researchers tested how well AI performed on a task where it had to find many possible uses for common items, a bit like how science-fiction author Douglas Adams described the many possible uses of a towel in The Hitchhiker’s Guide to the Galaxy.
“I’m reminded of MacGyver and his inventive use of everyday objects,” says study coauthor Simone Grassini at the University of Bergen and the University of Stavanger, both in Norway, referencing the 1980s TV show whose name became a synonym for improvisation.
Psychology researchers commonly ask subjects to think up alternate uses of objects to assess divergent thinking, a type of thought process commonly associated with creativity that involves generating many different ideas or solutions for a given task. Because current AI chatbots have a vast memory and the ability to quickly access large databases, Grassini and his colleague Mika Koivisto at the University of Turku, in Finland, theorized the machines could outperform humans on this task.
“With our work, we want to stimulate the social debate about the future of AI in the context of potential risks for the replacement of jobs that so far have not been at stake due to automation, as creative professions,” Grassini says.
The scientists experimented with three AI chatbots—ChatGPT-3, ChatGPT-4, and Copy.Ai. These are powered by large language models, which are essentially supercharged versions of the autocomplete feature that smartphones use to predict the rest of a word a person is typing.
What Creative Task Did the Chatbots Beat Humans At?
The researchers had the chatbots and 256 volunteers attempt to find uncommon and creative uses for four items—a rope, a box, a pencil, and a candle. They judged each response’s originality in two different ways.
In one, a computer model quantified a response’s semantic distance—how closely related it was to the object’s common use—on a scale from 0 to 2. In the other, six human judges subjectively rated the creativity of the responses from 1 to 5. For instance, “when asked to come up with creative use of a pen, an answer like ‘to write a letter’ would receive a very low score, while an answer like ‘to play darts’ would receive a higher score,” Grassini says.
On average, the scientists found the chatbots significantly outperformed the people in both semantic distance (0.95 versus 0.91) and creativity (2.91 versus 2.47). The human responses displayed a far greater range of scores in both measures, with the minimum human scores much lower than AI ones, but the maximum human scores typically higher.
“Our results showed that the best humans still outperform current AI models when it comes to creative thinking,” Grassini says. “However, this may change in the future.”
One potential implication of this work is that “jobs that require some degree or some type of creative thinking will possibly be done by AI,” Grassini says. “However I like to think that AI will be used by humans to improve their work, and not that AI will replace humans in creative jobs.”
Still the researchers do caution that they focused on only a single task associated with creativity.
“Creativity is a complex phenomenon, and we analyzed only one aspect of it,” Grassini says. “The current creative abilities of AI do not automatically mean that the current AI models are suitable for all creative tasks, or that will produce masterpieces when it comes to art.”
The scientists detailed their findings online 14 September in the journal Scientific Reports.
Charles Q. Choi is a science reporter who contributes regularly to IEEE Spectrum. He has written for Scientific American, The New York Times, Wired, and Science, among others.
