The large language models (LLMs) that power today’s chatbots have gotten so astoundingly capable, AI researchers are hard pressed to assess those capabilities—it seems that no sooner is there a new test than the AI systems ace it. But what does that performance really mean? Do these models genuinely understand our world? Or are they merely a triumph of data and calculations that simulates true understanding?
To hash out these questions, IEEE Spectrum partnered with the Computer History Museum in Mountain View, Calif., to bring two opinionated experts to the stage. I was the moderator of the event, which took place on 25 March. It was a fiery (but respectful) debate, well worth watching in full.
Emily M. Bender is a University of Washington professor and director of its computational linguistics laboratory, and she has emerged over the past decade as one of the fiercest critics of today’s leading AI companies and their approach to AI. She’s also known as one of the coauthors of the seminal 2021 paper “On the Dangers of Stochastic Parrots,” a paper that laid out the possible risks of LLMs (and caused Google to fire coauthor Timnit Gebru). Bender, unsurprisingly, took the “no” position.
Taking the “yes” position was Sébastien Bubeck, who recently moved to OpenAI from Microsoft, where he was vice president of AI. During his time at Microsoft he coauthored the influential preprint “Sparks of Artificial General Intelligence,” which described his early experiments with OpenAI’s GPT-4 while it was still under development. In that paper, he described advances over prior LLMs that made him feel that the model had reached a new level of comprehension.
With no further ado, we bring you the matchup that I call “Parrots vs. Sparks.”
Eliza Strickland is a senior editor at IEEE Spectrum, where she covers AI, biomedical engineering, and other topics. She holds a master’s degree in journalism from Columbia University.
