Grok & Groupthink: Why AI is getting less reliable
My latest in TIME magazine, written with Yale School of Management Prof Jeffrey Sonnenfeld
By Jeffrey Sonnenfeld and Joanne Lipman, from today's TIME magazine:
Last week, we conducted a test that found five leading AI models—including Elon Musk’s Grok—correctly debunked 20 of President Donald Trump’s false claims. A few days later, Musk retrained Grok with an apparent right-wing update, promising that users “should notice a difference.” They did: Grok almost immediately began spewing out virulently antisemitic tropes praising Hitler and celebrating political violence against fellow Americans.
Musk’s Grok fiasco is a wakeup call. Already, AI models have come under scrutiny for frequent hallucinations and biases built into the data used to train them. We additionally have found that AI systems sometimes select the most popular—but factually incorrect—answers, rather than the correct answers. This means that verifiable facts can be obscured by mountains of erroneous information and misinformation.
Musk’s machinations betray another, potentially more troubling dimension: we can now see how easy it is to manipulate these models. Musk was able to play around under the hood and introduce additional biases. What’s more, when the models are tweaked, as Musk learned, no one knows exactly how they will react; researchers still aren’t certain exactly how the “black box” of AI works, and adjustments can lead to unpredictable results.
The chatbots’ vulnerability to manipulation, along with their susceptibility to groupthink and their inability to recognize basic facts, should alarm all of us about the growing reliance on these research tools in industry, education, and the media.
AI has made tremendous progress over the last few years. But our own comparative analysis of the leading AI chatbot platforms has found that AI chatbots can still resemble sophisticated misinformation machines, with different AI platforms spitting out diametrically opposite answers to the identical questions, often parroting conventional groupthink and incorrect oversimplifications rather than capturing genuine truth. Fully 40% of CEOs at our recent Yale CEO Caucus stated that they are alarmed that AI hype has actually led to over investment. Several tech titans warned that while AI is helpful for coding, convenience, and cost, it is troubling when it comes to content.
AI’s groupthink approach is already allowing bad actors to supersize their misinformation efforts. Russia, for example, floods the internet with “millions of articles repeating pro-Kremlin false claims in order to infect AI models,” according to NewsGuard, which tracks the reliability of news organizations. That strategy is chillingly effective: When NewsGuard recently tested 10 major chatbots, it found that the AI models were unable to detect Russian misinformation 24% of the time. Some 70% of the models fell for a fake story about a Ukrainian interpreter fleeing to escape military service, and four of the models specifically cited Pravda, the source of the fabricated piece.
It isn’t just Russia playing these games. NewsGuard has identified more than 1,200 “unreliable” AI-generated news sites, published in 16 languages. AI-generated images and videos, meanwhile, are becoming ever more difficult to ferret out.
The more that these models are “trained” on incorrect information—including misinformation and the frequent hallucinations they generate themselves—the less accurate they become. Essentially, the “wisdom of crowds” is turned on its head, with false information feeding on itself and metastasizing. There are indications this is already happening. Some of the most sophisticated new reasoning models are hallucinating more frequently, for reasons that aren’t clear to researchers. As the CEO of one AI startup told the New York Times, “Despite our best efforts, they will always hallucinate. That will never go away.”
Please go to TIME more