What happened
Researchers at Wharton gave about 1,300 people a set of reasoning puzzles and an AI to help them. The people didn’t know the researchers controlled the AI. Sometimes it gave the right answer, and sometimes, on purpose, the wrong one. When the AI was right, people scored 25 points higher than people with no AI. When it was wrong, they scored 15 points lower. And their confidence went up both times.
So is AI making us dumber? Not exactly. It’s making it very easy to stop thinking without noticing. Steven Shaw and Gideon Nave have a name for that: cognitive surrender, “the uncritical acceptance of AI’s answers in place of one’s own reasoning.” They build on Daniel Kahneman’s two systems, fast gut feel and slow effort, and argue AI adds a third: thinking that happens outside your head, inside the AI.
I use AI every day, for this channel and for my work, so this isn’t an anti-AI file. I’ve been in situations where I struggled to explain code I had just written with AI. I wanted to know what it’s doing to my own head.
Where the thinking fails
The plus 25 and the minus 15 come from the same habit. If you take the answer without checking, you get its right answers and its wrong ones. You’ve stopped being the filter.
Who surrendered most? Not the least intelligent people. The people who trusted AI the most followed its wrong answers more often. The ones with a stronger habit of reasoning caught more of the mistakes. It isn’t only a lab effect either: Microsoft researchers surveyed 319 knowledge workers, and the more people trusted AI on a task, the less critical thinking they reported. The more they trusted themselves, the more they did. Surrender often starts with self-doubt, not laziness.
The strongest pushback is old. In Plato’s Phaedrus, Socrates tells of a king who warns that writing will make people forgetful: “They will appear to be omniscient and will generally know nothing.” The philosopher Andy Clark argues that tools have always been part of how we think, the extended mind, and psychologists call it cognitive offloading. Even the famous 2011 study that said Google was changing our memory is shakier than the headlines: when a big replication project reran its first experiment in 2018, the effect didn’t show up.
The software engineer who writes as dinos drew the line better than anyone I read. The question isn’t whether you offload. It’s whether you offload the infrastructure of the thinking or the thinking itself. A calculator does the arithmetic; you still pick the formula, read the result and decide what it means. AI is the first tool that offers to do all four.
That’s where Lisanne Bainbridge’s 1983 paper, “Ironies of Automation”, comes in. Automation takes the easy, everyday tasks and leaves the human the rare hard ones. The human gets less practice, so when the hard moment comes, they’re less prepared than ever. Anthropic calls its own version the paradox of supervision: supervising Claude “requires the very coding skills that may atrophy from AI overuse.” Then it tested it. Fifty-two mostly junior engineers learned a new coding tool, some with AI and some without. On the quiz afterwards, the AI group averaged 50%, the hand coding group 67%, and the AI group wasn’t meaningfully faster.
The evidence an auditor should ask for
In a high school in Turkey, nearly a thousand maths students used plain ChatGPT and their practice scores went up 48%. When the AI was taken away for the exam, they did 17% worse than students who never had it. dinos calls where this ends up executive atrophy: losing the habits of setting the goal, breaking down the problem and judging the result, which are the exact skills you need to manage an AI.
Then zoom out. Doshi and Hauser found writers who got ideas from an AI wrote stories rated more creative and better written, but the AI-assisted stories were more similar to each other. Each person got better; the group got narrower. MIT’s small Your Brain on ChatGPT study, fifty-four people released as a preprint, found people who wrote with ChatGPT felt the least ownership of their essays and struggled to quote what they had just written. Treat it as a warning light, not a verdict. If ten people bring back the same AI’s answer, I think the company is paying ten salaries for one opinion.
The same studies show the fix:
- Turkey: a third group’s AI was set up as a tutor that gave hints, not answers. Practice scores went up 127%, and when the AI was taken away, most of the harm was gone.
- Anthropic: engineers who handed the whole task to the AI scored around 39%; those who generated code and then asked the AI to explain it scored 86. The groups were tiny, two to seven people, so it’s a hint.
- Wharton: people rewarded for accuracy and told at once if they were right overruled the wrong AI far more often.
It’s not the tool. It’s which job you give it: substitution, where the AI does the thinking you needed to do, or scaffolding, where it helps you do it yourself. Addy Osmani describes a ladder: offloading at the top, where you still own the result, deferral in the middle, surrender at the bottom. Nobody jumps to the bottom. You slide one reasonable step at a time.
The test to run this week
Ask one question: could I explain this and defend it without the AI in the room? Be careful, because your brain says yes too fast. In 2002, Rozenblit and Keil asked people how well they understood everyday things like a zip or a flush toilet. People rated themselves high, then had to explain how it works step by step, and their ratings dropped. If you can’t explain it, you didn’t offload it. You surrendered it.
Four habits keep you at the top of the ladder:
- Guess first. Write down your own answer before you ask, even a bad one. In 1978, Slamecka and Graf showed words people generated themselves were remembered better than words they just read.
- Hunt the rival. Ask what would have to be true for this to be wrong, and have the AI argue the other side as hard as it can. Then you decide.
- Explain it back. Close the answer and say the main point in your own words, or tell the AI: “Don’t give me the answer, quiz me.”
- Match it to the stakes. Hand over decisions where being wrong is cheap, like picking a hotel. Keep the ones where someone else lives with the result.
And once a week, do one thing you’d hate to forget how to do without AI. As Gideon Nave puts it, the goal is to benefit from AI when it’s correct and not lose when it’s wrong. Keep the plus, refuse the minus.
Sources
- Wharton: Thinking, Fast, Slow, and Artificially (Shaw & Nave, cognitive surrender)
- Shaw & Nave, the paper (SSRN 6097646)
- Microsoft Research (CHI 2025): GenAI and critical thinking, 319 knowledge workers
- Frederick (2005): the Cognitive Reflection Test
- Addy Osmani: Critical thinking when agents do the work
- Plato, Phaedrus (Jowett translation)
- Andy Clark (2025): Extending Minds with Generative AI, Nature Communications
- Google effects on memory (2011) and its 2018 replication
- dinos: Executive Atrophy
- Bainbridge (1983): Ironies of Automation
- Anthropic: How AI is transforming work at Anthropic (the paradox of supervision)
- Anthropic: AI assistance and coding skills
- Bastani et al. (2025), PNAS: the Turkey high-school maths study
- Doshi & Hauser (2024), Science Advances: AI ideas and creative writing
- Kosmyna et al. (2025): Your Brain on ChatGPT (MIT, preprint)
- Rozenblit & Keil (2002): the illusion of explanatory depth
- Slamecka & Graf (1978): the generation effect
- Bill Burkey's reply on Donald Campbell and rival explanations