What Are You Losing While AI Makes You Faster?

A Joint Anthropic-Stanford Study Reveals the Hidden Cost of AI Assistance

A seemingly paradoxical phenomenon is emerging.

Over the past two years, we've witnessed an explosive growth in AI coding assistants. From GitHub Copilot to Cursor, from Claude to ChatGPT, nearly every developer has begun relying on these tools to some degree. The corporate rationale for deploying them is straightforward—boost efficiency, accelerate output. But a recent paper from researchers at Anthropic and Stanford University poses an unsettling question: When AI helps you complete tasks faster, what exactly is your brain learning?

The answer may be more troubling than we expected.

A Carefully Designed Experiment

The research team devised an elegant test scenario. They recruited 52 experienced Python programmers and randomly assigned them to two groups: one could use a GPT-4o-powered AI coding assistant, while the other could only rely on traditional documentation and search engines. The task was to learn and use an asynchronous programming library called Trio—a tool none of the participants had encountered before.

The brilliance of this design lies in what it measures: not the application of existing skills, but the acquisition of new ones. After 35 minutes, regardless of whether they had used AI, all participants had to complete a test with AI completely disabled, assessing their conceptual understanding of Trio, code reading ability, and debugging skills.

Figure 1: The Impact of AI Assistance on Coding Speed and Skill Acquisition

Source: Anthropic & Stanford University, "How AI Impacts Skill Formation", arXiv:2601.20245

The results were unexpected.

Participants who used AI assistance scored an average of 17% lower on the test—equivalent to a full two grade levels. And in terms of task completion time, the AI group was not significantly faster than the control group (p=0.391). In other words, AI neither made people faster nor helped them learn more.

This stands in stark contrast to the prevailing market narrative.

The Efficiency Illusion and the Skipped Learning

Why does this happen?

By analyzing participants' screen recordings frame by frame, researchers uncovered a key phenomenon: while AI can generate code instantly, participants spent considerable time on "prompt engineering"—figuring out how to phrase their questions, evaluating AI responses, deciding whether to adopt suggestions. Some participants spent over 10 minutes interacting with AI. This hidden cost offset any speed advantage AI might have provided.

Figure 2: AI Bypasses the Learning Phase

Source: Anthropic & Stanford University, "How AI Impacts Skill Formation", arXiv:2601.20245

But the more fundamental issue is this: AI creates a shortcut that bypasses learning entirely.

The diagram above illustrates this mechanism clearly. Without AI assistance, there's a "learning" phase between novice and task completion—you need to read documentation, understand concepts, attempt coding, encounter errors, debug and fix. Each step builds neural connections in your brain. With AI, this path is compressed into a straight line: ask, paste, done. The task is completed, but learning never happened.

The research data supports this. Control group participants encountered an average of 3 errors during the task, while the AI group encountered only 1. More importantly, those who encountered more Trio-specific errors (like RuntimeWarning and TypeError) performed better on subsequent tests.

Errors aren't bugs—they're features. At least for learning.

Six Personas, Two Destinies

If the story ended here, the conclusion would be simple: disable AI, return to basics. But the researchers' findings are far more nuanced.

By analyzing participants' interaction behaviors with AI, the team identified six distinctly different usage patterns. These six patterns showed striking differences in test scores and task completion times.

Figure 3: Performance Distribution of Six AI Interaction Patterns

Source: Anthropic & Stanford University, "How AI Impacts Skill Formation", arXiv:2601.20245

Three patterns led to severe loss of skill acquisition:

AI Delegation: Letting AI write the code directly, only responsible for pasting. Fastest task completion (19.5 minutes), but test scores of only 39%. These participants encountered almost no errors—because AI-generated code typically "just works."

Progressive AI Reliance: Initially trying to understand things independently, gradually surrendering after encountering difficulties, eventually depending entirely on AI. Completion time 22 minutes, score 35%.

Iterative AI Debugging: When code throws an error, directly throwing the error message at AI with "fix it." In this pattern, humans become mere couriers for AI. Longest completion time (31 minutes), lowest score (24%).

But the other three patterns painted an entirely different picture:

Conceptual Inquiry: Only asking AI "what's the principle behind this" or "what's the logic of this code," then writing code independently. Completion time 22 minutes, score 65%.

Hybrid Code-Explanation: Having AI generate code while requiring explanations for every line. Completion time 24 minutes, score 68%.

Generation-Then-Comprehension: After AI generates code, spending time carefully reading and manually modifying it rather than directly pasting. Completion time 24 minutes, score 86%—even exceeding some participants who didn't use AI at all.

These three high-scoring patterns share one common trait: maintaining cognitive engagement. AI was treated as a tutor or reference, not a subcontractor.

The Bigger Picture: The Bottom of the Career Ladder Is Being Eroded

This study's findings don't exist in isolation.

A few months ago, while analyzing another study, I wrote that generative AI is playing the role of a "seniority-biased technological change"—its primary impact isn't laying off senior employees, but closing the hiring gate for entry-level positions. In companies adopting AI, the ratio of junior to senior employees dropped by 12%, primarily through hiring cuts (averaging 22% reduction) rather than layoffs.

Putting these two studies together, a troubling picture emerges:

At the macro level, companies reduce entry-level hiring, leaving newcomers with fewer opportunities to enter the workforce. At the micro level, even those who secure jobs find AI assistance eroding their skill accumulation. This is a pincer movement.

What makes it more insidious is that neither problem is easily detected. Companies don't lay people off—they just stop hiring—which doesn't show up in unemployment statistics. Individuals using AI improve short-term efficiency without immediately realizing their skills are declining. Both are "boiling frog" crises. Jason Calacanis calls this phenomenon "Invisible Unemployment"—the shrinking of entry-level positions won't immediately appear in statistics, but it's fundamentally changing the underlying structure of the labor market.

A Paradox: Where Will the Supervisors Come From?

This leads to a deeper paradox.

The current AI narrative tells us that humans will transition from "executors" to "supervisors"—we no longer need to write code ourselves, just review and debug AI-generated code. But this study reveals a critical problem: If a person has never built deep technical understanding through hands-on coding, encountering errors, and debugging fixes, how will they possess the capability to review AI output?

The research data shows that debugging ability showed the largest gap among all skill dimensions. Participants who didn't use AI, because they had to handle errors themselves, actually remembered many pitfalls precisely because of the "struggle." Meanwhile, AI-assisted participants were helpless when facing errors once separated from AI.

This isn't merely an academic question. In safety-critical domains like finance, healthcare, and aviation, the ability to review and debug may be the last line of defense against disaster.

How to Use AI Correctly: The Cognitive Engagement Principle

The study's conclusion isn't "disable AI." Quite the opposite—it points toward a more refined usage strategy.

Based on the analysis of six interaction patterns, several approaches may help preserve learning outcomes while leveraging AI efficiency:

Explanation First: Require AI to explain the principle before providing code. "Why should it be done this way" promotes understanding better than "write the code for me."

Manual Intervention: Even when AI generates code, read it carefully, try typing or modifying it manually rather than directly pasting. The study found no significant difference between pasting and manual entry for skill formation—what matters is cognitive investment, not physical action.

Independent Debugging: When encountering errors, first try to locate the problem yourself, or ask AI "why is this wrong" rather than "fix it for me." Treat errors as learning opportunities, not obstacles to eliminate.

Define Boundaries: During the early stages of learning a new skill, consider consciously limiting AI usage, leaving space for your brain to build neural connections.

Conclusion

AI is not a substitute for skills—it's a skill amplifier.

If you lack foundational skills, AI can only provide short-term convenience while creating long-term capability deficits. Perhaps the most thought-provoking finding of this study is that the participants themselves recognized the problem. In post-study surveys, AI group participants commonly reported feeling "lazy," admitting there were "many gaps" in their understanding of Trio. Control group participants, meanwhile, found the task "very interesting," and despite facing challenges, reported more positive learning experiences.

In an era of widespread AI tools, we may need to revisit an ancient truth: There are no shortcuts on the road to mastery—or rather, paths that look like shortcuts often won't take you where you truly want to go.

Reference: Judith Hanwen Shen, Alex Tamkin et al., "How AI Impacts Skill Formation", arXiv:2601.20245, January 2026