Where did all the productivity in AI go?

In 2025, METR paid 16 experienced open-source developers $150 an hour to work on 246 real issues in their own mature repositories—codebases averaging over a million lines. Half the tasks allowed AI; the other half didn’t¹. The developers believed that the AI would make them 24% faster. After doing the work, they believed they were 20% faster. The test measured 19% slower².

That is a 39% difference between what they believed and what the test measured. When I read this report last year, I was shocked. While I don’t believe AI will take away huge swaths of jobs as some do, I still believe AI will offer significant productivity gains. Trinsic provides AI hosting and applications, so we have a vested interest in AI working.

Two types of evidence on AI productivity disagree. At the Micro level (one worker, one task), you can find abundant evidence of productivity gains. The Quarterly Journal of Economics reported³ in 2025: +15% more issues resolved per hour and a +30% productivity increase for the least experienced customer support agents. These are real productivity gains.

Yet when around 6,000 executives across four countries were surveyed about revenue gains from AI, they reported only a cumulative gain of 0.29%⁴. Worse, utilization-adjusted US total factor productivity grew only 0.07% over the four quarters ending Q1 2026.

The data isn’t asking whether AI increases productivity; it is asking why it disappears as you move up the ladder.

Before we answer this question, we need to take a step back and define what we mean by a ‘productivity increase.’ They generally measure productivity in two ways (there are others, but these are the most commonly cited). The first is Labor Productivity (LP), and the Second Is Total Factor Productivity (TFP).

The distinction matters. LP rises for reasons unrelated to anyone getting better at their job. If you buy your employees better equipment and each worker produces more per hour, great. But the equipment is a capital expense and could therefore be a wash for the bottom line. TFP is what’s left after you count what it cost you for the productivity gain. If AI is truly providing productivity gains, then we should see it in the TFP. Here is the problem: we don’t.

From 2024 through 2025, amid an AI-accelerated implementation, LP and TFP dropped, which isn’t what we should see if AI is providing large productivity gains. TFP growth halved in the heaviest year of AI deployment yet. The data center buildout held LP up, contributing nearly 39% of US GDP growth in the first nine months of 2025. This measures investment, not productivity gains, which is why it doesn’t show up in TFP⁸.

Let me give you an example. If one of our clients buys forty new laptops for their employees, and they are much faster than the old ones, there will be some productivity gains (LP). Still, when you add the capital expense, it will wash out at the bottom line (TFP). This seems to be what’s happening with AI. Making someone’s job easier (LP) doesn’t always translate to a revenue increase (TFP). GDP growth and a productivity boom don’t always go hand in hand.

What makes the data more complex is that it’s all over the place.

This is why you should evaluate vendor reports carefully. This includes even the author’s company (Trinsic Technologies Inc.). To see true ROI and productivity gains with AI, you need to measure work before and after AI implementation. This also includes understanding what AI is good at and what it isn’t.

In a pre-registered randomized controlled trial, 758 BCG consultants found that when AI was used within its capability range¹² it delivered +12.2% more tasks completed, +25% faster completion, and +29.9- 33.9% higher quality—real, replicated results.

However, on a task deliberately built to sit just past the edge of what AI can handle, consultants using GPT-4 were 19 percentage points less likely to be correct. Worse, those who received prompt-engineering training did better than those who did not. To add insult to injury, the wrong answers were rated more coherent and more persuasive than the unassisted wrong answers. AI hallucinations remain a real problem.

AI is jagged intelligence, doing well in some tasks and not well in others. While LLMs have improved, this remains a problem. The following costs don’t show up in the ROI models:

METR’s developers accepted fewer than 44% of AI suggestions and spent 9% of their total task time reviewing and cleaning AI output¹³.

In the Danish data, roughly 17% of workers in AI initiative workplaces report new AI-created tasks — and about 35% of those new tasks are AI quality review and AI ethics compliance¹⁴. The tool created a job whose purpose is to check the tool.

The clearest published measurement of skill erosion: across 1,443 unassisted colonoscopies at four Polish centers, adenoma detection fell from 28.4% before AI was introduced to 22.4% after¹⁵. Observational, not randomized — say so in the sentence.

What does this tell us? AI should be used carefully, particularly in regulated industries such as Law Firms, TAS, call centers, medical clinics, etc. Regulatory agencies have been slow to enforce privacy concerns related to AI. This will not last. In many of these industries, wrong answers can have serious consequences.

However, the skeptics don’t get a free pass either. METR walked back its own headline, saying it had to abandon the study after 30-50% of developers admitted they withheld exactly the tasks they thought AI would speed up. METR believes developers are working faster now, but they can’t measure it¹⁶.

Many people cite the MIT study that shows 95% of AI implementations fail (I have been guilty of this). However, that is not what the MIT study showed. It showed that only 5% of AI projects actually reach the implementation phase. This doesn’t mean the other 95% necessarily failed; they didn’t reach production for various reasons^17,18 .

The AI implementation is real. Some evidence suggests productivity gains at the micro level, but these gains aren’t translating into TFP or the bottom line. Why? Three potential answers fit the data.

The J-Curve – gains arrive only after organizations reorganize around the tool. We’ve seen this curve with other technology adaptations, including the Internet and manufacturing¹⁹.

Intensity – Firms haven’t actually bought much. 37% of organizations report any EBIT impact from AI and only 6% attribute more than 5% — both unchanged from 2025, while the share scaling AI across the enterprise rose from 38% to 44%²⁰. We are still early in this wave.

Ceiling – The gains are real but narrow — fewer than 3% of workplace tasks have AI adoption above 50%, and none exceed 70% — and verification, oversight, and rework eat a share of what’s left²¹.

It is 2026, and the question is what level of AI productivity gains will be realized. It is still too early to tell. Evidence suggests there will be gains, but they may not live up to the hyperbole. One thing we do know is that only time and the consumer will tell.