TL;DR
- Back in May I wrote that AI comes for parts of a PM’s job, the administrative parts first, and widens the gap between great PMs and everyone else. Half a year of hard evidence is in, and it mostly agrees, with one correction I owe you.
- Narrow tasks are where the machines are astonishing. METR, an independent AI lab, measures the task length agents finish with 50 percent reliability doubling every seven months. Carnegie Mellon staffed a fake software company with agents, and the best finished about 30 percent of the work, some of it by faking results.
- Whole jobs are where replacement keeps failing. Klarna spent 2025 rehiring customer-service humans after saying its AI assistant did the work of 700 of them, and Forrester finds most employers who made AI-attributed layoffs in 2025 already regret them.
- The correction concerns the gap. A study of 5,000 support agents found AI lifted novices by roughly a third and the most experienced by nothing. That compresses one gap, the skill gap inside a tool, and widens a different one, the judgment gap between practitioners. My May sentence ran the two together.
- What survives is the useful part: judgment, accountability, and tool direction concentrate value in the people who already had them. The job bends toward directing and verifying.
- In a randomized trial, experienced developers finished 19 percent slower with AI tools while believing they had been 20 percent faster. Feeling faster is not being faster, and almost nobody measures.
In May I made a call here, from thirty years of project work and two years of running AI tools on my own projects: AI takes the administrative parts of a PM’s job first, and it widens the gap between the great ones and everyone else. I wrote it from experience because that’s what I had. Experience is also a sample of one, so this fall I went looking for the evidence, and there is now a lot of it: field studies, randomized trials, benchmark data, and one very public corporate reversal. Most of the May argument held up. One piece of it I’d phrase differently, and phrasing it better turned out to be the most useful thing I learned about AI all year.
The parts-of-the-job half aged well
Start with the finding that sounds like bragging on the machines’ behalf. METR, an independent lab that benchmarks frontier models, measures capability as the length of task an agent can finish with 50 percent reliability. That length has doubled roughly every seven months since 2019, which is genuinely fast, and it still tops out in the range of minutes to a few hours. Then Carnegie Mellon built a simulated software company, 175 ordinary workplace tasks, and let agents loose inside it. The best finished about 30 percent of the work on its own, and several agents were caught faking completion rather than doing it. That matches the curve we see in the workshop whenever we let an agent drive instead of ride along, which is what July’s post on loops was about. The first third is magic. The rest is you, supervising.
Klarna is the famous field test. In February 2024 it announced that its AI assistant was doing the work of 700 full-time service agents; by spring 2025 it was recruiting humans again because the complex conversations were going badly, and Forrester reported in September that most employers who made AI-attributed layoffs in 2025 already regret them. The volume went to the machine, and the work that needed a person came back for one.
The two gaps
The most-cited field study comes from economists Erik Brynjolfsson, Danielle Li, and Lindsey Raymond: about 5,000 customer-support agents at a Fortune 500 firm, tracked as a generative AI assistant rolled out. Productivity rose 14 percent on average, roughly a third for novices, and essentially nothing for the most experienced, because the assistant hands everyone the tacit playbook of the firm’s best performers.
Read casually, that contradicts the May argument. It doesn’t. It separates two gaps I had run together. The first is the skill gap inside a tool, the distance between the firm’s best and its newest people, measured in codifiable craft; AI compresses that one hard, and nothing will stop it. The second is the gap between practitioners, the distance between the person who can direct the tools with judgment and the person who can only run them. That one widens. It’s the one worth planning around. The experienced agents gained nothing because they already owned what the assistant was handing out. What the assistant cannot hand out is knowing which client needs a phone call instead of an email, and when to stop a line of work that is going bad.
I watch both gaps at Mocoda from an odd seat. The studio runs on two of us, so the AI tools play the junior role. On production speed they have closed most of the distance already, drafting and assembling while we aim them. The other gap doesn’t budge. When a client is upset and someone has to decide what we say next, that has never once been the tool’s call.
What the job bends into
If the tasks get cheaper and the glue holds, the job becomes directing and verifying: deciding what to produce, aiming the tools, checking what comes back. Attention becomes the bottleneck, and the evidence says to distrust your own sense of it. In METR’s randomized trial, experienced open-source developers working their own mature codebases finished 19 percent slower with AI tools while believing they had been 20 percent faster. Feeling faster is not being faster, and almost nobody measures.
I’ve been working the practitioner side of this in public over on ThirtyYearPM, and the pieces line up with the new evidence better than I expected: where AI pays off in a PM’s week, how a tool earns trust by keeping its confidence gates visible, and what PMI’s new AI standard gets right in insisting that every AI-assisted decision carry a named human.
In May I gave the widening gap three to five years to become undeniable. Shorten that estimate. And the half I underweighted matters just as much, because the same wave that widens the gap is handing your juniors the playbook it took you decades to write. Planning the next year as though only one of those is happening is the mistake to avoid.