The Knowledge Problem: Why AGI Won't Win and Oracles Will

Picture a teenager working a Saturday shift at McDonald’s. They’re learning nothing about burgers, not really — those meaty slabs arrive pre-made. What they’re learning is systems: how a kitchen manages throughput under pressure, how a supply chain delivers consistency at scale, how a brand survives ten thousand individual acts of human error every day. Three years later they open a smash burger joint and they know things about running a kitchen that no culinary school teaches. They didn’t steal the recipe. They absorbed the logic.

This is how intelligence actually spreads. Not in one great equalising wave — where everyone gets the same firmware update and carries on with new abilities. But in a slow, steady leak of knowing — capability bleeding out from those who have it into those paying close enough attention. The teenagers become the chefs. The chefs become the restaurateurs. The OpenAI alumni become the OpenAI beaters. The knowledge travels, transformed and recombined, leaving behind the original but producing something better suited to the new context. Like fire passed from torch to torch.

I want to argue that AI will follow the same pattern. And that the dominant cultural story we’re telling about it — the one with a Singularity in it, a moment where general intelligence arrives and equalises everything — is the wrong story. It’s a seductive story. It’s also the same mistake we’ve made before. Call it Technological Communism.

The Perfect Idea That Fails on Imperfect Execution

Communism is not a bad idea because its goals are bad. Equality, shared resources, freedom from exploitation — these are genuinely good goals. Communism fails because it rests on an assumption that has never been true: that a central authority can hold enough knowledge to make good decisions for everyone. As a Berlin resident I can still sniff the residue of it in the city, and behind the eyeballs of the older generations. The good and bad of it lingers here.

Friedrich Hayek identified the underlying problem in 1945. In The Use of Knowledge in Society, he argued that the information required for efficient decisions is not just vast — it’s fundamentally distributed. It lives in the minds of shopkeepers and engineers and farmers, embedded in practice, resistant to being written down or transmitted upward. The factory foreman who knows which machine is about to fail. The analyst who senses a technology is about to inflect. The procurement lead who understands why a supplier relationship is worth more than the contract says. This knowledge is local, tacit, and irreducibly human.

Hayek’s conclusion: because you cannot centralise the knowledge, you should not centralise the decisions. The market works not because it aggregates information into one place, but because it processes information in parallel, everywhere at once.

Now apply this to AI. The dream of AGI — Artificial General Intelligence — is essentially a Hayekian nightmare. It assumes that a sufficiently large model, trained on sufficiently much data, will converge on a universal intelligence that can reason effectively about anything, for anyone, anywhere. The knowledge problem is assumed away. My argument is not that AGI won’t be smarter, but that it won’t be smart enough — not against a specialist system with sufficient compute, focused on a single domain, shaped by years of proprietary data. Specialism will be the natural consequence of scarcity.

The data is already bearing this out.

The Numbers Don’t Lie, But They Do Surprise

Here is the macro picture. According to McKinsey’s State of AI 2025 survey of nearly 2,000 organisations, 88% of companies are now using AI in at least one business function.¹ Nearly everyone has pulled up to the same trough.

And yet: only 39% report any measurable impact on EBIT at all. Of those, the majority attribute less than 5% of earnings to AI. Only 6% of organisations qualify as genuine AI high performers — those seeing meaningful, enterprise-wide financial returns.

McKinsey frames this explicitly as a new Solow Paradox, after the economist Robert Solow’s 1987 observation that “you can see the computer age everywhere but in the productivity statistics.”² We saw this with electrification. We saw it with computing. We are seeing it again. Crucially, the paradox resolved — but only after a significant lag. Erik Brynjolfsson, who coined the term “productivity paradox” in 1993, found that meaningful organisational returns from IT investments typically took years to materialise, and that productivity gains required complementary changes in how organisations actually worked, not just what technology they deployed.³ In a 2017 NBER working paper co-authored with Rock and Syverson, he applied the same framework to AI explicitly, arguing that implementation lags are the most likely explanation for why the numbers don’t yet match the hype.⁴ The gains are real. They take time. And they don’t distribute evenly.

The OECD has mapped where early concentration is forming. In 2024, 52% of large firms across the OECD were using AI, compared to just 17% of small firms.⁵ Crucially, the OECD notes that the recent acceleration in AI adoption has been driven more by leaders pulling further ahead than by laggards catching up.⁶ The gap is widening, not closing. Early adopters, the OECD warns, may build advantages in institutional knowledge and industry standards that later entrants will find very hard to overcome.⁷

This is not a technology story. It is a knowledge story. The model is not the moat.

McKinsey distinguishes three types of AI adopter: Takers, who use off-the-shelf tools via APIs and subscriptions; Shapers, who integrate foundation models with proprietary data to build custom applications; and Makers, who build their own models from scratch.⁸ The Maker path is prohibitively expensive for most. The Taker path is where most organisations currently sit — and it is, by definition, undifferentiated. The Shaper position is where competitive advantage actually lives: proprietary data, embedded institutional knowledge, customised intelligence that a competitor cannot simply purchase and replicate.

Right now, most organisations paste in, some fine-tune, the richest pre-train.

Oracles, Not Omniscience

Here is what I think actually happens. Not a Singularity. Not one intelligence that knows everything. Instead: thousands of companies building private oracles — AI systems shaped by their own data, their own curation decisions, their own institutional memory — that become progressively more valuable precisely because they reflect what no general model can know.

Consider what Tata Steel has built. Across its blast furnace operations, the company has deployed AI models trained on years of its own sensor data — temperature, pressure, chemical composition, energy flow — to predict equipment failures and optimise yield in ways that a general-purpose model simply cannot replicate. Not because the general model is stupid, but because the specific knowledge of this furnace, in this plant, under these operating conditions does not exist anywhere except in Tata Steel’s own data history. They are not using AI generically. They are building something that gets smarter about their specific problems, compounding over time, in directions that a late arrival cannot fast-follow.

Or consider AlphaFold. DeepMind built a system of extraordinary power for predicting protein structures, transforming structural biology and accelerating drug discovery.⁹ It is also, fundamentally, a narrow system — exquisitely good at one class of problems, shaped by the specific knowledge architecture of molecular biology. The pharmaceutical companies building on top of AlphaFold are not doing so generically. They are layering their own compound libraries, their own target hypotheses, their own experimental results — creating systems that get smarter in the specific directions their business needs. The general model enables the specialised oracle. The oracle becomes irreplaceable.

McKinsey’s high performers — that 6% seeing real returns — share one defining characteristic: their advantage is organisational, not technological. They do not have access to better models. They have fundamentally different approaches to embedding AI into their specific workflows, their specific data, their specific institutional knowledge. They are 3.6 times more likely than peers to be pursuing transformational, enterprise-level change rather than bolting AI onto existing processes.¹

This matches what David Teece has spent decades arguing about competitive advantage. In his original 1986 work on the appropriability of innovation, Teece identified that firms cannot fully capture the returns from their own knowledge, because knowledge leaks — through employee mobility, observation, and imitation.¹⁰ The question is never whether knowledge will leak. It always does. His later work on dynamic capabilities — developed with Pisano and Shuen and extended through multiple updates — went further: sustainable advantage comes not from holding a position, but from a firm’s ongoing capacity to sense new opportunities, seize them, and reconfigure its assets around them faster than competitors can follow.¹¹ An organisation that treats AI as a procurement event has a position. An organisation that builds AI into its sensing, learning, and reconfiguring has a capability. Only one of those compounds.

The companies that treat AI as a procurement decision — buy the same foundation model as every competitor, prompt it roughly the same way, call it a transformation — will discover that they have automated mediocrity. The companies that understand that the knowledge they hold that no one else holds is the asset worth building AI around will pull away. Slowly at first, then faster.

The Leak Is the Point

Back to the teenager in McDonald’s.

The smash burger they open in three years will be better than McDonald’s in the ways that matter to their customers. But McDonald’s will still exist to frustrate and inspire the next generation. And those cooks will take what they learn somewhere else. The knowledge leaks forward, recombined, specialised, improved. No one owns it permanently.

Capitalism isn’t the complete answer either. Pure markets, left entirely alone, concentrate knowledge in ways that calcify advantage and crowd out the new. The right model is something dynamic and multi-tenant: shared infrastructure at the foundation, fierce specialisation in the middle, and constant leakage at the edges that seeds the next generation. Grounded by the necessary tension between resource and regulation.

There is no Singularity coming. What is coming — what is already here — is a hierarchy of intelligence. General foundation models provide capability infrastructure. Proprietary knowledge layers — built slowly, through use, through curation, through accumulated institutional judgment — provide actual competitive differentiation. The AI that wins won’t be the most general. It will be the one that knows your business, your data, your problems — and has been quietly getting smarter about them for years.

There are thousands of oracles, growing in the dark, each one a map of what one organisation has learned to see.

The question is whether yours is one of them.

References

McKinsey & QuantumBlack, The State of AI (Nov 2025) — survey of ~2,000 organisations: 88% use AI in at least one function, yet only 39% report any EBIT impact and just 6% qualify as high performers; high performers are 3.6× more likely to pursue enterprise-wide transformation. Source: McKinsey. ↩ ↩²
McKinsey, “Where AI will create value — and where it won’t” (May 2026) — frames the AI-productivity gap as a new Solow Paradox. Source: McKinsey. ↩
Brynjolfsson, E. (1993). “The Productivity Paradox of Information Technology,” Communications of the ACM 36(12) — coined the term; found IT gains materialise only after a lag and complementary organisational change. Source: ACM. ↩
Brynjolfsson, E., Rock, D. & Syverson, C. (2017). “Artificial Intelligence and the Modern Productivity Paradox,” NBER Working Paper 24001 — argues implementation lags best explain why AI’s measured gains trail the hype. Source: NBER. ↩
OECD, “AI adoption by small and medium-sized enterprises” (Dec 2025) — documents a wide, widening AI-adoption gap between large and small firms across the OECD. Source: OECD. ↩
OECD, “Emerging divides in the transition to artificial intelligence” (Regional Development Papers No. 147, Jun 2025) — finds recent AI diffusion driven more by leaders pulling ahead than laggards catching up, with cross-firm gaps widening. Source: OECD. ↩
OECD, “Emerging divides in the transition to artificial intelligence” (2025) — warns that early adopters can lock in advantages in data, standards and institutional knowledge that late entrants find hard to overcome. Source: OECD. ↩
McKinsey, “A generative AI reset: Rewiring to turn potential into value in 2024” — distinguishes Takers, Shapers and Makers of AI. Source: McKinsey. ↩
Jumper, J. et al. (2021). “Highly accurate protein structure prediction with AlphaFold,” Nature 596 — the AlphaFold2 paper that transformed structural biology. Source: Nature. ↩
Teece, D.J. (1986). “Profiting from technological innovation,” Research Policy 15(6):285–305 — returns from innovation often accrue to owners of complementary assets rather than the innovator, because knowledge leaks. Source: ScienceDirect. ↩
Teece, D.J., Pisano, G. & Shuen, A. (1997). “Dynamic Capabilities and Strategic Management,” Strategic Management Journal 18(7):509–533 — sustainable advantage comes from sensing, seizing and reconfiguring assets faster than rivals. Source: RePEc. ↩