What is distributed cognition in AI systems?

A frame from cognitive science (Hutchins): cognitive processes span people, artifacts, and protocols, so the unit of intelligence is the system, not any component. Applied to AI: capability lives in the arrangement of model, precomputed knowledge, memory, verification loops, and humans - not primarily in the model.

Should I pay for a frontier model or invest in architecture?

Run a swap test: your current architecture and your previous (or a naive) one, crossed with a frontier and a mid-tier model, on your real eval suite. In mature task domains, architecture typically moves quality more than a model generation does - but on frontier tasks with no mature architecture, model capability is the whole game.

What's the difference between compensatory and constitutive architecture?

Compensatory architecture patches current model weaknesses (decomposition scaffolds, format enforcement) and becomes obsolete as models improve. Constitutive architecture provides properties no model can have from the inside - persistence, auditability, isolated verification, deletion guarantees, ground truth - and grows more valuable as models improve. Audit each component by asking: "if the model were twice as good, would this be pointless?"

Does the bitter lesson contradict architecture-first thinking?

No - it bounds it. The bitter lesson says learned general methods beat engineered cleverness at producing capability, and it keeps winning there: compensatory architecture dies on schedule. Constitutive architecture isn't producing capability; it's governing it (memory, audit, isolation, correction), and those properties live outside the learned component by definition.

Is a multi-agent system an example of distributed cognition?

Only when built like an organization: narrow roles, typed artifacts passed between agents, and trust boundaries - not a shared transcript that every agent reads. Shared-everything multi-agent systems distribute confusion, not cognition; the coordination protocol is where the intelligence lives.

What does this essay explain about the downgrade that didn't hurt?

The finance team did what finance teams do, and I will forever be grateful to them for it. Late last year, our inference costs crossed a threshold that triggered a review, and the directive came down: cut model spend by half. The obvious lever was the big one - move our document intelligence platform's primary model from the frontier tier to the mid tier. Roughly a generation of capability, on paper. The benchmarks…

What additional detail does the "The Downgrade That Didn't Hurt" section cover?

Late last year, our inference costs crossed a threshold that triggered a review, and the directive came down: cut model spend by half. The obvious lever was the big one - move our document intelligence platform's primary model from the frontier tier to the mid tier. Roughly a generation of capability, on paper. The benchmarks between those two models showed gaps of ten, fifteen points on reasoning suites.

What does this essay explain about the swap test?

Ruling out the eval-garbage hypothesis left the second possibility, and it suggested an experiment I now think every serious team should run. We had, sitting in version control, the *original* architecture of the same product - the naive RAG pipeline from before the knowledge graph, the derived tables, the verification passes, the memory layer, the context ledger. Eighteen months of essays five through eight, before…

What additional detail does the "The Swap Test" section cover?

So we ran the full two-by-two. Old architecture and new architecture, frontier model and mid-tier model, same eval suite, same corpus.

What does this essay explain about a ship knows where it is?

In the 1990s, the anthropologist Edwin Hutchins spent years aboard US Navy vessels studying how a ship navigates, and wrote up the findings in *Cognition in the Wild*. His observation sounds simple and is not. No individual aboard knows where the ship is.

What additional detail does the "A Ship Knows Where It Is" section cover?

No individual aboard knows where the ship is.

What does this essay explain about system iq and model iq?

Let me name the distinction the 2x2 forces. Model IQ is the capability of the reasoning component in isolation - what benchmarks measure. System IQ is the capability of the full arrangement: model, artifacts, protocols, loops, and humans, measured on the tasks the system actually exists to do.

What additional detail does the "System IQ and Model IQ" section cover?

Model IQ is the capability of the reasoning component in isolation - what benchmarks measure. System IQ is the capability of the full arrangement: model, artifacts, protocols, loops, and humans, measured on the tasks the system actually exists to do.

What does "Architecture multiplies; models add" mean in this essay?

The frontier model added 13 points to the naive architecture but only 2 to the good one - because the good architecture had already *removed most of the opportunities to fail* that the frontier model's extra capability was rescuing.

What does "Model IQ has a threshold, not a gradient" mean in this essay?

Below some competence floor, nothing helps - the 2019-era models couldn't run this system at any architecture level, because every component interaction assumes basic instruction-following and tool reliability.

What does "Benchmarks measure the wrong unit" mean in this essay?

Every leaderboard is a single-context, single-shot, no-artifact evaluation - model IQ by construction.

What key claim does the essay make about system iq and model iq?

AI capability is a property of arrangements, not components. Past a competence threshold, system IQ is dominated by architecture - precomputed knowledge, memory, verification loops, division of labor - rather than by model IQ. Benchmarks rank components; users experience arrangements. The two diverge, increasingly.

What does this essay explain about the cognitive division of labor?

If intelligence lives in the arrangement, the central design activity changes. It stops being "how do I get the most out of the model" and becomes something closer to organizational design: which substrate should perform each kind of cognition? Because here's the uncomfortable economics, stated plainly. For any cognition that *some other substrate can perform*, the LLM is almost always the most expensive and least r…

What additional detail does the "The Cognitive Division of Labor" section cover?

Because here's the uncomfortable economics, stated plainly. For any cognition that *some other substrate can perform*, the LLM is almost always the most expensive and least reliable place to perform it.

What does this essay explain about constitutive versus compensatory architecture?

Now the sharpest distinction in this essay - the one that determines whether everything above is durable insight or a snapshot of 2026's limitations. The obvious objection to architecture-first thinking: *models keep absorbing architecture*. Chain-of-thought prompting became reasoning models. Retrieval hacks became long context. Trajectory search (essay four's navigation budget) got internalized as test-time compute…

What additional detail does the "Constitutive Versus Compensatory Architecture" section cover?

The obvious objection to architecture-first thinking: *models keep absorbing architecture*. Chain-of-thought prompting became reasoning models. Retrieval hacks became long context. Trajectory search (essay four's navigation budget) got internalized as test-time compute. The OS essay already conceded providers keep absorbing layers. If the pattern continues, isn't today's architecture just scaffolding awaiting demoli…

What does "Compensatory architecture patches a model weakness" mean in this essay?

Decomposition scaffolds for models that lose the thread; format-enforcement layers for models that drift from schemas; the elaborate re-ranking dances that compensate for attention dilution.

How should readers understand: Constitutive architecture provides properties a model cannot have from the inside - at any capability level?

*Persistence*: weights are frozen and contexts evaporate; accumulated, governable experience requires an external store (essay three's whole argument).

What This Looks Like in Real Systems?

Document intelligence: the swap test's own domain. Everything essay six built - graph, derived tables, versioned states - is chart-making: cognition moved to write time, crystallized into artifacts, spent once instead of per-request. The mid-tier model scores 89 because answering "total exposure" requires *reading a number the arrangement already computed*, not deriving it in a diluted context. Multi-agent systems:…

What additional detail does the "What This Looks Like in Real Systems" section cover?

Multi-agent systems: organizations, not brains. The persistent temptation is to see a multi-agent system as one big distributed mind and share everything - which essay two showed is how agents poison each other. Hutchins' ship says the opposite: the power comes from *narrow roles, typed messages, and artifacts* - the bearing-taker doesn't stream consciousness to the plotter; he says "lighthouse, 084 degrees." Artifa…

What does "Document intelligence: the swap test's own domain" mean in this essay?

Everything essay six built - graph, derived tables, versioned states - is chart-making: cognition moved to write time, crystallized into artifacts, spent once instead of per-request.

What does "Multi-agent systems: organizations, not brains" mean in this essay?

The persistent temptation is to see a multi-agent system as one big distributed mind and share everything - which essay two showed is how agents poison each other.

What does "NL2SQL: the quiet prototype" mean in this essay?

NL2SQL is distributed cognition so mature nobody notices it: the model translates intent, the database *does the thinking*, the syscall boundary (essay five) governs the trust seam.

What does "The verification economy" mean in this essay?

Once you see checking as a substrate question, an asymmetry appears with real strategic weight: *verification is usually cheaper than generation* - a mid-tier model with the right artifacts can check what a frontier model produced (does the citation support the claim? does the number match the table? does the plan touch only permitted tools?).

What are the limits of the argument presented in this essay?

Capability thresholds are real, and they move. Some tasks have no architectural rescue: if the model can't follow a four-constraint instruction or produce sound code, no chart saves the voyage. And each new model generation *creates task categories where everyone is back at the naive corner of the 2x2* - agentic computer use in 2025 had no mature architecture to lean on; frontier model IQ was the whole game. Archite…

What additional detail does the "Where This Argument Breaks" section cover?

The 2x2 flatters architecture. Honest methodological note: our "naive" baseline was real but unloved, and eighteen months of eval-driven iteration had - unavoidably - tuned the architecture *to the eval*. I believe the direction generalizes; I'd bet less on the magnitude. Run your own swap test before reallocating your budget on mine.

What does "Capability thresholds are real, and they move" mean in this essay?

Some tasks have no architectural rescue: if the model can't follow a four-constraint instruction or produce sound code, no chart saves the voyage.

What does "The 2x2 flatters architecture" mean in this essay?

Honest methodological note: our "naive" baseline was real but unloved, and eighteen months of eval-driven iteration had - unavoidably - tuned the architecture *to the eval*.

What does "Arrangements have their own failure modes" mean in this essay?

Distributed cognition distributes *failure* too: Hutchins documented the bridge protocols breaking down under casualty conditions, and our arrangements deadlock (essay five), thrash, accumulate debt (essay eight), and rot in ways a single model call never does.

What does "Providers may sell the arrangement" mean in this essay?

The strongest commercial objection: model providers are shipping memory APIs, managed retrieval, computer-use sandboxes - maybe system IQ becomes something you *buy*, collapsing the distinction back into "use the best vendor."

What should readers take away about a forced model downgrade cost 1.8 eval points?

A forced model downgrade cost 1.8 eval points, and a full 2x2 swap test explained why: a mid-tier model in a mature architecture (89%) beat a frontier model in a naive one (74%) - architecture was worth more than a model generation.

What should readers take away about hutchins' distributed cognition names the reason: intelligence is a property of arrangements - models?

Hutchins' distributed cognition names the reason: intelligence is a property of arrangements - models, precomputed artifacts, memory, verification loops, protocols, and humans - and most cognition in a good system happened before the request arrived, crystallized into chart-like artifacts at write time.

What should readers take away about system iq diverges from model iq past a competence threshold?

System IQ diverges from model IQ past a competence threshold, because good architecture converts model weaknesses into non-problems; benchmarks measure components while users experience arrangements.

What should readers take away about design becomes a division of labor: computation to databases?

Design becomes a division of labor: computation to databases, structure to graphs, experience to memory, judgment to the model, checking to isolated passes, ground truth to humans.

What should readers take away about the durable half of this is constitutive architecture - persistence?

The durable half of this is constitutive architecture - persistence, audit, isolation, deletion, ground truth - which no model provides from inside and which appreciates as models improve; compensatory architecture, which patches model weaknesses, should be expected to die with model progress.

What should readers take away about the open frontier: arrangements are beginning to design themselves?

The open frontier: arrangements are beginning to design themselves.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to AI system architecture?

The Future of AI Is Distributed Cognition, Not Bigger Models uses AI system architecture as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to model selection LLM?

The Future of AI Is Distributed Cognition, Not Bigger Models uses model selection LLM as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to compound AI systems?

The Future of AI Is Distributed Cognition, Not Bigger Models uses compound AI systems as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to multi-agent systems?

The Future of AI Is Distributed Cognition, Not Bigger Models uses multi-agent systems as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to the bitter lesson?

The Future of AI Is Distributed Cognition, Not Bigger Models uses the bitter lesson as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to AI scaling limits?

The Future of AI Is Distributed Cognition, Not Bigger Models uses AI scaling limits as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to LLM cost optimization?

The Future of AI Is Distributed Cognition, Not Bigger Models uses LLM cost optimization as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

How does The Future of AI Is Distributed Cognition, Not Bigger Models relate to human in the loop?

The Future of AI Is Distributed Cognition, Not Bigger Models uses human in the loop as a lens for explaining production LLM system behavior, failure modes, and engineering tradeoffs.

June 28, 2026

The Future of AI Is Distributed Cognition, Not Bigger Models

A mid-tier model in a well-built system beat a frontier model in a naive one. Where intelligence actually lives in AI systems - and where to invest next.

Key Takeaways

A forced model downgrade cost 1.8 eval points, and a full 2x2 swap test explained why: a mid-tier model in a mature architecture (89%) beat a frontier model in a naive one (74%) - architecture was worth more than a model generation.
Hutchins' distributed cognition names the reason: intelligence is a property of arrangements - models, precomputed artifacts, memory, verification loops, protocols, and humans - and most cognition in a good system happened before the request arrived, crystallized into chart-like artifacts at write time.
System IQ diverges from model IQ past a competence threshold, because good architecture converts model weaknesses into non-problems; benchmarks measure components while users experience arrangements.
Design becomes a division of labor: computation to databases, structure to graphs, experience to memory, judgment to the model, checking to isolated passes, ground truth to humans.

Key Takeaways

FAQ

Related Expertise

Related Concepts

Related Projects

Related Research

Related Articles

Related Automations