The weight of the context window
Dex Horthy had analyzed one hundred thousand developer sessions with AI coding agents before he gave the problem a name. The agents were failing — not because the models were incapable, but because of where, in the sea of text they were given to read, the important information happened to land. He called it the dumb zone: the middle forty to sixty percent of a large context window, where transformer attention patterns degrade and recall falters. A file placed at position 100,000 in a 200,000-token context would often be treated, for all practical purposes, as if it had never been included. The fix for most failures had nothing to do with the prompt. It had to do with what went where.
On June 19, 2025, Tobi Lütke, the CEO of Shopify, posted a note on X that named the broader phenomenon. “I really like the term ‘context engineering’ over prompt engineering,” he wrote. “It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.” Andrej Karpathy endorsed it within the week: “in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step.” Neither man was proposing a new technique. They were naming something practitioners had been doing badly — and not yet thinking about clearly — for two years.
Prompt engineering, as a discipline, had always focused on the instruction: the wording of the request, the tone of the system prompt, the architecture of a few-shot example. Context engineering asks a harder question. Given that a model can only reason about what occupies its context window, and that the window is finite and nonuniform, what should be in there? Not just the instruction — but which documents, which conversation history, which tool schemas, and crucially, what gets cut when the token budget runs short. Context engineering is information architecture. The model is the CPU; the context window is working memory; and the practitioner’s job is to manage what lands there.
Horthy’s dumb zone formalized an insight that Stanford researchers had documented in 2023: language models perform best when relevant information sits at the beginning or end of the context. Bury the critical fact in a long middle section and the model behaves as if it was not there. The architectural response was immediate and practical: compressors that summarize older turns before the window fills; retrieval systems that pull documents at call time rather than pre-loading everything; XML-delimited formats that pack information more densely than the standard chat message structure. Position in the context window stopped being an accident. It became a design decision.
What changed was the mental model. Before context engineering, an AI application was a request-and-response system with a carefully worded system prompt. After it, an AI application is an information pipeline, and the model is a downstream consumer with documented constraints on what it can reliably use. Retrieval-augmented generation stopped being an optimization and became standard plumbing. Memory systems — short-term, episodic, semantic, long-term — became a recognized architectural layer rather than an afterthought. By 2026, every major AI framework had a context management API. The question was no longer whether to manage context, but how.
The context window is, in the end, working memory with a bill attached. Software architecture has always grappled with the gap between what a system knows and what it needs to know — that tension is as old as the first cache. The model made the gap visible, and started charging by the token.
Sources
- Tobi Lütke on X, June 19, 2025 — the tweet that named “context engineering” over “prompt engineering” as the core discipline
- Andrej Karpathy on X, June 2025 — endorsement and definition: “the delicate art and science of filling the context window with just the right information for the next step”
- 12-Factor Agents — Dex Horthy, HumanLayer — the dumb zone discovery, Factor 3 “Own Your Context Window,” analysis of 100,000 developer sessions at the AI Engineer Summit
- Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” — Stanford, 2023 — academic evidence that LLM recall degrades for information in the middle of long contexts
- Context Engineering in 2026: From Karpathy’s Tweet to Production Infrastructure — the 2026 consolidation of context engineering into standard framework APIs and production tooling