The Bigger the Window, the Quieter the Rot

The 30% cliff

Two years ago the models topped out around 200,000 tokens. Reasoning and attention were weaker, so the ceiling was not something we could escape from.

I remember that at around 30% full, the agent started to slip. It started to hallucinate well before the hard limit. It forgot instructions I had given it four steps earlier. It stopped following indirection, the “do X, then use the result to decide Y” chains that need the model to hold a thread. Back then there were no skills, no subagents, and no model that could really drive them, so the only lever I had was the window itself.

The debug loop was really the worst of it. When asked for a fix, the agent would run the tests and the wall of output alone would eat a chunk of what was left. Logs flood the window faster than the model can reason over them, so by the third iteration the thing that was supposed to be reading the error was drowning in it.

Then the windows grew. A million tokens. Reasoning caught up too. The pressure that forced the discipline just lifted off. Staying lean stopped being survival and quietly became a preference. And preferences slide.

A bigger window does not only tempt you to relax. It tempts you into bigger tasks. And in a big task, the rot hides: when the agent is chewing through forty files and a thousand lines of its own reasoning, you cannot read the run closely enough to catch the one place it went sideways. Abundance did not make the failure go away. It made it quiet.

ℹ️ Context window? The context window is the working set of tokens the model reads in one pass: your instructions, the open files, the running conversation, the tool output, all of it bounded by a capacity (say a million tokens). For the mechanics of how it is built and how it relates directly to costs, see what a context window actually is.

Better isn’t immunity

The windows are bigger now. But while that raised the ceiling on task complexity, it did not buy immunity to rot. Pile junk into a million-token window and you still get context rot. The room went up. The rot did not go away.

🔬 Measured, not guessed. In 2025, Chroma Research showed this directly: LLMs grow measurably less reliable as input length grows, even on trivial retrieval tasks, an effect they measured across 18 frontier models including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 (Chroma Research, Context Rot: How Increasing Input Tokens Impacts LLM Performance, 2025). Their report made “context rot” the standard name for the effect.

So it is the same fight it always was, just with more rope. Not the fewest tokens for their own sake. Not the most, because you can. The fewest and the best. That is the lean window, and keeping it lean is the job of the next generation of engineers and developers.

The flat whiteboard

Think of the window as a whiteboard, not a warehouse. A warehouse lets you keep a million things because they are boxed and shelved, out of sight, costing nothing to whatever you are working on right now. A whiteboard shows everything at once. Everything you wrote an hour ago is still up there, competing for the same wall. It charges rent on every mark, forever, whether you are still using it or not.

The model does weigh the marks differently. That is what attention is. But that weighting happens inside the model, not in front of you. At twenty thousand marks on the board, you cannot tell the load-bearing token from the dead one. They are all just marks. You can’t curate what you can’t distinguish, and that is exactly why the cleanup can’t be left to your eye in the moment.

There is a second cost, true no matter how the model weighs anything. Every mark on the board costs compute to read, on every step. So a fuller board is not only more error-prone. It is heavier and pricier to run, and it gets more so the fuller it gets (context-window primer). A board you keep clearing is a board that stays cheap and sharp.

Enforce leanness, don’t count on it

The governing move is this: build leanness into the environment. Do not trust the agent to police itself, and do not trust your own eyes as enough. You cannot see the rot coming at scale, make the harness do it. I organize that on two axes:

Control the flood. This is the influx inside the loop, the tool output that piles up while the agent works. Two moves handle most of it.

• Silence the happy path: emit nothing on success, so a green test run costs zero tokens instead of two hundred lines.
• Head-and-tail truncate of what remains. Keep the first N lines and the last N lines of a log, drop the middle. This is not arbitrary. Models neglect the middle of a long input (Liu et al., 2024), so the head and the tail are exactly the parts that survive attention anyway. The cut matches how the model actually reads.

Control the standing load. This is the persistent baseline, what sits on the board before the real work even starts.

• Progressive disclosure: load the one skill or doc a step actually needs, when it needs it, instead of the whole manual up front.
• Subagents: offload a sub-task to a worker so the main thread never sees the pile it generated, only the result. That is how you get a long session on a hard task without the main thread drifting.
• The handoff pattern: when I want to work on a task across multiple sessions or hand it to a colleague’s agent, I compress the state into a markdown file that contains decisions, the current goal, relevant file paths and line numbers, not pasted snippets. Any fresh session reads that file and picks up exactly where the last one stopped. A pointer costs a line. The pasted file costs the file.

Concretely, that handoff file can be as short as this:

## STATE: parser refactor
Decisions:
- Split the tokenizer out of parser.rs (it was doing two jobs). Kept the old
  error type; changing it breaks three callers.
Standing: steps 1 and 2 done. Step 3 (wire up the new tokenizer) is next.
Files: src/parser.rs:120 (entry point), src/tokenizer.rs (new), tests/parser_test.rs:40.
Open: does step 3 need a migration for the saved-session format?
Resume: run `just test parser`, then start at src/parser.rs:120.

Judge it by one test: could a fresh session, handed nothing but this file, carry on correctly? If not, it is missing something. In general you’ll want more information that the example above.

One caution on the word “enforce.” It is not an unbreakable rule. The agent is still the one doing the work, and the harness is a help to get the best of today’s models, not a cage. It’s a discipline where you want to stay agile. For now, there is no single all-in-one tool that covers all of this well, so you pick your pieces and wire your own, ideally as a team. Above this sits a heavier layer, specs a script can execute, orchestration, agents that remediate and escalate. That is its own story, not this one.

Don’t wait for it to go wrong

Watch the gauge. Don’t wait for the slip. The single most useful instrument I have is a status line that changes color as the window fills, so I can’t miss it even when I’ve stopped reading the raw number.

My redline is about 10% of a million-token window, roughly 100,000 tokens. That is the line I do not cross, not the level I run at. The color starts shifting well before it: yellow by around 50,000, half the redline, so the warning lands with plenty of room to act. These are my own numbers, a conservative margin I hold on frontier million-token models, not a law. Arrange your harness differently or route to different models and the thresholds move. Treat them as a rule of thumb.

The posture matters more than the exact figure. I’m not waiting for things to go wrong. And the gauge teaches a second lesson over time: sometimes the right read is not “trim the window.” It is “this session started wrong.” When the instructions were imprecise from the first message, adding context does not heal it, it just buries the mistake deeper. Restart fresh. Don’t salvage.

Fewest and best

The 200k era forced this discipline on everyone. The million-token era makes it optional. Which is why “how few tokens can I get away with?” is the wrong question to end on. It was never really the question.

The real one is the tension every serious harness lives inside. Give the agent the maximum information it needs, so it never burns tokens re-deriving the shape of a codebase it has already seen. But do not stuff the window to do it, or every session starts already 10% full and the board is never clean. Two failure modes: under-inform and it re-explores forever, over-stuff and it rots from the first message. One art threads between them: not the fewest tokens, not the most, but the most signal per token. Fewest and best.

Which leaves two open questions this piece deliberately does not answer. First, of everything you could always load, which tokens actually earn their place? Second, once you’ve chosen them, how do you keep that preload up to date as the code moves out from under it? Both are stories for another day. The window was never something to fill. It’s something you spend.

Sources

• Chroma Research, Context Rot: How Increasing Input Tokens Impacts LLM Performance, 2025. https://www.trychroma.com/research/context-rot
• Liu, N. F., et al., Lost in the Middle: How Language Models Use Long Contexts, TACL 2024. https://aclanthology.org/2024.tacl-1.9/