The Headroom teardown: zero tokens saved, then 88% - depending entirely on how you wire it up

Dave Slutzkin asked last week how Headroom - the token compressor doing the rounds - was actually working out for people. I spent a couple of nights finding out, and the honest answer is: it depends entirely on how you install it, to a degree that should worry you.

First pass: zero tokens saved, and the dashboard agreed

Used the way it’s built to be used, Headroom is sharp. Point your whole coding session through it and it does real work: it reads Anthropic’s cache markers, freezes the part of your context that’s already cached, and only rewrites the tail the model is about to respond to. It runs a small cost model to check whether compressing a given block would actually save more than the prompt-cache discount it would burn by invalidating that cache. Different logic for Claude and for Codex, because their caching works nothing alike. Good engineering - I went in expecting to find the opposite.

Then there’s how most people will actually install it. I wired its compress endpoint straight into my editor, which is the obvious way to use it without handing over your whole session. That path throws the clever stuff away entirely. No sense of where your cache boundary is. It tried to compress my entire message history, mangled some of my own past turns doing it, and a safety guard did the right thing and rejected the whole batch. Tokens saved on a real coding turn: zero.

Here’s the part that should actually bother you. The proxy sat there reporting it had saved 17,634 tokens on that turn - about 12%. What reached my session was nothing. And the tool’s own stats page, the thing I’d check to see if it was working, read zero requests and zero tokens saved for the endpoint I was actually hitting. From inside the session, everything looked healthy. It was doing nothing, and every number I could see told me it was fine.

That’s the real story, and it isn’t that Headroom is bad - it isn’t. It’s that you cannot see what your context tooling is doing to your context from the outside. One layer claimed 12%. The layer that mattered applied 0%. Nothing in my setup could tell me which of those was true without going in and tapping the wire myself.

I forked the extension and fixed it in three commits. It now applies roughly 88% of the savings the proxy was only claiming, and leaves prompts and assistant history byte-for-byte intact - no rewriting history, nothing left for the model to have to fetch back later. The fork isn’t really the point, though. I only found the gap because I went looking with my own instrumentation. Most people won’t. They’ll install it, watch a dashboard that reads zero, and never learn whether they’re saving 88% or nothing at all.

Second pass: what it does when you actually wire it up right

The whole thing comes down to one trick, and it’s a genuinely good one: footnotes.

When a big tool result comes back - a 500-row listing, a fat JSON blob - Headroom swaps it for a tiny reference before the model ever sees it, and keeps the real bytes in a local cache. It preserves the shape and a handful of sample rows, and folds the rest into a pointer. Sixty rows I ran through it looked like this afterward:

[{"id":0,"status":"ok"},{"id":1,"status":"ok"}, … 13 sample rows kept … ,{"_ccr_dropped":"<<ccr:703c19d4c0e245_rows_offloaded>>"}]

That’s 1,670 characters down to 415. Your conversation history only ever carries the footnote - never the full sixty rows.

The clever part is what happens when the model actually needs the dropped data. It calls a retrieve tool with that hash. Headroom intercepts the call itself - the agent never sees it happen - looks up the original, re-asks the model with the full data attached, and hands back only the final answer. The full data exists for one hidden round trip, then it’s gone again. What stays in your permanent context is the footnote plus whatever the model concluded from it. The bulk never gets carried forward.

The bit that took me longest to work out: none of that retrieval magic can happen unless the entire agent loop runs through the proxy. Call its compress endpoint like a library - which is how most bolt-on integrations do it, including my first attempt - and you get the lossy footnote and none of the fetch-back. You have to wrap the whole session: headroom wrap claude or headroom wrap codex, not a partial integration. That single architectural choice is the entire difference between the two halves of this post.

On cost, it’s smarter than I first assumed. When the model retrieves the dropped data, you don’t re-pay for the whole conversation - that history is still a cache hit. The full blob just gets appended for one pass. So a retrieval is cheap: one uncached read of that single blob, plus one extra model call. The real win is moving the cost of big context from carried forever to paid once, on demand, only when you actually need it.

Where it falls apart

There’s one clean failure mode. If the model keeps pulling back the same big blob turn after turn - say it works through 500 files fifty at a time instead of reaching for a single find command - it re-fetches every time, and you pay for it repeatedly. The saving depends entirely on how the model chooses to work, which is the whole problem with any tool that needs the model to behave a particular way to earn its keep.

Which is why I think coding agents will eventually just do this by default. The footnote-and-fetch pattern is far more natural built into the agent itself - which already knows what it looked at and what’s safe to drop - than bolted on as an external proxy that has to guess at agent intent from the outside. Dave’s response when I walked him through it: “They’ll do it if they actually care about token efficiency.” I think that’s right, and I think it’s a matter of when, not if.

The actual lesson

We’ve all agreed, in principle, that you have to read the code an agent writes. Almost nobody extends that same scepticism to the tools the agent runs on. Something like Headroom has a dozen levers, and every one of them changes what happens to your context - if you’re not checking what it’s actually doing, you’re trusting the label on the box, not the contents. In this case the label and the contents disagreed by a factor of infinity in one configuration, and matched closely in another, and the only way to tell which one you had was to go and look.