Cadence

Tooling. Workflows. Process.

A field perspective on AI coding - what we see working, and failing, across terabytes of real coding-agent sessions.

GB of coding-agent logs stored 4000 3000 2000 1000 0 Feb 5 Mar 5 Apr 2 Apr 30 Jun 3 ~3.4TB (12TB raw)
Stages of AI coding PHASE 1 Adoption PHASE 2 Effectiveness PHASE 3 Efficiency
Phase 1 · Adoption PHASE 1 Adoption using the tools PHASE 2 Effectiveness PHASE 3 Efficiency tokenmaxxing - use it as much as possible; familiarity is the goal
Phase 2 · Effectiveness PHASE 1 Adoption PHASE 2 Effectiveness producing output PHASE 3 Efficiency output: PRs merged · lines of code · commits · tickets closed
Phase 3 · Efficiency PHASE 1 Adoption PHASE 2 Effectiveness PHASE 3 Efficiency output vs spend the same output, with token burn and dollar spend in the denominator
The frame Model Engineer Codebase
1 / 3 · the model

Model choice matters.

2 / 3 · the engineer - real sessions, paraphrased
"We're using 0.7.9 of the library, which doesn't support that parameter, so we'll add workarounds at each call site." - maybe just bump the library to 0.8.0.
"Tests can't cover this because there's no testing set up for this project, so we'll just have to fix it and skip testing." - maybe the answer is to set up tests, not route around their absence.
3 / 3 · the codebase - legibility checklist
Files/modules that fit a context window
Predictable naming & structure
Enough tests to catch breakage
Few enough tests to stay fast
Static checking - lots of it
Minimal side effects / global state
Minimal metaprogramming
One way of doing any given thing
Language-native idioms
Docs - especially diagrams
How people actually use it L0 L1 L2 L3 "Build this, now build this, now build this" vibe-coding - errors compound silently until unrecoverable "You're so smart, do everything" a challenge dressed up as delegation → "AI is overhyped" Tiny pieces, no joined-up thinking Building without learning
How people actually use it L0 L1 L2 L3 "Build this, now build this, now build this" "You're so smart, do everything" Tiny pieces, no joined-up thinking offshore-team-you-don't-trust mode - works, but slow Building without learning shipping fast while quietly losing the grain of the codebase
The mind-meld HUMAN HOLDS The plan - invaluable guidance Task size & structure The grain of the code Intent & judgement AGENT CARRIES Implementation Iteration against tests Breadth & speed Judgement calls within the plan
Success rate: one-shot vs planning
Baseline One-shot
100% 75% 50% 25% 0% 89.1% 65.0% 81.6% 44.2% claude-code codex −24.1 pts −37.4 pts
Three documents, not one
  1. 1Spec - the goal, 50–200 linesread every line; if you don't agree with it, you're watching, not engineering
  2. 2Design doc - how it fits what already existsheavy reference to architecture and the current codebase
  3. 3Implementation plan - what the agent executesreviewed like a PR, discussed with the agent in the margin
% of sessions showing context rot (log-scale x)
Claude Opus Codex
30% 25% 20% 15% 10% 5% 0% <0.5M 0.5-1M 1-2M 2-5M 5-10M 10-25M 25-100M 100M+ 23.2% 11.2%
Backpressure: verification inside the loop
  1. 1Agent implements
  2. 2Fast local checks - types, lint, changed-file tests
  3. 3Push to CI - full verification, inside the loopCI is now a verify tool the agent calls repeatedly, not an end gate
  4. 4Agent reads failures, iterates - no human in this looptoo heavy → hours of churn · too light → the human becomes QA
  1. 1Plan with the agent
  2. 2Review the plan in detail
  3. 3Agent implements
  4. 4Review the tests it built
  5. 5Push to CI → automated adversarial review
  6. 6Guided human review
001

Real AI-assisted coding

An overview of what we see out there.

  • Levels of adoption.
  • Patterns you might notice in yourselves.
  • Our recommended workflow.
002

Where this comes from

We store and analyse terabytes of real coding-agent session logs from our customers - aggregated and anonymised.

003

Know where you are

Three rough phases of adoption.

  • Using the tools.
  • Getting productivity out of the tools.
  • Driving efficient outcomes.
004

Phase 1: Adoption

Basically tokenmaxxing - use the tools as much as possible, on as much as possible.

  • The goal is familiarity: building instinct for what agents can and can't carry.
  • Over-use is fine here. Spend at this phase is tuition, not waste.
  • You can't skip this phase by reading about it.
005

Phase 2: Effectiveness

Producing output - measured by PRs, lines of code, commits, tickets closed.

  • All flawed proxies - but at this phase, direction beats precision.
  • The question shifts from "am I using it?" to "is work actually landing?"
  • Most teams we see are somewhere in this phase.
006

Phase 3: Efficiency

Taking token burn and dollar spend into account too - outcomes per dollar, not just outcomes.

  • Same output for less, or more output for the same spend.
  • Needs both sides measured: what shipped and what it cost.
  • Very few teams are genuinely here yet.
007

Three things to line up

The model, the engineer driving it, and the codebase underneath it.

008

Model choice is increasingly interesting

Some tasks, some parts of each task, take bigger or smaller models.

  • fable-5 is good but very expensive.
  • sonnet-5 is surprisingly good.
  • gpt-5.6-sol is good but watch the reasoning mode.
  • gpt-5.6-luna can write very solid code on xhigh.
  • Try not to use copilot.
009

The engineer: shortest path to green

Models are smart, but dumb in specific, recognisable ways. They reliably take the shortest path - and it's often architecturally wrong.

  • Both of these are real sessions from our logs.
  • Backwards compatibility is another one.
  • Catching this is the engineer's job - and it's a new skill, distinct from traditional experience.
010

The codebase: legibility is leverage

The agent reads your codebase the way it reads a prompt.

  • Most of this is something you already wanted for humans.
  • If it's confusing to a new hire then it will be confusing to an agent..
011

Imperfect workflows

A rough taxonomy. The first two failure modes are asking too much of the agent:

  • L0: no planning, no judgement on output - errors compound silently.
  • L1: do a short prompt, go make a coffee, make the agent prove it can get it right first time.
012

More imperfect workflows

These are asking too little of the agent:

  • L2: tiny, tightly-specified tickets. Work ok, just way slower than what's possible.
  • L3: shipping fast while losing your own mental model. Suppresses your instincts.
013

Perfect workflow

Mind-meld with the agent.

  • Strong planning so the human provides invaluable guidance.
  • Picking the right size and structure for tasks.
  • Giving enough to the agent so it can add significant value.
  • Staying in touch with the grain of the code and architecture.

Familiar to experienced eng managers because this has always been the goal.

014

Planning beats one-shotting

Sessions that skip planning succeed much less often - on both major tools.

  • Correlational, and task mix is a confound - harder tasks may get one-shotted more.
  • But the gap is large, consistent, and matches everything we see qualitatively.
  • Planning is the job now. Get good at it.
015

A planning workflow that works

50%+ of agentic engineering time is up-front planning. It feels like waterfall. The cost-benefit changed: the spec is now the only lever over what gets built.

  • 60–90 minutes of back-and-forth before a reasonably-sized change.
  • Review the plan like you'd review a PR - that's where your judgement goes now.
  • Skipping it just moves the conversation to after the code exists, where it's expensive.
016

Context is finite

Sessions degrade as they grow - measured via dev-frustration and tool-failure signals. Note the log-scale x axis.

  • Fine in normal ranges; past ~100M tokens it gets worse.
  • Claude still isn't great at compacting.
  • Codex much better, don't worry so much about long sessions there.
017

Test architecture is crucial

Agents take a random walk through possibility space, need to constrain that.

  • "Backpressure" = automated testing that runs inside the agent's loop.
  • Run targeted/changed tests locally, the rest (e2e, integration) in CI.
  • CI changes role: from one build per branch to a verify tool called repeatedly.
  • CI speed becomes a constraint on agent throughput, run in parallel.
018

Recommended workflow

The job is to balance human judgement and agent speed.

  • Human contribution is significant.
  • Step 3 and step 5 are the agent's domain.
↑ ↓ · space · or clicker to advance