AI coding efficiency is not simply about reducing the number of tokens used in individual interactions. The more useful measure is whether an agent can complete a coding task with the right context, fewer unnecessary turns, and less repeated work. A shorter response can actually become more expensive when important information is removed and the agent has to retrieve it again.
GitHub Copilot has tested this principle across its coding workflows, including Copilot CLI, by evaluating changes in offline agentic coding benchmarks and then validating promising approaches through controlled online experiments. The focus has been on removing unnecessary work while preserving the information and behaviours agents need.
Why token count alone can be misleading
Reducing every tool response may look like an obvious way to lower AI costs, but local savings do not always translate into lower end-to-end costs. When useful information is omitted, an agent may reopen files, rerun commands or take additional turns to recover the missing context.
- A shorter tool response can trigger additional retrieval work.
- Repeated recovery adds model turns and carries more context forward.
- The complete task, rather than an individual tool call, is the meaningful efficiency measure.
- Efficiency improvements should be tested against both cost and task quality.
Compress repetitive output while preserving useful context
GitHub Copilot’s evaluation found that install, build, test and lint logs often contain repetitive information, while source-like output and arbitrary command results can be more important to an agent. This led to a selective approach that compresses predictable noise instead of shortening everything.
- Source-like and arbitrary command output is preserved.
- Search results can be reorganised without removing matches.
- Repetitive build, test and progress output can be compressed selectively.
- The complete original output remains available through a recovery path.
Early experiments showed why restraint matters. Overly aggressive compression caused agents to repeat work or retrieve the original output. For example, compressing git diff was abandoned after benchmark tasks showed that agents sometimes needed to reopen the uncompressed result. The final approach was shaped by these failures and by repeated evaluation.
Remove formatting that does not help the agent
Another optimisation came from the file-viewing workflow. The view tool had historically added line-number prefixes to every line of file content. Although line numbers had been useful for older editing workflows, current tools can identify changes using surrounding code, making the repeated prefixes unnecessary in ordinary file reads.
- Removing the prefixes leaves the underlying file content unchanged.
- The change avoids repeatedly sending unused formatting to the model.
- Offline benchmarks showed roughly 5% lower model-inference cost.
- An online Copilot CLI experiment reduced average daily model-inference cost per user by about 3%, with no material quality regression detected in tracked metrics.
Shorten prompts without changing intended behaviour
Prompts are another recurring source of token usage because instructions are passed to the model across turns. GitHub Copilot reduced the size of guidance for its task tool by roughly half while testing whether important behaviours remained intact.
The first online experiment exposed a problem that offline evaluation had not caught: an instruction rewrite changed cautious parallelism guidance into a harder scheduling rule, causing independent custom agents to run sequentially. The experiment was stopped and a regression test was added before the prompt was revised.
- Prompt compression must preserve the behaviours users rely on.
- Behavioural tests should cover important instructions before a shorter prompt is shipped.
- The revised guidance allowed independent agents to run in parallel while leaving room for the model to consider side effects.
- The shipped change removed about 1,300 task-tool prompt tokens per turn.
Deliver background results without an extra retrieval turn
Coding agents frequently run independent work in the background, such as shell commands or sub-agent investigations. Previously, completion notifications could wake the model without including the completed result, forcing the agent to spend another turn retrieving information the harness already had.
- Eligible background completions can now be batched.
- Completed results are delivered in the existing tool-result format.
- The agent can process multiple finished tasks in one model turn.
- The change reduces unnecessary retrieval calls and avoids carrying full session context through those extra turns.
In the example described by GitHub, two completed background tasks previously required four model calls before their results could be processed. With batching, both results can be supplied together for a single processing call. GitHub reports that this approach reduced average token-related usage, measured in AI Credits, by about 2.3%.
Measure efficiency changes in the workflow where they run
An optimisation that works in one Copilot workflow can behave differently in another. GitHub therefore evaluates changes in the specific product surface and workload where they are intended to ship.
- Offline agentic coding benchmarks help identify promising changes.
- Online experiments reveal effects under real usage conditions.
- Quality and behavioural metrics need to be monitored alongside token savings.
- Changes that increase cost or create regressions should not be shipped simply because they reduce local output.
Key lessons for building efficient AI coding agents
- Optimise the completed task rather than an individual tool call.
- Reduce unnecessary orchestration and retrieval turns where the harness can do the work deterministically.
- Prefer lossless transformations and preserve information that may be needed later.
- Test prompt rewrites for behavioural regressions, not just token reduction.
- Evaluate every optimisation in the workload and product surface where it will operate.
The broader lesson is that AI coding efficiency does not require making the model do less useful work. It requires removing work that the model never needed to perform in the first place. By preserving useful context, eliminating redundant formatting, tightening prompts carefully and delivering completed background results directly, coding agents can reduce unnecessary usage while maintaining the quality of the completed task.
Read more such articles from our Newsletter here.
