ReFold: How Reversible Context Folding Cuts Long-Horizon AI Agent Costs

Long-running AI agents have a cost problem that grows quietly with every step. A coding or terminal agent re-sends its entire interaction history (every file read, search result, and test log) to the model on each turn. Context length climbs, inference bills climb faster, and eventually the session hits the context window and must be compacted, often losing information along the way.

A new paper from researchers at UC Santa Barbara and Intel, ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon Agents (arXiv, October 2026), offers a simple fix that needs no extra model calls and no training. It reports up to 2.5x lower token use, about half the KV-cache memory per session, and no measurable loss in task success. Here is how it works and how to apply the ideas in your own agent stack.

Why agent context gets so expensive

In a standard ReAct-style loop, each step appends the agent's action and the tool's output to an append-only history, and that whole history becomes the next prompt. If turns are roughly the same size, the total tokens processed across a session grow quadratically with the number of steps.

The paper cites production evidence that this matters: an analysis of GitHub Copilot agent traces covering 95 trillion tokens found that only 7.8% of sessions trigger context compaction, yet those sessions account for 44.2% of total token usage. A small share of long sessions dominates serving cost.

What is wrong with existing fixes

The authors compare six context-management strategies in one harness on 50 SWE-bench Verified tasks under a 16k budget. Their findings point to three hidden costs:

1. Pruning tool output is not enough. Tool outputs are only about 59% of peak KV-cache use. The agent's own commands and reasoning traces keep piling up, so methods that trim only observations still run close to the budget.
2. Auxiliary work adds overhead. Summarization-based compaction (the approach used by tools like Claude Code and Codex) made a median of four summary calls per task at about 38 seconds each, adding roughly 152 seconds and about 14% of the task's cost.
3. Editing history breaks the prefix cache and loses data. Masking old observations dropped the prefix-cache hit rate to 23%, versus 84% with no context management, and the lost cache alone added about 31% to cost. Several strategies permanently discarded 17% to 48% of observation tokens with no way to recover them.

How ReFold works

ReFold is a rendering layer. It keeps the full, untouched history and changes only what gets rendered into the model's prompt. It uses two operators.

1. Deduplication

When the same file range, search result, or command output appears again, ReFold keeps the first copy and replaces the repeat with a short stub that points back to the original step. For long spans, the stub keeps top-level class and function signatures.

2. Folding

The agent is asked to begin each reply with two lightweight metadata lines:

• PROGRESS: a one-line description of what the current step does.
• STALE: the indices of recent steps whose output it no longer needs.

Steps marked stale are collapsed into a one-line note that keeps the progress line and key details such as function signatures. Only steps from the last few turns can be marked stale, which pushes the agent to clean up as it goes. These signals come inside the agent's normal response, so there are no extra model calls.

Two design choices that keep it cheap

• Chunked rendering. Instead of rewriting the prompt prefix at every step, ReFold advances its compression boundary once every k steps (the authors use k=3). Between boundary moves, the prompt is strictly append-only, so the serving engine's prefix cache keeps working. With chunking, the cache hit rate stayed at 87%, compared with 51% when re-rendering every step.
• Reversibility. Every stub and note names the steps it replaced. The agent can run a restore command to bring the original output back exactly, appended at the end of the context so the cached prefix stays intact. It can also simply re-run the original command to see the current state.

The rendering overhead is about 0.4 milliseconds per step, compared with 3.3 seconds for summarization-based compaction.

The results

The team tested ReFold in the mini-swe-agent harness with two open models, the dense Qwen3.6-27B and the mixture-of-experts Qwen3.6-35B-A3B, served with vLLM, on five benchmarks: SWE-bench Verified, SWE-bench Pro, Multi-SWE-bench, and Terminal-Bench 1.0 and 2.1 (50 tasks each).

At the full context window with eight concurrent sessions:

• Tokens dropped 18% to 53% on the 27B model and 42% to 60% on the MoE model.
• Peak KV-cache memory per session fell 30% to 58%.
• Estimated cost fell by up to 45%.
• Resolved-task counts changed by -1 to +3 out of 50, which falls within the run-to-run noise the authors measured.

The gains grow under pressure. With 48 concurrent sessions on SWE-bench Verified, the baseline solved only 7 of 50 tasks because of memory contention, while ReFold solved 32, with average queuing delay falling from 8.96 seconds to 0.02 seconds. Under capped budgets, ReFold avoided up to 92% of forced compactions.

In a smaller test with Claude Opus 4.8 on 21 tasks, ReFold cut tokens by 24% and cost by 11%. Savings were smaller there because that model finished tasks in fewer steps and the provider's prompt cache already served most input tokens.

An ablation found that folding does most of the work. Folding alone cut KV-cache use by 49%, while deduplication alone cut it by 16%.

Practical takeaways for agent builders

You do not need to adopt ReFold wholesale to benefit from its ideas.

1. Separate history from the prompt. Store the full transcript and treat the prompt as a rendered view of it. That one change makes every compression reversible.
2. Let the agent mark finished work. A cheap PROGRESS/STALE convention gives you a high-quality signal about what can be folded, with no extra model calls.
3. Protect your prefix cache. Batch context edits and apply them every few steps instead of every step. On most hosted APIs, cached input is much cheaper than uncached input.
4. Deduplicate repeated reads. Agents often re-read the same file or rerun the same failing test. Replacing exact repeats with stubs is nearly free.
5. Measure cost per resolved task, not just accuracy. Track tokens, cache hit rate, peak context, and queue delay alongside success rate, especially if you self-host and run many sessions at once.
6. Keep compaction as a backstop. ReFold runs underneath threshold-based summarization rather than replacing it, and reduces how often summarization fires.

Limits to keep in mind

The main experiments use 50-task samples per benchmark and two open models served on a specific GPU setup, and the cost figures are estimates based on representative hosted rates. The approach also depends on the agent following the metadata format; the authors add reminders when it forgets. Results on non-coding agents, such as web or customer-support agents, have yet to be shown. The authors say they will release code and scripts.

Key takeaways

• Long-horizon agents waste a large share of compute re-sending redundant and finished context.
• ReFold compresses only the rendered prompt, keeps the full history, and makes every removal reversible.
• Chunked rendering preserves the prefix cache, which is where much of the real cost saving comes from.
• Reported gains include up to 2.5x fewer tokens, roughly half the KV-cache memory, and much better throughput under concurrent load, with no measurable loss in task success.

Source

• Su, Tian, Zhang, and Kundu, "ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon Agents," arXiv:2610.07863 (October 2026)