Coding-agent harness study finds context management pays off as budgets tighten

single source· 1 articles · confidence: medium · first seen 2026-09-16 20:00 UTC

What this means for you

Nothing to buy or migrate. If you build an agent harness, the ordering is actionable: spend your effort on context management when budgets are tight, trim by rule before you summarise, and skip recoverable elision — it added machinery, not accuracy. Stronger models did not need predefined tools.

A study of 176 configurations of a coding-agent harness — the scaffolding around a model that sets how it works through a task and manages its context window — finds the three components pay off differently by model and budget. Across four models on SWE-Bench Verified and Terminal-Bench 2.1, context management mattered most as budgets tightened, mainly by preventing context-overflow failures rather than improving reasoning. Staging rule-based trimming before LLM-based summarisation was the most efficient strategy; making trimmed content recoverable added machinery models rarely used and no accuracy. Planning changed cost, not accuracy. Predefined tools helped models weak at bash; bash-capable models worked effectively with bash-only interfaces at substantially lower cost.

Key facts

  • ·The study evaluates 176 matched harness settings across four models, on SWE-Bench Verified and Terminal-Bench 2.1. source
  • ·It varies three harness components — planning, action space and context management — across five context-management strategies and four context-window budgets. source
  • ·Context management's benefit grew as the context-window budget tightened, with most of the gain coming from preventing context-overflow failures. source
  • ·Staging rule-based elision before LLM-based summarisation gave the strongest overall efficiency among the context-management strategies tested. source
  • ·Making elided content recoverable added machinery that models rarely used and produced no accuracy gain. source
  • ·Bash-capable models operated effectively with a bash-only interface at substantially lower cost, especially on command-line-centric tasks. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire