Frozen design agent improves by writing new skills, not retraining weights

single source· 1 articles · confidence: medium · first seen 2026-09-17 20:00 UTC

What this means for you

Nothing to adopt yet: this is a method, not a released model or API, and the skill bank is tied to one design-agent setup. The practical read is that agents facing many-tool workflows may gain more from a curated, gated library of written procedures than from retraining a frozen model.

A graphic-design agent improves without changing its underlying model: it accumulates written procedures, adding and revising natural-language skills from its own successes and failures, with a replay gate that rejects edits that fix one task while breaking another. Over five rounds on 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, the skill bank grew from 76 to 139. The authors report GenEval2 execution success on Claude-Sonnet-4 rising from 72.7% to 99.3%, with 61.8% and 67.6% win rates against a skill-less agent on four design benchmarks; the paper does not give a date or test setup for that evaluation.

Models in this story

Key facts

  • ·The agent operates professional design software through more than 230 tools while the underlying model stays frozen. source
  • ·Over five rounds on 1,406 real user briefs and 1,869 automatically graded trajectories, with no weight updates and no human labels, the skill bank grew from 76 documentation-derived skills to 139. source
  • ·GenEval2 execution success on Claude-Sonnet-4 rose from 72.7% to 99.3%; the paper gives no evaluation date for that harness. source
  • ·Win rates against the no-skill agent were 61.8% and 67.6% across four specialised design benchmarks on Claude-Sonnet-4 and Claude-Opus-4.6. source
  • ·On 200 held-out briefs, widening the skill bank alone reached a 49.4% win rate and deepening alone 48.6%, against 58.5% for the two combined (p = 0.025). source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire