Paper pins fine-tuning outcome on update direction, not distance

single source· 1 articles · confidence: medium · first seen 2026-09-11 20:00 UTC

What this means for you

Nothing to act on yet: no scores, no harness details, no API. The release points to code and a Hugging Face collection, so this is for teams that fine-tune their own models. The transferable idea is cheap to test — fix your drift budget first, then treat direction as the variable.

A preprint proposes setting a behavioural drift budget before fine-tuning an instruct model: cap how far the model may move from its reference, and the remaining free variable is the direction of the update, not its size. The authors test this on Qwen3-8B and Qwen3-14B by fine-tuning only on final answers while still requiring multi-step reasoning at inference — a setting that normally breaks. A coarse probe that alters only some layers, they report, reverses that failure and improves scientific reasoning and translation across more than 100 languages, matching or beating dedicated translation systems. No benchmark scores, harness details or evaluation dates are given. Code is at github.com/CONE-MT/DCO, with a Hugging Face collection at LLaMAX/dco.

Key facts

  • ·Preprint posted 11 September 2026 as arXiv 2609.13680. source
  • ·The method fixes a behavioural drift budget before fine-tuning and treats update direction as the remaining degree of freedom. source
  • ·Tested on Qwen3-8B and Qwen3-14B. source
  • ·The authors report the resulting models match or outperform dedicated translation systems over more than 100 languages. source
  • ·Code is released at github.com/CONE-MT/DCO, with a Hugging Face collection at LLaMAX/dco. source
  • ·The abstract gives no benchmark scores, evaluation harness or evaluation dates. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire