Mismatched stop tokens explain runaway length in distilled students

single source· 1 articles · confidence: medium · first seen 2026-09-16 20:00 UTC

What this means for you

If you run on-policy distillation, treat this as a token-mapping change rather than a new training recipe; the released implementation applies the correction. If you do not train student models this way, nothing to act on.

A 16 September arXiv preprint identifies a failure mode in on-policy distillation, where a student model generates its own answers and a larger teacher scores them: the student's outputs grow longer and can exhaust the generation budget. Across Qwen3, Llama and Gemma, student and teacher put their stopping probability on different end-of-sequence tokens even when they declare the same stopping set, which suppresses the student's stop without reliably transferring the teacher's. Aligning the declared set alone is insufficient; treating equivalent tokens as one shared stopping action reduces the inflation in all three families. A separate late-training inflation persists beyond that fix, and an implementation with the corrections is released.

Key facts

  • ·The preprint is arXiv 2609.20511, posted 16 September 2026. source
  • ·The stopping-token mismatch was observed across Qwen3, Llama and Gemma, between base students and post-trained teachers. source
  • ·The two models can place stopping probability on different end-of-sequence tokens even when their declared stopping sets are identical. source
  • ·Aligning the decoding stopping set alone was insufficient; treating functionally equivalent EOS tokens as one shared stopping action mitigated the length inflation in all three model families. source
  • ·A distinct length inflation appears late in on-policy distillation runs and persists beyond termination alignment. source
  • ·The authors released an implementation incorporating the proposed termination-handling corrections. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire