Token-level annotation tool reports 52% cut in labelling time

single source· 1 articles · confidence: medium · first seen 2026-09-20 20:00 UTC

What this means for you

If you pay annotators to clean model outputs, the number to look at is a 52% cut in median time — from the authors' own small study, not an independent trial. No hosted service or pricing is given, so nothing to migrate today; Panda-CVL and the correction benchmark are what you can inspect.

Researchers released onPanda, an annotation tool that lets a human fix a model's output one token at a time: the annotator marks the first wrong token, picks a replacement from the model's own candidates or types one, and generation resumes from that point. In a small study the authors ran, median annotation time fell 52% against manual post-editing. Because most of the final text is still the model's own, the corrections are on-policy (drawn from the distribution of the model being trained, not a different one), which the authors say suits supervised fine-tuning and preference data. The paper also ships Panda-CVL, a dataset, and a benchmark for token-level correction.

Key facts

  • ·onPanda's interaction is token-level correction: the annotator marks the first inappropriate token, picks a candidate or types a replacement, and the system truncates the rest of the response and regenerates from the corrected prefix. source
  • ·A small controlled study reported by the authors puts onPanda's median annotation time 52% below manual post-editing. source
  • ·The paper releases Panda-CVL, a dataset annotated with onPanda, plus a benchmark for token-level correction. source
  • ·onPanda connects to external tools and harnesses so agent trajectories can be annotated in interactive environments. source
  • ·The work is posted as arXiv 2609.24983, listed on Hugging Face Daily Papers dated 20 September 2026. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire