Preprint reports that training audio and video separately improves joint output

single source· 1 articles · confidence: medium · first seen 2026-09-23 20:00 UTC

What this means for you

Nothing to act on yet. The paper points to a code and data repository; it does not claim a released model, a hosted endpoint or an evaluation date. If you work on paired generation, the transferable idea is the frozen-tower trick — scoring one half of the pair without paying to update both.

Joint audio-video models are usually trained as one system, which tangles the feedback telling each half what it got wrong. A preprint posted on 23 September 2026, AV-GRPO, trains the two halves separately, freezes one while the other updates, and scores synchronisation on paired samples at controlled difficulty. The authors also release 5DAV, a dataset that splits its samples along five axes. On JavisBench and VABench they report gains over LTX-2.3 in generation quality, text alignment and cross-modal synchronisation, under LoRA adapters (small added weight sets) and full fine-tuning. No evaluation date is given and no outside group has checked the result.

Key facts

  • ·AV-GRPO is described as a modality-anchored online diffusion reinforcement-learning framework for joint audio-video generation, posted as arXiv 2609.29816 on 23 September 2026. source
  • ·The framework has three parts: modality-anchored rollouts, trajectory-locked frozen-tower optimisation, and modality-specific adaptive objectives and perturbation strengths. source
  • ·The authors release 5DAV, a training dataset that decouples samples across five dimensions. source
  • ·On JavisBench and VABench the paper reports AV-GRPO outperforming LTX-2.3 in generation quality, semantic alignment and cross-modal synchronisation, under both LoRA and full fine-tuning. source
  • ·Code and data are listed at github.com/zhiyuxu03/AV-GRPO. source

What the sources say

  • Hugging Face Daily Papers (research) — Abstract-only preprint claiming that treating each modality as its own subproblem tightens synchronisation, plus a dataset for difficulty control.

Sources

The original reporting. Follow these — they did the work.

← the wire