CodeMidas builds 5,545 coding tasks out of existing open-source projects

single source· 1 articles · confidence: medium · first seen 2026-09-17 20:00 UTC

What this means for you

Nothing to act on today: no dataset, weights, licence or evaluation dates are given, so none of this is reproducible yet. The reusable part is the recipe — mine finished source code for tasks rather than issue trackers, and let execution checks, not annotators, decide which tasks survive.

CodeMidas, a paper posted to arXiv on 17 September, builds reinforcement-learning environments out of existing open-source code. Its agents read what a codebase already implements, write behavioural specifications and tests, then filter candidate tasks by execution checks and repeated solution rollouts. No issue reports or commit history needed. The pipeline produced 5,545 training tasks from 3,185 repositories across 23 languages and 15 domains. Training MiMo-V2.5 on them with GRPO, a reinforcement-learning method, improved reported scores on five benchmarks, three of them named: DeepSWE +11.7%, ProgramBench +17%, Terminal-Bench v2.1 +8.5%. The paper gives no evaluation dates or harness details, and nothing here has been independently reproduced.

Models in this story

Key facts

  • ·CodeMidas produced 5,545 training tasks drawn from 3,185 open-source codebases, spanning 23 programming languages and 15 technical domains. source
  • ·Training MiMo-V2.5 on those tasks with GRPO improved reported scores by 11.7% on DeepSWE, 17% on ProgramBench and 8.5% on Terminal-Bench v2.1. source
  • ·The pipeline takes source code as its only task-specific input, rather than development artifacts such as issues and commits. source
  • ·The paper gives no evaluation dates or harness details for the five benchmarks it reports. source
  • ·Ablations found that increasing the number of high-quality training tasks improved performance. source
  • ·Trajectory analysis reports the RL-trained agent explored the codebase more and self-verified more diversely than the baseline. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire