Claude Code gets plugin evaluation with six graders and a baseline run

single source · 1 articles · capability · confidence: medium · first seen 2026-09-11 21:05 UTC

Anthropic has added a plugin evaluation command to Claude Code, its agentic coding tool. `claude plugin eval` runs a plugin against realistic prompts, grades what Claude produces, and compares that with a run where the plugin is not loaded. The workflow includes six grader types, a no-plugin baseline, and a CI gate for skills. It is meant to answer three questions plugin developers previously could not measure: whether the skill triggers, whether it changes the output, and whether that change is an improvement. Details of the grader types and CI integration were not included in the initial announcement.

What this means for you

If you build Claude Code plugins, use `claude plugin eval` to compare outputs with and without the plugin and gate releases on the results. If you don't build plugins, nothing to do.

Key facts

  • ·Anthropic added a plugin eval workflow to Claude Code. source
  • ·The command is `claude plugin eval`. source
  • ·It compares a run with the plugin against a run without the plugin. source
  • ·The workflow includes six grader types. source
  • ·It includes a CI gate for skills. source

What the sources say

  • MarkTechPostDetails the new eval command and its six grader types, baseline comparison, and CI gate.

Sources

The original reporting. Follow these — they did the work.

← the wire