Claude Code Plugin Evals

A Claude Code command that tests whether a plugin or skill makes Claude's output better, for people who build Claude Code plugins.

Runs your plugin against a set of test prompts, scores each result pass or fail, then runs the same prompts with no plugin loaded so you can see how much the plugin changed the outcome
Running claude plugin eval init has Claude ask what good and bad output looks like, draft the test cases and checks, try them once, and write the files
Checks can match text patterns, confirm a tool or skill was used, confirm a file was created, or have a second model judge the reply against a written rubric
Each test runs three times by default, and results come back as a summary table plus an HTML report
Every run is a real model call counted against your plan usage or API bill, and a cost ceiling option caps spending
Requires Claude Code v2.1.269 or later

Checking that a skill or plugin you built does what it's supposed to before you share it with a team.

Before rolling a plugin out to colleagues, after editing a skill, or when a new model ships and you want to confirm the plugin still helps.

Team leads and ops people who maintain shared skills and plugins in Claude Code.

Join the newsletter to get ahead with AI.

I share strategies, new modules, free guides, and tools to get you better at using AI in your work.