Skip to content
NewsProduct launch

Claude Code v2.1.269 Ships Plugin Evals, Output-Style Switching, and Agent Map in VS Code

· by Pondero Newsdesk

The short version

Anthropic released Claude Code v2.1.269 on September 11, 2026, adding a plugin eval command with six grader types and differential scoring, a new output-style switcher, a VS Code agent map for multi-agent workflows, and a concurrency cap for Workflow fan-outs.

Claude Code v2.1.269 Ships Plugin Evals, Output-Style Switching, and Agent Map in VS Code

The standout addition in Claude Code v2.1.269 is a quality gate for the plugin ecosystem. Anthropic shipped claude plugin eval on September 11, 2026: a command that runs a plugin's eval suite against Claude Code, scores each case, and tells you whether your plugin actually changes what Claude does versus what it would have done anyway.

What changed

The claude plugin eval command produces both a JSON and an HTML report, per Anthropic's changelog. Each case is graded against six types. Four of those graders (regex, tool_used, tool_order, and file_exists) run entirely from the session transcript and file system and cost nothing. The other two (llm and baseline) call a judge model and add to the run's cost, per the plugin-evals documentation.

The real signal is the differential score. Each case runs three times with the plugin loaded and three times without it by default. The resulting WITH and W/OUT columns and their difference (Delta) show what the plugin actually contributed. A near-zero delta on a tool_used: Skill grader is the most common first finding: it means Claude is not selecting the skill on natural phrasing, and the fix is to revise the skill's description rather than the logic.

claude plugin eval init generates an initial suite by asking about the plugin, proposing cases and graders, running them, and writing the files, so teams can gate CI on plugin score without hand-authoring every case from scratch.

Three other changes ship in the same release:

  • /output-style [name] lists and switches output styles, including over Remote Control and in cloud and headless sessions, per the changelog.
  • VS Code gains an agent map: an "N agents" footer pill opens a panel showing all sub-agents with per-agent cards, a Stop agent control, and read-only transcripts. A separate permission rules dialog lists, adds, and removes permission rules across user, project, and local settings.
  • CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS (accepts values 1 to 256) caps the Workflow tool's per-run concurrent agent limit, which Anthropic describes as targeting inference-bound fan-out workloads.

Why it matters

Plugin eval addresses the gap that made plugin quality opaque before this release. A plugin author could write a skill, see Claude pass a test, and still not know whether the skill caused the pass or Claude would have passed without it. The WITH/W/OUT differential makes that distinction measurable and repeatable. For teams shipping plugins into shared Claude Code environments, a CI gate on the delta score is now a practical option rather than a manual review process.

The concurrent-agents cap serves a different function: cost control. Workflow fan-outs can spawn many sub-agents in parallel, and without a ceiling the spend on a single run can grow faster than a team expects. Setting CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS to a number appropriate for the team's budget creates a hard limit the Workflow tool respects, independent of how the underlying tasks are structured.

The agent map in VS Code addresses observability during multi-agent runs. Previously, understanding which sub-agents were active during a fan-out required reading transcripts sequentially. The map makes the live sub-agent topology visible at a glance.

What to watch next

Whether Anthropic integrates the plugin eval framework as a publish gate in a future Claude Code Skill Marketplace is the main open question. The differential scoring infrastructure maps directly to how a centralized marketplace would enforce a minimum-quality threshold before listing a skill. Adoption of CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS in enterprise deployments will also indicate how broadly teams are running multi-agent fan-outs in production.

Sources