Can you prove the extension works
before the team installs it?_

A test-readiness console for the five Claude Code extension types — skills, subagents, hooks, plugins, MCP servers. Each one sits at a different point on a determinism gradient, and the gradient decides what kind of test you're even allowed to write.

Extension testability is a gradient, not a checkbox. Hooks take fully-deterministic CI ^[1]. Skills & subagents are model-in-the-loop — their description field is the unit under test, gated on hit-rate, never one green run ^[2]. MCP & plugins add a supply chain you don't author that can rug-pull after install ^[3]. No purpose-built harness exists yet — the working stack is improvised from claude -p, Promptfoo, snapshot tests, and JSON-schema linters.

topic: extension-artifact testing date: 2026-06-24 depth: expedition

The determinism gradient

green → amber → red = less testable

Three lanes. The further right, the less a single green run means — and the more of the risk lives outside your own diff.

Deterministic

Hooks

Plain scripts with a fixed stdin / exit-code / JSON contract. The one extension you can test like any other CLI.

Unit under teststdout / exit code / JSON on stdin

Harnessbats · pytest · any CI ^[1]

Ceiling: none worth noting — green means green. This is the only lane where CI is the whole answer.

Model-in-the-loop

Skills & Subagents

Claude fuzzy-matches the description against the prompt to decide whether to fire ^[4]. The description is the test.

Unit under testthe description field — does it trigger?

HarnessPromptfoo skill-used assertions ^[2] · skill-creator eval loop ^[6]

Ceiling: autonomous triggering is only ~50% reliable — must-fire workflows fall back to a deterministic hook calling Use Skill() explicitly ^[7].

Supply-chain

MCP & Plugins

A third axis you don't author. The code can change after you approve it — and MCP has no native re-approval when a tool mutates ^[3].

Unit under testschema/contract drift + dependency provenance

HarnessBellwether ^[8] · agent-scan ^[12] · pinner-mcp ^[13]

Ceiling: the malicious change lands after install, in someone else's repo — there is no per-PR answer here.

hooks · deterministic CIskills/subagents · hit-ratemcp/plugins · provenance

One test, written twice

"Does my skill trigger?" and "does the parent route to my subagent?" are the same test ^[5] — positive prompts that must hit, plus near-miss prompts that must route to a sibling instead.

assertion · trigger classifier

Promptfoo expresses this directly with skill-used / not-skill-used assertions ^[2]. Anthropic's skill-creator upgrade (shipped 3 March 2026) treats triggering as a false-positive / false-negative classifier and auto-tunes the description against sample prompts ^[6].

# tests/skill_trigger.yaml — gate on hit-rate, not a single run
prompts:  must-fire set (N) + near-miss set (M)
assert:
  - skill-used      → expected on every must-fire prompt
  - not-skill-used  → expected on every near-miss prompt
# PASS = hit-rate over threshold across the whole set

Two regressions hide under one word

Opposite failures, opposite defenses. Conflating them is how teams test the one they can see and miss the one that ships the breach.

functional drift · your own artifact

The schema changed under you

Your tool's contract shifts between versions. Caught by snapshotting tools/list and failing CI on a schema diff.

defense
Bellwether ^[8], the official MCP conformance suite ^[9], or FastMCP in-memory client tests ^[10].

rug-pull · a dependency you trusted

Identical for 15 versions, then a backdoor

postmark-mcp behaved identically for 15 releases, then shipped a BCC email-harvesting backdoor in v1.0.16 ^[11].

defense
Hash-pin tool descriptions & alert on change (agent-scan ^[12]), pin servers to a sha256 digest (pinner-mcp ^[13]). Provenance is no silver bullet — 633 malicious npm versions passed Sigstore with stolen certs in May 2026 ^[14].

The realistic per-PR floor

deterministic first · one cheap model call second

What a small team can actually gate on every pull request. The first three steps need zero model calls; the fourth is one cheap headless run; the fifth has no per-PR answer at all.

●

Validate plugin manifest, strict mode

claude plugin validate --strict — warnings become errors ^[15]

deterministic

●

JSON-schema check plugin.json / marketplace.json

schemas now official on SchemaStore ^[16]

deterministic

●

Lint frontmatter & SKILL.md

pulser ^[18] · skill-validator ^[19] — tiny, single-maintainer

deterministic

●

"Does it load" smoke test

claude -p --output-format stream-json → read plugin_errors on the system/init event ^[17]

one cheap call

○

Catch a rug-pull

the malicious change lands after install, in someone else's repo, long after your diff was green ^[3]

no per-PR answer

The surface itself is unstable

2026 broke working assertions

2026-01-12ERR #17591 open — TaskOutput returns raw JSONL instead of the subagent summary, breaking transcript-based dispatch checks ^[20]

2026-01-12WARNTask → Agent tool rename — transcript matchers must now accept both strings ^[20]

2026-03-03INFOskill-creator ships description auto-tuning as a FP/FN classifier ^[6]

2026-05-xxERR 633 malicious npm versions pass Sigstore provenance with stolen certificates ^[14]

consensusWARNconsolidation signal is schemas landing on SchemaStore ^[16] — not a dominant tool. This space is still emerging.

Drill into a job

5 child runs · all green

Each extension type gets its own deep-dive. Start with hooks (the easy win) and the per-PR floor; escalate into the MCP/plugin expedition for the supply-chain surface.

passedrecon

Testing Claude Code Hooks before shipping

Hooks are plain scripts with a documented stdin/exit-code/JSON contract — the one extension you can test with fully-deterministic CI.

6 citations~3 min

passedsurvey

Testing an authored Skill before you ship it

Two separable tests — does the description trigger, does the body behave — over the skill-creator eval loop plus a with/without baseline. Everything else is thin.

11 citations~6 min

passedsurvey

Smoke-testing a subagent's routing

Promptfoo's claude-agent-sdk provider plus headless stream-json are the only real ways to assert routing. The description is the trigger; parent_tool_use_id is the proof.

11 citations~8 min