Claude Code Plugin Release Automation in GitHub Actions

The tag pushed, the marketplace refreshed, and two of the plugin's three skills were simply not there. The install printed no error, and claude plugin list showed the plugin enabled at the new version. I had added a third skill under an extras directory and declared that directory in the manifest, and the commands field replaces the default commands scan instead of adding to it, so the two skills already sitting in commands stopped being loaded the moment the new one arrived.

That is the class of bug plugin CI exists to catch: not a crash, but a bundle that loads and does less than it says. This is the GitHub Actions pipeline I run over a plugin repository now. What claude plugin validate covers, how to turn its report into a gate, where the version bump and claude plugin tag belong in the job graph, and what actually has to change before a user's plugin update offers them anything. Every field name and flag below is checked against the Claude Code plugins reference.

What the loader requires, and what it merely tolerates

The name field is the only required one in .claude-plugin/plugin.json, and it is the failure you will never ship: a manifest without it is rejected with a validation error naming the field as expected string, received undefined. Everything else is optional, including version, description, the component path fields and dependencies. That optionality is the whole problem, because a field allowed to be absent is also allowed to be wrong in ways nothing has to complain about.

// .claude-plugin/plugin.json — the only file that belongs in that folder.
// name is the sole required field. Everything below it is optional, which
// is exactly why a wrong value here fails quietly rather than loudly.
{
  "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
  "name": "receipt-tools",
  "displayName": "Receipt Tools",
  "version": "2.3.1",
  "description": "ESC/POS receipt layout skills and a print-queue hook",
  "author": { "name": "Matthews Wong", "url": "https://www.matthewswong.com" },
  "repository": "https://github.com/example-org/receipt-tools",
  "license": "MIT",
  "keywords": ["escpos", "receipts", "pos"],
  "skills": ["./domain/skills/"],
  "commands": ["./commands/", "./extras/"],
  "hooks": "./hooks/hooks.json",
  "dependencies": [{ "name": "secrets-vault", "version": "~2.1.0" }]
}

Two rules decide whether a mistake is loud or quiet. Claude Code ignores top-level fields it does not recognise, so a misspelled key is a warning from claude plugin validate and nothing at all at runtime, and a plugin whose only findings are unrecognised fields passes validation and loads. A wrong type is usually louder: for most fields the plugin fails to load outright, though a non-object experimental or metadata value is ignored with a warning instead. So the manifest above is what CI checks, and --strict is what makes the warnings count for something.

The failures that do not error

Five ways a plugin repository ships a bundle that installs cleanly and does less than its manifest claims. Not one of them fails the install, which is why each needs a step of its own in CI rather than a careful reviewer.

// Wrong: commands REPLACES the default commands/ scan. The two skills
// already sitting in commands/ stop being loaded, and nothing errors.
"commands": ["./extras/"]

// Right: name the default explicitly to keep it, then add your own.
"commands": ["./commands/", "./extras/"]

// skills is the exception — it ADDS to the default skills/ scan, so this
// loads both ./skills/ and ./domain/skills/ with no second entry needed.
"skills": ["./domain/skills/"]

// Wrong: a path outside the plugin root is refused with
// "path escapes plugin directory" — and the plugin still loads, minus that
// one component. Nothing in the install output mentions the omission.
"agents": ["../shared/agents/"]

The path fields are worth learning by heart, because they do not all behave the same way. commands, agents, workflows and outputStyles replace their default directory; skills adds to it. That asymmetry is the most expensive line in the plugin schema, and it is documented, so getting it wrong costs you a release rather than a bug report.

The whole pipeline in one workflow file

Three jobs, and the split matters more than the contents. The bundle job is the cheap deterministic half: schema, paths, permissions, version agreement, and it needs no model credentials because nothing in it calls a model. The behaviour job is the half that costs money and can be flaky, so it depends on bundle and never runs once bundle has failed. The release job runs only on a manual dispatch, because cutting a version should be a decision rather than a side effect of merging.

# .github/workflows/plugin-release.yml
name: plugin-release

on:
  pull_request:
  workflow_dispatch:
    inputs:
      bump:
        description: patch, minor or major
        required: true
        default: patch

permissions:
  contents: write # only the tag push needs this; the checks need nothing

concurrency:
  group: plugin-release # two release runs racing is how a tag gets moved
  cancel-in-progress: false

env:
  # validate --json needs 2.1.259 or later, so pin the CLI rather than let
  # the report shape change under the workflow on an unrelated day
  CLI_VERSION: "2.1.259"

jobs:
  bundle:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "22"
      - run: npm install -g @anthropic-ai/claude-code@$CLI_VERSION

      # Exit 0 passes, 1 fails, 2 means the validate run itself failed on an
      # unreadable path and wrote nothing to stdout. pipefail is what stops
      # the pipe from swallowing all three.
      - name: validate --strict
        run: |
          set -o pipefail
          claude plugin validate . --strict --json | tee validate.json
          test "$(jq -r .success validate.json)" = "true"

      # Every path the manifest names must exist in the checkout. One that
      # does not registers no component and reports nothing at install time.
      - name: declared paths exist
        run: |
          FIELDS='[.skills, .commands, .agents, .hooks] | flatten
                  | map(select(type == "string")) | .[]'
          for p in $(jq -r "$FIELDS" .claude-plugin/plugin.json); do
            test -e "$p" || { echo "manifest names a missing path: $p"; exit 1; }
          done

      # Hook commands are shell scripts. Shipped without the execute bit, the
      # plugin loads, the hook registers, and it never once fires. The
      # documented command form quotes the variable inside the string, so
      # strip quotes before testing the path.
      - name: hook scripts are executable
        run: |
          HOOKS='.hooks | to_entries[] | .value[] | .hooks[]
                 | select(.type == "command") | .command'
          for s in $(jq -r "$HOOKS" hooks/hooks.json | tr -d '"' \
                     | sed 's|${CLAUDE_PLUGIN_ROOT}/||'); do
            test -x "$s" || { echo "hook not executable: $s"; exit 1; }
          done

      # Prints, does not assert. This is the log you read when a component
      # goes missing and every other step is green.
      - name: what registered
        run: claude --plugin-dir . plugin list --json

      # Validates the bundle and checks plugin.json against the marketplace
      # entry version, without creating anything. Safe on a pull request.
      - name: tag dry run
        run: claude plugin tag --dry-run

  behaviour:
    needs: bundle
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "22"
      - run: npm install -g @anthropic-ai/claude-code@$CLI_VERSION
      - run: bash .github/scripts/behaviour-gate.sh
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

  release:
    needs: [bundle, behaviour]
    if: github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # the tag command reads the tags already on the repo
      - uses: actions/setup-node@v4
        with:
          node-version: "22"
      - run: npm install -g @anthropic-ai/claude-code@$CLI_VERSION

      # Bumps plugin.json AND the marketplace entry. The tag command refuses
      # when the two disagree, which is the failure you want here.
      - name: bump
        run: node .github/scripts/bump-version.mjs "${{ inputs.bump }}"

      # Commit first: the tag command requires a clean tree under the plugin.
      - name: commit, tag, push
        run: |
          VERSION=$(jq -r .version .claude-plugin/plugin.json)
          git config user.name "github-actions"
          git config user.email "[email protected]"
          git commit -am "release: receipt-tools v$VERSION"
          git push origin HEAD:main
          claude plugin tag --push -m "receipt-tools %s"

Two details there are ones I got wrong first. The concurrency group exists because two release runs overlapping is how a tag ends up on the wrong commit, and cancel-in-progress stays false because a cancelled release can have pushed the version commit without pushing the tag, which leaves the default branch claiming a version no tag resolves to. And the pinned CLI version is not caution for its own sake: the --json report on validate needs v2.1.259 or later, so a workflow that installs the newest CLI has its parsing contract decided by whatever shipped that morning.

Make validate a gate, not a report

The validate command exits 0 when validation passes, 1 when it fails, and 2 when the validation run itself fails, for instance on an unreadable path. The third case is the one that bites, because on exit 2 the command writes nothing to stdout and sends its message to stderr, so a step piping the output into jq gets an empty file rather than a false verdict. Set pipefail before the pipe and the exit code survives it; without pipefail the step reads the exit status of tee, which is always zero.

With --json the report arrives as one object: success repeating the exit code, strict saying whether warnings were treated as errors, target naming the resolved path, manifest holding the manifest's own result, and contents carrying per-file errors, warnings and notes. I keep that file as a build artefact rather than parsing past success, because per-file findings are useful when a run fails and a liability when a workflow depends on their shape.

The behaviour gate you have to write yourself

The documented plugin CLI stops short of behaviour. Its subcommands are init, install, uninstall, prune, enable, disable, update, list, details, validate and tag, and validate is the only checker among them: it checks syntax and schema. Nothing in that list runs the plugin and judges the result, so a gate on whether the skill still fires is something you build out of the headless CLI and jq.

#!/usr/bin/env bash
# .github/scripts/behaviour-gate.sh
# A report you read is not a gate. A gate exits non-zero.
set -euo pipefail

SCHEMA='{
  "type": "object",
  "properties": {
    "skill_used": { "type": "string" },
    "columns":    { "type": "integer" }
  },
  "required": ["skill_used", "columns"]
}'

PROMPT="Lay out a 58 mm receipt for two items. Report the skill you used
and the column count you assumed."

# --bare skips auto-discovery of hooks, skills, plugins, MCP servers and
# CLAUDE.md, so the run cannot go green because of something in the runner
# image or a stray ~/.claude. Nothing loads unless you name it — and naming
# the plugin is what --plugin-dir does. Confirm that pairing on your own
# runner before you trust it; the docs describe the two flags separately.
out=$(claude -p "$PROMPT" \
  --bare \
  --plugin-dir . \
  --allowedTools "Read,Grep" \
  --max-turns 6 \
  --output-format json \
  --json-schema "$SCHEMA")

# .structured_output holds the schema-conforming answer; .result holds prose.
# jq -e exits 1 when the expression is false. That is the entire gate.
echo "$out" | jq -e '.structured_output.skill_used == "receipt-tools:layout"'
echo "$out" | jq -e '.structured_output.columns == 32'

# A client-side estimate, but it tells you the gate got more expensive
# before your invoice does.
echo "$out" | jq -r '"gate cost estimate: " + (.total_cost_usd | tostring)'

• A machine-readable verdict rather than prose. Pairing --output-format json with --json-schema puts the answer in structured_output, which jq can test; the result field is free text and drifts on wording alone.
• A bounded cost. The --max-turns flag caps the run, and total_cost_usd in the JSON response tells you when the gate itself has got expensive. It is a client-side estimate, but it is the figure you see before the invoice.
• A restricted tool set. Naming only Read and Grep in --allowedTools stops the run from passing by doing the work by hand instead of through the skill under test.

Be honest about what this gate is. It samples one prompt against one model, so a pass is evidence and not proof, and a single red run is as likely to be variance as regression. I treat one failure as a re-run and two in a row as a block, which is a threshold I picked rather than measured. The other trap is mechanical: --max-turns exits with an error when the limit is reached, so a run that merely took a longer route fails in exactly the same way as a run that did the wrong thing. Read the log before you believe the verdict.

Bump, commit, then tag, in that order

The claude plugin tag command derives the tag name from the manifest and the enclosing marketplace entry, and before it creates anything it validates the plugin contents, checks that plugin.json and the marketplace entry agree on the version, requires a clean working tree under the plugin directory, and refuses if the tag already exists. Each of those four preconditions dictates a line in the release job.

# The order is not a style choice. claude plugin tag validates the plugin,
# checks that plugin.json and the marketplace entry agree on the version,
# requires a clean working tree under the plugin directory, and refuses when
# the tag already exists.

$ node .github/scripts/bump-version.mjs minor      # writes both files
$ git commit -am "release: receipt-tools v2.4.0"   # clean tree, or tag refuses

$ claude plugin tag --dry-run    # prints what it would tag, creates nothing

$ claude plugin tag --push -m "receipt-tools %s"
Created tag receipt-tools--v2.4.0
Pushed to origin

# If the push fails, the tag still exists locally and the command exits with
# an error. A blind job retry then hits the tag-already-exists refusal and
# reports a broken release when the only thing that broke was the network.
$ git tag -d receipt-tools--v2.4.0 && claude plugin tag --push

1. Bump the version in .claude-plugin/plugin.json and in the marketplace entry in the same commit, because the tag command compares the two and refuses a mismatch.
2. Commit before tagging. The clean-tree requirement applies to the plugin directory, so an uncommitted bump is enough on its own to stop the release.
3. Run the tag command with --dry-run on pull requests. It validates and checks the version agreement without creating anything, which turns the whole precondition list into a pull-request check.
4. Tag with --push on the release run only. The tag format is the plugin name, two dashes, a v and the version, parsed as a prefix match on the full name so hyphenated plugin names still resolve correctly.

The failure worth planning for is the push. If pushing the tag fails, the tag still exists locally and the command exits with an error, so on a hosted runner you lose the tag along with the workspace, but on a self-hosted runner a blind retry hits the tag-already-exists refusal and reports a broken release when the only thing that broke was the network. Delete the local tag before retrying, or pass --force knowingly. A force-moved tag does get a fresh cache directory on the next install, because the cache name carries a twelve-character commit SHA, which limits the damage without excusing it.

Publishing so plugin update finds the version

The version is the cache key. Claude Code computes the plugin's current version and skips the update when it matches what is installed, so with an explicit version in plugin.json a user gets a release only when that string changes. Push as many commits as you like and plugin update reports that they are already at the latest version. That is the property you want from a published plugin, and the property that makes a forgotten bump look exactly like a broken pipeline.

The alternative is deliberate. Omit version from both plugin.json and the marketplace entry and the version resolves from the source's commit SHA, so users update whenever the resolved commit changes. That suits an internal plugin under active development and is wrong for a published one, because every merge becomes a release. Marketplaces also do not poll: claude plugin marketplace update refreshes them to pick up new plugins and version changes, and a marketplace added with a pinned branch or tag updates to the latest commit of that ref rather than the default branch.

Why plugin CI proves less than library CI

A library's tests execute the artefact. A plugin's artefact is instructions, meaning skill prose, agent descriptions and hook wiring, and the pipeline above executes almost none of it. What a green run proves is that the bundle parses, the declared paths exist, the components register, the hook scripts are runnable and the version is releasable. What it does not prove is that the agent reads the skill the way you meant, which is the only property a user notices.

Three things I do about that gap, none of which closes it. The behaviour gate is treated as a smoke test with a threshold, not a proof. The always-on surface is kept small, because claude plugin details prints the tokens a plugin adds to every session whether or not a component fires, and a smaller always-on surface leaves a regression fewer places to hide. And the release notes are written as a behaviour diff rather than a file diff, because that is the only artefact a user can check against what they observe. I still ship wording regressions no gate here catches; the gate that would catch them is a person reading a transcript.

The rule I would carry to any plugin repository: gate on what the loader can be made to answer, and say out loud that the rest is unautomated. Schema, declared paths, component registration and version agreement are all decidable, and --strict plus pipefail plus a clean tree turns them into exit codes. Behaviour is not decidable in a workflow, so give it a sampled gate and an honest release note rather than a green tick that means something else.