Skip to content

feat(skills): give every pipeline skill a machine-checked contract - #50

Merged
glennfaison merged 1 commit into
mainfrom
feat/skill-contracts
Nov 2, 2026
Merged

glennfaison merged 1 commit into
mainfrom
feat/skill-contracts

Conversation

@glennfaison

Copy link
Copy Markdown
Owner

What

Every pipeline skill now opens with a ## Contract: a YAML block that says what the skill is as a function. This is the first step toward evaluating the skills as functions (design in ADR 0010).

kind: item            # or group
step: Triaging
target: { param: issue, type: issue }
params: [...]         # exactly the rows of the Inputs table
requires: [...]       # preconditions; false means skipped, nothing claimed
returns: [...]        # the closed set of result lines
ensures: [...]        # true on every way out
effects:
  reads: [...]
  writes: { claimed: [...], repository: [...], other: [...], shared: [...] }   # anything not listed is forbidden
  external: [...]
never: [...]
calls: [...]
idempotent: "..."
terminates: "..."

The contract replaces the prose Eligibility, Context read, Outputs, Result, Never and Success criteria sections. Process stays prose.

Behaviour: unchanged

The contracts describe what the skills do today, including the awkward parts. Two documented results are corrected: triage-github-issue and implement-github-issue now list #N escalated: <reason>, which their steps could already produce.

Everything I found that looks like a real behaviour problem is listed under Known gaps in the ADR, deliberately not fixed here, so the contracts stay a clean baseline for evaluation.

Checks

  • scripts/contracts.mjs, run by npm run validate, checks each contract against its skill: params equal the Inputs table, every #N … result the text returns is in returns, item skills list skipped and failed (and escalated if they cite escalate.md), write scopes are valid, group skills never claim, calls and dispatches resolve, and the folded prose sections are gone.
  • tests/contracts.test.mjs: 17 tests on synthetic skills. Added to npm test.
  • Negative check by hand on implement-github-issue: removing escalated, adding a bogus param, re-adding a ## Result section and returning an undeclared result each fail validation.
  • Local: validate, lint, sync:check and check-plugin-version pass. 70 node tests pass. 148 shell tests pass (needed jq on PATH).

Other changes

  • file-github-issues: its Inputs table was split by a prose line into a headed table and a headless one. Now two valid tables. This reorders one line (unattended) in the generated runner prompt.
  • AGENTS.md listed three meta skills and one user-invoked skill. runner-prompts is both. Fixed, and a Contracts rule added.
  • agent-workflow-protocol: new Contracts section. The group and item frame now points at the contract's requires and returns.
  • Changeset added.

For review

Read the contract of one item skill against its Process. The thing to judge is whether ensures, effects and never say what the skill really promises. I wrote them from the skill text, not from runs.

Utility skills are next, and so are scenario tests built on these contracts.

…DR 0010)

Each pipeline skill now opens with a ## Contract (YAML): requires, returns,
ensures, effects by scope, never, calls, idempotent, terminates. It replaces the
Eligibility, Context read, Outputs, Result, Never and Success criteria sections.
Behaviour is unchanged. Two documented results are corrected (escalated on
triage-github-issue and implement-github-issue).

scripts/contracts.mjs checks each contract against its skill in npm run validate,
with tests in tests/contracts.test.mjs. Known behaviour gaps found while writing
the contracts are listed in the ADR and left for separate changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants