Skip to content

Add sell-api skill, sell-oneshot skill eval, and radius-cli sell check - #60

Merged
erikzrekz merged 9 commits into
mainfrom
feat/seller-skill-and-sell-check
Oct 15, 2026
Merged

erikzrekz merged 9 commits into
mainfrom
feat/seller-skill-and-sell-check

Conversation

@claude

@claude claude Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Requested by Eriks Reks · Slack thread

Before: Seller guidance was one section inside the x402 skill. It was Hono-only, and it shipped with a long hand-rolled protocol illustration. The one-shot seller eval's prompt lived in evaluations/ and pointed the agent straight at the SDK README and the x402 skill. Nothing could check a seller's 402 before deploy short of decoding the header by hand.

After: A top-level sell-api skill covers selling an API. It detects the framework (Hono, web-standard/Next route handlers, Express, Next middleware marked untested), installs and configures radius-sdk, runs locally, verifies with radius-cli sell check, and hands off deployment. It has one reference per topic: frameworks, protocols, networks/facilitators, settlement, and gotchas. The gotchas are taken from the seller eval's XFAILs and observations, each with a mitigation. The x402 skill is now the buyer side plus the protocol. The SELL-ONESHOT prompt is now a plugin eval case. The agent has to find sell-api on its own, and the deterministic grader is still pnpm eval:payments:sell -- --case SELL-ONESHOT. The new radius-cli sell check <url> audits an unpaid 402 and exits non-zero when a check fails.

How: sell check decodes PAYMENT-REQUIRED with @x402/core/http (the same decoder radius-sdk uses; it is now a CLI dependency, pinned to the SDK's 2.27.0). It parses MPP with the SDK's parseMppChallenges, reads /supported through RadiusFacilitatorClient, and asks createRadiusFetch (with a throwaway key that declines before signing) whether a radius-sdk buyer would pay. For each offer it checks the network, the asset and EIP-712 domain, the amount (against --price, which names the decimals a wrong amount implies), payTo, resource.url, maxTimeoutSeconds, and facilitator support. Options: --probe-head, and --pay --threshold (testnet unless --network mainnet is passed explicitly). Unit tests use the real SDK seller and a mock facilitator. SELL-ONESHOT now scaffolds a package.json that lists only one framework, reads the prompt from plugins/radius/evals/sell-oneshot/prompt.md, and runs sell check as an extra grader step. Plugin bumped 0.0.5 → 0.1.0. CLI changeset is minor. No SDK runtime changes.

Verified locally: pnpm build, pnpm typecheck and pnpm test pass (SDK 218, CLI 46 tests). eval:payments passes 20/20 and eval:payments:agent passes 36/36. eval:payments:sell gives 6 pass, 2 XFAIL, 0 fail, the same as before. validate_plugin.py --base-ref, claude plugin validate --strict (plugin and marketplace), and docs prettier, eslint and validate also pass. SELL-ONESHOT was also run once with a real Claude Code integrator (--plugin-dir plugins/radius, the README example), and it passed on Express and Hono. The paid plugin-eval workflow was not run.

Release order: the skill tells users to run radius-cli sell check (radius-cli 0.5.0) and gives a curl fallback. Publish the CLI before releasing this plugin version.

Stacked on #48: the base is feat/agent-payments-evaluation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq


Generated by Claude Code

- plugins/radius/skills/sell-api: a top-level seller skill (detect the
  framework, install, configure radius-sdk, run locally, verify with
  `radius-cli sell check`, hand off deployment), with references for
  frameworks, protocols, networks and facilitators, settlement, and the
  gotchas the seller eval records. The x402 skill keeps the buyer side
  and the protocol and points sellers to sell-api; the manual seller
  illustration (x402-server.md) is removed and the seller scenario
  fixtures move to sell-api. Plugin 0.0.5 -> 0.1.0.
- plugins/radius/evals/sell-oneshot: the SELL-ONESHOT task as a Claude
  plugin eval case (skill-fired + behavior graders). It names no
  framework or repo path; sell.mjs reads it from there, scaffolds a
  package.json listing one framework, and also runs `radius-cli sell
  check` as a grader step.
- radius-cli `sell check <url>`: audits a seller's unpaid 402 (x402
  offers decoded with @x402/core as the SDK does, MPP challenges,
  facilitator /supported, SDK buyer selection), with --price,
  --facilitator, --probe-head and --pay --threshold (testnet unless
  --network mainnet). Checklist or --json, exit 0/1/2. Unit tests use
  the real SDK seller and a mock facilitator.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq
@erikzrekz
erikzrekz added this pull request to stack #61 October 13, 2026 17:22
@claude

claude Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

CI: Docs / preview is red, and the cause is not this PR's changes

The check job passes (format, lint, typecheck, docs validate, build, agent-markdown and sitemap checks). The failure is in the preview job, at wrangler versions upload. Cloudflare rejects the site's _redirects file:

A request to the Cloudflare API (/accounts/***/workers/scripts/radius-docs/versions) failed.
  Invalid _redirects configuration:
  Line 142: Maximum number of dynamic _redirects rules limit of 100 exceeded [code: 100324]

This PR doesn't modify packages/docs/public/_redirects. The file is inherited from the base branch feat/agent-payments-evaluation (#48), which was cut before #57 ("chore: remove old redirects", e6d5c11) landed on main. #57 removed 50 rules, so main has 73 non-comment rules and this branch still has 108. main's Docs runs have passed since then.

Fix: bring #57 into the base branch, by merging or rebasing feat/agent-payments-evaluation onto main. After that, this PR's preview should upload. Re-running the job without that change will fail the same way, because the input is identical.


Generated by Claude Code

(cherry picked from commit e6d5c11)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🦋 Changeset detected

Latest commit: c7fb137

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
radius-cli Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

…Worker proxy, selective gating)

SELL-NEXT-MW runs radiusPayments in Next 15 edge middleware, SELL-ASTRO-MW in Astro 4.15
and 5.18 SSR middleware, SELL-WORKER-PROXY in a Worker (workerd via wrangler dev) in front of
an origin, and SELL-SELECTIVE a humans-free, agents-pay wrapper. Next and Astro are installed
on demand into a cache directory and the cases report SKIP when that is not possible; the
runner gains a SKIP status. Known gaps are XFAIL: paid missing pages, localhost resource.url,
prerendered pages, build-time 402 corruption, and CDN headers passing through paid responses.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq
…igin gates in sell-api

Replace the untested @x402/next row with the radiusPayments middleware pattern for Next 15
and 16, add Astro middleware (with the prerender guard), a Cloudflare Worker in front of an
origin and Fastly Compute, and a publishers reference (agents versus humans, CDN caching,
paywalls, off-domain assets, discovery, paid 404s). Add gotchas for Astro prerender build
corruption, paid 404, localhost resource.url, CDN header pass-through and Fastly AbortSignal,
broaden the skill description, and add a skill scenario. Claims are limited to what the
seller evals or hand runs verified; untested items are marked.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq
@erikzrekz
erikzrekz marked this pull request as ready for review October 13, 2026 20:03
@azf20

azf20 commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Review: sell-api skill + radius-cli sell check

This covers only the diff on top of #48. I reviewed it and then re-checked each finding by building the CLI and running radius-cli sell check against a local node server that sends crafted 402s and acts as a fake facilitator. Nothing was paid.

Worth fixing before merge

1. A malformed accepts entry crashes sell check with no report (packages/cli/src/lib/sellCheck.ts:259-262)
decodePaymentRequiredHeader only checks base64 and JSON. The Array.isArray guard at :240 still lets [null] and other non-object entries through to presetFor(req.network).

  • Repro: PAYMENT-REQUIRED = {"x402Version":2,"accepts":[null],…} gives error: Cannot read properties of null (reading 'network'), exit 1, and no report.
  • A tool for diagnosing broken 402s shouldn't crash on one. Fix: shape-check each offer and report a FAIL for any that are malformed.

2. The MPP mirror check skips the accepts array guard (sellCheck.ts:397)

  • Repro: when the response also carries a valid WWW-Authenticate: Payment challenge, both of these crash with exit 1:
    • accepts as an object: error: pr.accepts.some is not a function
    • accepts missing: error: Cannot read properties of undefined (reading 'some')
  • The same inputs without the MPP header are handled cleanly by the guard at :240.
  • Fix: reuse the already-checked accepts from :240.

3. An x402 v1 challenge in the JSON body is missed (sellCheck.ts:199)
sell check reads the challenge only from the PAYMENT-REQUIRED header. The SDK buyer (packages/sdk/src/client/index.ts readChallenge, ~728-745) also parses a v1 challenge from the 402 body.

  • Repro: a 402 with no header and a v1 JSON body gives FAIL x402: no PAYMENT-REQUIRED header… and exit 1. createRadiusFetch against the same URL prints WOULD PAY x402 0.001 USDC base.
  • sell check therefore fails sellers that real buyers would pay, and the buyer probe is skipped entirely (:408), so the report has no buyer line.

Lower priority / follow-ups

  • --pay timeout message (:479/:497): each request, the paid retry included, gets its own --timeout budget (15s by default). A seller that settles and then takes longer than that to respond gets aborted after the payment was sent. The report then says payment failed: … aborted with no txHash, which reads as "nothing was paid". Consider a longer timeout on the paid leg, or a message that says the payment may have settled. (Checked by reading the code only.)
  • MPP mirror only matches v2 offers (:397): it compares amount and the CAIP-2 network, so a v1 offer (maxAmountRequired, network: "base") that MPP mirrors exactly still gets warn mpp[0].mirror: matches no x402 offer. Reproduced; it's only a warning.
  • fixtures/sell-next/middleware.js:33-36: isAgent treats any UA containing Mozilla/ as human, so ClaudeBot and ChatGPT-User read for free. The skill's publishers.md and sell-selective/gate.mjs check AI_AGENT_UA first, and the eval doesn't depend on this, but the fixture's header calls it the skill's pattern. Copying the AI_AGENT_UA check in would make that true.
  • Smaller items:
    • --price compares the amount as strings (:311), so "01000" fails against 1000. Compare BigInts instead.
    • A facilitator kind with no extra.assetTransferMethod is treated as eip3009-only (:351).
    • Bad flags (-H 'bad', -d @missing.json, --network foo) exit 1, not the documented 2, and --threshold isn't validated up front (sell.ts:81).
    • The Radius asset override is duplicated at :284/:383/:415, and the request-body logic at :166/:495.

Review run with Claude Code; each "worth fixing" item above was reproduced, not just read.

…check

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq
- Shape-check every accepts entry; a non-object offer is a FAIL for that offer, not a crash.
- MPP mirror check reuses the validated offers (object/missing accepts no longer throws).
- Read an x402 challenge from the 402 JSON body when there is no PAYMENT-REQUIRED header,
  as the SDK buyer's readChallenge does (v1, and an off-spec v2 body with a warning).
- Compare --price amounts as BigInt (also for MPP); MPP mirror matches v1 offers
  (maxAmountRequired, network aliases).
- Bad -H, unreadable -d @file, unknown --network/--networks and invalid --threshold exit 2.
- --pay: the paid request gets a longer timeout budget, and a failure after the payment
  was sent says the payment may have settled.
- sell-next fixture: check AI_AGENT_UA (and signature-agent) before the Mozilla/ human test.
- Tests for each; README, docs and changeset updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq
@claude

claude Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor Author

Addressed azf20's review in 11c7e10. The branch now includes the #48 fixes (2ded934) through merge 8472cf6, which also brings main (#62) in via #48. pnpm test passes (CLI 64 tests), eval:payments 20/20, eval:payments:agent 36/36, eval:payments:sell 7/12 pass + 5 XFAIL, 0 fail. The docs checks and both plugin validations also pass.

Each crash and behaviour was reproduced first with the built CLI against a local server sending crafted 402s, and now has a unit test that fails on the old code.

Fixed:

  • 1. Malformed accepts entry: each entry is shape-checked, and a non-object (null, string, number) is a FAIL for that offer (offer[i]) instead of a crash. The other offers are still checked.
  • 2. MPP mirror check: it reuses the validated offers, so accepts as an object, missing, or [null] no longer throws. The mirror check reports skip, and x402 fails with "accepts is not an array" where relevant.
  • 3. v1 challenge in the JSON body: with no PAYMENT-REQUIRED header, an x402 v1 body is read as the SDK buyer's readChallenge does. The offers and the buyer probe line now appear. readChallenge is private to createRadiusFetch, so I mirrored its logic. An off-spec v2 body is also read, with a warning.
  • 4. Smaller items:
    • --price is compared as BigInt for x402, and for MPP too.
    • A v1 offer that an MPP challenge mirrors exactly no longer warns. It compares maxAmountRequired and network aliases.
    • -H 'bad', -d @missing.json, an unknown --network/--networks, and an invalid --threshold now exit 2, validated up front in sell.ts.
    • On --pay, the paid request gets a longer budget (4x --timeout, at least 60s). A failure after the payment was sent now says it "may have settled", though the paid leg fails with no response, so there is no hash to quote. That part is covered by a unit test only.
  • 5. fixtures/sell-next/middleware.js: it now checks AI_AGENT_UA and signature-agent before the Mozilla/ human test, like publishers.md and sell-selective/gate.mjs.
  • README, the radius-cli docs page and .changeset/sell-check.md are updated.

Skipped:

  • Missing extra.assetTransferMethod on a facilitator kind: left as eip3009. The SDK's own convention (server/scheme.ts, and the buyer's offer filter) treats a missing value as eip3009, so the check already matches it.
  • Duplicate-code refactors: the Radius asset override and the request-body logic are left as is, as agreed.

Generated by Claude Code

…check

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MTVVAF1bbGhWkfLjATWseq

# Conflicts:
#	plugins/radius/.claude-plugin/plugin.json
kylecrawshaw
kylecrawshaw previously approved these changes Oct 14, 2026
Base automatically changed from feat/agent-payments-evaluation to main October 14, 2026 19:47
@claude
claude Bot dismissed kylecrawshaw’s stale review October 14, 2026 19:47

The merge-base changed after approval.

@erikzrekz
erikzrekz merged commit 70ff44c into main Oct 15, 2026
8 checks passed
@erikzrekz
erikzrekz deleted the feat/seller-skill-and-sell-check branch October 15, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants