Skip to content

release: post-Milestone 3 pricing fixes and the move to sorobanscan.com - #418

Draft
karczuRF wants to merge 165 commits into
masterfrom
develop
Draft

karczuRF wants to merge 165 commits into
masterfrom
develop

Conversation

@karczuRF

@karczuRF karczuRF commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

What it carries

Pricing and ingest:

Pools and coverage sweep:

Protocol:

Hosts and portal:

Docs and tooling:

Checked on production

Checked with cdk diff against production (2026-11-10, develop @ b24425b)

  • Compute, EventBridge: the only differences are the Lambda code hashes. No change to environment variables, IAM, schedules, memory or timeouts. The hashes come from a build on a different machine, so they neither confirm nor rule out a match.
  • ApiGateway: no differences.
  • Observability: three alarms on develop are not deployed: LedgerProcessorSacUnprovenAlarm, LedgerProcessorSacCandidatesAlarm and SacContractIdentitiesAlarm (dashboard alarm count 87 → 90). The stack was last deployed on 2026-10-07. This is deliberate: task 0242 decision D8 holds them until heal runbook §6, after the §5 cleanup. Until then master describes alarms that do not exist yet, and SacUnprovenSkipped is read by hand.
  • Secrets: one parameter description differs (wording only); the stack was last deployed on 2026-08-28.

Not verified here

adamkoot and others added 30 commits October 5, 2026 16:45
…n unproven one

A SAC leg was minted as a Contract identity whenever its classic asset
had not been interned yet, giving one token two asset_ids.

learn_sac accepts a contract -> SEP-11 mapping only when the sha256 SAC
derivation reproduces the address. The live path feeds it from the
transaction's own transfer/mint/burn/clawback events before the swaps
are classified. A trade whose leg BE flags is_sac but carries no proof
is skipped, interns nothing and is counted as SacUnprovenSkipped. The
is_sac set is a fourth cold-start read and fails Init on error.
…on an unproven one

events-backfill reads only pool events, so it never sees the SAC event
that proves a leg's classic asset. One start-of-run query fetches a proof
per is_sac contract assets.sac_address cannot resolve, and learn_sac
verifies each before the first chunk. A swap on an is_sac leg that is
still unproven is skipped and printed as `unproven sac swaps: N` (0
included); the 0286 orchestrator STOPs on N > 0 and passes a pre-0242
binary that prints no such line.
…AC skips

The resolver stops new SAC mints, but nothing watched for one. The
rollup-freshness probe now publishes SacContractIdentities: contract rows
of assets FINAL that are some row's sac_address or a BE is_sac contract,
refused on an empty scan or an empty BE set. Production reads 36 until
the heal, so its alarm ships with actions off behind
opsAlarms.sacContractIdentitiesActionsEnabled. The live SacUnprovenSkipped
counter gets its alarm, enabled, shaped like the 0291 one.
A classic asset's SAC address answered 404, or, for a SAC we had
minted as a second identity, a near-empty listing of the same token.
Resolve a C... identifier against the stored assets.sac_address once at
the handler boundary, so detail, price, ohlcv, oracles and batch answer
the classic (XLM's SAC as native), and the classic wins over a pre-heal
Contract row (D5).
The code stops new mints; the 36 existing SAC contract identities need a
data heal. The runbook orders it: 29-row seed after stage B, stage C to
202609, deploy, residual cleanup with an only-copy gate and rollback
copies, then the alarm actions. It also covers a STOP on
`unproven sac swaps:`, the --amm wait path, and keeps AC3 open for the
five U3 assets. The 0286 runbook points to it.
…wers

docs(0294): shorter Milestone 3 form answers
U3 was the 0139 re-key hole, not dust or an ingest bug: the five SAC
ids collided with classic ids and their candles stayed in __pre0139.
Stage C with the 29-row seed rebuilds them; U3 stays inside 0242.
Every item re-checked on 2026-10-06, with the observable beside each:
the reviewer key's plan read from API Gateway, the release run's jobs,
the alarm states and the /api/ redirects.
0327 fixed getAssignee in the generator, but board.html kept its own
rule: the who of the last history entry. On a newest-first history
that is the author, so 0149 showed okarcz while board.json said akot.
…s-the-computed-assignee

fix(0328): board page reads the assignee from board.json
Re-measured on prod 2026-10-06: no coarse row on any of the six tiers
holds a USD value its children do not support. 0286's rate-form
close_usd (ADR 0287 §5) removed TEST E's zero; the version arithmetic
is unchanged and documented in ADR 0317. No follow-up tasks.
…ion-checklist

docs(0294): tick the pre-submission checklist
…en tiers

Since 0229 the /ohlcv clamp repairs a crossed candle on the way out, so
nothing noticed a corrupt stored row. The probe now reads every
price_ohlcv tier FINAL over a per-tier bucket window (2 days widened by
one bucket) and publishes CandleBandViolations: priced rows outside the
band or with any price <= 0, zero tolerance. Seven independent reads so
one failing tier cannot hide the others; a zero is refused unless all
seven were read and the 1m window was non-empty.
…nst ClickHouse

Seven ITs on the production queries: high below close, low above open,
low above high (counted once), a zero low beside a positive close, _1w
and _1M violations older than 2 days inside their widened windows,
violations outside the window, and a bad row superseded by a good
higher version. Each was shown red with its clause removed.
… pinned at 1

Alarms prices-{env}-ohlc-band-{1,100,10000} on the probe's new metric,
through the shared usdSanityRungs helper with the first rung fixed at 1
as zeroInvariantAlarms does. The escalation-counts doc now names the
fourth ladder those rungs drive.
…urce is asserted

The 0229 clamp stays: stored-row consistency is now asserted at the
source with zero tolerance by rollup-freshness-probe. Records that the
as-stored QuoteLeg arm does not clamp O/H/L, the 2026-09-28 prod
baseline, and why there is no API clamp metric.
On develop client() became async: it now stops the shared prices
reconcile MVs before a test seeds the tiers (0203 review WR-05).
The sac-contract-identities alarm reads 36 until the heal, so deployed
with the PR it would sit red on the Row-0 alarm strip, which is the SCF
evidence of 0294. Observability now ships in §6 with its actions on (D8);
until then the operator reads SacUnprovenSkipped by hand.
Observability ships after the cleanup so the red baseline alarm stays off
the SCF alarm strip. The seed ran on 2026-10-06; bak_0242_seed was
rebuilt from its rows because the staged variant was not used.
The ingest needs decimals() from the same simulateTransaction call the
asset-discovery symbol stage makes. Move the envelope builder and the
Absent/Transient classification into prices_ingest_core::soroban_rpc so
both readers share one boundary. symbols.rs re-exports the old names.
amm_trade_to_tick scaled both legs at a fixed 7, so a contract token
with d decimals priced at 10^(7 - d) of market. Classic identities and
their SACs stay at 7; a pure Soroban token uses its decimals() from the
asset registry, and a token whose decimals are unknown yields no tick
and is reported in LedgerSoroban::missing_decimals instead of being
guessed.
Add prices.asset_decimals and a DecimalsResolver that asks a token's
SEP-41 decimals() by simulation, rejects anything above rust_decimal's
scale, and leaves a failed contract alone for ten minutes. The writer
loads the table into the asset registry and writes new answers.
The live processor, events-backfill and sdex-backfill (combined mode)
now resolve the tokens a decode reported missing, persist them, and
decode that ledger again. Cold starts load prices.asset_decimals first.
events-backfill prints the trades it still could not reprice.
Schema overview gains section 3.10b, amm-trades-schema closes its open
decimals question, and the 0286 re-ingest runbook explains the new
trades dropped (decimals) line events-backfill prints.
Reported by SBE: contract-token prices are off by 10^(7 - decimals).
All seven contract tokens the API lists have decimals != 7. Design:
SEP-41 decimals() over RPC, resolved by the ingest on a cache miss.
Was committed as 0328, which another task took while this sat unpushed.
adamkoot and others added 28 commits November 2, 2026 14:28
…econd-identity

fix(0242): resolve a SAC to its classic asset instead of a second identity
The product portals gather under /api/ on the explorer's host
(soroban-block-explorer task 0660, PR #699). The task carries the
deploy order: sync, then the explorer, then Compute.
The product portals gather under /api/ on the explorer's host: the
Prices portal now, x402-facilitator and sorobanscan later, /api/ itself
a hub (soroban-block-explorer task 0660, PR #699).

Base path, router basename, OAuth landing (PORTAL_HOME) and the sync
prefix move to /api/prices-api/. The backend's /api/ on the API host,
the session cookie's Path=/api/, the Discord redirect URI and
portalWebOrigin stay: that is another host.

Deploy only after explorer PR #699 (order in the task); Compute must
not go before the explorer's Delivery deploy.
…-api-prices-api

feat(0339): serve the portal at /api/prices-api/
sorobanscanApiDomain adds a second custom domain on the same stage and
root base path, beside apiDomain, which keeps serving. Its DNS is
Cloudflare's (soroban-infrastructure), so the stack writes no record and
outputs the regional target instead. The certificate is the explorer's
*.sorobanscan.com one (its task 0661), referenced by ARN: in use, so ACM
keeps renewing it.
…ices-api-sorobanscan-com

feat(0340): serve the API on prices-api.sorobanscan.com too
apiBaseUrl is https://prices-api.sorobanscan.com and portalWebOrigin https://sorobanscan.com, so the gateway's CORS, the handler's links and the portal's sign-in landing name the new hosts. The portal bundle is built against sorobanscan ApiDomain's host, same-site with its page: the session cookie is SameSite=Lax. The old API host keeps serving.
PUBLIC_API_BASE_URL and the committed openapi.json servers must equal apiBaseUrl (links.spec, the openapi diff in CI).
…nscan-com

feat(0340): switch the API and the portal to sorobanscan.com
The API answers on prices-api.sorobanscan.com and the portal on
sorobanscan.com since the switch. The endpoint reference, the overview,
the OAuth runbook, the skill, the portal's logo link and the comments
that state the current hosts now name them.

Dated SCF evidence and runbooks stay as submitted. The cache
verification and the k6 load test stay on the old host, because they
measure API Gateway and the new host puts Cloudflare in front.
docs(lore-0340): name the new hosts in docs and the skill
The explorer's ClickHouse endpoint moved to ch.sorobanscan.com on
2026-11-06 (soroban-block-explorer task 0665); every Prices Lambda
already uses it. Runbooks, reingest_0286.py's --ch-url default and the
sdex-backfill flag's doc now name it too, before the old name goes.
Dated SCF evidence stays as submitted.
docs: name ch.sorobanscan.com in runbooks and tool defaults
…m Inferera

Hackathon feedback from Dominik (Inferera). External requests tracked in GH #416 (SAC alias) and #417 (NAV). Notify him when each ships.
No code written. The alias already exists on develop via 0242 (PR #390); what production runs is unclear. Findings recorded in the task.
@karczuRF
karczuRF marked this pull request as draft November 10, 2026 16:54
Production is deployed by hand from develop and master trails it.
Describe the develop -> deploy/production -> master flow; no code yet.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants