Skip to content

feat: verifiable downtime evidence with optimistic challenges and consumer pausing - #63

Open
giunatale wants to merge 14 commits into
mainfrom
giunatale/feat/offline-detection
Open

feat: verifiable downtime evidence with optimistic challenges and consumer pausing#63
giunatale wants to merge 14 commits into
mainfrom
giunatale/feat/offline-detection

Conversation

@giunatale

Copy link
Copy Markdown
Contributor

Closes #38

Downtime on a consumer chain is unprovable on the provider, but it is disprovable: a single validator signature for a claimed-missed height, sealed under a light-client-verified header, indicts the evidence source.
This PR builds the downtime pipeline around that asymmetry.

  • Consumers track missed blocks over tumbling windows (x/slashing downtime
    handling becomes log-only) and report a per-window bitmap to the provider.
    Window parameters are provider-owned and distributed via consumer genesis
    and VSC packets, with staged activation
  • The provider verifies and prices the infraction (validator's epoch fee
    share x missed fraction, converted via photon), then queues the slash
    behind a challenge window instead of executing it. DowntimeSlashFraction
    acts as a per-window ceiling (default 0.0001), repeated
    windows queue independently and can compound
  • MsgChallengeConsumerDowntime lets anyone cancel a validator's pending
    slashes by proving a claimed-missed block was actually signed. A
    successful challenge refunds the withheld fee shares (escrowed in the
    consumer fee pool for the window's duration) and moves the consumer to a
    new CONSUMER_PHASE_PAUSED: no VSC packets, fee accrual stopped, resumable
    by governance (MsgResumeConsumer, with a forced snapshot resync), and
    auto-stopped after MaxPauseDuration.
  • Inbound VSC packets are now authenticated by source port and a pinned
    provider chain id.

The first two commits are standalone fixes for 2 pre-existing bugs on main (export at a zero height panicked after any slash & the consumer stored the provider's client id instead of its own)

Full design and operational notes in docs/consumer-downtime.md.

giunatale added 12 commits July 16, 2026 20:32
app.NewContext(true) builds a context from an empty header, so the
export context reported block height 0. x/distribution's
CalculateDelegationRewards replays validator slash events between the
delegation's creation height and the context height, so at height 0 it
replayed none: for any validator slashed after its delegation was
created, the recomputed final stake exceeded the current stake and the
export panicked in prepForZeroHeightGenesis. Use NewContextLegacy with
LastBlockHeight, matching upstream simapp.
…vidence packets

the consumer stored packet.SourceClient (the provider's own client) as
its ProviderClientID on first VSC recv, guarded by a "set once" check.
that value is meaningless for the consumer's own outbound sends -- it
needs packet.DestinationClient, its own client id, which is guaranteed
by ibc-go's RecvPacket to already have a registered counterparty. this
was invisible until now because nothing before the downtime evidence
feature ever needed the consumer to send an IBC v2 packet back to the
provider; the genesis-time self-created client (never linked to a
counterparty by the relayer) was silently latched onto forever, so
every evidence packet failed to send with "counterparty not found".

discovery now resyncs on every accepted VSC packet instead of once, so
a stale value from a placeholder client heals itself.

@julienrbrt julienrbrt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some tiny nits i'll share, but amazing work! so ACK


GLM 5.2 review

PR Review: feat: verifiable downtime evidence with optimistic challenges and consumer pausing (#63)

Overall assessment: approve with minor comments. This is a high-quality, well-reasoned PR that implements a subtle consensus-critical feature with rare attention to soundness and edge cases. The design document is excellent, and the implementation traces it faithfully. I verified the build compiles and all new and existing tests pass.

Scope

12 commits, 100 files, +18986/−1846. The first two commits (d25a408, a48a7e0) are standalone bug fixes against main; the rest build the downtime pipeline. New Go code is concentrated in:

  • x/vaas/consumer/keeper/downtime.go — consumer-side tumbling-window detection
  • x/vaas/provider/keeper/downtime*.go — evidence validation, slash pricing/execution, challenge verification, pruning
  • x/vaas/provider/keeper/fees.go — fee exclusion + pool-as-escrow
  • x/vaas/provider/keeper/consumer_lifecycle.go — PAUSED phase + auto-stop/resume
  • x/vaas/consumer/ibc_module.go, relay.go — source-port + chain-id pinning

Soundness of the core design

The asymmetry the PR builds around (downtime is disprovable, not provable; only chain-sealed signatures are unforgeable) is correctly translated into code:

  • Challenge soundness (verifySealedCommitSignature): the commit.Hash() == header.Header.LastCommitHash check is the load-bearing seal. The VoteSignBytes(chainID, int32(sigIdx)) uses the array index of the matching signature, which is what cometbft expects. Accepting both Commit and Nil flags is correct — Nil still proves liveness. Pubkey self-authentication via ed25519.PubKey(pubKey).Address() == valAddr correctly decouples from key-assignment state.
  • Re-acceptance prevention: the DowntimeWindowFloors + AcceptedDowntimeWindows interaction is sound. The floor advances monotonically to the max pruned window end (guarded by key.K3() > floor), and the floor write precedes the record delete so a mid-prune failure cannot make a pruned window re-acceptable. Ancient windows hit the floor, live windows hit the retained records.
  • BeginBlock ordering: the explicit TestBeginBlockOrdering_UnresponsiveStopCancelsPendingSlashBeforeSweep encodes the contract that SweepUnresponsiveConsumers / BeginBlockAutoStopPausedConsumers run before SweepPendingDowntimeSlashes. Same-block cancellation wins over execution.
  • Resume atomicity: ResumeConsumerChain uses sendVSCPacketsToChainStrict (propagates errors) rather than the EndBlock wrapper (swallows them), with a clear explanation of why a silent queue-only "success" would corrupt the consumer's validator set under out-of-order IBC v2 delivery.

What I verified

  • go vet clean on consumer and provider packages
  • go test ./x/vaas/provider/keeper/ — pass (93s)
  • go test ./x/vaas/consumer/... — pass
  • Targeted run of all downtime/pause/challenge/resume/withheld-fee tests — pass

Minor comments / questions

These don't block merge but are worth considering.

  1. Genesis import of StagedDowntimeParams panics on invalid input (x/vaas/consumer/keeper/genesis.go ~L105–128). The comment justifies this ("Halt InitChain on unusable staged params"), and GenesisState.Validate also rejects them, so an honest operator never hits it. But a state-export with a future schema change could halt InitChain. The defensive stance is defensible; flagging in case you prefer a log-and-skip.

  2. recordWithheldFee's expired-but-not-swept branch (fees.go L337–354). When existing.ExpiresAt is in the past but the record hasn't been swept yet, the code overwrites amount (discarding existing.Amount) and sets a fresh expiry. This is consistent with the doc ("an expired-but-not-yet-swept record is replaced outright"), but the expired funds are still in the pool — the consumer kept them when the record expired. Re-escrowing a fresh amount against funds that were never moved is fine, but the validator loses the previously-escrowed claim. Intended? Given fee exclusion is a side effect of accusation (not a right), probably acceptable, but worth confirming this matches the intent.

  3. liveEpochShare recomputes numBonded at pricing time (fees.go L141–151). DistributeConsumerFees and liveEpochShare both call GetBondedValidatorsByPower, but the consumer's numBonded can change between pricing (receipt of evidence in the current epoch) and distribution (epoch boundary). If a validator unbonds in between, the share recorded at distribution diverges from the share used to price. The doc acknowledges P resolves "live" for current-epoch windows; just noting the small window of inconsistency is inherent.

  4. DowntimeEvidenceMaxAge + DowntimeChallengeWindow < trusting is checked against DefaultConsumerUnbondingPeriod only (ValidateInfractionParamsAgainst, params.go L207–220). The comment acknowledges this is "per-consumer deviations are operator guidance." Since the default is the only checked bound, a chain that configures longer consumer unbonding periods gets a stricter-than-needed constraint, and one with shorter is unprotected. The doc is honest about this; consider whether the operator guidance (section 9) is the right place vs. a runtime per-consumer check.

  5. E2e test genesis patch note (e2e_setup_test.go): the comment about slash_fraction deserializing from nil to zero is an important footgun. Since InfractionParameters is "unmarshaled directly into InfractionParameters with no defaulting pass," any operator writing genesis by hand who omits slash_fraction silently gets zero-cap downtime slashes. Worth either (a) defaulting at genesis unmarshal time, or (b) a louder callout in the genesis documentation. The same applies to MinSignedPerWindow / SignedBlocksWindow at the consumer genesis.

  6. findPendingDowntimeSlashContaining iterates pending slashes linearly (downtime_challenge.go L156–175). Bounded by the number of pending windows for a single (consumer, validator) pair — typically 1–2. Fine in practice; flagging only because a validator with a large backlog (e.g. long network partition) would make each challenge O(n) in pending windows.

Non-issues I checked and confirmed safe

  • The BitmapSet bounds in the consumer's TrackMissedBlocks are safe: stale bitmaps are padded to (window+7)/8 before indexing.
  • MaxMissed formula W − ceil(M·W) matches the doc and is used identically on consumer (close) and provider (threshold check).
  • SlashTokens units work out: P (fee tokens) · M / C (photons/bond_token) → bond tokens; fraction = slashTokens / totalTokens is dimensionless and capped by SlashFraction ∈ [0,1].
  • Chain-id pinning is pre-seeded at genesis from the trusted provider client state, closing the "first packet teaches the pin" window.
  • PendingDowntimeSlashes keyed by (consumer, validator, window_end_height) correctly coexists with multiple windows per pair; deletion-on-last-execute correctly leaves the WithheldFeeRecord alive while any window is still pending.

Recommendation

Approve. The design is thoughtful, the implementation is careful and matches the doc, tests cover positive paths, negative controls, ordering contracts, and the full queue-then-execute lifecycle. The six minor items above are worth a follow-up issue or a quick reply, none are blockers.

@tbruyelle tbruyelle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incredible work, the design is very solid.

Me and my friend claude just found a couple of bugs that needs to be fixed IMO, see the comments.

Comment thread docs/consumer-downtime.md Outdated
Comment thread x/vaas/provider/keeper/consumer_lifecycle.go
Comment thread x/vaas/provider/keeper/downtime_challenge.go
Comment thread x/vaas/consumer/keeper/downtime.go
…atus, and fix paused-consumer and staged-param holes

Four defects raised in review, each with a test that fails without the fix.

The consumer's provider client could be replaced by any client tracking the
same chain id. IBC attaches no meaning to chain-id uniqueness, so an attacker
chain reporting the provider's chain id passed the only check there was and took
over the pin -- redirecting every downtime evidence packet to a chain that drops
them, and stranding the real provider below the dedup watermark with a large
valset_update_id. The established client is now permanent: once the pin has a
registered IBC v2 counterparty, packets over any other client are rejected. A
pin without one is the genesis client nothing can be delivered over, and the
first client that does deliver replaces it. Recovery from an expired or frozen
pin is a governance MsgRecoverClient under the same client id.

Downtime challenges were verified through the light client's VerifyClientMessage,
which checks the trusted state but not the client's status: an expired client
failed only incidentally via the trusting period, and a frozen one not at all.
Frozen is what a client becomes once its chain is proven to have equivocated, so
its headers now cannot cancel a slash or pause a consumer.

Paused consumers were invisible to the consensus-key collision guard, so a new
validator could take a key already assigned on a paused consumer and the resume
snapshot would carry one consensus address twice. PAUSED now counts as active,
which also opens key assignment during a pause -- replacing a key there prunes
the old consumer address instead of deleting it, since a paused consumer is the
one with downtime state in flight. MsgUpdateConsumer stays refused: it can
rewrite the parameters the challenge was judged under.

Staged downtime params leaked a reverted change. Staging compares against the
active params, so a revert arriving before the window closed read as a no-op and
left the abandoned value to activate at the boundary, leaving the consumer
measuring windows the provider prices differently. A revert now drops the stage.

Also from review: a zero double_sign slash fraction is rejected, since zero
passes the shared fraction check and silently removes the entire penalty for
equivocation, while zero downtime slashing stays valid as a jail-only policy.
@giunatale

Copy link
Copy Markdown
Contributor Author

@julienrbrt answering the six minor items from your review, since they are in the review body
rather than inline threads. Two led to changes, one is already handled in #65, three are
answers with no change.

1 · Genesis panics on invalid StagedDowntimeParams. No change. The scenario needs a
state export whose schema has drifted, and this module has no migration path by design: it is
pre-release with nothing deployed, so there is no earlier version of the schema to drift
from. GenesisState.Validate rejects the same input first, so InitChain is reached only by an
operator who bypassed validation. Halting is the right outcome there, because staged params
that cannot be parsed mean the next window boundary would activate something unusable, and a
consumer measuring downtime under unusable params is worse than a chain that will not start.

2 · recordWithheldFee dropping an expired claim. Intended, and your reasoning for why
it is acceptable is the reason it is correct. The escrow exists to make a validator whole if
it challenges successfully
. Once the challenge window closes unchallenged, the accusation
stood and the claim is extinguished: the amount stays with the consumer and is released on the
next sweep. Carrying the old amount forward would re-escrow funds nobody can claim. Since
this needed asking, the code now says so rather than only the doc.

3 · liveEpochShare recomputing numBonded at pricing time. No change, this is
inherent. P for a current-epoch window has no recorded value to resolve, so it is priced
live and the bonded count can move before distribution. The alternative, pricing at
distribution time, would make a slash depend on state well after the infraction, which is
worse. The window is bounded by one epoch.

4 · ValidateInfractionParamsAgainst checking only DefaultConsumerUnbondingPeriod.
Already addressed in #65, which is where it belongs since that PR is what makes the client's
trusting period observable. Adoption there rejects a client whose TrustingPeriod is not
above DowntimeEvidenceMaxAge + DowntimeChallengeWindow, and a MinConsumerUnbondingPeriod
floor stops a consumer being registered with an unbonding period too short to support the
window. That turns the default-only compile-time bound into a real per-consumer runtime check.

5 · slash_fraction deserialising to zero. Half right, and the half that was right is now
fixed. A nil fraction is already rejected: GenesisState.Validate reaches
ValidateFraction, which refuses nil, and there is a test for the omitted-key case. Zero
was the actual hole, since it sits inside [0,1] and passes. double_sign.slash_fraction is
now required to be positive, because zero there removes the entire economic deterrent for
equivocation while the parameters still read as configured, and nobody selects that on purpose
by leaving a field at zero. downtime.slash_fraction still accepts zero, since jail without
slashing is a legitimate policy a chain may want.

I did not take the "default at genesis unmarshal time" option. Substituting a value the
operator did not write is worse than refusing the input: it would mean a genesis that reads
one way and runs another, which is the failure mode this whole PR is about.

6 · Linear scan in findPendingDowntimeSlashContaining. No change. It is bounded by the
pending windows for a single (consumer, validator) pair, which is one or two in practice, and
a long partition raises it only as far as the acceptance floor allows before older windows are
pruned. A challenge is a single transaction paying its own gas, so the cost lands on the
challenger.

On the verification you ran, the same three are clean here, along with make lint. The four
items from @tbruyelle's review are fixed in d7b4149 with 11 new tests, each written failing
first and each production guard mutation-checked. The e2e suites still need one run against
those fixes before this merges.

Comment thread x/vaas/provider/keeper/msg_server.go Outdated

@tbruyelle tbruyelle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving with just one last thing to fix (see my last comment).

…merActive

IsConsumerActive counts a paused consumer as active, so the update gate
read as active-but-not-paused, a composition the review found confusing.
The site is phase-specific, so it now names the three phases it admits;
paused stays excluded for the reason the comment keeps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: handle offline validators on the consumer chain side and punish them on provider

3 participants