Skip to content

feat: add chunk rpc, upgrade and evm rpc traffic attribution - #230

Open
jacderida wants to merge 1 commit into
WithAutonomi:mainfrom
jacderida:chrisoneil/v2-834-traffic-accounting-final-ground-truth-instrumentation
Open

jacderida wants to merge 1 commit into
WithAutonomi:mainfrom
jacderida:chrisoneil/v2-834-traffic-accounting-final-ground-truth-instrumentation

Conversation

@jacderida

@jacderida jacderida commented Sep 16, 2026

Copy link
Copy Markdown
Member

Summary

V2-834 Part D.

Part D of the traffic-accounting ground truth. Three new process-global
counter tables, same relaxed-atomic style as the V2-623 replication
table, emitted on the existing 300s summary loop.

  • storage::traffic — the largest attribution gap by volume: chunk RPC
    bytes were indistinguishable from DHT messaging in wire_tx_bytes.
    Inbound bytes are counted at ChunkMessage::decode by request kind
    (get / put / quote variants / other / decode_error); outbound bytes at
    send-success in answer_one_request, keyed by kind × outcome (get:
    success/not_found/error; put: success/already_exists/payment_required/
    error) via a traffic_key carried on HandledChunkRequest; failed
    sends itemised separately. chunk rpc traffic summary (cumulative)
    in two groups, target ant_node::storage::traffic.
  • upgrade::traffic — archive / binary / signature / manifest response
    body sizes across all three reqwest clients; the releases poll now
    reads bytes() then serde_json::from_slice so the body length is
    observable. upgrade traffic summary (cumulative).
  • payment::traffic — EVM RPC call counts and outcomes at the two
    on-chain read sites. Bodies are serialised inside alloy behind a
    fresh per-call evmlib provider, so bytes are not observable from
    ant-node; EVM RPC is HTTPS/TCP and outside the UDP reconciliation
    invariant. evm rpc summary (cumulative).

Linear issue

Closes V2-834

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Counters and INFO lines on the existing replication summary loop. The releases poll now reads the body with bytes() then serde_json::from_slice instead of .json() — same parse, body length made observable.

Compatibility

  • Wire: none
  • Storage: none
  • API: none (new modules are pub(crate); HandledChunkRequest gains a pub(crate) field)

Semver impact

  • breaking
  • feature
  • fix

Test evidence

  • cargo clippy --all-features -- -D clippy::panic -D clippy::unwrap_used -D clippy::expect_used (CLAUDE.md): clean
  • cargo clippy --all-targets -- -D warnings: clean apart from the pre-existing file_store.rs:2005 redundant_pub_crate lint on main
  • cargo fmt --check: clean
  • New storage::traffic unit tests (2) pass
  • Emitted lines counted against the 32-field cap (largest: 28)

New dependency

none

ADR

n/a

Mitigation / rollback

Revert the PR. Purely additive counters and INFO summary lines at the existing 300s cadence; no config, no wire, no storage change — nothing to migrate.

🤖 Generated with Claude Code

V2-834 Part D.

Part D of the traffic-accounting ground truth. Three new process-global
counter tables, same relaxed-atomic style as the V2-623 replication
table, emitted on the existing 300s summary loop.

- storage::traffic — the largest attribution gap by volume: chunk RPC
  bytes were indistinguishable from DHT messaging in `wire_tx_bytes`.
  Inbound bytes are counted at `ChunkMessage::decode` by request kind
  (get / put / quote variants / other / decode_error); outbound bytes at
  send-success in `answer_one_request`, keyed by kind × outcome (get:
  success/not_found/error; put: success/already_exists/payment_required/
  error) via a `traffic_key` carried on `HandledChunkRequest`; failed
  sends itemised separately. `chunk rpc traffic summary (cumulative)`
  in two `group`s, target `ant_node::storage::traffic`.
- upgrade::traffic — archive / binary / signature / manifest response
  body sizes across all three reqwest clients; the releases poll now
  reads `bytes()` then `serde_json::from_slice` so the body length is
  observable. `upgrade traffic summary (cumulative)`.
- payment::traffic — EVM RPC call counts and outcomes at the two
  on-chain read sites. Bodies are serialised inside alloy behind a
  fresh per-call evmlib provider, so bytes are not observable from
  ant-node; EVM RPC is HTTPS/TCP and outside the UDP reconciliation
  invariant. `evm rpc summary (cumulative)`.

Test evidence: `cargo clippy --all-features -- -D clippy::panic -D clippy::unwrap_used -D
clippy::expect_used` clean, `cargo clippy --all-targets -- -D warnings` clean apart from the
pre-existing `file_store.rs:2005` lint on main, `cargo fmt --check` clean, new `storage::traffic`
unit tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jacderida
jacderida force-pushed the chrisoneil/v2-834-traffic-accounting-final-ground-truth-instrumentation branch from 77c6a6e to 601740f Compare September 16, 2026 21:49

@dirvine dirvine left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Genuinely observability-only: chunk-RPC attribution (rx at decode by kind, tx at send-success by kind×outcome, failed sends itemised), upgrade download body sizes, and EVM RPC call counts. The response.json()response.bytes() + from_slice refactor is behaviour-equivalent with size checks preserved. Non-blocking note: this sits on the chunk-serving hot path (2 relaxed atomic adds per inbound + 2 per outbound on shared global arrays) — consistent with the existing replication table and dwarfed by chunk decode + ML-DSA verify, but worth revisiting with sharded/thread-local counters if it ever shows up at peak. Note: Security Audit CI failure is pre-existing baseline, not introduced here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants