feat(asap-tools): generate records metrics in the paced feeder

[zzylol]

Stacked on the workload-table feeder PR (base branch feat/paced-feeder-facts).

Summary

The feeder now generates sketch-bench's multi-grouping data: type: "records" metrics (http, flows, from sketch-bench#209's e2e.metrics). Each record carries labels and several value fields. Record n depends only on the seed, the metric name and n.

• Labels: value index i has weight (i+1)^-skew, a child is drawn given its parent, and child values name their parent. This mirrors sketch-bench's label_shares.
• Keys (user_id, src_ip): Zipf rank over keys, drawn by binary search over a precomputed CDF with a guide table.
• Burst (flows): with probability share, the record's dst_subnet is set to the burst value and its key is drawn uniformly from a separate range of keys burst keys.
• Latency: 1000 · Pareto(a) (scale from configs/datagen/hydra_http_latency.yaml), times the service's factor 10^u (aqpbm-datagen's scale_range [1, 10]), times 10 for the anomaly endpoint "0.0".
• Time: a second's records are spread over 100 slots of 10 ms, at t + (s+1)·10 ms, and each tick is sent concurrently (fix(asap-tools): send paced feeder ticks concurrently, ordered per series, in (t, t+1s] #840).
• Remote write: each record is one sample per value field, named <metric>.<field> (e.g. http.latency), which is how ASAPQuery's SQL-mode router matches table.value_column. Prometheus is not a target for records metrics.
• ClickHouse: one table per metric: ts, the labels, then the value fields (Float64). See the README.
• CLI: --records-per-sec scales records metrics down. --verify-facts also checks each label's top shares and the burst share over 1e6 generated records, within 5 standard errors.
• One feeder can send a whole all table (classic and records metrics together).

Testing

• Facts check on multigroup-shared1: every label subset matches (http 15, up to 10,000 combinations; flows 7, up to 3e6). The shares match label_shares (e.g. service 0: 0.29841 expected, 0.29847 generated). The burst gets 5.03% of the records.
• CloudLab, ClickHouse 26.10, full rate, 10 s:
  • http: 20M rows, 1,000 distinct timestamps (first at t+10 ms), every endpoint under its service, all 10,000 label combinations.
  • The anomaly endpoint's median latency is 9.97× that of the other endpoints of its service. Service medians differ (scaling).
  • flows: 40M rows, dst_subnet=999 share 0.0500, exactly 10,000 burst keys.
• CloudLab, ASAPQuery precompute_engine (default lateness), http at 20k records/s via remote write: 0 late drops, and all 200,000 http.latency samples were routed to the aggregation.
• Throughput limits are the sinks', not the feeder's:
  • Against a local stub, ticks are 171 ms (http) and 195 ms (flows) for remote write, and about 110 ms for ClickHouse.
  • On one node, a single ClickHouse taking http and flows together (6e6 rows/s) and a single engine taking http's 4e6 samples/s both fell behind (ticks about 1.1 s). End-to-end runs should scale with --records-per-sec or spread the sinks.
• 35 unit tests: shares, child conditionals, burst share and keys, anomaly and scaling, determinism across sinks, remote-write names and labels, RowBinary columns, facts check. fmt/clippy clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae

[milindsrivastava1997]

Code review

1. Medium — records.rs ~426 (timestamp_ms): many samples per series share a timestamp. Records land in 10 ms slots, so records with the same label tuple become samples of one series at one timestamp (~400 per timestamp for the heaviest http tuple at 2M/s). Fine for ASAPQuery's engine today (no per-series timestamp dedupe), but Prometheus/VictoriaMetrics or any timestamp-keyed consumer silently keeps one value. "Never send records to Prometheus" is only a comment (main.rs ~1229) — consider a runtime guard or doc warning.
2. Medium — records.rs ~578-595: label_values can OOM before its size check. Child labels build every "parent.i" string before ensure!(len <= u32::MAX). Parent and child each at 1e5 pass the per-label cap and then try 1e10 strings. Check the cardinality product first.
3. Low — main.rs ~322 (verify_facts): label coverage relaxed to actual <= expected, so missing/collapsed tuples still pass. When expected is far below the 1e6 sample size, equality could be required.
4. Low — records.rs ~374: a burst on a child label overrides only the child index, so it spreads across all parents. Documented as intended and the shipped burst is on a root label; worth confirming it matches sketch-bench semantics.
5. Low (cost) — records.rs 450-483: label_coverage and share_checks each regenerate the 1e6 records, and coverage allocates a Vec<u32> per record per subset (~15M allocs for http). Generate once; pack tuples into an integer key.
6. Low — main.rs ~1229: records batches are cut anywhere and sent concurrently, so a series' samples can arrive out of order within a tick. OK for ASAPQuery, same caveat as (1).

Nits: Rows::{Samples, Records} mismatch panics at runtime (encode.rs ~23); verify_facts returns Result<bool> but never Err; workload.rs module doc says "either … or" though mixed workloads are supported.