feat(asap-tools): generate the paced feeder's data from a workload table

[zzylol]

Stacked on #840 (ticks sent concurrently, samples in (t, t+1s]).

Summary

The paced feeder can now generate its data from a sketch-bench #777 workload table, so an end-to-end run gets the data the optimizer assumed (#824 B4). --workload-table FILE reads workloads[0].e2e.metrics and sends every metric in it, or one with --metric NAME. The existing --groups/--series-per-group mode is unchanged.

• Labels: a label has cardinality values, or with child_of that many under each parent value. A child value names its parent (job="3", label_0="3.17"), so it is unique across parents. Labels without child_of form a cross product.
• Values: the series of Zipf rank r is scaled by N·(r+1)^-θ / Σ_j (j+1)^-θ, so scales average 1. Ranks are a seeded shuffle, so heavy series spread across parents.
  • A gauge is scale_i · Pareto(a).
  • A counter is (n+1)·scale_i: stateless, strictly increasing, never resets.
  • Metrics are seeded per name, so two feeders with the same flags send identical data.
• --verify-facts: for every non-empty label subset it compares the generated series' distinct tuples with the expected cardinality (the product of fan-outs, closed under child_of ancestors), then exits without sending. It is non-zero on a mismatch.
• ClickHouse: one table per metric, named after it, with columns ts, the labels in order, then value. The CREATE TABLE statements are in the README.
• --samples-per-sec scales a run down.

Testing

• Facts check on sketch-bench's classic table: job 10, label_0 10,000 and (job, label_0) 10,000 for both data and data_total; exit 0.
• CloudLab (clnode109), ClickHouse 26.10, full rate, one feeder per metric, 10 s: 20M rows each. Every cardinality matches, and every label_0 sits under its job. The counter strictly increases per series with a constant increment. The 32 heaviest label_0 sums spread over all 10 jobs.
• Prometheus 2.53 remote-write receiver, 10 samples/s: all 400 requests returned 204, with 0 out-of-order samples. Its [t−5s, t] sums match ClickHouse's on the same window.
• ASAPQuery precompute_engine: all 20M samples arrived, and with allowed lateness 0 none was dropped as late.
• Timing against a local stub, full rate (4e6 rows/s): about 0.33 s per tick for remote write and 0.25 s for ClickHouse.
• 18 unit tests: fan-out, unique child values, cross product, facts check pass and fail, Zipf ranks and weights, counter monotonicity, sorted remote-write labels, RowBinary columns. fmt/clippy clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae

[milindsrivastava1997]

Code review

1. Medium-high — main.rs:407: early stop misses a metric that always fails. It only fires when rows == 0 across the whole tick. E.g. --sink clickhouse where data exists but data_total doesn't: every data_total insert fails, data keeps rows > 0, and the feeder runs the full --duration-secs sending half the workload, reporting errors only at exit. Track failures per metric.
2. Medium — main.rs:291: --verify-facts sends nothing but runs after the rate checks (samples_per_sec == 0, > 1000, mismatched per-metric rates in table::load), so a valid table with mixed rates can't be fact-checked without an unrelated --samples-per-sec.
3. Medium — table.rs:267: facts() does assert!(n < 32) on user input and iterates 2^n label subsets × all series. ≥32 labels panics; ~20 effectively hangs. Validate label count in load and return an error.
4. Medium (perf) — main.rs:361: per-metric rows for a tick are generated sequentially on the main loop and wait behind earlier metrics' sends holding permits. Multi-metric single-feeder at full rate is untested (the 0.33 s/tick measurement was one feeder per metric). Consider generating inside each spawned batch task.
5. Medium (regression) — workload.rs:72: Workload::grouped now materializes all label strings (Vec<Vec<String>>) for the run; previously computed on the fly. At 1e7 series that's ~1 GB extra RSS, so flags that worked before may OOM.
6. Low-medium — main.rs:171: in table mode the ClickHouse table is the bare metric name and --table conflicts with --workload-table, so there's no way to target db.table.
7. Low — main.rs:247, table.rs:121: unwrap_or_default() turns missing values into 0 / "" (e.g. "type `` is not supported"), against the fail-loud rule in AGENTS.md.
8. Low — encode.rs:21: RemoteWriteEncoder::new still takes name though every caller passes &metric.name; two sources for __name__.
9. Low — table.rs:135: bare 1.0 Pareto scale literal.
10. Low — table.rs:436: write_table writes to the system temp dir, never cleans up, and identical content shares a path so parallel tests can race. A load_str/from_value split would avoid the filesystem.

Tests: none cover findings 1–3.