Skip to content

Tracking: RQE frontend for the paper eval #886

Closed

Description

@milindsrivastava1997

Goal

For the paper eval, run ASAPQuery's side of an experiment without PromQL/SQL parsing: a query is an RQE expression tree (self-contained RQE leaves, combined by aggregations and binary ops), POSTed to a new engine route that looks up each leaf's sketch directly by RQE id. Baselines (Prometheus, ClickHouse, …) keep PromQL/SQL. Covers sketch-bench eval tables and the random workloads, PromQL- and SQL-style queries.

sketch-bench side: ProjectASAP/sketch-bench#224

Steps (one PR each, stacked PRs, worktrees via gwt.sh)

# Issue What Stacked on
A1 #880 tree type + validation (asap_types) main
S1 sketch-bench#222 expr in eval-table e2e queries; regenerate tables main
A2 #881 expr + feeder-format data_facts in random workloads main
S2 sketch-bench#223 eval table from a random manifest needs A2
A3 #882 --planner saved reads trees, writes rqe_configs A1
A4 #883 engine rqe.rs + POST /api/v1/rqe_query A3
A5 #884 rqe client protocol, side file, query_engine.frontend switch, e2e check A4

Decisions

  • Use: paper eval only; the experiment scripts call the RQE route for ASAPQuery. The PromQL/SQL paths are unchanged.
  • Tree: leaf rqe is self-contained (id, metric/table, capability, lookback_ms, grouping, phi, k, value). agg = avg/min/max/sum with by only, by ⊆ child labels. binary = + - * / (scalar operands allowed), equal label sets. Top-k only at the root. Leaf capabilities: sum, quantile, cardinality, topk_by_value (rate and topk_by_count out of scope; queries using them get no expr).
  • Where the tree comes from: written by the generators next to promql/sql (expr), for existing query shapes only; new templates (e.g. avg by (G)(quantile_over_time), binary in random workloads) are a follow-up.
  • Sketch lookup: direct by RQE id via a new rqe_configs section of inference_config.yaml, written by --planner saved (roll-up deployments included); query_configs kept.
  • Random workloads: planned like eval tables: sketch-bench converts a manifest to a table (S2), solved with --target p95 (any accuracy_sla ≠ 0.95 fails), converted by --planner saved. data_facts use the paced feeder's metrics format so the feeder and the planner agree: data Pareto α 2.0, Zipf θ 1.1 over 10,000 series, 10 samples/s; events uniform labels, user_id Zipf θ 1.1 over 10,000 keys, 10 records/s per leaf.
  • Route: POST /api/v1/rqe_query {expr, time} → Prometheus vector JSON with exactly Prometheus' output labels. No fallback. No data → empty result; unknown RQE id / invalid tree → 400.
  • Client: rqe protocol; configs keep PromQL/SQL text, a side file maps it to expr, so results and calculate_fidelity are unchanged. query_engine.frontend: promql|rqe (default promql).
  • Out of scope: SQL accuracy scoring (asap-tools: calculate_fidelity never scores SQL queries #885), filtered RQEs, Prometheus range queries, new templates.

Done when

One classic sketch-bench table and one random workload run end to end through the RQE route with the paced feeder, matching the PromQL route for the same plans wherever the PromQL route works.

Activity

  1. milindsrivastava1997 commented on Oct 15, 2026

    @milindsrivastava1997
    ContributorAuthor

    Decision (2026-10-14): random-workload tables (sketch-bench#228) need #879's per-series-avg planning (af8888a) in the planner. We wait for #879 to merge, then merge main up the stack (#887 → #888 → #889 → #890). No cherry-pick.

  2. milindsrivastava1997 commented on Oct 15, 2026

    @milindsrivastava1997
    ContributorAuthor

    Known and deferred (review of #889): an agg over a child with Inf/NaN (e.g. avg(a / b) with b == 0) returns 500, where Prometheus propagates the value. No current workload aggregates over a division.

  3. milindsrivastava1997 commented on Oct 15, 2026

    @milindsrivastava1997
    ContributorAuthor

    All steps are merged: #887, #888, #889 and #890 on ASAPQuery main (dd574c7 … 4db156f); sketch-bench#225, #226 and #228 on sketch-bench main.

    CloudLab check (scratch3, node0/node1, --planner saved --expr-only, then sketch_bench_e2e_configs.py --frontend rqe):

    Run Result
    Classic e2e-rate10 table, 1h 23,760 RQE-route queries, 0 errors, no lost repetitions. Sums exact; quantiles within DDSketch α=0.1; top-k 31–32 of 32. ASAP latency p50 14 ms vs Prometheus 3.2 s.
    Random workload-01 pilot, 1h, sketchdb only 14,880 of 14,880 answered, 0 errors. Sums exact; avg-of-per-series quantiles worst 9.9%; per-group top-k 80 of 80 keys.

    Run overrides required: prometheus.scrape_interval=100ms, controller.punting=false, providers.cloudlab.*.

    Left open:

    • 2h runs (classic and random), to fill the 1h windows
    • run 2's latency compared with the PromQL frontend on the same plan
    • the runner leaves blackbox_exporter running on the coordinator
    • random-run feeder seed: 42 in these runs vs manifest feeder_seed 862
    • deferred: an agg over Inf/NaN returns 500
  4. milindsrivastava1997 commented on Oct 15, 2026

    @milindsrivastava1997
    ContributorAuthor

    Done: the RQE frontend is on main (#887–#890; sketch-bench #225, #226, #228), and the CloudLab 1h runs passed (see above). Not done here: 2h runs, a PromQL-frontend latency comparison for the random workload, and the random runs' feeder seed (42 vs the manifest's 862). The blackbox_exporter leftover is filed separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions