Repository navigation
Tracking: RQE frontend for the paper eval #886
Copy link
Copy link
Closed
Labels
Description
Goal
For the paper eval, run ASAPQuery's side of an experiment without PromQL/SQL parsing: a query is an RQE expression tree (self-contained RQE leaves, combined by aggregations and binary ops), POSTed to a new engine route that looks up each leaf's sketch directly by RQE id. Baselines (Prometheus, ClickHouse, …) keep PromQL/SQL. Covers sketch-bench eval tables and the random workloads, PromQL- and SQL-style queries.
sketch-bench side: ProjectASAP/sketch-bench#224
Steps (one PR each, stacked PRs, worktrees via gwt.sh)
| # | Issue | What | Stacked on |
|---|---|---|---|
| A1 | #880 | tree type + validation (asap_types) |
main |
| S1 | sketch-bench#222 | expr in eval-table e2e queries; regenerate tables |
main |
| A2 | #881 | expr + feeder-format data_facts in random workloads |
main |
| S2 | sketch-bench#223 | eval table from a random manifest | needs A2 |
| A3 | #882 | --planner saved reads trees, writes rqe_configs |
A1 |
| A4 | #883 | engine rqe.rs + POST /api/v1/rqe_query |
A3 |
| A5 | #884 | rqe client protocol, side file, query_engine.frontend switch, e2e check |
A4 |
- A1 feat(asap-common): RQE expression tree type and validation #880 (PR feat(asap-common): RQE expression tree type and validation #887)
- S1 sketch-bench#222 (PR sketch-bench#225)
- A2 feat(asap-tools): RQE trees and feeder-format data facts in random workloads #881
- S2 sketch-bench#223 (covered by sketch-bench#226 + feat(asap-tools): add accelerated random E2E workload artifacts #879; trees: sketch-bench#227, PR sketch-bench#228)
- A3 feat(planner): --planner saved reads RQE trees and writes rqe_configs #882 (PR feat(planner): --planner saved reads RQE trees and writes rqe_configs #888)
- A4 feat(query-engine): RQE tree query route #883 (PR feat(query-engine): answer RQE expression trees on POST /api/v1/rqe_query #889)
- A5 feat(asap-tools): rqe query protocol and frontend switch for e2e runs #884 (PR feat(asap-tools): rqe query protocol and frontend switch for e2e runs #890, code; e2e check pending)
Decisions
- Use: paper eval only; the experiment scripts call the RQE route for ASAPQuery. The PromQL/SQL paths are unchanged.
- Tree: leaf
rqeis self-contained (id,metric/table,capability,lookback_ms,grouping,phi,k,value).agg= avg/min/max/sum withbyonly,by⊆ child labels.binary=+ - * /(scalar operands allowed), equal label sets. Top-k only at the root. Leaf capabilities: sum, quantile, cardinality, topk_by_value (rate and topk_by_count out of scope; queries using them get noexpr). - Where the tree comes from: written by the generators next to
promql/sql(expr), for existing query shapes only; new templates (e.g.avg by (G)(quantile_over_time), binary in random workloads) are a follow-up. - Sketch lookup: direct by RQE id via a new
rqe_configssection ofinference_config.yaml, written by--planner saved(roll-up deployments included);query_configskept. - Random workloads: planned like eval tables: sketch-bench converts a manifest to a table (S2), solved with
--target p95(anyaccuracy_sla≠ 0.95 fails), converted by--planner saved.data_factsuse the paced feeder'smetricsformat so the feeder and the planner agree:dataPareto α 2.0, Zipf θ 1.1 over 10,000 series, 10 samples/s;eventsuniform labels,user_idZipf θ 1.1 over 10,000 keys, 10 records/s per leaf. - Route:
POST /api/v1/rqe_query {expr, time}→ Prometheus vector JSON with exactly Prometheus' output labels. No fallback. No data → empty result; unknown RQE id / invalid tree → 400. - Client:
rqeprotocol; configs keep PromQL/SQL text, a side file maps it toexpr, so results andcalculate_fidelityare unchanged.query_engine.frontend: promql|rqe(default promql). - Out of scope: SQL accuracy scoring (asap-tools: calculate_fidelity never scores SQL queries #885), filtered RQEs, Prometheus range queries, new templates.
Done when
One classic sketch-bench table and one random workload run end to end through the RQE route with the paced feeder, matching the PromQL route for the same plans wherever the PromQL route works.
Reactions are currently unavailable
Activity
eval: RQE expression trees for random-manifest tables
milindsrivastava1997 commentedon Oct 15, 2026
Decision (2026-10-14): random-workload tables (sketch-bench#228) need #879's per-series-avg planning (af8888a) in the planner. We wait for #879 to merge, then merge main up the stack (#887 → #888 → #889 → #890). No cherry-pick.
milindsrivastava1997 commentedon Oct 15, 2026
Known and deferred (review of #889): an
aggover a child with Inf/NaN (e.g.avg(a / b)withb == 0) returns 500, where Prometheus propagates the value. No current workload aggregates over a division.eval: RQE expression trees for random-manifest tables (#228)
eval: export ASAPQuery random workload manifests (#226)
milindsrivastava1997 commentedon Oct 15, 2026
All steps are merged: #887, #888, #889 and #890 on ASAPQuery main (dd574c7 … 4db156f); sketch-bench#225, #226 and #228 on sketch-bench main.
CloudLab check (scratch3, node0/node1,
--planner saved --expr-only, thensketch_bench_e2e_configs.py --frontend rqe):Run overrides required:
prometheus.scrape_interval=100ms,controller.punting=false,providers.cloudlab.*.Left open:
blackbox_exporterrunning on the coordinatorfeeder_seed862aggover Inf/NaN returns 500milindsrivastava1997 commentedon Oct 15, 2026
Done: the RQE frontend is on main (#887–#890; sketch-bench #225, #226, #228), and the CloudLab 1h runs passed (see above). Not done here: 2h runs, a PromQL-frontend latency comparison for the random workload, and the random runs' feeder seed (42 vs the manifest's 862). The blackbox_exporter leftover is filed separately.