feat(planner): read MILP accuracy from sketch-bench's saturation curves; heap from the plan

[zzylol]

Depends on ProjectASAP/sketch-bench #178, #179, #180, #182 and #185, all merged. Rebuilt on main after #798, #801, #803, #806 and #807.

Why

• Accuracy came from the wrong source. The MILP took accuracy from the cost table (table_accuracy). Since sketch-bench Moved some files from asa-query-engine to asap-common #171/[Discussion] Explore better sketch storage format #178, sketch accuracy comes from the saturation study's curves, and a cost-table row is only the point it was measured at.
• The heap was too small. sketch-bench #182 prices a top-k deployment's heap at m · k for the m windows a query merges. The emitted CMS-with-heap config still used the query's literal k, smaller than what the plan priced and what a merged answer needs.

What

• Pin. rqe-optimizer moves to sketch-bench main (a2e0455).
• Accuracy.
  • asap-optimizer-cli takes a required --saturation-dir, the study directory with out_grid_1e7_cost/ and out_1e9/, and passes SaturationCurves::accuracy to solve_milp.
  • solve_milp now takes the accuracy function as an argument. The tests pass table_accuracy.
  • The cost table still comes from --atomic-costs.
• Workload facts. Each group may carry shape: {zipf_s, distinct_keys, tail_index}, fitted by the caller as in sketch-bench's small_problem. It maps to MetricFacts.data_shape.
  • It is validated like value_range: all values finite, zipf_s >= 0, distinct_keys >= 1, tail_index > 0. The error names the metric and labels.
  • A RAQE that a sketch family with cost rows could serve (a candidate under --allow-undeployable-families), on a grouping without a shape, fails before solving with MissingShape, naming the RAQE, metric and grouping. Without a shape, sketches get no accuracy. Families without rows are left to the normal Unservable path.
• Curve loading. A failure to load the curves names the --saturation-dir path.
• Top-k k. Each top-k query's literal k now flows into Raqe.topk_k, using sketch-bench Benchmark the precompute engine accuracy of accumulators #185's per-query k. The plan prices the heap at m · k and scales query cost by the answered k. A top-k query without a literal k fails with TopkWithoutK.
• Heap. heapsize is the plan's own heap: the deployment's heap param, which is m · k, or the smallest measured heap that already holds it. A heap family without one is a MissingParam error.
  • This ships with the bump because the plan now prices that heap, so the emitted config has to build it.
  • The engine reads each query's k from the query itself, so the config carries no separate k.
• Cost rows. accuracy_metric and measured_at are now required, and merge_accuracy is gone. Fixtures follow. The top-k fixtures carry rows at heap 32 and 2048, as the cost table does.

Verification

• cargo test -p asap_planner passes: 126 unit tests plus the integration tests.
• New tests:
  • topk_plans_with_its_own_k_and_heap_m_times_k: top-10 and top-100 plan with their own k and emit the planned heap. For k = 100 that is exactly m · k; for k = 10 it is the smallest measured heap, 32, which holds it.
  • a_shared_heap_is_the_planned_heap
  • a_heap_family_without_a_heap_is_a_missing_param
  • topk_carries_its_literal_k_and_needs_one
  • rejects_an_invalid_shape
  • a_sketch_served_grouping_without_a_shape_is_named: also covers skipping a family without rows.
  • a_group_with_a_shape_keys_the_curves
  • atomic_cost_entry_requires_accuracy_metric_and_measured_at
• cargo clippy -p asap_planner --all-targets -D warnings and cargo fmt --check are clean.

Open

• End-to-end run. asap-optimizer-cli against real curves and the regenerated cost table waits for the cost-table and curve rerun on sketch-bench main with asap_sketchlib 0.3.0 (sketch-bench Tracking issue for security monitoring/detection query demo #183, merged; running).
• No cross-check. The CLI doesn't run sketch-bench's check_cost_table against the curves.

🤖 Generated with Claude Code