Repository navigation
feat(planner): emit streaming and inference configs from the MILP plan - #803
Merged
Merged
Conversation
plan_to_planner_output maps each planned deployment's sketch-bench variant to an ASAPQuery aggregation and writes the planner's streaming and inference YAML: exact-sum (sum/count) -> MultipleSum, exact-min/max -> MultipleMinMax, exact-increase -> MultipleIncrease, kll-percall -> DatasketchesKLL, hll -> HLL, cms-heap -> CountMinSketchWithHeap with heapsize = the largest k it serves. Any other variant (e.g. hydra-kll), avg queries, and a query split across deployments are errors. Cleanup is NoCleanup. asap-optimizer-cli --milp gains --output-dir and --allow-undeployable-families (prints the plan, writes no configs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- asap-optimizer-cli --output-dir rejects avg queries before the MILP solve; print-only runs still plan them. - Both YAML strings are serialized before either file is written. - Each item is visited once per deployment, not once per occurrence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements A3 (plan → StreamingConfig / InferenceConfig) of #790 (session 3a). It builds on #801, which has the S8 plan output and the capability split.
What
optimizer/milp_output.rs:plan_to_planner_output(config, workload, solution) -> PlannerOutputproduces streaming and inference YAML inasap-planner's format.rollup_labelsis left empty (the legacy planner fills it; the engine doesn't read it). It reuses the PromQL generator'sbuild_streaming_yaml/build_inference_yaml.asap-optimizer-cli --milp:--output-dir <dir>writesstreaming_config.yamlandinference_config.yaml.--allow-undeployable-familiespassestruetobuild_all_candidates. It prints the plan, writes no configs, and can't be combined with--output-dir.solve_milptakesallow_undeployable_families.Mapping
exact-sum(Sum / Count)sum/countexact-min/exact-maxmin/maxexact-increasekll-percallK=khllprecision=lg_kcms-heap-topk-fastpath-vector2d(TopKByValue / TopKByCount)sum/countdepth=rows,width=cols,heapsize= largest k among served queriesset_subpopulation_labels, the same split the legacy planner uses.window == slide, otherwise sliding.Design decisions
hydra-kll(not deployed: its DeltaSet key tracker has no ASAPQuery pairing). No DeltaSet pairing is built.--output-dirrejects them before the solve; print-only runs still plan them. The avg rewrite plans sum and count, but the user's avg query would get no query_config, and the engine has no Avg (Support Avg (and other multi-statistic aggregations) in query engine #463).NoCleanup, the same as the greedy translator. Under NoCleanup the YAML carries no retention, soretained_instance_countandmerged_instance_countaren't emitted yet and retention is unbounded. Moving to ReadBased is tracked in planner: MILP plan → InferenceConfig should use ReadBased cleanup instead of NoCleanup #800.ddas deployable; the follow-up, including value-range facts and the accuracy metric, is planner: deploy DDSketch from MILP plans #802.Known risk
Count is served by
MultipleSumwith sub-typecount. That works because each query_config names its aggregation id. The engine's fallback matching (compatible_agg_types(Count)) doesn't list MultipleSum, so a Count query without a query_config wouldn't find it. No end-to-end engine run yet.Interface for 4a (A4+A5)
plan_to_planner_outputis the plan → configs entry point. ItsPlannerOutputhasto_streaming_yaml_string/to_inference_yaml_string.promql::generator::{build_streaming_yaml, build_inference_yaml}are nowpub(crate).Test plan
milp_output.rs:InferenceConfig/StreamingConfigas the engine parses it, checking type, sub_type, params, labels and no retention for sum, count, quantile, and value- and count-ranked topk.cargo test -p asap_planner, clippy with-D warnings, fmt; pre-commit ran workspace check, clippy and test.sketch-bench/out_10_6_26_1342/rqe_atomic_costs.json(scrape 15s; sum, count, rate, quantile, both topk kinds, max[1h]) wrote 7 aggregations as in the table.--allow-undeployable-familiesprints the plan and is rejected with--output-dir.🤖 Generated with Claude Code