Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions crates/devtools/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ asap-logical-optimizer = { path = "../logical-optimizer" }
# plans with. A devtool, not a stage crate, so the #572 guards allow it.
asap-executor = { path = "../executor" }

# Used by the show_*_ir / stage_pipeline / variant_coverage bins (catalog schemas,
# async SQL path, JSON output) and by the topk_ir / canonical_examples
# examples. Regular deps, not dev-deps: `[[bin]]` targets can't see dev-deps.
# Used by the show_logical_dag / stage_pipeline / variant_coverage bins
# (catalog schemas, async SQL path, JSON output) and by the topk_ir /
# canonical_examples examples. Regular deps, not dev-deps: `[[bin]]` targets can't see dev-deps.
asap-types = { path = "../types" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
// cargo run -p asap-devtools --bin show_pre_asap_ir -- queries.txt
// (or pipe via stdin: cargo run -p asap-devtools --bin show_pre_asap_ir < queries.txt)
// cargo run -p asap-devtools --bin show_logical_dag -- queries.txt
// (or pipe via stdin: cargo run -p asap-devtools --bin show_logical_dag < queries.txt)
//
// Lowers a batch of ad-hoc SQL/PromQL queries to **pre-ASAP IR** (the
// sketch-agnostic intent algebra: an `OperatorNode` DAG of `NonASAPOp`
// operators with `AggIntent` measures) and prints them.
// See `show_post_asap_ir` for the post-ASAP sketch-bound IR one layer
// downstream — this tool never picks a sketch, it only shows what a query
// means.
// operators with `AggIntent` measures) and prints them. This tool never
// picks a sketch, it only shows what a query means; `stage_pipeline` shows
// how the planner plans it.
//
// File format: one query per line, prefixed with "sql>" or "promql>".
// Blank lines and lines starting with '#' are ignored.
Expand Down Expand Up @@ -49,7 +48,7 @@ async fn main() {
assert_eq!(
args.next().as_deref(),
Some("--data-ingestion-interval-ms"),
"usage: show_pre_asap_ir --data-ingestion-interval-ms <ms> [queries.txt]"
"usage: show_logical_dag --data-ingestion-interval-ms <ms> [queries.txt]"
);
let interval_ms = args
.next()
Expand Down
205 changes: 0 additions & 205 deletions crates/devtools/src/bin/show_post_asap_ir.rs

This file was deleted.

2 changes: 1 addition & 1 deletion docs/develop_docs/library-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ async lower_sql_dialect(query: &str, catalog: &SqlCatalog,
| `ElasticSQL` | Returns `UnsupportedDialect` |

The catalog is required and describes your tables. For a complete schema-building
example, see [the CLI frontend example](../../crates/devtools/src/bin/show_pre_asap_ir.rs).
example, see [the CLI frontend example](../../crates/devtools/src/bin/show_logical_dag.rs).

## Generate and rank candidates

Expand Down
53 changes: 7 additions & 46 deletions docs/user_guide_docs/run-a-query.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ tool on first use.

| Command (`cargo run -p asap-devtools --bin … -- …`) | Input / options | Result |
| --- | --- | --- |
| `show_pre_asap_ir --data-ingestion-interval-ms 1000 queries.txt` | File path, or stdin when omitted | Prints canonical Pre-ASAP IR |
| `show_post_asap_ir --data-ingestion-interval-ms 1000 queries.txt` | Same query file format | Prints all sketch-strategy Post-ASAP candidates using a fixed approximate target, in cost-model order |
| `show_logical_dag --data-ingestion-interval-ms 1000 queries.txt` | File path, or stdin when omitted | Prints canonical Pre-ASAP IR |
| `analyze_corpora --corpora --data-ingestion-interval-ms 1000 --out-dir <dir>` | Repository PromQL corpora, output directory | Writes successful/error IR dumps and summary reports |
| `analyze_corpora --sql-corpora --out-dir <dir>` | Repository SQL corpora, output directory | Writes SQL corpus reports |
| `variant_coverage --data-ingestion-interval-ms 1000` | Repository corpora | Reports Pre-ASAP IR variant coverage |
Expand All @@ -38,8 +37,8 @@ promql> quantile(0.99, rate(http_requests_total[5m]))
sql> SELECT service, COUNT(*) FROM metrics GROUP BY service
```

Blank lines and lines beginning with `#` are ignored. The two file/stdin tools
accept `sql>` and `promql>`; MetricsQL is available through the library frontend.
Blank lines and lines beginning with `#` are ignored. The file/stdin tool
accepts `sql>` and `promql>`; MetricsQL is available through the library frontend.
SQL examples use the fixed catalog
`metrics(ts: Timestamp, service: Utf8, region: Utf8, latency: Float64, bytes: Int64)`.
For your own schema, provide a `SqlCatalog` through the library API.
Expand All @@ -49,13 +48,13 @@ For your own schema, provide a `SqlCatalog` through the library API.
Run:

```sh
cargo run -p asap-devtools --bin show_pre_asap_ir -- --data-ingestion-interval-ms 1000 queries.txt
cargo run -p asap-devtools --bin show_logical_dag -- --data-ingestion-interval-ms 1000 queries.txt
```

You can also provide the queries through stdin:

```sh
cargo run -p asap-devtools --bin show_pre_asap_ir -- --data-ingestion-interval-ms 1000 < queries.txt
cargo run -p asap-devtools --bin show_logical_dag -- --data-ingestion-interval-ms 1000 < queries.txt
```

To dump and compare every PromQL corpus, run:
Expand All @@ -76,44 +75,6 @@ The corresponding SQL corpus analysis is:
cargo run -p asap-devtools --bin analyze_corpora -- --sql-corpora --out-dir artifacts/sql_pre_asap
```

### Inspect Post-ASAP IR candidates

Run:

```sh
cargo run -p asap-devtools --bin show_post_asap_ir -- --data-ingestion-interval-ms 1000 queries.txt
```

Or through stdin:

```sh
cargo run -p asap-devtools --bin show_post_asap_ir -- --data-ingestion-interval-ms 1000 < queries.txt
```

`show_post_asap_ir` uses an approximation target of ε = 0.01 and prints every
available binding from the sketch strategy for each query, numbered in cost-model
order. If no candidate is available, it prints the pre-ASAP fallback as candidate
1. It does not show the complete ranked workload candidate set or choose a
deployment. Its SQL examples use a fixed demonstration catalog, not
your database schema. Use the
[library workflow](../develop_docs/library-api.md) to retain workload alternatives
and provide your own models.

The default strategy generates DDSketch quantile-ratio candidates even when no
input-domain evidence is available. Such candidates have `guarantee: None`:
they do not claim a certified end-to-end accuracy bound. They also remain
visible in a target-aware `CandidateLogicalASAPDAGs` so the downstream backend can decide
whether to select them using its own evidence. Planner's automatic
`global_selection` skips them; their presence alone does not show that they
meet the requested target.

Each input line is followed by its debug IR or an `ERR:` message. Pre-ASAP and
Post-ASAP output use the same node format: Post-ASAP output adds summary nodes
(state, readouts) and keeps the original exact operators wherever no summary
replaces them. An
approximate target permits approximation; it does not guarantee a legal or
certified sketch. The tool prints plans, not query results.

## More inspection commands

### See how the planner plans a workload
Expand Down Expand Up @@ -166,5 +127,5 @@ cargo run -p asap-devtools --example canonical_examples

PromQL commands require `--data-ingestion-interval-ms` with the nonzero source
sample cadence in milliseconds. The examples use a one-second cadence; supply
the interval for your data. The mixed-input `show_pre_asap_ir` and
`show_post_asap_ir` tools require this option even for SQL-only input files.
the interval for your data. The mixed-input `show_logical_dag` tool requires
this option even for SQL-only input files.