feat(ir): derive summary coverage as definition + selection

[zzylol]

Closes #570.

Why

A summary state's schema says what kind of state it is (KLL, k=200, by job), not what it summarizes. #560 makes SummaryMerge structurally valid, but structure cannot tell a KLL over latency from one over size, nor show that two inputs share no row. This PR derives what each summary state covers from its sub-DAG and uses it to decide whether a merge is valid. The design is in #573 (§4), based on Goldstein & Larson's view matching (SIGMOD 2001).

Before this PR (main + #560): coverage was declared by whoever built the node and never checked against the sub-DAG.

A = KLL(latency) over Filter(region = 'us', Scan t)   declared {region: eu}   ← wrong
B = KLL(latency) over Filter(region = 'us', Scan t)   declared {region: us}
SummaryMerge(A, B) → accepted; every US row counted twice

After this PR: nothing is declared. Both states derive selection {region ∈ {us}} over the same definition, so the merge fails with PossibleOverlap.

What

Coverage = definition (what the state computes) + selection (which output rows of that computation it took):

KLL(latency) by[job] over Filter(region = 'us' AND latency < 100, Scan t)
  definition: KLL(latency) by[job] over Scan t
  selection:  [{ region ∈ {us}, latency ∈ (−∞, 100) }]

KLL(value) over TimeRange(1m, range) over TimeShift(2m) over Scan m{job="api"}
  definition: KLL(value) over Scan m
  selection:  [{ job ∈ {api}, relative time (−3m, −2m] }]

• Selection: a predicate conjunct is lifted when it passes Filter, a range TimeRange, a TimeShift without @, and direct-column Project items up to the SummaryAgg (including SummaryAgg.filter and Scan.predicates), and it is a value set (=, !=, IN, NOT IN, OR of equalities) or an interval (<, <=, >, >=) on one column. Columns are identified by (table, name), so shipping.region and billing.region stay distinct. A column whose (table, name) is not unique in the agg child (two items aliased k) is not lifted, and value sets compare literals by type, so 1 and 1.0 are never proven different. One range TimeRange(w) over TimeShift(s) gives the relative window (−(s+w), −s]; PromQL ranges are left-open.
• Definition: the SummaryAgg with what was lifted removed. Everything else (arithmetic, regex, rate, instant selectors, …) stays in it as a residual, so states that differ there do not merge.
• Merge: valid only when all inputs have structurally equal definitions (ignoring timing and guarantee, so ingestion-time and query-time panes can merge) and pairwise disjoint selections. Its coverage is the shared definition and the union of the selections; adjacent windows and value sets join, gaps stay separate boxes. Merges nest.

Not in this PR (each follows when a caller needs it, per #573 §4.4): SummaryMerge { group_by } (rollup), family-specific overlap rules (HLL may overlap), hash-partition constraints, lifting through Aggregate/window/rate partition columns, and SQL timestamp intervals (the IR has no timestamp literal yet).

Key code interfaces

summary_coverage.rs, node.rs

impl OperatorNode {
    /// `Some` for a SummaryAgg and a valid SummaryMerge; derived on first use.
    pub fn coverage(&self) -> Option<&SummaryCoverage>;
}

impl SummaryCoverage {
    /// The single entry point for every summary node.
    pub fn derive(node: &OperatorNode) -> Result<Self, CoverageError>;
}

pub struct SummaryCoverage {
    pub definition: Rc<OperatorNode>,
    pub selection: Vec<SelectionBox>,
}

pub struct SelectionBox {
    pub columns: BTreeMap<ColumnIdentity, Constraint>,
    pub relative_time: Option<(Bound<i64>, Bound<i64>)>,
}

pub enum Constraint {
    In(Vec<ScalarValue>),
    NotIn(Vec<ScalarValue>),
    Interval { lower: Bound<ScalarValue>, upper: Bound<ScalarValue> },
}

pub enum CoverageError { NotSummary, EmptyMerge, DefinitionMismatch, PossibleOverlap }

• OperatorNode::new and validate_structure reject an invalid SummaryMerge.
• The node's coverage field is now a private cache: ignored by equality, skipped by serde, emptied on clone.
• Removed: with_coverage, requires_coverage, CoverageError::Missing, and the coverage fields of CSE keys and FlatNode.

Tests

crates/types/tests/summary_coverage.rs covers each rule: filters and scan predicates lifting, population and value-range merges, overlap rejection, different inputs and different scan schemas, qualified columns, time panes (join, gap, overlap), residuals, instant selectors, nested merges, timing, a forged merge caught by validate_structure, and three regressions from an independent review (ambiguous column names, literals of different types, partly lifted scan predicates), each confirmed failing before the fix. Workspace tests, cargo fmt and workspace/all-targets Clippy with warnings denied pass.

---

Base: #560 · Next: #539 · Tracker: #528 · Design: #573

🤖 Generated with Claude Code

[Selvomega]

The naming of time_ms seems not good. I assume you want conceptually a "range" here?

[zzylol]

yes, I will rename to "time_range"

[zzylol]

Done in fea4cc0: CoverageRegion.time_ms is now time_range, and so is the parameter of derive/with_time_range.

[Selvomega]

Bug here: If derived_population() returns None, it will be converted to the default legal value of population.
The problem is in derived_population() semantic, None means I cannot handle this population so it should not be used. While this treacherous unwrap_or_default() silently convert this into a legal value!

Spotting this bug actually makes me feel better: At least it proves reading code is still somehow useful.

[zzylol]

Fixed in fea4cc0. An unreadable population is now an error, UnprovenPopulation, never the empty (unrestricted) map. Such a SummaryAgg is still valid but has no coverage, so it cannot be merged, and a merge with it reports UnprovenPopulation. The same-input exception for unreadable populations is dropped.

[Selvomega]

Same. Check if there is bug here

[zzylol]

Same fix: check_against is removed. validate_structure compares the retained coverage with SummaryCoverage::derive, which fails with UnprovenPopulation instead of defaulting.

[Selvomega]

Seemingly a bug. Check other comments related

[zzylol]

Yes, same bug, fixed in fea4cc0. The rebuild (now with_new_children, #648) calls with_time_range with the old time range, i.e. SummaryCoverage::derive, and returns its error when the population can no longer be read, instead of defaulting it.

[Selvomega]

This function is also treacherous and shady. It is called for_summary but practically only works for SummaryAgg nodes. Some problems in PR #560 is related to this.

[zzylol]

Fixed in fea4cc0. for_summary is gone, and so is of_merge from #560. Coverage is now computed by one function for every summary node:

pub fn derive(node: &OperatorNode, time_range: Option<Range<i64>>) -> Result<SummaryCoverage, CoverageError>

SummaryAgg → source, population and columns read from its subtree over time_range; SummaryMerge → the disjoint union of its inputs (time_range must be None); anything else → NotSummary. OperatorNode::new, with_time_range (replaces with_coverage), validate_structure and with_new_children all call it; check_against, same_input and the public scanned_source/derived_population are removed.