[fix](fe) Keep MAP key/value decimal slots independent in default deimal v3 precision promotion - #67154
[fix](fe) Keep MAP key/value decimal slots independent in default deimal v3 precision promotion#67154starocean999 wants to merge 1 commit into
Conversation
…imal v3 precision promotion
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
/review |
There was a problem hiding this comment.
The fixed-argument MAP path now preserves independent DecimalV3 leaves, and the added unit/regression expectations are deterministic and internally consistent. However, the shared promotion helper still introduces three blocking correctness failures: linked Any/Follow slots can diverge across Decimal storage widths; field can retain different representations for its fixed operand and repeated search operands; and complex vararg signatures still collapse independent nested leaves into one global type. The two inline comments cover those three root causes.
Checkpoint summary: the four-file scope is focused, but the generic helper affects many built-ins and extension functions. Parallel Follow consumers and scalar/complex vararg paths were traced through FE cast insertion and BE dispatch. NULL/empty inputs and all Decimal32/64/128/256 boundaries were checked. There are no new concurrency, lifecycle, static-initialization, configuration-definition, transaction, persistence, data-write, wire/protocol, rolling-upgrade, observability, or status-propagation surfaces; the bounded signature-tree traversal has no actionable performance concern. Test coverage is useful but omits all three blocking triggers. No build or test command was run because this runner's review instructions prohibit them. No additional reviewer focus was supplied. The three-round review converged with no unresolved or undeduplicated candidates.
| List<DataType> newArgTypes = Lists.newArrayListWithCapacity(signature.argumentsTypes.size()); | ||
| for (int i = 0; i < signature.argumentsTypes.size(); i++) { | ||
| DataType sigType = signature.argumentsTypes.get(i); | ||
| if (signature.hasVarArgs && i == signature.argumentsTypes.size() - 1) { |
There was a problem hiding this comment.
[P1] Model logical Decimal groups inside vararg signatures
This blanket last-slot replacement breaks both relationships that the new promotion needs to preserve:
Fielddeclares.varArgs(DecimalV3Type.WILDCARD, DecimalV3Type.WILDCARD), so its fixed first operand and repeated tail are one comparison type. With a nonconstantDECIMAL(3,2)first operand equal to aDECIMAL(4,3)search operand, they instead remain split; BE dispatches from argument 0 and compares raw values 120/1200, returning 0 (and cross-width cases hit an incompatible column cast).- Variadic user functions can declare a repeated MAP type and use this default precision chain. For an exactly typed
MAP<DECIMAL(76,0), DECIMAL(76,76)>argument, the scan computesDECIMAL(76,6)globally and this line rewrites both independent leaves to it before execution, overflowing the key and truncating the value.
Please preserve common scalar families such as field while aggregating each corresponding nested leaf independently across repeated complex arguments, with regressions for both shapes.
| // keep the previous behavior for the vararg slot | ||
| newArgTypes.add(TypeCoercionUtils.replaceDecimalV3WithTarget(sigType, widerType)); | ||
| } else { | ||
| newArgTypes.add(promoteDecimalV3Slot(sigType, arguments.get(i).getDataType(), |
There was a problem hiding this comment.
[P1] Keep linked Decimal slots on one promoted type
This now promotes each declared argument separately after Any/Follow resolution has already tied logical slots together. For ElementAt's MAP<Any(0), Any(1)>, Follow(0) signature, a nonliteral MAP<DECIMAL(9,2), ...> column and an equal DECIMAL(10,3) lookup are first resolved to the map-key type, but this call leaves the key as Decimal32 while independently widening the lookup to Decimal64. expectedInputTypes() preserves that split, and BE's map lookup calls ColumnDecimal32::compare_at on the Decimal64 RHS even though it requires the same concrete column class. The NullType fallback has the same identity loss because it can replace a resolved Follow slot with widerType from an unrelated MAP leaf. Please promote by logical Any/Follow type-variable group, and cover storage-width boundaries plus NULL.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Nereids' default DECIMALV3 precision promotion
(
ComputeSignatureHelper.defaultDecimalV3PrecisionPromotion) recursively collects everyDECIMALV3 found in the argument types, computes a single global "wider" type, and then
replaces ALL decimal slots with that one type via
TypeCoercionUtils.replaceDecimalV3WithTarget.For
MAP<K, V>where K and V are independent type variables with differentprecision/scale, merging them into one wider type breaks correctness in two ways
(they share the same type-derivation path):
MAP_KEYS(and other map consumers) silently turns legal keys into NULL.Repro: a
MAP<DECIMAL(76,0), DECIMAL(76,18)>built from a 76-digit integral keyand
0.125000000000000000. Widening the key slot to scale 18 overflows the bigintegral key, so
MAP_KEYSreturns[null]instead of the key.UNNEST(MAP)(rewritten toEXPLODE_MAP) fails type analysis. The promoted MAPargument type no longer matches the original input MAP type.
Fix: promote each decimal slot independently instead of merging them. Each slot
(MAP key/value, ARRAY item, scalar) keeps its own precision/scale derived from its
own argument type; the wider type is only used for NULL-typed slots and the
placeholder return type. Vararg slots keep the previous behavior to avoid changing
vararg decimal functions (e.g.
field).None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)