Add bfloat16 and float8e4m3 to the spec - #956
Open
mklimenko-nv wants to merge 1 commit into
Open
mklimenko-nv wants to merge 1 commit into
mklimenko-nv wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds WebNN spec wording to support
bfloat16andfloat8e4m3, according to #938Detailed changes:
MLOperandDataType: new values"bfloat16"and"float8e4m3", plus a definition list describing every value.float8e4m3is the OCP OFP8 E4M3 encoding (1/4/3 bits, bias 7, no infinities, max 卤448, NaN =S.1111.111), mirroring the PyTorchfloat8_e4m3fnand ONNXFLOAT8E4M3FN. Neither new type is a required data type for any operation, support is discoverable throughopSupportLimits().bfloat16is added to the allowed data types whereverfloat16is allowed. The required columns are untouched.float8e4m3is allowed ondequantizeLinear(input,zeroPoint),quantizeLinear(zeroPoint, output) andcast, and is implicitly allowed on operands whose allowed types areany. It is intentionally not listed on compute operators:fp8compute is expressed through the Q/DQ pattern and realized by backend fusion, which a Note next to the enum spells out.cast(): the behavior table gainsbfloat16and afloat8e4m3target column. Conversions tofloat8e4m3saturate (out-of-range and infinite values become 卤448, NaN is preserved), matching OFP8's saturating mode and the ONNXCastdefault. NaN handling is now stated in every float-source cell.ConvertToFloat(used forMLNumbercasting) takes the data type instead of a bit length and handles the two new formats, RNE and saturation match OFP8 1.1 Appendix A.quantizeLinear/dequantizeLinear: states the type promotion (arithmetic happens in the scale/input float type) and the float8 formulacast(input / scale + cast(zeroPoint, input.dataType), zeroPoint.dataType). ThequantizeLinearemulation branches on integer vs. floating point output and clamps to the output type's range instead of a hardcoded 0...255. A note points out thatfp8quantization is typically symmetric (zeroPoint= zeros).bfloat16<->Uint16Array,float8e4m3<->Uint8Array(native-endian bit patterns;Uint8Arrayremains the generic view for any type).validate buffer with descriptorneeded no change since it is table-driven. The stale "Float16Array is Stage 3" note is refreshed.Also fixed while here: the isNaN/isInfinite tensor-limits table said
anywhile the method steps restricted to floating point types; the table now matches the steps.Preview | Diff