Skip to content

fix(stdlib): renormalize %e/%g mantissa carry and handle precision 0 - #1336

Open
jdymitarai wants to merge 1 commit into
google:masterfrom
jdymitarai:fix-format-e-g-rounding
Open

fix(stdlib): renormalize %e/%g mantissa carry and handle precision 0#1336
jdymitarai wants to merge 1 commit into
google:masterfrom
jdymitarai:fix-format-e-g-rounding

Conversation

@jdymitarai

Copy link
Copy Markdown

Description

This PR fixes two related formatting bugs in stdlib/std.jsonnet for %e and %g conversions (reported in #1327):

  1. Mantissa Carry Renormalization:
    When formatting floating-point numbers in scientific form (%e / %E / %g / %G), rounding to prec decimal places can round the mantissa up to 10.0 or higher (for example: "%.1e" % 9.99 yielded "10.0e+00" instead of "1.0e+01", and "%.0e" % 9.5e10 yielded "10e+10" instead of "1e+11").
    When mantissa * 10^prec + 0.5 >= 10 * 10^prec, we now detect the carry, increment the exponent by 1, and divide the mantissa by 10 to maintain standard normalized scientific representation (1 <= mantissa < 10).

  2. %g Precision of 0 & Boundary Transition:
    Per Python format specifications (which Jsonnet follows), a precision of 0 for %g is treated as equivalent to a precision of 1. Previously, precision 0 resulted in passing negative precision to render_float_dec, leading to corrupted output such as "%.0g" % 1.0 -> "5e+00".
    In addition, when rounding produces an exponent carry (e.g. "%.1g" % 9.9), the effective exponent is updated to 1, correctly switching to scientific notation "1e+01" instead of decimal overflow.

Fixes #1327.

Verification

  • Added automated regression tests to test_suite/format.jsonnet for %e, %E, %g, and %G testing mantissa carries, precision 0, and boundary exponent transitions.
  • Verified against Python standard library formatting across comprehensive test cases (test_suite/format.jsonnet returns true).
  • C++ test suite passes without regressions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

std.format %e/%g do not follow documented Python rounding semantics

1 participant