Skip to content

fix: use rounding for float-to-integer conversions - #191

Open
roderickvd wants to merge 1 commit into
RustAudio:masterfrom
roderickvd:fix/round-float-to-int
Open

fix: use rounding for float-to-integer conversions#191
roderickvd wants to merge 1 commit into
RustAudio:masterfrom
roderickvd:fix/round-float-to-int

Conversation

@roderickvd

Copy link
Copy Markdown
Member

This PR replaces truncating casts with proper rounding in float-to-integer sample conversions to eliminate systematic bias and nonlinear distortion.

Problem

The current implementation uses truncating casts (e.g. as i16), which creates two issues:

  1. Nonlinear distortion: All signal values in the interval (-1.0, 1.0) map to zero, creating an output bin twice as large as any other integer value. This violates the uniform quantization assumption and introduces harmonic distortion.

  2. Systematic bias towards zero: Small signals that should map to ±1 are instead lost to zero, introducing DC bias and reducing effective dynamic range by about 8 dB.

One publication that documents this is Dannenberg's "Danger in Floating-Point-to-Integer Conversion" letter to Computer Music Journal in 2002, which warns against truncation in audio applications.

Solution

Replace (s * scale) as {integer} with (s * scale).round() as {integer} for float-to-integer conversions.

Before (truncation):

s to_i16 { (s * 32_768.0) as i16 }  
// 0.1 * 32768 = 3276.8 → 3276 (truncated)
// 0.00002 * 32768 = 0.65536 → 0 (small signal lost)

After (rounding):

s to_i16 { (s * 32_768.0).round() as i16 }  
// 0.1 * 32768 = 3276.8 → 3277 (rounded)
// 0.00002 * 32768 = 0.65536 → 1 (small signal preserved)

The performance impact is minimal, because LLVM generates efficient code for round() intrinsics with dedicated instructions on most targets.

@roderickvd

roderickvd commented Sep 6, 2025

Copy link
Copy Markdown
Member Author

The failing build is unrelated and already present on master since cd8d893.

➡️ PR #192

@roderickvd
roderickvd force-pushed the fix/round-float-to-int branch 2 times, most recently from 0d048b3 to 78bf2c3 Compare September 9, 2025 06:13
@roderickvd

Copy link
Copy Markdown
Member Author

Rebased and passing the CI now.

This comment was marked as resolved.

Replace truncating casts with proper rounding in float-to-integer sample
conversions to eliminate bias and preserve small signals.

Changes:
- Use f32::round() and f64::round() instead of truncating `as` casts
- Eliminates bias towards zero from truncation behavior
- Preserves small audio signals that would otherwise be truncated to zero
- Removes nonlinear distortion caused by signal values in (-1.0, 1.0)
  all mapping to zero, creating an interval twice as large as any other

Inlines sqrt and round functions for performance.

Additional tests verify proper rounding behavior for cases that would
fail with truncation.
@roderickvd
roderickvd force-pushed the fix/round-float-to-int branch from 78bf2c3 to 4ce63ed Compare January 10, 2026 21:56
@hey-jj

hey-jj commented Aug 22, 2026

Copy link
Copy Markdown

Reproduced on dasp_sample 0.11.0 from crates.io, through the documented Sample::to_sample, ToSample::to_sample_ and FromSample::from_sample_ entry points. All three agree on every input. Output of the run (exact is the scaled product, round() is the expected value):

f32 -> i16 (scale 32768)
         input          exact  to_sample   ToSample    round()
    0.50000000    16384.00000      16384      16384      16384
    0.75000000    24576.00000      24576      24576      24576
    0.99996948    32767.00000      32767      32767      32767
    0.99996901    32766.98438      32766      32766      32767  <-- TRUNCATED
    0.49998999    16383.67188      16383      16383      16384  <-- TRUNCATED
    0.00002000        0.65536          0          0          1  <-- TRUNCATED
   -0.00002000       -0.65536          0          0         -1  <-- TRUNCATED
   -0.50000000   -16384.00000     -16384     -16384     -16384
   -0.75000000   -24576.00000     -24576     -24576     -24576
    0.00002747        0.90000          0          0          1  <-- TRUNCATED

f64 -> i16
    0.50000000    16384.00000      16384      16384
    0.99996900    32766.98419      32766      32767  <-- TRUNCATED
    0.00002000        0.65536          0          1  <-- TRUNCATED
   -0.00002000       -0.65536          0         -1  <-- TRUNCATED
   -0.75000000   -24576.00000     -24576     -24576
    0.99999000    32767.67232      32767      32767

f32 -> i8 (scale 128)
    0.50000000       64.00000         64         64
    0.75000000       96.00000         96         96
    0.99000001      126.72000        126        127  <-- TRUNCATED
    0.00780000        0.99840          0          1  <-- TRUNCATED
   -0.00780000       -0.99840          0         -1  <-- TRUNCATED

f64 -> i32 (scale 2147483648)
  0.5000000000   1073741824.00000   1073741824   1073741824
  0.7500000000   1610612736.00000   1610612736   1610612736
  0.9999999990   2147483645.85252   2147483645   2147483646  <-- TRUNCATED
  0.0000000004            0.85899            0            1  <-- TRUNCATED
 -0.0000000004           -0.85899            0           -1  <-- TRUNCATED

dead zone: every f32 in (-1/32768, 1/32768) -> i16 0; count distinct outputs for inputs in (-2/32768, 2/32768)
{-2: 1, -1: 1000, 0: 1999, 1: 1000, 2: 1}

The last line is a sweep of 4001 evenly spaced f32 values across (-2 LSB, +2 LSB) into i16. The zero bin holds twice as many inputs as the -1 or +1 bin. That matches the two effects the PR description names: values inside (-1 LSB, +1 LSB) collapse to zero, and everything else is biased toward zero by up to one LSB.

This path runs in published consumers. web-audio-api calls S::from_sample_ with an f32 input in src/render/thread.rs:456 for every frame rendered to an integer-format output device. lasprs calls i16::from_sample, I24::from_sample and i32::from_sample on an f64 in src/measurement/wav_export.rs:271-273 when writing WAV files. bwavfile calls sample.to_sample() into u8, i16, I24 and i32 in src/wavewriter.rs:73-93 for any S: Sample.

Could this get a review and merge, followed by a dasp_sample release? The only version on crates.io is 0.11.0 from 2020, so no published build carries the fix yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants