Skip to content

fix(still, sheet): analyse the audio track before exporting a frame - #422

Merged
LeadcodeDev merged 1 commit into
mainfrom
fix/audio-reactive-on-synthesised-score
Sep 29, 2026
Merged

LeadcodeDev merged 1 commit into
mainfrom
fix/audio-reactive-on-synthesised-score

Conversation

@LeadcodeDev

Copy link
Copy Markdown
Owner

Closes #349.

The diagnosis in the issue is wrong, the symptom is real

The issue reads the flat bars as the synthesised score (#331) being incompatible
with the audio-reactive components, since that is the combination it was found on.
It rules out two causes and lands on "check source_fingerprint and
decode_audio_file against the synth's own WAV format".

None of that is it. Two measurements:

result
full render of the issue's scenario (synthesised score) bars move — 2011 to 6655 lit pixels across four instants
still of the same scene with an explicit WAV src flat, identical at every instant

The score was never the variable. analyze_scenario_audio runs from the two
encoders and the studio, and from nowhere else — still and sheet went
straight to painting, so every audio-reactive component in an exported frame sat
at its min.

The fix

still and sheet shared a three-step preamble with the encoders, and it had
drifted: load the custom fonts, prefetch the icons, and now analyse the audio.
preload_scenario_assets is that preamble in one place, so the next thing a frame
needs before it can be painted has one place to be added rather than three.

On the issue's own repro

120 BPM, a kick on every beat, audio_spectrum with 48 bars:

lit pixels topmost
before, every instant 1472 508 (the min_height floor)
after, between kicks (t=0.30) 5172 479
after, on a kick (t=0.05) 10788 420
What the still looks like now

A real low-frequency spectrum of the kick, where it used to be a flat dotted line.

Test

the_shared_preload_analyses_the_track_the_way_an_encode_does — the cache is
cleared, the preamble runs, the track must be in it. Removing the analysis call
turns it red with the message naming the symptom.

Gate

cargo fmt --all --check clean · cargo clippy --workspace --all-targets --features rustmotion/studio -D warnings clean · cargo test --workspace 1812 passed.

analyze_scenario_audio ran from the two encoders and from the studio,
and from nowhere else. still and sheet went straight to painting, so
every audio-reactive component in an exported frame sat at its min.
waveform and audio_spectrum rendered a flat line, with no diagnostic.

The issue read this as the synthesised score (#331) being incompatible
with the audio-reactive components, since that is the combination it was
found on. It is not: a still of the same scene with an explicit WAV src
is flat in exactly the same way, and a full render of the synthesised
score has always animated correctly. The score was never the variable --
the export path was.

still and sheet shared a three-step preamble with the encoders, and it
had drifted: load the custom fonts, prefetch the icons, and now analyse
the audio. preload_scenario_assets is that preamble, so the next thing a
frame needs before it can be painted has one place to be added rather
than three.

Verified on the issue's own repro, a 120 BPM kick on every beat: the
bars now move between a beat (5172 lit pixels, top at y=479) and the
kick (10788, top at y=420), where all three sampled instants were
previously identical.
@LeadcodeDev LeadcodeDev added the bug Something isn't working label Sep 29, 2026
@LeadcodeDev LeadcodeDev self-assigned this Sep 29, 2026
@LeadcodeDev
LeadcodeDev merged commit 1fd3960 into main Sep 29, 2026
4 checks passed
@LeadcodeDev
LeadcodeDev deleted the fix/audio-reactive-on-synthesised-score branch September 29, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

audio_spectrum and waveform stay flat on a synthesised score

1 participant