Repository navigation
fix(still, sheet): analyse the audio track before exporting a frame - #422
Merged
Merged
Conversation
analyze_scenario_audio ran from the two encoders and from the studio, and from nowhere else. still and sheet went straight to painting, so every audio-reactive component in an exported frame sat at its min. waveform and audio_spectrum rendered a flat line, with no diagnostic. The issue read this as the synthesised score (#331) being incompatible with the audio-reactive components, since that is the combination it was found on. It is not: a still of the same scene with an explicit WAV src is flat in exactly the same way, and a full render of the synthesised score has always animated correctly. The score was never the variable -- the export path was. still and sheet shared a three-step preamble with the encoders, and it had drifted: load the custom fonts, prefetch the icons, and now analyse the audio. preload_scenario_assets is that preamble, so the next thing a frame needs before it can be painted has one place to be added rather than three. Verified on the issue's own repro, a 120 BPM kick on every beat: the bars now move between a beat (5172 lit pixels, top at y=479) and the kick (10788, top at y=420), where all three sampled instants were previously identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #349.
The diagnosis in the issue is wrong, the symptom is real
The issue reads the flat bars as the synthesised score (#331) being incompatible
with the audio-reactive components, since that is the combination it was found on.
It rules out two causes and lands on "check
source_fingerprintanddecode_audio_fileagainst the synth's own WAV format".None of that is it. Two measurements:
srcThe score was never the variable.
analyze_scenario_audioruns from the twoencoders and the studio, and from nowhere else —
stillandsheetwentstraight to painting, so every audio-reactive component in an exported frame sat
at its
min.The fix
stillandsheetshared a three-step preamble with the encoders, and it haddrifted: load the custom fonts, prefetch the icons, and now analyse the audio.
preload_scenario_assetsis that preamble in one place, so the next thing a frameneeds before it can be painted has one place to be added rather than three.
On the issue's own repro
120 BPM, a kick on every beat,
audio_spectrumwith 48 bars:min_heightfloor)What the still looks like now
A real low-frequency spectrum of the kick, where it used to be a flat dotted line.
Test
the_shared_preload_analyses_the_track_the_way_an_encode_does— the cache iscleared, the preamble runs, the track must be in it. Removing the analysis call
turns it red with the message naming the symptom.
Gate
cargo fmt --all --checkclean ·cargo clippy --workspace --all-targets --features rustmotion/studio -D warningsclean ·cargo test --workspace1812 passed.