Skip to content

DiffSinger: match word duration rounding and tail SP grouping to training data - #2565

Open
KakaruHayate wants to merge 1 commit into
openutau:masterfrom
KakaruHayate:fix/ds-word-div-duration
Open

KakaruHayate wants to merge 1 commit into
openutau:masterfrom
KakaruHayate:fix/ds-word-div-duration

Conversation

@KakaruHayate

Copy link
Copy Markdown
Contributor

What

Two small consistency fixes for the word-level duration/word-division inputs used by DiffSinger variance/pitch models:

  1. word_dur uses cumulative rounding. DiffSingerBasePhonemizer used loor(end_ms / frameMs) - floor(start_ms / frameMs) for each word interval. Training builds ph_dur with
    ound(cumsum(ms) / frame + 0.5) and derives the word budget from that, so the old per-endpoint truncation was off by one frame at boundaries (e.g. 100 ms notes at 11.609977 ms/frame: 8/9 vs 9/9). It now uses DiffSingerUtils.DurationsMsToFrames over the intervals.
  2. Tail SP is its own word. PaddedWordDivAndDur merged the tail SP into the last vowel group (Append(segments.Count - vowelIds[^1])), while the .ds format and DiffSingerScript.ph_num keep it separate (SP sh ir n i SP → 2 2 1 1, not 2 2 2). Runtime pitch/variance therefore had one word fewer than training. The tail segment is now appended as a separate word.

Impact

  • dur_type == attn / graphs consuming word_div + word_dur: word_dur now matches the training budget.
  • predict_dur == true pitch/variance models: word_div / word_dur now have the same word count as .ds training data.
  • �bs duration models and models without word inputs are unaffected.

Verification

  • dotnet build OpenUtau.Test: 0 errors.
  • DiffSingerWordDivTest (new): 3 passed.
    • [SP, sh, ir, n, i, SP] → wordDiv [2,2,1,1], wordDur [12,9,4,8].
    • no-vowel fallback.
    • cumulative rounding: 100/100 ms at 512/44100 → [9,9].
  • Existing OpenUtau.Core.DiffSinger tests: 20 passed, 0 failed.

No changes to the .ds variance clamp (will be a follow-up PR), and this branch is based on upstream master, not on the #2388 revert PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants