Skip to content

fix(document): memoize per-language USFM book-name lookup in localize_non_usfm_book_names - #407

Merged
linearcombination merged 1 commit into
WycliffeAssociates:doc-dev.walink.orgfrom
linearcombination:memoize-book-name-lookup
Nov 17, 2026
Merged

linearcombination merged 1 commit into
WycliffeAssociates:doc-dev.walink.orgfrom
linearcombination:memoize-book-name-lookup

Conversation

@linearcombination

Copy link
Copy Markdown
Contributor

Closes #363

Intent

Use case: a user selects only the TN (translation notes) resource, for all books in both OT and NT, for one language, requesting DOCX output.

Approved follow-up: implement the fix for the investigation's dominant finding. When generating a DOCX for this request shape, the code localizes each of the 66 requested TN books' name headers by calling document_generator.localize_non_usfm_book_names -> resource_lookup.book_codes_for_lang_from_usfm_only(lang_code) once per book. That function's result depends only on lang_code, which is identical across all 66 calls, yet it is recomputed from scratch every time: each call re-reads and re-parses every USFM file for that language's Bible-text repo (for pt-br, 66 files/6.0MB), an O(book-count x usfm-book-count) cost instead of the O(usfm-book-count) the work actually needs. Fix it so this per-language result is computed once per request instead of once per book (e.g. memoize the lookup keyed on lang_code, or hoist the call out of the per-book loop in localize_non_usfm_book_names), with no change to the localized output.

What Changed

  • In backend/doc/domain/document_generator.py, localize_non_usfm_book_names now caches the result of resource_lookup.book_codes_for_lang_from_usfm_only(lang_code) in a local usfm_names_by_lang dict keyed by lang_code, computing it once per distinct language per request instead of once per book.
  • This removes the redundant re-read/re-parse of every USFM file in a language's Bible-text repo on each of the 66 per-book iterations for a full OT+NT TN request, cutting that step from O(book-count × usfm-book-count) to O(usfm-book-count), with no change to the localized replacement output.
  • tests/unit/test_document_generator.py gains coverage asserting the lookup is invoked at most once per language across multiple books, alongside existing localization-output assertions.

Risk Assessment

✅ Low: The change is a small, request-scoped memoization of a pure per-language lookup (keyed correctly by lang_code, handles falsy-but-cacheable empty results, no cross-request global state), produces no behavioral change to localized output, exactly matches the authorized fix, and is backed by two new tests that assert real call-count bounding and correct localized output rather than source-text matching.

Testing

test

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • test
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Cache per-language USFM book-name lookup in localize_non_usfm_book_names
book_codes_for_lang_from_usfm_only(lang_code) re-reads and re-parses every
USFM file for that language's Bible-text repo. It was being called once
per TN/TNC/TQ book (O(books requested x usfm books)) even though its
result depends only on lang_code, which is identical across all books in
a request. Cache it per lang_code so it's computed once per language
instead of once per book, with no change to the localized output.
@linearcombination
linearcombination merged commit e372f70 into WycliffeAssociates:doc-dev.walink.org Nov 17, 2026
24 of 25 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in DOC Project Management Nov 17, 2026
@linearcombination
linearcombination deleted the memoize-book-name-lookup branch November 17, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

2 participants