Merge bundle tables without copying the base table - #107
Open
rotundtapir wants to merge 1 commit into
Open
rotundtapir wants to merge 1 commit into
rotundtapir wants to merge 1 commit into
Conversation
ApkBundle.mergeModules() merged the base into an empty module, which copied the whole base table entry by entry into a new TableBlock while the base kept its own parsed copy, and TableBlock.merge() refreshed the whole table after every module. On a large app (Google app, 87 MB resources.arsc) that is two full tables on the heap and one full refresh per split. A table that is only parsed for the merge is now taken over as the merged table when the target has none, a split's parsed table is dropped once merged, and TableBlock.merge(table, refresh) lets the bundle refresh once at the end. The public merge() methods still refresh. The largest table is picked by entry size, so a bundle without a base does not parse every table up front. Taking the base table over also keeps its strings as they are: the copy merged the string pool by XML text, which could turn a plain string containing markup into the styled string of the same text. Also keep a stored dex stored under the name it is renumbered to, instead of deflating it while writing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ApkBundle.mergeModules()merges the base into an empty module, which copies the whole base table entry by entry into a newTableBlockwhile the base keeps its own parsed copy.TableBlock.merge()then refreshes the whole table after every split. On a large bundle that means two full tables on the heap and one full refresh per split. A user of Morphe (which uses ARSCLib to merge splits on the phone) ran out of memory at this step with a 1280 MB heap on the Google app (com.google.android.googlequicksearchbox).Changes:
TableBlock.merge(table, refresh)lets the bundle refresh once at the end. The existingmerge()methods still refresh.Split bundle of the Google app (com.google.android.googlequicksearchbox 17.61.20: base 268 MB + 8 splits), JDK 21:
Taking the base table over also fixes string corruption. The copy merges the string pool by XML text, so a plain string containing markup could turn into the styled string with the same text. In the Google app,
assistant_onboarding_account_ineligible_shell_app_error_messagebecame styled text with its link lost in 36 locales. With this PR all 36 matchbase.apkagain.Otherwise the decoded tables match main, and dex and lib entries are byte-identical. One visible difference: when
validateResourcesDir()runs after the merge, a few files shared between the default and-nightconfigurations can land in the other directory (11 in Instagram). Their content is identical and every reference points at them. Tested on the Google app, Reddit (34 splits), Instagram, Google Photos, Twitch, SoundCloud and Duolingo bundles.ApkBundleMergeTestcovers the merge and the stored dex.Independent of #104. Second of the split-merge changes after #106; the per-entry memory change will follow separately.