Skip to content

warm-root: list the prefetch from a launch that restores the snapshot - #1397

Merged
ejc3 merged 3 commits into
mainfrom
perf/wall-prefetch-hit
Oct 27, 2026
Merged

ejc3 merged 3 commits into
mainfrom
perf/wall-prefetch-hit

Conversation

@ejc3

@ejc3 ejc3 commented Oct 27, 2026

Copy link
Copy Markdown
Owner

On the first host booted from the ce08eee AMI the prefetch list held 6163
files (334 MiB) and fcvm-fleet-prefetch.service ran for 20 s (17.8 s of
CPU) beside cloud-init-local (9.3 s, against 7.4 s on the image before)
and the store (13.6 s, against 10.8 s). warm-root.sh traced its only
launch, which also boots the VM and exports the image into root's podman
store, so the list held image layers no slot reads. On a leased
g4dn.metal, the traced launch of a slot that restores the snapshot read
120 files (124 MiB).

warm-root.sh now launches twice: the first takes the pre-start snapshot,
the second runs under strace, must log a pre-start snapshot hit, and is
what the list comes from.

Test: fleet/ami/check.sh asserts the traced launch follows an untraced
one and the hit is checked (fails against the previous warm-root.sh).
the_ami_warm_up_launches_its_vm_as_a_slot_is_launched passes.

Contract: the host AMI's prefetch list names what a woken slot's launch reads, not what the AMI build's first launch reads. Downstream: the next host AMI build. Evidence above; the list size on a built image is checked when that AMI is built.

On the first host booted from the ce08eee AMI the prefetch list held 6163
files (334 MiB) and fcvm-fleet-prefetch.service ran for 20 s (17.8 s of
CPU) beside cloud-init-local (9.3 s, against 7.4 s on the image before)
and the store (13.6 s, against 10.8 s). warm-root.sh traced its only
launch, which also boots the VM and exports the image into root's podman
store, so the list held image layers no slot reads. On a leased
g4dn.metal, the traced launch of a slot that restores the snapshot read
120 files (124 MiB).

warm-root.sh now launches twice: the first takes the pre-start snapshot,
the second runs under strace, must log a pre-start snapshot hit, and is
what the list comes from.

Test: fleet/ami/check.sh asserts the traced launch follows an untraced
one and the hit is checked (fails against the previous warm-root.sh).
the_ami_warm_up_launches_its_vm_as_a_slot_is_launched passes.
@ejc3

ejc3 commented Oct 27, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 38 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f679e569-247f-4ee4-a274-28a7619f5c92
Commits

Reviewing files that changed from the base of the PR and between ac9166d and 76eed0e.

Files selected for processing: 2
  • fleet/ami/check.sh
  • fleet/ami/host/warm-root.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T14:41:46.396863Z 76eed0e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 41cdc347f1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread fleet/ami/host/warm-root.sh Outdated
fcvm's default filter is warn and it logs 'Pre-start snapshot hit' at
info, so the hit check would fail every AMI build after two good
launches (Codex, #1397). The launches set RUST_LOG=info.

Test: fleet/ami/check.sh requires RUST_LOG=info on the launches (fails
against 41cdc34).
@ejc3

ejc3 commented Oct 27, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@ejc3 ejc3 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RED-VERIFIED: the Codex review's one finding (warm-root.sh:71, fcvm logs the hit at info under a warn default) is fixed in 8acd837, pinned by fleet/ami/check.sh, which failed against 41cdc34.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8acd837c92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread fleet/ami/host/warm-root.sh Outdated
fcvm logs 'Pre-start snapshot hit' before it loads the snapshot, and a
load that fails is logged as 'Snapshot miss' and cold-boots, which also
reaches a first frame (Codex, #1397). Each launch now logs to its own
file, and the traced launch's must hold the hit and no miss.

Test: fleet/ami/check.sh requires that check (fails against 8acd837).

@ejc3 ejc3 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RED-VERIFIED: the second Codex review's finding (warm-root.sh:72, a failed load falls back to a cold boot after the hit line) is fixed in 76eed0e, pinned by fleet/ami/check.sh, which failed against 8acd837.

@ejc3

ejc3 commented Oct 27, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 76eed0e815

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ejc3
ejc3 merged commit 7f5fb7e into main Oct 27, 2026
15 checks passed
@ejc3
ejc3 deleted the perf/wall-prefetch-hit branch October 27, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant