Skip to content

fix(sandbox): reject stale exit during restart - #2857

Merged
drew merged 1 commit into
mainfrom
fix/sandbox-start-exit-race/drew
Aug 21, 2026
Merged

fix(sandbox): reject stale exit during restart#2857
drew merged 1 commit into
mainfrom
fix/sandbox-start-exit-race/drew

Conversation

@drew

@drew drew commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Prevent a delayed canonical-process exit from the stopped sandbox instance from racing with restart and moving the sandbox from Starting to Error.

The Kubernetes v1beta1 combined E2E reproduced the same failure twice on the unpatched commit:

Related Issue

No issue required: this is a localized lifecycle race with a focused regression test.

Changes

  • Preserve the previous main-process instance ID as a tombstone through stop/start transitions
  • Ignore a matching exit report from the stopped instance while the sandbox is Starting
  • Continue accepting an exit from a different instance so genuine failures of the new process remain terminal
  • Add a regression test for both the stale-old and legitimate-new exit paths

Testing

  • mise run pre-commit passes
  • cargo test -p openshell-server --lib passes (1,406 passed, 8 ignored)
  • Unit test added for the restart race
  • E2E checks run in GitHub Actions

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew
drew requested review from a team, derekwaynecarr, mrunalp and sjenning as code owners August 21, 2026 01:47
@drew drew added the test:e2e Requires end-to-end coverage label Aug 21, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 8fc8528. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@drew

This comment was marked as outdated.

@drew drew closed this Aug 21, 2026
@drew drew reopened this Aug 21, 2026
@drew
drew enabled auto-merge August 21, 2026 03:21
@drew
drew added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit 20d2e86 Aug 21, 2026
144 of 148 checks passed
@drew
drew deleted the fix/sandbox-start-exit-race/drew branch August 21, 2026 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants