sim2real

Pipeline for transferring simulation-discovered routing algorithms from inference-sim to production llm-d.

Install

git submodule update --init --recursive
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

Set SIM2REAL to this repo's path for the rest of the walkthrough, then change into your experiment repo. All subsequent pipeline commands are run from the experiment repo root — pipeline/*.py defaults --experiment-root to the current working directory.

export SIM2REAL=$(pwd)
cd <path-to-your-experiment-repo>

Walkthrough

This is the canonical single-path flow for transferring one experiment from BLIS into a real cluster. Run all of these commands from inside the experiment repo. Each step's optional flags and variations are documented in pipeline/README.md. For recovery from failures, see the Troubleshooting section below.

1. Bootstrap the experiment bundle

If your experiment repo was generated by BLIS and does not yet have a transfer.yaml, run the bootstrap skill from a Claude Code session in the experiment repo:

claude --dangerously-skip-permissions --add-dir $SIM2REAL

/sim2real-bootstrap

For BYO operators — pre-built EPP image(s) + baseline scenario + per-algorithm overlay(s), no BLIS artifacts — use --byo mode instead: /sim2real-bootstrap --byo --baseline ... --algorithm ... --algorithm-image ... --algorithm-config .... See .claude/skills/sim2real-bootstrap/SKILL.md for the full invocation.

Skip this step if your experiment repo already contains transfer.yaml, baselines/, and (BLIS only) a component submodule.

2. Set required environment variables

export HF_TOKEN=<huggingface token with access to required models>

3. Bootstrap the cluster (one-time per cluster)

Bootstraps cluster identity, RBAC, PVCs, and Tekton resources in the primary namespace. Add additional namespace slots with slot add (safe to run while a deployment is in flight).

# One-time bootstrap (creates primary namespace):
python $SIM2REAL/pipeline/cluster.py init <cluster_id> <primary_namespace>

# Add additional namespace slots (repeat as needed):
python $SIM2REAL/pipeline/cluster.py slot add <cluster_id> <namespace>

4. Configure the workspace (one-time per experiment repo)

Records the registry and orchestrator image in workspace/setup_config.json.

python $SIM2REAL/pipeline/setup.py

5. Translate

Skill-driven (default). Checkpoint the translation, run the translation skill, then validate:

python $SIM2REAL/pipeline/sim2real.py translate

claude --dangerously-skip-permissions --add-dir $SIM2REAL

/sim2real-translate

python $SIM2REAL/pipeline/sim2real.py translate --resume

Alternative (BYO): if you already have a pre-built image, skip the skill and register it directly — python $SIM2REAL/pipeline/sim2real.py translation register --algorithm <name> --image <ref> --config <treatment-overlay-path>. See pipeline/README.md § Register a translation.

6. Build images

Compiles per-algorithm plugin sources into container images and pushes them to your registry. Skip when using BYO — the image is already built.

python $SIM2REAL/pipeline/sim2real.py build --translation <alias-or-hash>

7. Assemble the run

Materializes workspace/runs/<run-name>/ from the translation and the experiment repo's transfer.yaml.

python $SIM2REAL/pipeline/sim2real.py assemble \
    --translation <alias-or-hash> \
    --cluster <cluster_id> \
    --run <run-name>

8. Deploy

Dispatches Tekton PipelineRuns across the provisioned namespace slots. Add --remote to run the orchestrator as an in-cluster Job instead of locally.

python $SIM2REAL/pipeline/deploy.py --run <run-name> run

Check pair status while it runs:

python $SIM2REAL/pipeline/deploy.py --run <run-name> status

9. Collect results

Pulls per-pair metrics and logs from the cluster PVC. Safe to run repeatedly — only collects what is new.

python $SIM2REAL/pipeline/deploy.py --run <run-name> collect

10. Validate and analyze

Re-run /sim2real-check against the collected results to confirm the deployed run matches the simulation, then /sim2real-analyze to produce comparison tables and charts.

claude --dangerously-skip-permissions --add-dir $SIM2REAL

/sim2real-check
/sim2real-analyze

For pipeline details and flag reference, see pipeline/README.md.

Troubleshooting

For pipeline-level recovery (rerun failed pairs, stop orchestration, clean up cluster artifacts, wipe data), see pipeline/README.md § Troubleshooting.

For experiment-config issues (EPP RBAC, logging verbosity), see docs/troubleshooting.md.

Tests

python -m pytest pipeline/ -v

See CLAUDE.md for artifact contracts and exit code semantics.