feat(simulation): add badminton rl environment - #179
Draft
aicheye wants to merge 31 commits into
Draft
Conversation
MuJoCo badminton-receive environment (stationary arm): validated shuttle aero, inverse launcher, interception predictor, scripted FSM baseline, and the mjlab (MuJoCo Warp) teacher-student RL task package. Rework the mjlabs Dockerfile around uv + Python 3.12 for the headless training container, and add scripts/slurm_train.sbatch for the WATcloud SLURM cluster. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Feed the EKF one noisy measurement per 50 Hz tick and draw its position
estimate and rolled-out trajectory prior as an orange overlay ('e' toggles).
Validates the early-flight jitter and convergence visually.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Run 1 (a1vagllq): policy sat at the ready pose; exp(-d^2/0.4^2) at the median 0.69 m face-to-p* distance pays 0.05 with near-zero gradient, so holding still beat reaching and the exploration std collapsed. sigma 0.8 maximizes the kernel gradient at that distance. Add TRAINING_LOG.md and an EXTRA_ARGS passthrough in the sbatch script for probe-length runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Probe 2 (i70q1pab): sigma 0.8 gave the expected 4x approach jump but plateaued by iter 64 with mean_std collapsing 0.44 -> 0.084 again. The concave exp kernel raises expected reward as action noise shrinks, so 0.005 entropy cannot keep exploration alive long enough to find contact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
scancel's SIGTERM killed python outright, leaving wandb runs marked crashed. Trap it and signal the training child with SIGINT so wandb syncs and closes the run; checkpoints already save every 100 iterations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Job 593623 landed on the gtx_1080 node, whose sm_61 arch has no kernel images in the cu128 torch wheels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Run 3 (j2h0aizn): entropy 0.02 keeps std at 0.25 and exploration finds contacts in 70% of iterations, but at weight 10 a hit pays an effective 0.2 — about 2x one episode of hovering near p* — and the hit rate crawls at 0.15%. Effective 2.0/hit makes discovered hits dominate the gradient. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Probe 4 (k0zoennl) refuted the bonus-scale hypothesis: 10x the contact bonus left the hit rate at ~0.1%. The binding constraint is the flat gradient of the sigma-0.8 kernel over the last 30 cm to p*. A second approach term with sigma 0.15 / weight 5 puts slope there; since p* is on the flight path, precise parking produces contact. Entropy 0.02 also overshot once approach saturated (std 0.49 -> 0.60, reward bleeding into action penalties): 0.01. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
scancel signals every process in the job, so the sbatch-level trap never mattered: python received SIGTERM directly and died uncleanly. Raise KeyboardInterrupt from the handler instead, which wandb already exits cleanly on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Five reward-side probes plateaued in similar hover-and-lucky-swing behavior; before further reward surgery, measure the geometry. Logs per-episode min face->p* distance and the distance at t* under Metrics/perception/. Probe 6 reruns run 5's exact reward config with 2000 iterations to also test whether its contact growth compounds with runway. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
InitialStateCfg.pos/rot COMPOSES with the root body pose already in the scene XML (it does not overwrite, as the removed arm_base_pose comment claimed), so every training run so far had the arm mounted ~1.2 m above its real position — the racket could never reach the flight band, which is the root cause behind all five reward-probe plateaus (TRAINING_LOG run 6). Verified by zero-action Warp rollouts: face at reset now matches the CPU env to 3 mm; shuttle-through-p* was already correct (<= 5 cm). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Run 7 (iwn6h18p, 2000 iters): 31% hit rate, face parked 2 cm off the flight path — the reward structure works. Entropy 0.01 drifted std to 0.85 after reward saturation, costing precision (d@t* 0.14 vs 0.066 at low std). The old 0.005 collapses were the broken arm mount, not the coefficient. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Plain scancel SIGTERMs every process in the job simultaneously, killing wandb's sync subprocess before it can upload — which is why scancelled runs kept showing as crashed despite the SIGTERM handler. slurm_stop.sh signals only the batch shell (USR1), the shell SIGINTs just the python process, the wandb service child survives, and train_rl.py now calls wandb.finish() on KeyboardInterrupt so the run is marked finished. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Every run now records Metrics/feasibility/qvel_peak_jN, tau_peak_jN and tau_duty_jN (fraction of ticks above the datasheet rated torque, newly in params as arm.torque_rated). Torque reads qfrc_actuator — the joint-space force after the jnt_actfrcrange clamp that build_scene.py sets from the datasheet peaks; verified in CPU Warp that the clamp holds (peaks saturate at exactly 53/0.73 Nm) and that data.actuator_force is the unclamped servo request, not the applied torque. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Viewer review of run 8 suggested misses cluster on body-line shuttles. eval_rl.py rolls a checkpoint over the launcher bank and reports hit rate by p*'s distance in front of the chest plane, height, and lateral offset. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Training uses < 1 GB of VRAM; an exclusive gpu:1 request cannot coexist with anyone holding a shard on the node and queued for hours. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…ized) mjlab divides logged episode sums by max_episode_length_s, so logged face_contact 0.657 is a ~98.5% hit rate, not 33%. Record the run-8 bank eval: 99.3% overall, weakest bins body-line (0.944) and far-left (0.954). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Run-8 feasibility eval: joint speeds peak at 6-9 rad/s (far under rated) but the GL40 wrist sits above its 0.25 Nm rated torque on 95% of ticks, pinned at the 0.73 Nm peak — largely the gravity moment of the racket at its 0.45 m offset. The sim has no heat model, so add mdp.torque_over_rated = sum_j max(|tau_j|/rated_j - 1, 0) at weight -0.5 (a full-episode wrist overload ~ -1.2 vs the 2.0 contact bonus). Physics-grounded, no posture opinion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
The sim's PD gains (kp up to 600, kv up to 30) exceeded what the AK drives accept over MIT control (kp <= 500, kd <= 5, mit_profiles.yaml), and kp 60 / kv 3 on the 0.73 Nm GL40 wrist saturated it at any tracking error above 0.7 deg — the real source of run 8's 95% wrist overload (gravity moment at the wrist is 0.000 Nm at READY). New gains kp [400,400,300,300,100,8] / kv [5,5,5,5,2,0.3]; scene rebuilt. Run 9's per-tick thermal penalty is removed from the reward set: it suppressed the transient swing peaks the strong joints are rated for (hit rate 97 -> 88%, return quality 6x worse) while barely moving the wrist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
trpro-slurm1 (4x 3090) and delta-slurm1 (2x 2080 Ti) are up and supported by the cu128 wheels; only the gtx_1080 node is excluded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…mary tools BadmintonAction now applies joint_command's pipeline to every target: per-tick cap (control.target_velocity_max * step_dt), low-pass q <- 0.85*q_prev + 0.15*q, position clamp, prev reset to the ready pose. Runs 8/10 showed the policy exploiting bang-bang targets the real arm cannot receive (every joint at its torque clamp most of the episode). Values: rated motor speeds as the physical ceiling until the team sets an operational velocity_max (hardware runs a 40 deg/s bring-up crawl). Also: scripts/slurm_view.sh (one-command viser session, prints the tunnel) and scripts/wandb_summary.py (numpy-free run summary for the login node). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…U, 16 GB) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…very notes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…nd model Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…u,v) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hDkDZ7eJFkdhNf1LaGgE
…e u,v,d) instead of plane crossings A clean hit rebounds before crossing the face plane and the infinite plane is crossed far from the racket whenever the face rotates, so the crossing test caught 27% of hits nowhere and put points metres off the face. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nx9KChDhmGzcqKtNQ3HEie
…igin Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nx9KChDhmGzcqKtNQ3HEie
…he run-11 miss analysis Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nx9KChDhmGzcqKtNQ3HEie
This was referenced Sep 1, 2026
wilsonchenghy
added a commit
that referenced
this pull request
Sep 2, 2026
Superseded by the mjlab badminton environment (#179) — a full rewrite on the simulation_mj stack with a v2 arm + racket, court, and aerodynamics. The Isaac version was the earlier attempt and its only robot config (arm_hand.ARM_CFG) points at a USD that was deleted in #183. `arm_hand.py` / `ARM_CFG` is now used only by src/teleop/camera_teleop/, which is being archived in a follow-up. Claude-Session: https://claude.ai/code/session_01EtusD8QWFBLpdMUZPpye5B Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
wilsonchenghy
added a commit
that referenced
this pull request
Sep 2, 2026
* Archived: remove the Isaac Lab badminton task Superseded by the mjlab badminton environment (#179) — a full rewrite on the simulation_mj stack with a v2 arm + racket, court, and aerodynamics. The Isaac version was the earlier attempt and its only robot config (arm_hand.ARM_CFG) points at a USD that was deleted in #183. `arm_hand.py` / `ARM_CFG` is now used only by src/teleop/camera_teleop/, which is being archived in a follow-up. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EtusD8QWFBLpdMUZPpye5B * Archived: remove camera_teleop + the dead arm_hand.py config camera_teleop (webcam hand landmarks -> DexRetargeting -> single-arm sim) was "not wired yet" and is unmaintained. Removing it retires its only dependency, pioneer_humanoid/arm_hand.py (ARM_CFG / LEFT_ARM_CFG) — a single-right-arm+hand config whose USD was deleted in #183. Everything else is on BIMANUAL_ARM_CFG. Stacked on the Isaac-badminton archival (the other ARM_CFG consumer). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EtusD8QWFBLpdMUZPpye5B * chore: retrigger CI after base retarget to main Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EtusD8QWFBLpdMUZPpye5B --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
wilsonchenghy
added a commit
that referenced
this pull request
Sep 2, 2026
…n arm dir (#267) - UsdModelAssets/{block,box}.usd + Table/table.usd -> src/simulation/assets/ (consumers: push_block scene, quest_isaac_teleop) - pioneer_badminton_arm/ + empty arm_assembly/ deleted — zero references (badminton is mjlab now, #179) pioneer_boxing_arm/ kept for now — pending confirmation it's unused by Ramy's AMP / motion-prior work. Part 1 of dissolving src/simulation/Humanoid_Wato/. wato_hand/ and the V1 humanoid model fold into pioneer_humanoid/ next. Claude-Session: https://claude.ai/code/session_01EtusD8QWFBLpdMUZPpye5B Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP