Skip to content

Fix the from-assets arm's fitting and level defaults, and give SceneBase engine materials - #207

Merged
yichao-liang merged 1 commit into
masterfrom
from-assets-fixes
Oct 3, 2026
Merged

yichao-liang merged 1 commit into
masterfrom
from-assets-fixes

Conversation

@yichao-liang

Copy link
Copy Markdown
Collaborator

Why

In the EMPIRIC from-assets arm (agent_continual_from_assets), the agent builds its own scene from SceneBase, the scene manifest and the asset files.
It is not handed a per-domain visible core such as PyBulletBridgeBaseEnv.
The Sept 21 pilot won 6 of 10 runs: Bridge 2/2, Boil 2/2, Balloons 1/2, Fan ramp 1/2, Domino 0/2.
Every failed run built a scene that loaded and reconstructed its observations, with at least 7,686 of 10,000 steps left.
The failures trace to the harness gaps below, which also affect the other scene-built and model arms where noted.

What changes

  • Fit scope (sim.fit).
    The scene contract told agents to declare RESIDUAL_FEATURES = {} "if none", and an empty scope gave the fit nothing to score: every segment read as unexplainable at any parameters ("NO FIT RAN", RMS inf).
    Both Domino agents hit this, and the test-level model was unfitted in all 10 pilot runs.
    An empty scope now falls back to the inferred features of the scene's objects, as an undeclared one already did, without the commanded robot.
    A "NO FIT RAN" report names the features it scored, and the contract and the deployment gate describe the scope as the fit's loss scope.
  • Edited declarations.
    Editing a ParamSpec init value was ignored: the carried value from the last fit or round won, so a friction sweep by edits produced identical rollouts.
    A carried value now holds only while its declaration is unchanged, and the file a fit was published for still runs at that fit when the agent reverts to it.
    The "PARAMS UNFITTED" notice lists the values in use and marks the carried ones.
    This applies to the subclass and rule model forms of every model arm.
  • Restart warning.
    A restart logged that a subclass model's parameters fell back to their declared inits.
    They did not: the check compared them with the model's empty rule specs.
    The check now counts AGENT_PARAM_SPECS.
  • Current-level defaults.
    On a test level, sim.reset() and sim.task() raised an obsolete phased-synthesis guard that told the agent to pass task_idx=0, which staged the first training level.
    evaluate_trajectory and is_goal_state also scored level 0 by default.
    All four now default to the level in progress, as the play prompt already documented, and task_idx is described as indexing the levels reached so far.
  • Engine materials in SceneBase.
    For every observed type with bodies, the base offers <type>_mass, _lateral_friction, _spinning_friction, _rolling_friction, _restitution, _linear_damping and _angular_damping, plus support_* contact materials for static bodies without an observed name.
    A name declared in AGENT_PARAM_SPECS is applied to every body of its group after each reset and on change.
    An undeclared one only describes the scene, so the fit's pinning of unestimated parameters cannot override the agent's own code.
    Every Domino agent left spinning friction at 0.005 or less (truth 0.5), and that friction decides how a struck domino twists on a turn.
  • Visual-only bodies in the manifest.
    Bodies without a collision shape are listed with their visual shapes and visual_only: true, for example the Balloons ceiling that marks the burst height, which the Balloons seed-1 agent had to guess.
    The generic backdrop walls stay out: SceneBase builds them itself, as it does the ground plane.

Evidence

An end-to-end replay drives the real continual loop and tools with a scripted agent on the benchmark Domino (high-friction turn, seed 1).
It uses the seed-1 pilot agent's own simulator.py and replays its winning level-1 plan (Pick and Place two blue dominoes, then Push the green one), which wins level 1 in 194 steps, as in the pilot.

Check master 864ed48c7 this branch
sim.fit() with the pilot's RESIDUAL_FEATURES = {} NO FIT RAN, RMS [inf] fit runs on 7 segments over the inferred domino poses: 45% lower error, domino_friction 0.70 to 0.73
Edit domino_friction 0.7 to 3.0, rerun the same push identical rollouts, world stays at the fitted 0.728 rollouts differ, world at 3.0
Restart from the level-1 checkpoint values kept, false "falling back to declared init values" warning values kept, no warning
Level 2: sim.reset(), sim.task() both raise, pointing at task_idx=0 stage and describe level 2 ("Task 1 (current level)")
Level 2: evaluate_trajectory default task_idx=0 current level
Material menu of the agent's world none domino_* and support_* entries

Tests

  • New: fit-scope fallback, edited declarations (subclass form), restore without the false warning, current-level defaults for the probe and the goal queries, SceneBase materials (declared set and kept across resets, undeclared never set), manifest visual-only bodies without the backdrop.
  • Updated: the stale-current-task guard tests and the candidate provider test, which pinned the carry semantics this changes; the provider test now resets the config it reads, and the evaluate_trajectory stubs carry the current level.
    Prompt goldens regenerated.
  • Local CI replay of this commit: static checks (yapf, isort, docformatter, mypy, pylint) and all 8 shards pass (2,532 tests).

Not in this PR

  • A rerun of the from-assets arm on current master; the pilot ran before the principled belief.
  • Determinism of repeated probe rollouts in an agent-built scene: real-to-sim Domino seed 2 saw the same layout go from 0/16 to 15/16.

🤖 Generated with Claude Code

…ase engine materials

The Sept 21 EMPIRIC from-assets pilot won 6 of 10 runs. Replaying its
failures through the real tools found harness problems, not scene
building:

- sim.fit scored nothing when RESIDUAL_FEATURES was empty, which the
  scene contract asked for ("{} if none"): every segment read as
  unexplainable at any parameters (NO FIT RAN, RMS inf). An empty scope
  now falls back to the inferred features of the scene's objects, as an
  undeclared one does, leaving out the commanded robot. A NO FIT RAN
  report names the features it scored, and the contract and the
  deployment gate describe the scope as the fit's loss scope.
- An edited ParamSpec init_value was ignored: the carried value of the
  last fit or round won, so a friction sweep by edits gave identical
  rollouts. A carried value now holds only while its declaration is
  unchanged, the fitted file still runs at its fit, and the UNFITTED
  notice lists the values in use.
- A restart warned that a subclass model's parameters fell back to
  their declared inits. They did not; the check compared them against
  the empty rule specs.
- On a test level, sim.reset() and sim.task() raised and pointed at
  task_idx=0, the first training level, and evaluate_trajectory and
  is_goal_state scored level 0 by default. All four now default to the
  current level. The provider test now resets the config it reads.
- SceneBase offers the engine materials of every observed type with
  bodies (mass, lateral, spinning and rolling friction, restitution,
  damping) and of the static support as parameters, set on every body
  of the group when declared. The Domino agents never modelled spinning
  friction, which decides how a struck domino twists on a turn.
- The scene manifest lists bodies without a collision shape, such as
  the Balloons ceiling that marks the burst height, with their visual
  shapes. The backdrop walls stay out: the scene base builds them
  itself, as it does the ground plane.
@yichao-liang
yichao-liang merged commit 2fbf7de into master Oct 3, 2026
14 checks passed
@yichao-liang
yichao-liang deleted the from-assets-fixes branch October 3, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant