Add batched visualization markers and asset attachment

[yuecideng]

Description

Add render-only mesh markers shared by native DexSim and Viser, with RGBA transparency and reusable box, sphere, cylinder, capsule, cone, arrow, frame, and custom-mesh prototypes.

Groups default to all SimulationManager.num_envs; updates, visibility and clear accept selected env_ids. scope="world" creates one global batch. Groups attach to registered object/robot/articulation roots or links, follow host updates with local offsets, and detach while preserving world poses. Existing legacy drawing APIs remain available; there is no new public arena_index.

Automatic attachment refresh preserves browser capture cadence and isolates visual failures from simulation accounting. Manual mutations validate and publish synchronously with rollback. Native markers are excluded from camera outputs by default; opt-in overlay inclusion retains the documented Hybrid NRD limitation.

Docs CI also fetches the architecture snapshot's pinned evidence commit when squash history omits it. Exact-source assertions and generated data remain unchanged.

Refs #129.

Dependencies: native rendering requires DexSim MR !1423, engine revision 1acf56f57f7126f8eaa91d27c2eb15c8ca9ff916. The adapter creates ordinary MeshObjectDesc(physics=None) with RenderDesc/MaterialDesc through Scene.add_mesh_object, retains existing stable SpawnedRigidBody handles, and removes them by Scene path. Lightweight Scenes exist before physics finalization on both backends; marker operations do not prepare physics, and handles survive Newton rebuild. Cleanup tolerates prior Scene-close invalidation. No DebugMesh or standalone public factory is added.

The engine dependency now removes its temporary _owned_materials list and all new owned API parameters. Ordinary material assignment holds references; actor removal and Scene-handle invalidation are immediate, while unused material instances are reclaimed at existing manager cleanup boundaries. Live bodies, SHARE clones and external material handles remain valid until their final references are released. A retained material template also preserves its default parameters; unused named instances remain collectable. The engine ports the RenderService cleanup scheduling from unmerged DexSim !1431, source snapshot 219bb7ba, and covers Filament default/Scene managers too. Collection runs after 60 seconds of accumulated render update dt, not a wall-clock timeout. This does not change the marker group's public API or introduce consumer-side GC.

Merged current main (7ae9a501), including its COM/body-frame inertia adaptation. Resolved the simulation-update overlap by retaining the substep observer before marker attachment refresh and visualization capture; preserved both test suites. The registered-articulation attachment smoke now runs without the obsolete com_quaternion capability skip. Keep this PR draft pending the qualified engine release; no unpublished package version is pinned.

Type of change

• New feature
• Documentation update

Validation

Current consumer submitted commit: d893318. Current consumer CI: CI #35559124254 — cancelled after the sim lane stopped producing output. Current engine submitted commit/pipeline: 1acf56f57f7126f8eaa91d27c2eb15c8ca9ff916 / DexSim #71280 — SUCCESS, all six jobs.

Current revision	State
Marker CPU/transaction suite, format, API/context checks	PASS — 304 manager/visualization tests in 5.85 s, including callback/marker/capture ordering. Black 26.3.1 passed (1126 files); API inventory 2184/2184. Context lifecycle/refresh ordering guidance updated.
Native marker/material integration against revised engine	PASS — all 4 tests/sim/test_marker_rendering.py cases passed with --run-gpu in 12.71 s against engine 1acf56f57: RGBA/sensor isolation, selected environments without preparation, repeated handle retirement with retained material references, registered-articulation root/link attach and detach.
Current consumer CI	INCOMPLETE — CPU fast lane: 4860 passed, 1 skipped, 49 warnings in 180.49 s; distributed lane selected no tests; GPU lane: 175 passed, 7 skipped in 1187.30 s; the sim lane selected 362 tests and last logged workspace analyzer at 30% (05:01:03 UTC). After about 27 minutes without output, the run was cancelled at 05:28:51 UTC; its test gate failed. No sim totals or runtime stack were produced; the cause is unestablished and this lane is not counted as passing. The four marker GPU cases skip on the released SDK and are covered by the private-engine integration run above. Lint, documentation tests and full documentation build passed; workflow conclusion is cancelled.
Workspace stall diagnostic against revised engine	PASS — the four analyzer cases plus the actual next cache-hit case passed in 53.40 s with verbose setup/teardown and a 60 s stack-dump timer (240 s outer bound). All teardown phases completed. This local five-case run uses engine 1acf56f57; it does not qualify the full sim lane or establish the cause of the remote released-SDK stall.
Current engine formal CI / release	Formal CI PASS at engine 1acf56f57: all six jobs in #71280 succeeded, including renderer/Python matrices, six lifecycle configurations × five rounds, Python 3.10–3.12 packaging and docs. Engine release remains pending. H1 performance is not qualified: teardown spread 10.44% exceeded the 10% stability limit (warning-only gate).

Remaining integration work

• The former COM/inertia blocker is resolved by the changes already merged in main (fix(sim): Adapt mass properties to mat33 and batch Newton reads #639). The current branch preserves those conversions, and the two-environment registered-articulation marker smoke passes. Current-head CPU/GPU and docs passed, but sim qualification remains blocked by the stall. Source review found no marker groups in the analyzer cases; that does not prove a baseline bug. Local collection identifies the next selected case as TestWorkspaceResultsCacheIntegration::test_cache_hit_skips_recompute. A bounded local run of all four analyzer cases and that next cache case completed with all five passing against the revised engine. The remote released-SDK stall was not reproduced locally; obtain remote phase/stack evidence before changing production code or declaring the full sim lane qualified.
• The revised engine passed formal CI at its final SHA; publish a release containing the required capabilities before completing dependency integration. The engine MR records its remaining performance and threading qualification limits.
• Native publication is still per mesh handle. Python batch semantics do not imply one native bulk call or GPU instancing, and no throughput improvement is claimed.
• Arbitrary USD marker import, text and new streaming point-cloud machinery remain follow-up scope.

Screenshots

No screenshots attached; focused tests and the runnable example document the behavior.

Checklist

• I have run the black . command to format the code base.
• I reviewed affected documentation and agent context and updated the simulation/visualization guidance and API inventory.
• Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py).
• I have added tests that prove the feature works.
• Dependencies have been updated, if applicable. Pending a qualified engine release.

[acrlw]

Changes requested. See the three inline findings for the affected code ranges, evidence and suggested fixes.

[acrlw]

Changes requested: two P2 findings and one P3 documentation correction, attached to the affected lines. All three require EmbodiChain changes.

Validation: 304 focused tests passed; both defects reproduced; live two-environment Viser attachment, following and cleanup passed. Four native overlay tests remain skipped pending runtime capabilities, and the complete sim CI lane remains unverified.

[acrlw]

[P2] Include marker environments in visibility controls and preserve their state

An environment containing only markers gets no Environment N control. The environment inventory collects ordinary nodes, cameras, gizmos and joint controls from the manifest, while markers live in SceneOverlays.meshes. Their environment IDs are therefore omitted, and this lookup treats the missing visibility state as visible.

This also breaks visibility after asset removal. In a two-environment scene, hide Environment 0, then remove the sole parent asset so that only markers remain. The manifest refresh clears _env_visibility, and the next frame makes the hidden markers visible again. A backend probe reproduced this transition; a live Viser scene also confirmed that markers for both environments survive parent removal while the environment controls and visibility state disappear.

Please include marker environments when creating and refreshing the controls, and retain visibility state for environments that still exist. The existing visibility test inserts an ordinary SceneNode, so it misses this case. Extend that test file with marker-only environments and removal of the last parent asset.

[acrlw]

[P2] Read the current environment's marker count directly during updates

update() calls _updated_state() for each selected environment, but each self.counts[env_id] first evaluates the counts property, traversing every environment and constructing a full tuple. Updating all E environments therefore adds O(E²) counting work. Color-only or orientation-only updates perform that full traversal twice per environment.

With 4096 environments and one box per environment, the CPU state update in group.update(translations=...) took 390.13 ms at the median of three runs after warm-up. A control that changed only the two count lookups to read the current environment's array length took 59.03 ms, with count and position assertions passing in both cases. This extra work runs on the caller's thread and slows simulations that update markers each step.

Please read old_count = len(self._states[env_id]["translations"]) once at the start of _updated_state() and reuse it for omitted translations and count-change detection. Reuse the existing full/partial-environment, count-change and attribute-preservation tests, then repeat the timing comparison at the same scale.

[acrlw]

[P3] Update the native marker documentation to match the Scene descriptor API

This paragraph describes manually adding an object to an Arena, assigning a material and building it, and explicitly says that no Spawn descriptor is required. The current NativeMarkerRenderer._create() instead constructs MaterialDesc, RenderDesc and MeshObjectDesc, then calls Scene.add_mesh_object(). Initialization also checks that these descriptor and Scene APIs are available.

Users checking runtime compatibility against this documentation would miss required Scene/descriptor capabilities. Please update the creation, stable-handle and removal flow, including Scene ownership of object registration and the engine's reference-aware material collection. The visualization context already describes this behavior; the API documentation should agree with it.