Repository navigation
Conversation
This was referenced Oct 20, 2026
TomCC7
added this pull request to stack #4743
October 20, 2026 22:26
TomCC7
force-pushed
the
cc/feat/cdr-unit-perception
branch
2 times, most recently
from
October 21, 2026 18:43
bbf56f8 to
f3bfd2c
Compare
Member
Author
Review reportUpdated: 2026-10-21. Direct predecessor: #4733.
What changes and why the diff is largeThis is a broad camera/perception migration, not a codec-only addition. It introduces domain value helpers, changes Python TF to generated messages, migrates camera/detection/tracking/localization consumers and adds native Rust SDK support used by sensor adapters. Review mapSuggested entry points: dimos/message_utils/image.py, dimos/message_utils/pointcloud.py, dimos/protocol/tf/tf.py, native/rust/dimos-module/src/cdr.rs, dimos/hardware/sensors/camera/module.py. Complete file mapImplementation (104)
Build and dependency metadata (48)
Tests (51)
Documentation and replay inputs (4)
Review focus and scope reductions
ValidationFresh checks on At cleaned stack tip Reproduction commands, not executed here unless stated aboveRun from this PR's own checkout root after installing its documented test dependencies. These are selected review entry points, not a claim of full-stack coverage. uv run pytest dimos/message_utils dimos/visualization/rerun/test_cdr.py dimos/visualization/rerun/test_cdr_image_values.py dimos/visualization/rerun/test_cdr_entities.py dimos/visualization/rerun/test_cdr_e2e.py dimos/protocol/tf/test_tf.py dimos/protocol/tf/test_static_tf_publisher.py dimos/teleop/hosted/test_camera_mux.py dimos/web/test_cdr_codec.pyRemaining workRestacked onto the current viewer head. Native sensor builds, cross-language SDK checks, Nix and hardware were not rerun. The 142 pre-existing mypy errors and remaining business graphs are unresolved; helper consolidation is not completion of the migration. |
TomCC7
force-pushed
the
cc/feat/cdr-unit-perception
branch
from
October 22, 2026 02:34
e4ad8ef to
f59b609
Compare
TomCC7
commented
Oct 22, 2026
| cdr::encode(value).expect("serialize a valid camera value") | ||
| } | ||
|
|
||
| fn encode_cdr_tf(values: &[Transform]) -> std::io::Result<Vec<u8>> { |
Member
Author
There was a problem hiding this comment.
super weird we need manual encode/decode definition?
TomCC7
removed this pull request from stack #4743
October 22, 2026 18:51
TomCC7
added this pull request to stack #4776
October 22, 2026 18:52
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrate camera, image, pointcloud, TF and perception boundaries to generated CDR messages. Keep source headers through detection, tracking, localization and camera adapters.
Base: #4733. Shared SDK support is reviewed separately in #4774 below this layer. Native sensors bind the public fallible CDR encoders and shared TF adapters directly; only device data conversion stays local. This removes repeated expect wrappers and fixes RealSense timestamp precision without adding a codec to cdrgen.
The final combined stack passed 158 SDK and 2 adapter tests, including exact epoch/negative/adjacent-nanosecond timestamps and malformed input handling. V4L2 passed 6 tests. Nix source staging from the actual immutable archive passed locked Cargo checks for all seven migrated consumers; archive and CDR git-source hashes are verified.
RealSense requires librealsense2 and depth2depth requires model weights, which are unavailable in this environment. Their full native builds, hardware operation and full Nix builds remain unverified. Broad Python perception tests were not rerun for this codec change. Existing boundary review: #4734 (comment)
CDR dependencies and TF conventions
The stack consumes published dimos-cdrgen 0.1.2 and the immutable CDR library fix. SDK #4774 now uses standard ROS
TFMessage,TransformStampedand integerTime, with no per-driver TF codec callbacks or parallel public transform type. The externaltransformslibrary, validated by its complete std/no_std/serde and ARM verification gate, owns the frame tree, common-time lookup, interpolation and static history. Its atomic batch patch is pinned to5ae9563de0c02638c04e393aadfad28444a4398e.Combined-stack local validation: SDK 121, macros 22, mapper 101, MLS 135, local planner 91, follower 63, V4L2 6 and depth2depth 12 Rust tests; all 24 native/DimSim/port-contract acceptance tests pass without skips. The mapper exchange includes Python CDR
tf_staticinput; the follower exchange covers drive, arrival, a new goal and empty-path stop. Missing native binaries fail the dedicated job. RealSense passescargo check --all-targets --features docs-only; device linking and hardware execution were not tested. Seven real source-staging layouts pass locked Cargo resolution. Native acceptance for tipcc463c163aed465f7c1d6808165bf640820b0182: workflow run. These are combined-tip results, with the SDK additionally tested standalone at its own layer.Native Nix inputs use verified source archive
f7723ad4d5b9aa5dfe63614b6b537d1ccc2d212e, SHA256f90af8fa794a34d53ee9481c6d0bdb63138853b30f851d92960fd34224ff62b2. Full Nix builds remain unverified. SDK static edges persist after receipt; LCM/Zenoh do not acquire DDS transient-local durability from CDR. Existing Python mount publishers retain dynamictfcompatibility. Legacy modules remain explicitly pinned to SDK 0.1.0.Stack #4776 preserves existing history and PR bases. Current main still has conflicts with the original stack; this change does not rebase onto main. No DimOS PR has been merged.