Skip to content

feat(messages): add built-in CDR types alongside legacy messages - #4366

Open
TomCC7 wants to merge 3 commits into
mainfrom
cc/feat/cdr-message-packages
Open

TomCC7 wants to merge 3 commits into
mainfrom
cc/feat/cdr-message-packages

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

What this adds

  • Build built-in .msg definitions with cdrgen during package installation.
  • Package generated Python types at dimos.messages, plus schemas and native source artifacts.
  • Keep existing dimos.msgs and business code unchanged.

Flow: definitions → cdrgen → installed artifacts → runnable examples. Base: main.

How to test

From this PR's checkout, with uv and a C++ compiler:

uv venv .venv-msg-review --python 3.12
DIMOS_ALLOW_MISSING_COCKPIT=1 uv pip install --python .venv-msg-review/bin/python .
.venv-msg-review/bin/python -I examples/message-package/demo_installed_wheel.py

The example prints the generated message and schema directories and checks CDR round trips.

For Python → C++ → Rust → Python, install CMake and Rust/Cargo, then run:

uv pip install --python .venv-msg-review/bin/python 'dimos-cdrgen[native]==0.1.2'
.venv-msg-review/bin/python -I examples/message-package/demo_native.py

The native example checks map/cpp/rust and the unchanged Header timestamp. Full instructions.

DIMOS_ALLOW_MISSING_COCKPIT=1 permits installation without prebuilt web assets. The Rust relay uses the complete external CDR library directly, pinned to the tested library fix in TomCC7/cdr-rs#1. WIP: transport, SDK and business integration follow in later PRs.

CDR dependencies and TF conventions

The stack consumes published dimos-cdrgen 0.1.2 and the immutable CDR library fix. SDK #4774 now uses standard ROS TFMessage, TransformStamped and integer Time, with no per-driver TF codec callbacks or parallel public transform type. The external transforms library, validated by its complete std/no_std/serde and ARM verification gate, owns the frame tree, common-time lookup, interpolation and static history. Its atomic batch patch is pinned to 5ae9563de0c02638c04e393aadfad28444a4398e.

Combined-stack local validation: SDK 121, macros 22, mapper 101, MLS 135, local planner 91, follower 63, V4L2 6 and depth2depth 12 Rust tests; all 24 native/DimSim/port-contract acceptance tests pass without skips. The mapper exchange includes Python CDR tf_static input; the follower exchange covers drive, arrival, a new goal and empty-path stop. Missing native binaries fail the dedicated job. RealSense passes cargo check --all-targets --features docs-only; device linking and hardware execution were not tested. Seven real source-staging layouts pass locked Cargo resolution. Native acceptance for tip cc463c163aed465f7c1d6808165bf640820b0182: workflow run. These are combined-tip results, with the SDK additionally tested standalone at its own layer.

Native Nix inputs use verified source archive f7723ad4d5b9aa5dfe63614b6b537d1ccc2d212e, SHA256 f90af8fa794a34d53ee9481c6d0bdb63138853b30f851d92960fd34224ff62b2. Full Nix builds remain unverified. SDK static edges persist after receipt; LCM/Zenoh do not acquire DDS transient-local durability from CDR. Existing Python mount publishers retain dynamic tf compatibility. Legacy modules remain explicitly pinned to SDK 0.1.0.

Stack #4776 preserves existing history and PR bases. Current main still has conflicts with the original stack; this change does not rebase onto main. No DimOS PR has been merged.

@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.01626% with 59 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/message_codegen/test_bundled.py 19.44% 57 Missing and 1 partial ⚠️
dimos/message_codegen/build.py 98.57% 0 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##             main    #4366      +/-   ##
==========================================
- Coverage   82.37%   82.33%   -0.04%     
==========================================
  Files        1674     1677       +3     
  Lines      156721   156940     +219     
  Branches    13144    13161      +17     
==========================================
+ Hits       129101   129220     +119     
- Misses      23965    24048      +83     
- Partials     3655     3672      +17     
Components Coverage Δ
Tests 96.78% <67.04%> (-0.10%) ⬇️
Flag Coverage Δ
OS-macos-latest 76.28% <76.01%> (-0.01%) ⬇️
OS-ubuntu-24.04-arm 76.05% <76.01%> (+<0.01%) ⬆️
OS-ubuntu-latest 76.39% <76.01%> (-0.01%) ⬇️
Py-3.10 ?
Py-3.11 76.40% <76.01%> (+<0.01%) ⬆️
Py-3.12 76.41% <76.01%> (-0.01%) ⬇️
SelfHosted-Large 32.14% <25.20%> (-0.02%) ⬇️
SelfHosted-Linux 40.71% <25.20%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/message_codegen/test_build.py 100.00% <100.00%> (ø)
dimos/message_codegen/test_integration.py 100.00% <ø> (ø)
dimos/message_codegen/build.py 98.57% <98.57%> (ø)
dimos/message_codegen/test_bundled.py 19.44% <19.44%> (ø)

... and 18 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TomCC7 TomCC7 changed the title feat: package native messages for external applications (2/7) feat: package native messages for external applications (2/8) Oct 2, 2026
@TomCC7
TomCC7 marked this pull request as ready for review October 21, 2026 00:17
@greptile-apps

greptile-apps Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The change reaches DimOS installs and message-package builders.

The confirmed problems affect maintainer generation and native packaging workflows and are non-blocking; neither establishes a security vulnerability.

Findings

  1. P2 Schema edits get left out ▶
  2. P2 Packaging fails on Python 3.10 ▶

Summary

Adds built-in CDR messages under dimos.messages while keeping dimos.msgs available.

  • DimOS installs built-in CDR messages under dimos.messages.
  • Installed message examples round-trip in Python and across native languages.
  • Message-codegen checks no longer run in a dedicated CI workflow.

The two maintainer-workflow issues do not block merging.

Reviews (1) · Last reviewed commit: "docs(messages): document complete built-..." · Reviewed by Greptile

Comment thread scripts/generate_builtin_messages.py Outdated
Comment thread scripts/package_messages.sh Outdated
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 21, 2026
Comment thread packages/dimos-generated/catalog.json Outdated
paul-nechifor
paul-nechifor previously approved these changes Oct 21, 2026
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Oct 21, 2026
@TomCC7
TomCC7 requested a review from paul-nechifor October 21, 2026 03:53
paul-nechifor
paul-nechifor previously approved these changes Oct 21, 2026
@TomCC7
TomCC7 force-pushed the cc/feat/cdr-message-packages branch from 9e0d7b3 to 0f3c1f6 Compare October 21, 2026 04:39
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 21, 2026

This branch was successfully deployed

1 active (outdated) deployment
cachix — 0f3c1f6c Deployed Oct 21, 2026 by TomCC7 via cachix-build #13726
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants