evals: store patrol cases on dim-go2-sim's Go2 with lidar and camera, and --robot-on-host

[jeff-hykin]

Store patrol sandbox cases on the dim-go2-sim Go2 (lidar + camera), plus --robot-on-host to run the robot outside Docker.

[github-actions]

go2 replay realtime (arm64)

Details

Benchmark suite	Current: 0037169	Previous: 4bee180	Ratio
peak memory	1635.309 MB	1566.871 MB	1.04
peak threads	369 threads	364 threads	1.01
network (transport)	2724.695 MB	2730.605 MB	1.00
disk write	1.672 MB	1.449 MB	1.15
instructions	115.626 G	115.402 G	1.00

This comment was automatically generated by workflow using github-action-benchmark.

[codecov]

❌ 5 Tests Failed:

Tests completed	Failed	Passed	Skipped
6306	5	6301	85

View the top 3 failed test(s) by shortest run time

::dimos.evals.sandbox.test_go2_home

Stack Traces | 0s run time

ImportError while importing test module '.../evals/sandbox/test_go2_home.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../evals/sandbox/test_go2_home.py:22: in <module>
    from dimos.evals.sandbox.go2_home import (
.../evals/sandbox/go2_home.py:47: in <module>
    from dim_go2_sim import Go2Sim
E   ModuleNotFoundError: No module named 'dim_go2_sim'

::dimos.evals.sandbox.test_runner

Stack Traces | 0s run time

ImportError while importing test module '.../evals/sandbox/test_runner.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../evals/sandbox/test_runner.py:21: in <module>
    from dimos.evals.sandbox.go2_home import SUITE
.../evals/sandbox/go2_home.py:47: in <module>
    from dim_go2_sim import Go2Sim
E   ModuleNotFoundError: No module named 'dim_go2_sim'

dimos.robot.test_all_blueprints::test_blueprint_is_valid[sandbox-go2-home]

Stack Traces | 0.004s run time

blueprint_name = 'sandbox-go2-home'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x1692cb9d0>

    @pytest.mark.parametrize("blueprint_name", UBUNTU_BLUEPRINTS)
    def test_blueprint_is_valid(blueprint_name: str, monkeypatch: pytest.MonkeyPatch) -> None:
        """Validate blueprints that should import on the ubuntu-latest runner."""
        # The multi-robot blueprints read ROBOT_IPS at import time.
        monkeypatch.setattr(global_config, "robot_ips", "192.0.2.10,192.0.2.11")
>       _check_blueprint(blueprint_name)

blueprint_name = 'sandbox-go2-home'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x1692cb9d0>

dimos/robot/test_all_blueprints.py:94: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/robot/test_all_blueprints.py:75: in _check_blueprint
    blueprint = load_blueprint(blueprint_name)
        blueprint_name = 'sandbox-go2-home'
dimos/robot/get_all_blueprints.py:72: in load_blueprint
    return get_blueprint_by_name(name)
        name       = 'sandbox-go2-home'
dimos/robot/get_all_blueprints.py:65: in get_blueprint_by_name
    module = __import__(module_path, fromlist=[attr])
        attr       = 'sandbox_go2_home'
        module_path = 'dimos.evals.sandbox.go2_home'
        name       = 'sandbox-go2-home'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    # Copyright 2026 Dimensional Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    """Plant care, built from scratch by the agent: map a home, stop at each of its plants, then come
    back to the living room, while people walk about.
    
    The robot is dim-go2-sim's Go2, with its Mid-360 and front camera, in a scanned ranch home
    (SuperSplat 3f89bbd3, CC BY 4.0) with two people walking slow loops through it. The agent sees
    only plain zenoh topics (raw-robot-bridge); grading reads the simulator's ground truth, which
    stays in the robot container.
    
        dimos evals sandbox dimos.evals.sandbox.go2_home --robot-gpu --max-steps 2000
    
    The plants and the living room were found by eye in the scan: a top-down render of its splats
    and eye-height views from every room (dim_go2_sim.splat.SplatRenderer), then each plant placed
    at the middle of its green splats. Every plant has a spot within 0.75 m of it where the dog,
    0.25 m clear of anything, sees it at its own height; REACH allows for the dog's length. Not
    graded: a plant on the bathroom vanity (the doorway leaves the dog under 0.25 m either side,
    so the prompt's "skip it" covers it) and the cut flowers on the dining table.
    """
    
    from __future__ import annotations
    
    from collections.abc import Callable, Iterator
    from contextlib import AbstractContextManager, nullcontext
    from dataclasses import dataclass
    from functools import cache
    import json
    import math
    from pathlib import Path
    import shlex
    import subprocess
    import time
    from typing import TYPE_CHECKING
    
>   from dim_go2_sim import Go2Sim
E   ModuleNotFoundError: No module named 'dim_go2_sim'

AbstractContextManager = <class 'contextlib.AbstractContextManager'>
Callable   = <class 'collections.abc.Callable'>
Iterator   = <class 'collections.abc.Iterator'>
Path       = <class 'pathlib.Path'>
TYPE_CHECKING = False
__builtins__ = <builtins>
__cached__ = '.../sandbox/__pycache__/go2_home.cpython-311.pyc'
__doc__    = 'Plant care, built from scratch by the agent: map a home, stop at each of its plants, then come\nback to the living ro...es the dog under 0.25 m either side,\nso the prompt\'s "skip it" covers it) and the cut flowers on the dining table.\n'
__file__   = '.../evals/sandbox/go2_home.py'
__loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x13d1af9d0>
__name__   = 'dimos.evals.sandbox.go2_home'
__package__ = 'dimos.evals.sandbox'
__spec__   = ModuleSpec(name='dimos.evals.sandbox.go2_home', loader=<_frozen_importlib_external.SourceFileLoader object at 0x13d1af9d0>, origin='.../evals/sandbox/go2_home.py')
annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
cache      = <function cache at 0x106a70540>
dataclass  = <function dataclass at 0x106d2db20>
json       = <module 'json' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/json/__init__.py'>
math       = <module 'math' (built-in)>
nullcontext = <class 'contextlib.nullcontext'>
shlex      = <module 'shlex' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/shlex.py'>
subprocess = <module 'subprocess' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/subprocess.py'>
time       = <module 'time' (built-in)>

.../evals/sandbox/go2_home.py:47: ModuleNotFoundError

dimos.codebase_checks.test_blueprint_kwargs::test_blueprint_atom_kwargs_match_module_config[sandbox-go2-home]

Stack Traces | 0.005s run time

blueprint_name = 'sandbox-go2-home'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x172444990>

    @pytest.mark.parametrize("blueprint_name", _blueprint_params())
    def test_blueprint_atom_kwargs_match_module_config(
        blueprint_name: str, monkeypatch: pytest.MonkeyPatch
    ) -> None:
        """Fail when blueprint kwargs cannot be consumed by their target module."""
        # The multi-robot blueprints read ROBOT_IPS at import time.
        monkeypatch.setattr(global_config, "robot_ips", "192.0.2.10,192.0.2.11")
>       blueprint = _get_blueprint_or_skip(blueprint_name)

blueprint_name = 'sandbox-go2-home'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x172444990>

dimos/codebase_checks/test_blueprint_kwargs.py:85: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/codebase_checks/test_blueprint_kwargs.py:33: in _get_blueprint_or_skip
    return load_blueprint(blueprint_name)
        blueprint_name = 'sandbox-go2-home'
dimos/robot/get_all_blueprints.py:72: in load_blueprint
    return get_blueprint_by_name(name)
        name       = 'sandbox-go2-home'
dimos/robot/get_all_blueprints.py:65: in get_blueprint_by_name
    module = __import__(module_path, fromlist=[attr])
        attr       = 'sandbox_go2_home'
        module_path = 'dimos.evals.sandbox.go2_home'
        name       = 'sandbox-go2-home'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    # Copyright 2026 Dimensional Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    """Plant care, built from scratch by the agent: map a home, stop at each of its plants, then come
    back to the living room, while people walk about.
    
    The robot is dim-go2-sim's Go2, with its Mid-360 and front camera, in a scanned ranch home
    (SuperSplat 3f89bbd3, CC BY 4.0) with two people walking slow loops through it. The agent sees
    only plain zenoh topics (raw-robot-bridge); grading reads the simulator's ground truth, which
    stays in the robot container.
    
        dimos evals sandbox dimos.evals.sandbox.go2_home --robot-gpu --max-steps 2000
    
    The plants and the living room were found by eye in the scan: a top-down render of its splats
    and eye-height views from every room (dim_go2_sim.splat.SplatRenderer), then each plant placed
    at the middle of its green splats. Every plant has a spot within 0.75 m of it where the dog,
    0.25 m clear of anything, sees it at its own height; REACH allows for the dog's length. Not
    graded: a plant on the bathroom vanity (the doorway leaves the dog under 0.25 m either side,
    so the prompt's "skip it" covers it) and the cut flowers on the dining table.
    """
    
    from __future__ import annotations
    
    from collections.abc import Callable, Iterator
    from contextlib import AbstractContextManager, nullcontext
    from dataclasses import dataclass
    from functools import cache
    import json
    import math
    from pathlib import Path
    import shlex
    import subprocess
    import time
    from typing import TYPE_CHECKING
    
>   from dim_go2_sim import Go2Sim
E   ModuleNotFoundError: No module named 'dim_go2_sim'

AbstractContextManager = <class 'contextlib.AbstractContextManager'>
Callable   = <class 'collections.abc.Callable'>
Iterator   = <class 'collections.abc.Iterator'>
Path       = <class 'pathlib.Path'>
TYPE_CHECKING = False
__builtins__ = <builtins>
__cached__ = '.../sandbox/__pycache__/go2_home.cpython-311.pyc'
__doc__    = 'Plant care, built from scratch by the agent: map a home, stop at each of its plants, then come\nback to the living ro...es the dog under 0.25 m either side,\nso the prompt\'s "skip it" covers it) and the cut flowers on the dining table.\n'
__file__   = '.../evals/sandbox/go2_home.py'
__loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x13202dfd0>
__name__   = 'dimos.evals.sandbox.go2_home'
__package__ = 'dimos.evals.sandbox'
__spec__   = ModuleSpec(name='dimos.evals.sandbox.go2_home', loader=<_frozen_importlib_external.SourceFileLoader object at 0x13202dfd0>, origin='.../evals/sandbox/go2_home.py')
annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
cache      = <function cache at 0x109e7c540>
dataclass  = <function dataclass at 0x10a139b20>
json       = <module 'json' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/json/__init__.py'>
math       = <module 'math' (built-in)>
nullcontext = <class 'contextlib.nullcontext'>
shlex      = <module 'shlex' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/shlex.py'>
subprocess = <module 'subprocess' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/subprocess.py'>
time       = <module 'time' (built-in)>

.../evals/sandbox/go2_home.py:47: ModuleNotFoundError

dimos.hardware.sensors.lidar.pointlio.test_wiring::test_every_pointlio_input_has_one_producer

Stack Traces | 0.012s run time

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x3cabb90d0>

    def test_every_pointlio_input_has_one_producer(monkeypatch: pytest.MonkeyPatch) -> None:
        monkeypatch.setattr(global_config, "robot_ips", "192.0.2.10,192.0.2.11")
        checked = []
        skipped = []
        for name in UBUNTU_BLUEPRINTS:
            try:
>               blueprint = load_blueprint(name)

atom       = BlueprintAtom(kwargs={'frame_id': 'world'}, module=<class 'dimos.hardware.sensors.lidar.pointlio.module.PointLio'>, st...me='tf', type=<class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>, direction='out')), module_refs=(), instance_name=None)
blueprint  = Blueprint(blueprints=(BlueprintAtom(kwargs={'blueprint': <function _layout at 0x17286a200>, 'pubsubs': [<dimos.protoco...=mappingproxy({}), remapping_map=mappingproxy({}), requirement_checks=(), configurator_checks=(), hosted_placements=())
checked    = ['alfred-keyboard-teleop', 'demo-virtual-mid360-pointlio', 'go2-dds-nav', 'mid360-pointlio', 'mid360-pointlio-ray-trace', 'mid360-pointlio-replay', ...]
driver     = BlueprintAtom(kwargs={'frame_id': 'mid360_link', 'point_format': 'full'}, module=<class 'dimos.hardware.sensors.lidar...., type=<class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>, direction='out')), module_refs=(), instance_name=None)
driver_frame = 'mid360_link'
lio        = []
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x3cabb90d0>
name       = 'sandbox-go2-home'
names      = ['mid360.imu_raw']
port       = 'imu_raw'
producers  = [BlueprintAtom(kwargs={'frame_id': 'mid360_link', 'point_format': 'full'}, module=<class 'dimos.hardware.sensors.lidar... type=<class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>, direction='out')), module_refs=(), instance_name=None)]
sensor_frame = 'mid360_link'
skipped    = ['demo-object-scene-registration (ZED SDK not installed. Please install pyzed package to use ZED camera functionality.)']

.../lidar/pointlio/test_wiring.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/robot/get_all_blueprints.py:72: in load_blueprint
    return get_blueprint_by_name(name)
        name       = 'sandbox-go2-home'
dimos/robot/get_all_blueprints.py:65: in get_blueprint_by_name
    module = __import__(module_path, fromlist=[attr])
        attr       = 'sandbox_go2_home'
        module_path = 'dimos.evals.sandbox.go2_home'
        name       = 'sandbox-go2-home'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    # Copyright 2026 Dimensional Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #     http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    
    """Plant care, built from scratch by the agent: map a home, stop at each of its plants, then come
    back to the living room, while people walk about.
    
    The robot is dim-go2-sim's Go2, with its Mid-360 and front camera, in a scanned ranch home
    (SuperSplat 3f89bbd3, CC BY 4.0) with two people walking slow loops through it. The agent sees
    only plain zenoh topics (raw-robot-bridge); grading reads the simulator's ground truth, which
    stays in the robot container.
    
        dimos evals sandbox dimos.evals.sandbox.go2_home --robot-gpu --max-steps 2000
    
    The plants and the living room were found by eye in the scan: a top-down render of its splats
    and eye-height views from every room (dim_go2_sim.splat.SplatRenderer), then each plant placed
    at the middle of its green splats. Every plant has a spot within 0.75 m of it where the dog,
    0.25 m clear of anything, sees it at its own height; REACH allows for the dog's length. Not
    graded: a plant on the bathroom vanity (the doorway leaves the dog under 0.25 m either side,
    so the prompt's "skip it" covers it) and the cut flowers on the dining table.
    """
    
    from __future__ import annotations
    
    from collections.abc import Callable, Iterator
    from contextlib import AbstractContextManager, nullcontext
    from dataclasses import dataclass
    from functools import cache
    import json
    import math
    from pathlib import Path
    import shlex
    import subprocess
    import time
    from typing import TYPE_CHECKING
    
>   from dim_go2_sim import Go2Sim
E   ModuleNotFoundError: No module named 'dim_go2_sim'

AbstractContextManager = <class 'contextlib.AbstractContextManager'>
Callable   = <class 'collections.abc.Callable'>
Iterator   = <class 'collections.abc.Iterator'>
Path       = <class 'pathlib.Path'>
TYPE_CHECKING = False
__builtins__ = <builtins>
__cached__ = '.../sandbox/__pycache__/go2_home.cpython-311.pyc'
__doc__    = 'Plant care, built from scratch by the agent: map a home, stop at each of its plants, then come\nback to the living ro...es the dog under 0.25 m either side,\nso the prompt\'s "skip it" covers it) and the cut flowers on the dining table.\n'
__file__   = '.../evals/sandbox/go2_home.py'
__loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x3ca9e34d0>
__name__   = 'dimos.evals.sandbox.go2_home'
__package__ = 'dimos.evals.sandbox'
__spec__   = ModuleSpec(name='dimos.evals.sandbox.go2_home', loader=<_frozen_importlib_external.SourceFileLoader object at 0x3ca9e34d0>, origin='.../evals/sandbox/go2_home.py')
annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
cache      = <function cache at 0x109e7c540>
dataclass  = <function dataclass at 0x10a139b20>
json       = <module 'json' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/json/__init__.py'>
math       = <module 'math' (built-in)>
nullcontext = <class 'contextlib.nullcontext'>
shlex      = <module 'shlex' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/shlex.py'>
subprocess = <module 'subprocess' from '.../_temp/uv-python-dir/cpython-3.11.17-macos-aarch64-none/lib/python3.11/subprocess.py'>
time       = <module 'time' (built-in)>

.../evals/sandbox/go2_home.py:47: ModuleNotFoundError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}