Repository navigation
Add LeRobot support (convert, train, infer, review) + SmolVLA, Train tab, config builder and settings page - #5
Merged
Conversation
added 8 commits
October 8, 2026 09:01
…ve container management - Updated replay_parity.py to allow dataset and checkpoint paths to be set via environment variables. - Added error handling for required arguments in replay_parity.py. - Modified pyproject.toml to relax numpy version constraints. - Improved Qt application initialization in app.py by sanitizing the Qt environment. - Refactored topic_containers.py to clean up comments and improve readability. - Replaced CvBridge usage with custom image conversion functions in container_visualizer.py and data_recorder.py. - Introduced container_manager.py to encapsulate container lifecycle management in the record page. - Enhanced demo_playback_reconstructor.py to use custom image message conversion. - Updated obs_builder.py to remove CvBridge dependency and utilize custom image decoding. - Added qt_env.py to manage OpenCV and PyQt5 environment conflicts.
…ataNormalizer class
…ties - Implement tests for output-format buttons to ensure single-choice toggle behavior. - Create end-to-end tests for converting recorded HDF5 data to LeRobotDataset format. - Add tests for the LeRobotDataset writer to validate data integrity and structure. - Introduce tests for the shared logger to verify output format and level control. - Develop tests for offline configuration loading to handle various scenarios and errors. - Implement tests for post-process options to ensure correct handling of configuration files. - Add tests for the RecordingController to validate button states and recording logic.
- Introduced tests for settings application, ensuring correct handling of ROS domain IDs and startup page defaults. - Added tests for settings consumers to validate that user settings are correctly applied and overridden by YAML configurations. - Implemented tests for the settings page, verifying UI interactions, saving behavior, and reset functionality. - Created tests for the training CLI, covering argument parsing, dry runs, and error handling. - Developed tests for the train page, focusing on dataset loading, parameter editing, and run initiation. - Added extensive tests for the training launcher, validating dataset detection, command generation, and run management. - Included tests for user settings to ensure defaults, validation, and migration are functioning as expected.
- Implemented `run_in_background` function for executing tasks in a separate thread while updating the GUI. - Created `reexec.py` to handle re-execution of the inference node in the correct Python environment. - Developed `envsetup.py` for managing dedicated Python environments for LeRobot and robomimic, including installation scripts and status checks. - Introduced `tf_filter.py` to filter out stale TF transforms and log warnings for ignored transforms. - Added unit tests for the configuration builder page, including saving and loading configurations, handling errors, and validating configurations. - Implemented tests for environment setup scripts, ensuring they handle existing environments and validate bash syntax. - Created tests for the first run folder selection and ROS catalog functionality, ensuring proper topic classification and message field extraction. - Added tests for user settings persistence and validation of configuration drafts, ensuring correct behavior of the settings page. - Included tests for URDF limits parsing and validation, ensuring correct extraction of joint limits and error handling.
…g and validation - Added _check_mapping method in ActionDispatcher to validate message mappings during initialization. - Enhanced dispatch logic to prevent sending non-finite values after transformations. - Updated ObsBuilder to utilize new utility functions for input synchronization and trigonometric operation stripping. - Introduced message encoding module to handle various ROS message types for DataRecorder. - Improved environment setup scripts to manage process states and prevent concurrent builds. - Added tests for new functionalities in ActionDispatcher, ObsBuilder, and message encoding.
- Added validation checks for normalized actions in test_config_draft.py. - Updated joint series saving logic to include empty rows in test_data_recorder_save.py. - Improved action template mappings and inference script execution in test_dispatch_templates.py. - Adjusted package list assertions in test_envsetup.py to include new dependencies. - Enhanced lerobot conversion tests to handle broken files and missing action streams in test_lerobot_conversion.py. - Added checks for task sentence handling in batch building in test_lerobot_runner.py. - Introduced new tests for action generation and mask splitting in test_numerics.py. - Updated postprocess options tests to ensure output folder naming aligns with dataset format in test_postprocess_options.py. - Improved recording controller tests to handle save failures and retries in test_recording_controller.py. - Enhanced session folder naming tests to include recording suffix in test_settings_consumers.py. - Added tests for fine-tuning SmolVLA from pretrained models in test_training_launcher.py. - Introduced new tests for AppAPI session handling and demo validation in test_app_api.py and test_demo_validator.py. - Added tests for inference session handling of stale inputs and rate warnings in test_inference_session.py.
|
pascd seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds LeRobot as a second end-to-end format next to robomimic: convert recordings to a LeRobot dataset, train ACT / Diffusion / SmolVLA from the new Train tab (or
relay-d-train), run the checkpoint withlfd-inference-node, and browse datasets in Review. Verified against LeRobot 0.6.1.Added
dispatch/lerobot_runner.py);--backend autopicks robomimic or LeRobot from the checkpoint. The custom backend (model_runner.py) is removed.lerobot/smolvla_base) with a task sentence at inference (--task, default from the dataset).relay-d-train: start/list/stop robomimic and LeRobot runs; per-format environments under~/.relayd/envs/built by a "Set up environment" button.acquisition_config.yamlfrom the live ROS graph, with a shared config check.utils/user_settings.py), also read by the CLI tools.relay-d-convert.stop: [...]inaction_config.yaml, sent once when inference stops.recording_frequencyfail validation.ros:jazzycontainer.Changed
action[t] = value[t+1])._Recording,_Robomimic,_Lerobot.inference_script.pyis a thin wrapper aroundlfd-inference-node.resources_rc.pyloses ~250k lines.Fixed (highlights)
AppAPI.shutdown()no longer loses an unsaved recording.in_min == in_maxscales stop startup instead of failing at runtime.TF_OLD_DATAwarning spam filtered.StopIterationon small datasets.pyarrow<21so LeRobot conversion works with ROS Jazzy's NumPy 1.26.Full list in
CHANGELOG.md→ [Unreleased].Upgrade notes
lerobot[smolvla]).relay_d.dispatch.DispatcherHelpersremoved → usesave_dispatch_templates(output_dir).lfd-inference-node --hzdefaults to the LeRobot dataset fps; for robomimic pass--hz <recording_frequency>.robotiq_grippermust mapposition: 0, not7.scipydependency,dispatchextra,--model-type,--obs-keys,--action-horizon,infer_backend: custom.Test plan
pytest(ROS-free) passespytestwith ROS sourced passes (CIros:jazzyjob)lfd-inference-nodeon the robot--task