dora-rs
/
dora
Not watched
1
0
Fork 0
Code
Releases 124 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
Browse Source

Improving overall readme

tags/test-git
haixuantao 10 months ago
parent
de21dc1241
commit
9406d3e892
5 changed files with 253 additions and 50 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +159
    -49
      README.md
  2. +40
    -1
      examples/llm/README.md
  3. +9
    -0
      examples/object-detection/README.md
  4. +27
    -0
      examples/object-detection/yolo.yml
  5. +18
    -0
      examples/speech-to-speech/README.md

+ 159
- 49
README.md View File

@@ -55,9 +55,10 @@ An extremely fast and simple **dataflow oriented robotic** framework to manage y
<details open>
<summary><b>2025</b></summary>

- \[2025/03\] Add support for Meta SAM2, Kokoro(TTS), Improved Qwen2.5 Performance using `llama.cpp`.
- \[2025/02\] Add support for Qwen2.5(LLM), Qwen2.5-VL(VLM), outetts(TTS)

- \[2025/03/05\] dora-rs has been accepted to [**GSoC 2025 🎉**](https://summerofcode.withgoogle.com/programs/2025/organizations/dora-rs-tb), with the following [**idea list**](https://github.com/dora-rs/dora/wiki/GSoC_2025).
- \[2025/03/04\] Add support for Zenoh for distributed dataflow.
- \[2025/03/04\] Add support for Meta SAM2, Kokoro(TTS), Improved Qwen2.5 Performance using `llama.cpp`.
- \[2025/02/25\] Add support for Qwen2.5(LLM), Qwen2.5-VL(VLM), outetts(TTS)
</details>

## Highlights
@@ -72,16 +73,146 @@ An extremely fast and simple **dataflow oriented robotic** framework to manage y
- 🤖 Simplifies building robotic applications by integrating hardware, algorithms, and AI models to facilitate seamless communication.
- ⚙️ Eases integration of hardware and software by supporting Python, C, C++, and ROS2, while ensuring low-latency communication with zero-copy Arrow messages.

## Installation
## Packaged Nodes

Install dora with our standalone installers, or from [crates.io](https://crates.io/crates/dora-cli):
### Camera

| Title | Description | Shields |
| ---------------------------------------------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [PyOrbbeckSDK](https://github.com/dora-rs/dora/blob/main/node-hub/dora-pyorbbecksdk) | Image and depth from Orbbeck Camera | ![Downloads](https://img.shields.io/pypi/dm/dora-pyorbbecksdk) ![License](https://img.shields.io/pypi/l/dora-pyorbbecksdk) ![Release](https://img.shields.io/pypi/v/dora-pyorbbecksdk) |
| [PyRealsense](https://github.com/dora-rs/dora/blob/main/node-hub/dora-pyrealsense) | Image and depth from Realsense | ![Downloads](https://img.shields.io/pypi/dm/dora-pyrealsense) ![License](https://img.shields.io/pypi/l/dora-pyrealsense) ![Release](https://img.shields.io/pypi/v/dora-pyrealsense) |
| [Video Capture](https://github.com/dora-rs/dora/blob/main/node-hub/opencv-video-capture) | Image stream from Camera | ![Downloads](https://img.shields.io/pypi/dm/opencv-video-capture) ![License](https://img.shields.io/pypi/l/opencv-video-capture) ![Release](https://img.shields.io/pypi/v/opencv-video-capture) |

### Peripheral

| Title | Description | Shields |
| ----------------------------------------------------------------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Keyboard](https://github.com/dora-rs/dora/blob/main/node-hub/dora-keyboard) | Keyboard char listener | ![Downloads](https://img.shields.io/pypi/dm/dora-keyboard) ![License](https://img.shields.io/pypi/l/dora-keyboard) ![Release](https://img.shields.io/pypi/v/dora-keyboard) |
| [Microphone](https://github.com/dora-rs/dora/blob/main/node-hub/dora-microphone) | Audio from microphone | ![Downloads](https://img.shields.io/pypi/dm/dora-microphone) ![License](https://img.shields.io/pypi/l/dora-microphone) ![Release](https://img.shields.io/pypi/v/dora-microphone) |
| [PyAudio(Speaker)](https://github.com/dora-rs/dora/blob/main/node-hub/dora-pyaudio) | Output audio from speaker | ![Downloads](https://img.shields.io/pypi/dm/dora-pyaudio) ![License](https://img.shields.io/pypi/l/dora-pyaudio) ![Release](https://img.shields.io/pypi/v/dora-pyaudio) |

### Actuator

| Title | Description | Shields |
| ---------------------------------------------------------------------------------------- | ---------------- | ------- |
| [Feetech](https://github.com/dora-rs/dora-lerobot/blob/main/node-hub/feetech-client) | Feetech Client | |
| [Dynamixel](https://github.com/dora-rs/dora-lerobot/blob/main/node-hub/dynamixel-client) | Dynamixel Client | |

### Chassis

| Title | Description | Shields |
| ------------------------------------------------------------------------------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Agilex - UGV](https://github.com/dora-rs/dora/blob/main/node-hub/dora-ugv) | Robomaster Client | ![Downloads](https://img.shields.io/pypi/dm/dora-ugv) ![License](https://img.shields.io/pypi/l/dora-ugv) ![Release](https://img.shields.io/pypi/v/dora-ugv) |
| [DJI - Robomaster S1](https://huggingface.co/datasets/dora-rs/dora-robomaster) | Robomaster Client | |
| [Dora Kit Car](https://github.com/dora-rs/dora/blob/main/node-hub/dora-kit-car) | Open Source Chassis | ![Downloads](https://img.shields.io/pypi/dm/dora-kit-car) ![License](https://img.shields.io/pypi/l/dora-kit-car) ![Release](https://img.shields.io/pypi/v/dora-kit-car) |

### Arm

| Title | Description | Shields |
| ------------------------------------------------------------------------------------------------ | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Alex Koch - Low Cost Robot](https://github.com/dora-rs/dora-lerobot/blob/main/robots/alexk-lcr) | Alex Koch - Low Cost Robot Client | |
| [Lebai - LM3](https://github.com/dora-rs/dora-lerobot/blob/main/node-hub/lebai-client) | Lebai client | |
| [Agilex - Piper](https://github.com/dora-rs/dora/blob/main/node-hub/dora-piper) | Agilex arm client | ![Downloads](https://img.shields.io/pypi/dm/dora-piper) ![License](https://img.shields.io/pypi/l/dora-piper) ![Release](https://img.shields.io/pypi/v/dora-piper) |

### Robot

| Title | Description | Shields |
| -------------------------------------------------------------------------------------------- | --------------- | ------- |
| [Pollen - Reachy 1](https://github.com/dora-rs/dora-lerobot/blob/main/node-hub/dora-reachy1) | Reachy 1 Client | |
| [Pollen - Reachy 2](https://github.com/dora-rs/dora-lerobot/blob/main/robots/reachy) | Reachy 2 client | |
| [Trossen - Aloha](https://github.com/dora-rs/dora-lerobot/blob/main/robots/aloha) | Aloha client | |

### Voice Activity Detection

| Title | Description | Shields |
| ------------------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Silero VAD](https://github.com/dora-rs/dora/blob/main/node-hub/dora-vad) | Silero Voice activity detection | ![Downloads](https://img.shields.io/pypi/dm/dora-vad) ![License](https://img.shields.io/pypi/l/dora-vad) ![Release](https://img.shields.io/pypi/v/dora-vad) |

### Speech to Text

| Title | Description | Shields |
| --------------------------------------------------------------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Whisper](https://github.com/dora-rs/dora/blob/main/node-hub/dora-distil-whisper) | Transcribe audio to text | ![Downloads](https://img.shields.io/pypi/dm/dora-distil-whisper) ![License](https://img.shields.io/pypi/l/dora-distil-whisper) ![Release](https://img.shields.io/pypi/v/dora-distil-whisper) |

### Vision Language Model

| Title | Description | Shields |
| -------------------------------------------------------------------------------- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Qwen2.5-vl](https://github.com/dora-rs/dora/blob/main/node-hub/dora-qwen2-5-vl) | Vision Language Model using Qwen2.5 VL | ![Downloads](https://img.shields.io/pypi/dm/dora-qwen2-5-vl) ![License](https://img.shields.io/pypi/l/dora-qwen2-5-vl) ![Release](https://img.shields.io/pypi/v/dora-qwen2-5-vl) |
| [InternVL](https://github.com/dora-rs/dora/blob/main/node-hub/dora-internvl) | InternVL is a vision language model | ![Downloads](https://img.shields.io/pypi/dm/dora-internvl) ![License](https://img.shields.io/pypi/l/dora-internvl) ![Release](https://img.shields.io/pypi/v/dora-internvl) |

### Large Language Model

| Title | Description | Shields |
| ----------------------------------------------------------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Qwen2.5](https://github.com/dora-rs/dora/blob/main/node-hub/dora-qwen) | Large Language Model using Qwen | ![Downloads](https://img.shields.io/pypi/dm/dora-qwen) ![License](https://img.shields.io/pypi/l/dora-qwen) ![Release](https://img.shields.io/pypi/v/dora-qwen) |

### Vision Language Action

| Title | Description | Shields |
| ------------------------------------------------------------------------ | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [RDT-1B](https://github.com/dora-rs/dora/blob/main/node-hub/dora-rdt-1b) | Infer policy using Robotic Diffusion Transformer | ![Downloads](https://img.shields.io/pypi/dm/dora-rdt-1b) ![License](https://img.shields.io/pypi/l/dora-rdt-1b) ![Release](https://img.shields.io/pypi/v/dora-rdt-1b) |

### Object Detection

| Title | Description | Shields |
| ---------------------------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Yolov8](https://github.com/dora-rs/dora/blob/main/node-hub/dora-yolo) | Object detection | ![Downloads](https://img.shields.io/pypi/dm/dora-yolo) ![License](https://img.shields.io/pypi/l/dora-yolo) ![Release](https://img.shields.io/pypi/v/dora-yolo) |

### Segmentation

| Title | Description | Shields |
| -------------------------------------------------------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [SAM2](https://github.com/dora-rs/dora/blob/main/node-hub/dora-sam2) | Segment Anything | ![Downloads](https://img.shields.io/pypi/dm/dora-sam2) ![License](https://img.shields.io/pypi/l/dora-sam2) ![Release](https://img.shields.io/pypi/v/dora-sam2) |

### Translation

| Title | Description | Shields |
| --------------------------------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [ArgosTranslate](https://github.com/dora-rs/dora/blob/main/node-hub/dora-argotranslate) | Open Source translation engine | ![Downloads](https://img.shields.io/pypi/dm/dora-argotranslate) ![License](https://img.shields.io/pypi/l/dora-argotranslate) ![Release](https://img.shields.io/pypi/v/dora-argotranslate) |
| [Opus MT](https://github.com/dora-rs/dora/blob/main/node-hub/dora-opus) | Translate text between language | ![Downloads](https://img.shields.io/pypi/dm/dora-opus) ![License](https://img.shields.io/pypi/l/dora-opus) ![Release](https://img.shields.io/pypi/v/dora-opus) |

### With pip
### Text to Speech

| Title | Description | Shields |
| -------------------------------------------------------------------------------- | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Kokoro TTS](https://github.com/dora-rs/dora/blob/main/node-hub/dora-kokoro-tts) | Efficient Text to Speech | ![Downloads](https://img.shields.io/pypi/dm/dora-kokoro-tts) ![License](https://img.shields.io/pypi/l/dora-kokoro-tts) ![Release](https://img.shields.io/pypi/v/dora-kokoro-tts) |

### Recorder

| Title | Description | Shields |
| --------------------------------------------------------------------------------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Llama Factory Recorder](https://github.com/dora-rs/dora/blob/main/node-hub/llama-factory-recorder) | Record data to train LLM and VLM | ![Downloads](https://img.shields.io/pypi/dm/llama-factory-recorder) ![License](https://img.shields.io/pypi/l/llama-factory-recorder) ![Release](https://img.shields.io/pypi/v/llama-factory-recorder) |
| [LeRobot Recorder](https://github.com/dora-rs/dora-lerobot/blob/main/node-hub/lerobot-dashboard) | LeRobot Recorder helper | |

### Visualization

| Title | Description | Shields |
| ---------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [Plot](https://github.com/dora-rs/dora/blob/main/node-hub/opencv-plot) | Simple OpenCV plot visualization | ![Downloads](https://img.shields.io/pypi/dm/dora-yolo) ![License](https://img.shields.io/pypi/l/opencv-plot) ![Release](https://img.shields.io/pypi/v/opencv-plot) |
| [Rerun](https://github.com/dora-rs/dora/blob/main/node-hub/dora-rerun) | Visualization tool | ![Downloads](https://img.shields.io/pypi/dm/dora-rerun) ![License](https://img.shields.io/pypi/l/dora-rerun) ![Release](https://img.shields.io/pypi/v/dora-rerun) |

### Simulator

| Title | Description | Shields |
| ---------------------------------------------------------------------------------- | ------------------------------------ | ------- |
| [Mujoco](https://github.com/dora-rs/dora-lerobot/blob/main/node-hub/mujoco-client) | Mujoco Simulator | |
| [Carla](https://github.com/dora-rs/dora-drives) | Carla Simulator | |
| [Gymnasium](https://github.com/dora-rs/dora-lerobot/blob/main/gym_dora) | Experimental OpenAI Gymnasium bridge | |

## Quick Start

### Installation

```bash
pip install dora-rs-cli
```

<details open>
<summary><b>Additional installation methods</b></summary>

Install dora with our standalone installers, or from [crates.io](https://crates.io/crates/dora-cli):

### With cargo

```bash
@@ -100,6 +231,17 @@ curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/dora-rs/d
powershell -c "irm https://raw.githubusercontent.com/dora-rs/dora/main/install.ps1 | iex"
```

### With Source

```bash
git clone https://github.com/dora-rs/dora.git
cd dora
cargo build --release -p dora-cli
PATH=$PATH:$(pwd)/target/release
```

</details>

## Documentation

The full documentation is available on [our website](https://dora-rs.ai/).
@@ -107,14 +249,12 @@ A lot of guides are available on [this section](https://dora-rs.ai/docs/guides/)

## Getting Started

1. Run some Python examples (A venv must be activated):
1. Run some Python examples:

```bash
cd examples/python-dataflow
uv venv --seed
uv pip install -e ../../apis/python/node --reinstall
dora build dataflow.yml --uv
dora run dataflow.yml --uv
uv venv --seed -p 3.11
dora build https://raw.githubusercontent.com/dora-rs/dora/refs/heads/main/examples/object-detection/yolo.yml --uv
dora run yolo.yml --uv
```

> Make sure to have a webcam
@@ -176,16 +316,6 @@ dora-rs uses Opentelemetry to record all your logs, metrics and traces. This mea

Opentelemetry is language independent, backend agnostic, and easily collect distributed data, making it perfect for dora-rs applications.

### Hot-Reloading

dora-rs implements Hot-Reloading for python which means you can change code at runtime in Python while keeping your state intact.

Using the feature flag: `--attach --hot-reload`, dora-rs watch for code change and reload nodes that has been modified.

You can check fail-safe mechanism at: https://github.com/dora-rs/dora/pull/239.

See [this demo](http://www.youtube.com/watch?v=NvvTEP8Jak8).

### ROS2 Bridge

**Note**: this feature is marked as unstable.
@@ -236,36 +366,16 @@ Self-Coding Robot is just the tip of the iceberg of robotics combined with llm,
- [memory](https://github.com/cpacker/MemGPT)
- [function calling](https://github.com/ShishirPatil/gorilla)

### Cool Hardwares

Cool hardware that we think might be good fit to try out dora-rs 🙋 We are not sponsored by manufacturers:

| | Price | Open Source | Github | type | Dora Project |
| --------------------------------- | ----- | ------------------ | ---------------------------------------------------- | ---------- | ------------------------------------------------------- |
| DJI Robomaster S1 | 550$ | SDK | https://github.com/dji-sdk/RoboMaster-SDK | Rover | https://huggingface.co/datasets/dora-rs/dora-robomaster |
| DJI Robomaster EP Core | 950$ | SDK | https://github.com/dji-sdk/RoboMaster-SDK | Rover, Arm | |
| DJI Tello | 100$ | | | Drone | |
| BitCraze Crazyflies | 225$ | Firmware, Lib, SDK | https://github.com/bitcraze | Drone | |
| AlexanderKoch-Koch/low_cost_robot | 250$ | Everything | https://github.com/AlexanderKoch-Koch/low_cost_robot | Arm | |
| xArm 1S | 200$ | | | Arm | |
| Wavego | 250$ | | | Quadruplet | |
| AINex | 800$ | | | Humanoid | |

> For more: https://docs.google.com/spreadsheets/d/1YYeW2jfOIWDVgdEgqnMvltonHquQ7K8OZCrnJRELL6o/edit#gid=0

## Support Matrix

| | dora-rs | Hoped for |
| --------------------------------- | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| **Tier 1 Support** | Python, Rust | C, C++, ROS 2 |
| | dora-rs | Hoped for |
| --------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| **Tier 1 Support** | Python, Rust | C, C++, ROS 2 |
| **Tier 2 Support** | C, C++, ROS2 |
| **Hot-reloading** | Python | Rust (https://github.com/orgs/dora-rs/discussions/360) |
| **Message Format** | Arrow | Native |
| **Local Communication** | Shared Memory | Custom Middleware, [zero-copy GPU IPC](https://arrow.apache.org/docs/python/api/cuda.html), intra-process `tokio::channel` communication |
| **Remote Communication** | TCP | Custom Middleware, [Zenoh](https://zenoh.io/) |
| **Metrics, Tracing, and Logging** | Opentelemetry | Native logging libraries into Opentelemetry |
| **Data archives** | Parquet ([dora-record](https://github.com/dora-rs/dora/tree/main/node-hub/dora-record)) |
| **Visualization and annotation** | OpenCV | [rerun.io](rerun.io) |
| **Message Format** | Arrow | Native |
| **Local Communication** | Shared Memory, [Cuda zero-copy IPC](https://arrow.apache.org/docs/python/api/cuda.html) | Custom Middleware, intra-process `tokio::channel` communication |
| **Remote Communication** | [Zenoh](https://zenoh.io/) | Custom Middleware |
| **Metrics, Tracing, and Logging** | Opentelemetry | Native logging libraries into Opentelemetry |
| **Supported Platforms (x86)** | Windows, macOS, Linux |
| **Supported Platforms (ARM)** | macOS, Linux |
| **Configuration** | YAML |


+ 40
- 1
examples/llm/README.md View File

@@ -2,10 +2,49 @@

Make sure to have, dora, pip and cargo installed.

## System dependencies:

- MacOS

```bash
brew install portaudio
brew install espeak-ng
```

- Linux

```bash
# Make sure to not be in a virtual environment
sudo apt-get install portaudio19-dev
sudo apt-get install espeak
```

## Installation

```bash
# Make sure to have the latest the development dora version
cargo build -p dora-cli --release
## Alternatively
# uv pip install -e ../../binaries/cli --reinstall

uv venv --seed -p 3.11
# Make sure to have the latest the development dora node api version
uv pip install -e ../../apis/python/node --reinstall
dora build qwen-dev.yml --uv
dora run qwen-dev.yml --uv
```

## With prepackaged version

```bash
pip install dora-rs-cli==3.10
uv venv --seed -p 3.11
# Make sure to have the latest the development dora node api version
dora build qwen-dev.yml --uv
dora run qwen-dev.yml --uv
```

## To run

```bash
dora run qwen-dev.yml --uv
```

+ 9
- 0
examples/object-detection/README.md View File

@@ -0,0 +1,9 @@
## Yolo Example

To run:

```bash
uv venv --seed -p 3.11
dora build https://raw.githubusercontent.com/dora-rs/dora/refs/heads/main/examples/object-detection/yolo.yml --uv
dora run yolo.yml --uv
```

+ 27
- 0
examples/object-detection/yolo.yml View File

@@ -0,0 +1,27 @@
nodes:
- id: camera
build: pip install opencv-video-capture
path: opencv-video-capture
inputs:
tick: dora/timer/millis/20
outputs:
- image
env:
CAPTURE_PATH: 0
IMAGE_WIDTH: 640
IMAGE_HEIGHT: 480

- id: object-detection
build: pip install dora-yolo
path: dora-yolo
inputs:
image: camera/image
outputs:
- bbox

- id: plot
build: pip install dora-rerun
path: dora-rerun
inputs:
image: camera/image
boxes2d: object-detection/bbox

+ 18
- 0
examples/speech-to-speech/README.md View File

@@ -2,6 +2,24 @@

Make sure to have, dora, pip and cargo installed.

## System dependencies:

- MacOS

```bash
brew install portaudio
brew install espeak-ng
```

- Linux

```bash
sudo apt-get install portaudio19-dev
sudo apt-get install espeak
```

## To Run:

```bash
uv venv --seed -p 3.11
uv pip install -e ../../apis/python/node --reinstall


Write Preview
Loading…
Cancel
Save