Browse Source

updated readme, restructured and renamed the files

tags/v0.3.12-rc0
ShashwatPatil 7 months ago
parent
commit
a079bde6a2
19 changed files with 308 additions and 1133 deletions
  1. +0
    -225
      examples/franka-mujoco-control/02_target_pose_control/README.md
  2. +0
    -280
      examples/franka-mujoco-control/03_gamepad_control/README.md
  3. +0
    -139
      examples/franka-mujoco-control/04_gamepad_collision_avoidance/README.md
  4. +0
    -35
      examples/franka-mujoco-control/04_gamepad_collision_avoidance/collision_avoidance.yml
  5. +0
    -366
      examples/franka-mujoco-control/04_gamepad_collision_avoidance/nodes/franka_collision_controller.py
  6. +0
    -46
      examples/franka-mujoco-control/README.md
  7. +21
    -0
      examples/mujoco-sim/README.md
  8. +0
    -0
      examples/mujoco-sim/URDF/franka_panda/panda.urdf
  9. +0
    -0
      examples/mujoco-sim/URDF/kuka_iiwa/model.urdf
  10. +4
    -10
      examples/mujoco-sim/basic_simulation/README.md
  11. +1
    -1
      examples/mujoco-sim/basic_simulation/basic.yml
  12. +75
    -0
      examples/mujoco-sim/gamepad_control/README.md
  13. +6
    -6
      examples/mujoco-sim/gamepad_control/gamepad_control.yml
  14. +5
    -6
      examples/mujoco-sim/gamepad_control/nodes/gamepad_controller.py
  15. +99
    -0
      examples/mujoco-sim/target_pose_control/README.md
  16. +0
    -0
      examples/mujoco-sim/target_pose_control/nodes/controller.py
  17. +0
    -0
      examples/mujoco-sim/target_pose_control/nodes/pose_publisher.py
  18. +7
    -7
      examples/mujoco-sim/target_pose_control/target_pose_control.yml
  19. +90
    -12
      node-hub/gamepad/README.md

+ 0
- 225
examples/franka-mujoco-control/02_target_pose_control/README.md View File

@@ -1,225 +0,0 @@
# 02. Target Pose Control

This example demonstrates Cartesian space control by creating a dedicated Franka controller node that processes target pose commands and outputs joint commands using inverse kinematics with **PyTorch Kinematics**.

## Running the Example

```bash
cd 02_target_pose_control
dora build target_pose_control_pytorch.yml
dora run target_pose_control_pytorch.yml
```

You should see:
1. Robot moves to predefined target poses automatically
2. Smooth Cartesian space motion with differential inverse kinematics
3. End-effector following target positions accurately
4. GPU-accelerated kinematics computation (if CUDA available)

## Interactive Control

While the simulation is running, you can send custom poses:

```python
# In another terminal
python3 -c "
import pyarrow as pa
from dora import Node
import time

node = Node()

# Move to position (0.5, 0.2, 0.6) with downward orientation
target_pose = [0.5, 0.2, 0.6, 180.0, 0.0, 90.0] # x, y, z, roll, pitch, yaw

node.send_output(
'target_pose',
pa.array(target_pose, type=pa.float64()),
metadata={'timestamp': time.time()}
)
"
```

## How It Works

### Node Breakdown

#### 1. **Pose Publisher Node** (`pose_publisher.py`)
```python
class PosePublisher:
def __init__(self):
# Predefined sequence of target poses [x, y, z, roll, pitch, yaw]
self.target_poses = [
[0.5, 0.5, 0.3, 180.0, 0.0, 90.0], # Position + RPY orientation
[0.6, 0.2, 0.5, 180.0, 0.0, 45.0], # Different orientation
# ... more poses
]
```

**What it does**:
- Sends target poses every 10 seconds
- Cycles through predefined positions and orientations
- Can be replaced with path planning, user input, or any pose generation logic
- **Output**: `target_pose` array `[x, y, z, roll, pitch, yaw]`

#### 2. **Franka Controller Node** (`franka_controller_pytorch.py`)

**Key Components**:

```python
class FrankaController:
def __init__(self):
urdf_path = "path to the file/panda.urdf"
with open(urdf_path, 'rb') as f:
urdf_content = f.read()
self.chain = pk.build_serial_chain_from_urdf(urdf_content, "panda_hand")
self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
self.chain = self.chain.to(device=self.device)
```

**What it does**:
- **Input**: Target poses `[x, y, z, roll, pitch, yaw]` and current joint positions from MuJoCo
- **Processing**: Differential inverse kinematics using PyTorch Kinematics
- **Output**: Joint position commands for the robot

**Control Algorithm (Differential IK)**:
```python
def apply_differential_ik_control(self):
# 1. Get current end-effector pose using PyTorch Kinematics FK
current_ee_pos, current_ee_rot = self.get_current_ee_pose(self.current_joint_pos)
# 2. Calculate position and orientation errors
pos_error = self.target_pos - current_ee_pos
rot_error = (desired_rot * current_rot.inv()).as_rotvec()
# 3. Create 6D twist vector [linear_vel, angular_vel]
twist = np.zeros(6)
twist[:3] = self.Kpos * pos_error / self.integration_dt
twist[3:] = self.Kori * rot_error / self.integration_dt
# 4. Compute Jacobian using PyTorch Kinematics
jac = self.compute_jacobian_pytorch(self.current_joint_pos) # (6, 7)
# 5. Solve differential IK with damped least squares
diag = self.damping * np.eye(6)
dq = jac.T @ np.linalg.solve(jac @ jac.T + diag, twist)
# 6. Add nullspace control to prefer home position
N = np.eye(7) - np.linalg.pinv(jac) @ jac # Nullspace projection
dq_null = self.Kn * (self.home_pos - self.current_joint_pos)
dq += N @ dq_null
# 7. Integrate to get new joint positions
new_joints = self.current_joint_pos + dq * self.integration_dt
```

#### 3. **MuJoCo Simulation Node** (`dora-mujoco`)
- **Input**: Joint commands from controller
- **Processing**: Physics simulation, rendering, forward kinematics
- **Output**: Joint positions, velocities, sensor data

## Technical Implementation Details

- **Main Simulation** (`dora-mujoco` node): Physics, rendering, joint state
- **Controller** (`franka_controller_pytorch.py`): PyTorch Kinematics for FK/Jacobian
- **Single source of truth**: MuJoCo simulation provides all joint states / sensor feedback in case of hardware

```python
class FrankaController:
def __init__(self):
# Load kinematics model for computation only
self.chain = pk.build_serial_chain_from_urdf(urdf_content, "panda_hand")
self.chain = self.chain.to(device=self.device) # GPU acceleration
def get_current_ee_pose(self, joint_angles):
"""PyTorch Kinematics FK"""
q = torch.tensor(joint_angles, device=self.device, dtype=torch.float32).unsqueeze(0)
tf = self.chain.forward_kinematics(q)
# ... extract position and rotation
```

### Key Advantages

**Performance Benefits**:
- **GPU Acceleration**: PyTorch Kinematics can leverage CUDA for faster computation
- **Optimized Gradients**: Built-in automatic differentiation


**Control Benefits**:
- **Differential IK**: Smoother motion than discrete IK solving
- **Nullspace Control**: Avoids joint limits and singularities
- **Real-time Performance**: currently 500Hz control loops

### Differential vs. Analytical IK

**Differential IK** (Current Implementation):
```python
# Compute small joint movements based on end-effector error
dq = jac.T @ np.linalg.solve(jac @ jac.T + diag, twist)
new_joints = current_joints + dq * dt
```

**Analytical IK** (Alternative):
```python
# Solve for exact joint configuration to reach target [sometimes exact solution is not available can result in `None return`]
ik_solver = pk.PseudoInverseIK(chain, ...)
solution = ik_solver.solve(target_transform)
```

**Why Differential IK**:
- **Smoother motion**: Continuous trajectory without jumps
- **Better convergence**: Less likely to get stuck in local minima
- **Singularity handling**: Graceful behavior near workspace boundaries

### PyTorch Kinematics Features Used

1. **Forward Kinematics**: `chain.forward_kinematics(q)`
2. **Jacobian Computation**: `chain.jacobian(q)`
3. **GPU Support**: `.to(device=device)`
4. **Batch Processing**: Handle multiple configurations simultaneously
5. **Automatic Differentiation**: Could enable learning-based control

## Production Recommendations

For real robot deployment:
1. **Kinematics Library**: Currently the urdf is used to create the model for pytorch, for you robot you need to make it manually
2. **Use Quaternions**: Replace RPY with quaternion orientation representation
3. **Add Safety Monitors**: Joint limit monitoring, collision checking
4. **Real Robot Interface**: Replace MuJoCo with actual robot drivers
5. **Advanced Control**: Force/torque control, compliant motion

## Extensions

This example can be extended with:
- **Learning-Based Control**: Use PyTorch's autodiff for learned components
- **Multi-Robot Coordination**: Leverage GPU parallel processing
- **Advanced IK Solvers**: Try different PyTorch Kinematics IK algorithms
- **Collision Avoidance**: Integrate with [pytorch-volumetric](https://github.com/UM-ARM-Lab/pytorch_volumetric) for SDF queries
- **Real Robot**: Replace MuJoCo with actual robot drivers

## Troubleshooting

**"PyTorch Kinematics not found"**
```bash
pip install pytorch-kinematics
```

**"CUDA out of memory"**
- Set `device = "cpu"` in controller initialization
- Reduce batch sizes if using advanced features

**"Robot moves erratically"**
- Check joint limits are correctly applied
- Verify URDF file path is correct
- Try reducing control gains if oscillating

**"Controller slower than expected"**
- Ensure PyTorch is using GPU: check `torch.cuda.is_available()`
- Profile the forward kinematics and Jacobian computation times

## Performance Notes

- **GPU Acceleration**: ~10x speedup for kinematics computation with CUDA
- **Memory Usage**: Minimal - only loads kinematic chain, not full physics
- **Scalability**: Can handle multiple robot arms with batch processing

+ 0
- 280
examples/franka-mujoco-control/03_gamepad_control/README.md View File

@@ -1,280 +0,0 @@
# 03. Gamepad Control

This example demonstrates real-time interactive control by connecting a gamepad to the Franka controller. It builds upon the target pose control example by adding gamepad input processing while using **PyTorch Kinematics** for efficient computation. Shows how to integrate multiple input sources and implement teleoperation with GPU-accelerated kinematics.

## Controls

- **Left Stick X**: Move along X-axis (forward/backward)
- **Left Stick Y**: Move along Y-axis (left/right)
- **Right Stick Y**: Move along Z-axis (up/down)
- **LB/RB**: Decrease/Increase movement speed (0.1-1.0x scale)
- **START**: Reset to home position
- **X Button**: Close gripper
- **Y Button**: Open gripper

## Running the Example

1. **Connect a gamepad** (Xbox/PlayStation controller via USB or Bluetooth)
2. **Run the simulation**:
```bash
cd 03_gamepad_control
dora build gamepad_control.yml
dora run gamepad_control.yml
```

You should see:
1. Robot responds to gamepad input in real-time
2. Smooth incremental movement based on stick input
3. Speed control with bumper buttons
4. Gripper control with face buttons
5. GPU-accelerated kinematics computation (if CUDA available)

## How It Works

### Node Breakdown

#### 1. **Gamepad Node** (`gamepad`)
Built-in Dora node that interfaces with system gamepad drivers.

**What it does**:
- Polls connected gamepad at 100Hz (`tick: dora/timer/millis/10`)
- Converts raw gamepad input to standardized JSON format
- **Outputs**:
- `raw_control`: Raw gamepad data with axes and button states
- `cmd_vel`: Velocity commands (unused in this example)

**Raw Control Format**:
```json
{
"axes": [stick_x, stick_y, trigger_l, stick_rx, stick_ry, trigger_r],
"buttons": [X, A, B, Y, LB, RB, ..., START, ...],
"mapping": {"button_names": {...}, "axis_names": {...}}
}
```

#### 2. **Enhanced Franka Controller** (`franka_gamepad_controller.py`)

**Key Enhancement**: Extends the target pose controller with gamepad input processing using PyTorch Kinematics.

**Gamepad Input Processing**:
```python
def process_gamepad_input(self, raw_control):
axes = raw_control["axes"]
buttons = raw_control["buttons"]
# 1. Button handling
if buttons[9]: # START button
# Reset target to home position using PyTorch Kinematics FK
home_ee_pos, _ = self.get_current_ee_pose(self.home_pos)
self.target_pos = home_ee_pos.copy()
print("Reset target to home position")
# 2. Gripper control
if buttons[0]: # X button - Close gripper
self.gripper_state = 0.0
print("Gripper: CLOSED")
elif buttons[3]: # Y button - Open gripper
self.gripper_state = 1.0
print("Gripper: OPEN")
# 3. Speed scaling
if buttons[4]: self.speed_scale = max(0.1, self.speed_scale - 0.1) # LB
if buttons[5]: self.speed_scale = min(1.0, self.speed_scale + 0.1) # RB
# 4. Incremental position control with deadzone
dx = self.apply_deadzone(axes[0]) * self.speed_scale * 0.1
dy = -self.apply_deadzone(axes[1]) * self.speed_scale * 0.1 # Inverted Y
dz = -self.apply_deadzone(axes[3]) * self.speed_scale * 0.1 # Right stick Y
# 5. Update target position incrementally
if abs(dx) > 0 or abs(dy) > 0 or abs(dz) > 0:
self.target_pos += np.array([dx, dy, dz])
self.last_input_source = "gamepad"
print(f"Gamepad control: dx={dx:.3f}, dy={dy:.3f}, dz={dz:.3f}")
```

**Control Flow with PyTorch Kinematics**:
```python
def apply_cartesian_control(self, current_joints):
# Filter to first 7 joints (arm only)
self.current_joint_pos = current_joints[:7]
# Get current end-effector pose using PyTorch Kinematics FK
current_ee_pos, current_ee_rot = self.get_current_ee_pose(self.current_joint_pos)
# Calculate position error
pos_error = self.target_pos - current_ee_pos

# Construct 6D twist (3 position + 3 orientation)
twist = np.zeros(6)
twist[:3] = self.Kpos * pos_error / self.integration_dt
# Orientation control
current_rot = Rotation.from_matrix(current_ee_rot)
desired_rot = Rotation.from_matrix(self.get_target_rotation_matrix())
rot_error = (desired_rot * current_rot.inv()).as_rotvec()
twist[3:] = self.Kori * rot_error / self.integration_dt
# Get Jacobian using PyTorch Kinematics
jac = self.compute_jacobian_pytorch(self.current_joint_pos) # (6, 7)
# Damped least squares + nullspace control
diag = self.damping * np.eye(6)
dq = jac.T @ np.linalg.solve(jac @ jac.T + diag, twist)
# Nullspace control - drive towards home position
jac_pinv = np.linalg.pinv(jac)
N = np.eye(7) - jac_pinv @ jac # Nullspace projection matrix
dq_null = self.Kn * (self.home_pos - self.current_joint_pos)
dq += N @ dq_null
# Integrate to get new joint positions
new_joints = self.current_joint_pos + dq * self.integration_dt
# Return 8-dimensional control: 7 arm joints + gripper
full_commands = np.zeros(8)
full_commands[:7] = new_joints
full_commands[7] = self.gripper_range[0] if self.gripper_state < 0.5 else self.gripper_range[1]
return full_commands
```

#### 3. **MuJoCo Simulation Node** (`dora-mujoco`)
- **Input**: 8D joint commands (7 arm + 1 gripper) from enhanced controller
- **Processing**: Physics simulation, rendering, forward kinematics
- **Output**: Joint positions, velocities, sensor data

## Technical Implementation Details

### Control Modes Comparison

| Feature | Target Pose Control | Gamepad Control |
|---------|-------------------|-----------------|
| **Input Type** | Absolute positions | Incremental movements |
| **Update Rate** | Configurable | Real-time |
| **Control Precision** | Exact target poses | Controller/gamepad positioning |
| **Gripper Control** | None | X/Y button control |
| **Speed Control** | Fixed | Variable (LB/RB buttons) |

### Incremental vs Absolute Control

**Target Pose Control** (Absolute):
```python
# Direct target assignment
self.target_pos = np.array([0.5, 0.2, 0.6]) # Go exactly here
```

**Gamepad Control** (Incremental):
```python
# Relative movement from current target
dx = gamepad_input * speed_scale * 0.1
self.target_pos += np.array([dx, dy, dz]) # Move relative to current
```

### Deadzone Implementation

```python
def apply_deadzone(self, value, deadzone=0.05):
"""Prevent controller drift by ignoring small inputs."""
return 0.0 if abs(value) < deadzone else value
```

**Why needed**: Analog sticks rarely return exactly 0.0, causing unwanted drift.

## Extension Ideas

### Easy Extensions
1. **Add Orientation Control**:
```python
# Use right stick X for yaw rotation
dyaw = self.apply_deadzone(axes[2]) * self.speed_scale * 0.1
self.target_rpy[2] += dyaw # Update yaw angle
```

2. **Workspace Limits with FK Validation**:
```python
# Validate target position is reachable using PyTorch Kinematics
def validate_target_position(self, target_pos):
# Use FK to check if any joint configuration can reach target
# Could use IK solver to verify reachability
pass
```

3. **Custom Button Mapping**:
```python
# Load button mappings from config file
button_config = {
'gripper_close': 'X',
'gripper_open': 'Y',
'speed_up': 'RB',
'home_reset': 'START'
}
```

### Advanced Extensions
1. **Force Feedback**: Rumble controller when approaching limits or singularities
2. **Multi-Robot Control**: Leverage PyTorch Kinematics batch processing for multiple arms
3. **Recording/Playback**: Record gamepad sessions with precise end-effector trajectories
4. **Learning Integration**: Use PyTorch's autodiff for learning-based assistance
5. **Collision Avoidance**: Integrate with [pytorch-volumetric](https://github.com/UM-ARM-Lab/pytorch_volumetric) for SDF-based collision checking

### Multi-Modal Input Example
```python
def main():
# Controller can handle both gamepad and programmatic input
for event in node:
if event["id"] == "raw_control":
# Gamepad input - incremental control
controller.process_gamepad_input(raw_control)
elif event["id"] == "target_pose":
# Programmatic input - absolute positioning
controller.process_target_pose(target_pose)
# Same differential IK handles both input types seamlessly
commands = controller.apply_cartesian_control(joint_positions)
```

## Troubleshooting

### "Gamepad not detected"
```bash
# Check if gamepad is connected
ls /dev/input/js*

# Test gamepad input
jstest /dev/input/js0
```

### "Robot doesn't respond to gamepad"
- Check that gamepad node is outputting `raw_control` data
- Verify controller is receiving gamepad events
- Ensure PyTorch Kinematics is properly initialized


### "Slow response / choppy movement"
- Enable GPU acceleration: check `torch.cuda.is_available()`
- Reduce gamepad polling rate if CPU-limited
- Profile FK/Jacobian computation times

### "Robot moves erratically with gamepad"
- Adjust deadzone: increase `self.deadzone = 0.1` for less sensitive sticks
- Reduce speed scale: lower `self.speed_scale = 0.2` for finer control
- Check for controller drift: test with `jstest`

## Performance Notes

- **Real-time Control**: PyTorch Kinematics enables smooth 100Hz gamepad response
- **GPU Acceleration**: Significant speedup for FK/Jacobian computation
- **Memory Efficiency**: Minimal memory overhead compared to dual MuJoCo
- **Scalability**: Could theoretically control multiple robots with one gamepad

## Next Steps

This gamepad control example demonstrates a complete teleoperation system with modern kinematics computation. Consider exploring:

- **Direct Robot Control**: Replace MuJoCo with real robot drivers (FrankaEmika SDK)
- **Advanced Input Devices**: 3D mice, haptic devices, VR controllers
- **Autonomous + Manual**: Blend autonomous behaviors with manual override
- **Multi-Modal Control**: Voice commands, gesture recognition, eye tracking
- **Learning-Based Assistance**: Use PyTorch for adaptive control behaviors
- **Collaborative Control**: Multiple operators controlling different aspects

+ 0
- 139
examples/franka-mujoco-control/04_gamepad_collision_avoidance/README.md View File

@@ -1,139 +0,0 @@
# Franka Real-Time Collision Avoidance Controller

Franka Panda robot control with gamepad input and real-time self-collision avoidance using repulsive forces. The robot provides smooth, collision-aware motion.

## Features

- **Real-Time Gamepad Control**: Intuitive joystick control for precise robot movement
- **Self-Collision Avoidance**: Continuous collision detection and avoidance using repulsive forces
- **PyTorch Kinematics**: GPU-accelerated forward/inverse kinematics for high performance
- **Smooth Motion**: Natural robot behavior with collision-aware motion planning

## Controls

### Gamepad Layout
- **Left Stick X/Y**: Move end-effector in X/Y plane
- **Right Stick Y**: Move end-effector up/down (Z axis)
- **X Button**: Close gripper
- **Y Button**: Open gripper
- **LB/RB**: Decrease/Increase movement speed (0.1x - 1.0x)
- **START**: Reset to home position with downward orientation

## Collision Avoidance System

The system implements a sophisticated collision avoidance algorithm:

### Real-Time Detection
- **Continuous Monitoring**: Checks for potential self-collisions every control cycle
- **Geometric Collision Pairs**: Monitors 18 non-adjacent link pairs for potential collisions
- **Distance-Based Forces**: Generates repulsive forces based on proximity to collision

### Repulsive Force Model
```python
# Force magnitude calculation
if distance < min_safe_distance:
force = collision_gain * (1/distance - 1/min_safe_distance)
else:
force = 0 # No force outside influence zone
```

### Key Parameters
- **`min_link_distance`**: 0.02m - Minimum safe distance between robot links
- **`collision_force_gain`**: 1000.0 - Strength of repulsive forces
- **`collision_influence_distance`**: 0.05m - Distance at which collision avoidance activates

## Installation

### Prerequisites
```bash
# Install required Python packages
pip install pytorch-kinematics scipy numpy torch pyarrow dora-rs
```

### Running the System
```bash
# Navigate to the project directory
cd /path/to/dora-rs/examples/franka-mujoco-control/04_gamepad_collision_avoidance/

# Start the Dora runtime
dora up

# Launch the collision avoidance system
dora start collision_avoidance.yml
```

## System Architecture

<!-- ### Node Communication Flow
```
┌─────────────┐ raw_control ┌─────────────────────────────┐
│ gamepad │ ──────────────────▶ │ franka_collision_controller │
└─────────────┘ │ │
│ • Processes gamepad │
┌─────────────┐ joint_positions │ • Calculates IK/FK │
│ mujoco_sim │ ──────────────────▶ │ • Applies collision │
│ │ │ avoidance │
│ │ ◀────────────────── │ • Outputs joint cmds │
└─────────────┘ joint_commands └─────────────────────────────┘
``` -->

### Collision Detection Pipeline
1. **Forward Kinematics**: Calculate all link positions using PyTorch
2. **Distance Calculation**: Check distances between all collision pairs
3. **Force Generation**: Generate repulsive forces for close pairs
4. **Force Integration**: Convert Cartesian forces to joint space
5. **Motion Blending**: Combine tracking and collision avoidance motions

## Configuration

### Controller Parameters
```python
# Control gains
Kpos = 0.95 # Position control gain
Kori = 0.95 # Orientation control gain
max_angvel = 0.785 # Maximum joint velocity (rad/s)

# Collision avoidance
min_link_distance = 0.02 # 2cm minimum safe distance
collision_force_gain = 1000.0 # Repulsive force strength
collision_influence_distance = 0.05 # 5cm activation distance
```

### Link Geometry Configuration
The system models each robot link as a cylinder with specified radius and offset:
```python
link_geometries = {
'panda_link1': {'radius': 0.08, 'offset': [0, 0, 0.075]},
'panda_link2': {'radius': 0.08, 'offset': [0, 0, 0]},
# ... etc for all 7 links + hand
}
```

## Troubleshooting

### Common Issues
1. **Robot Not Moving**: Check gamepad connection and button mappings
2. **Jerky Motion**: Reduce collision force gains
3. **GPU Errors**: Ensure CUDA-compatible PyTorch installation
4. **Joint Limits**: Robot automatically respects joint limits and stops at boundaries

### Debug Information
The system provides real-time collision status:
```bash
# Console output shows active collisions
Active collisions: panda_link1<->panda_link4: 0.03m; panda_link2<->panda_link5: 0.04m
Collision forces cleared - normal motion resumed
```

## Performance Notes

- **Control Frequency**: 500Hz for smooth, responsive control
- **Collision Detection**: Runs every control cycle (no artificial delays)
- **GPU Utilization**: Automatic GPU acceleration when CUDA available
- **Memory Usage**: Efficient tensor operations minimize memory footprint

## Safety Features

- **Joint Limit Protection**: Automatic velocity limiting near joint boundaries
- **Velocity Clamping**: Maximum joint velocities enforced at all times
- **Self-Collision Avoidance**: Repulsive force approach to avoid collision

+ 0
- 35
examples/franka-mujoco-control/04_gamepad_collision_avoidance/collision_avoidance.yml View File

@@ -1,35 +0,0 @@
nodes:
- id: gamepad
build: pip install -e ../../../node-hub/gamepad
path: gamepad
outputs:
- cmd_vel
- raw_control
inputs:
tick: dora/timer/millis/10


- id: mujoco_sim
build: pip install -e ../../../node-hub/dora-mujoco
path: dora-mujoco
inputs:
tick: dora/timer/millis/2 # 500 Hz simulation
control_input: franka_collision_controller/joint_commands
outputs:
- joint_positions
- joint_velocities
- actuator_controls
- sensor_data
env:
MODEL_NAME: "panda"

- id: franka_collision_controller
path: nodes/franka_collision_controller.py
inputs:
tick: dora/timer/millis/2
joint_positions: mujoco_sim/joint_positions
raw_control: gamepad/raw_control
outputs:
- joint_commands
- ee_position
- collision_status

+ 0
- 366
examples/franka-mujoco-control/04_gamepad_collision_avoidance/nodes/franka_collision_controller.py View File

@@ -1,366 +0,0 @@
"""Enhanced Franka controller with gamepad support and collision avoidance using repulsive forces."""

import json
import time
import numpy as np
import pyarrow as pa
import torch
from dora import Node
import pytorch_kinematics as pk
from scipy.spatial.transform import Rotation

class FrankaCollisionController:
"""Franka controller with gamepad support and self-collision avoidance using repulsive forces."""
def __init__(self):
"""Initialize controller with PyTorch Kinematics and collision avoidance."""
# Load the robot model for IK and FK
urdf_path = "../franka_panda/panda.urdf"
with open(urdf_path, 'rb') as f:
urdf_content = f.read()
# Build serial chain for kinematics
self.chain = pk.build_serial_chain_from_urdf(urdf_content, "panda_hand")
# Move to GPU if available
self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
self.chain = self.chain.to(device=self.device)
# Control parameters
self.integration_dt = 0.1
self.damping = 1e-4
self.Kpos = 0.95
self.Kori = 0.95
self.Kn = np.array([10.0, 10.0, 10.0, 10.0, 5.0, 5.0, 5.0])
self.max_angvel = 0.785
# Collision avoidance parameters
self.collision_check_enabled = True
self.min_link_distance = 0.02
self.collision_force_gain = 1000.0
self.collision_influence_distance = 0.05
# Define which links can collide with each other (non-adjacent only)
self.collision_pairs = [
('panda_link1', 'panda_link3'), ('panda_link1', 'panda_link4'), ('panda_link1', 'panda_link5'), ('panda_link1', 'panda_link6'), ('panda_link1', 'panda_link7'),
('panda_link1', 'panda_hand'), ('panda_link2', 'panda_link4'), ('panda_link2', 'panda_link5'), ('panda_link2', 'panda_link6'), ('panda_link2', 'panda_link7'),
('panda_link2', 'panda_hand'), ('panda_link3', 'panda_link5'), ('panda_link3', 'panda_link6'), ('panda_link3', 'panda_link7'), ('panda_link3', 'panda_hand'),
('panda_link4', 'panda_link6'), ('panda_link4', 'panda_link7'), ('panda_link4', 'panda_hand'),
# ('panda_link5', 'panda_hand'), # Enabling this results in very jaggy motion
]
# Approximate link geometries for distance calculation
self.link_geometries = {
'panda_link1': {'radius': 0.08, 'length': 0.15, 'offset': np.array([0, 0, 0.075])},
'panda_link2': {'radius': 0.08, 'length': 0.12, 'offset': np.array([0, 0, 0])},
'panda_link3': {'radius': 0.07, 'length': 0.32, 'offset': np.array([0, 0, 0.16])},
'panda_link4': {'radius': 0.07, 'length': 0.28, 'offset': np.array([0, 0, -0.14])},
'panda_link5': {'radius': 0.06, 'length': 0.22, 'offset': np.array([0, 0, 0.11])},
'panda_link6': {'radius': 0.06, 'length': 0.12, 'offset': np.array([0, 0, 0.06])},
'panda_link7': {'radius': 0.05, 'length': 0.08, 'offset': np.array([0, 0, 0.04])},
'panda_hand': {'radius': 0.07, 'length': 0.07, 'offset': np.array([0, 0, 0.035])},
}
# Gamepad control parameters
self.speed_scale = 0.5
self.deadzone = 0.05
# Gripper control parameters
self.gripper_range = [0, 255]
self.gripper_state = 0.0
# Robot state
self.current_joint_pos = None
self.target_pos = np.array([0.55, 0.0, 0.6])
self.target_rpy = [180.0, 0.0, 90.0]
self.home_pos = np.array([0, 0, 0, -1.57079, 0, 1.57079, -0.7853])
# Collision state tracking
self.is_in_collision_risk = False
self.collision_force_applied_count = 0
self.current_collision_info = ""
# print("Franka Collision Controller initialized")
# print(f"Chain DOF: {len(self.chain.get_joint_parameter_names())}")
# print(f"Using device: {self.device}")
def calculate_collision_repulsive_forces(self, joint_angles):
"""Calculate repulsive forces to avoid collisions between robot links."""
if not self.collision_check_enabled:
return np.zeros(7), False, ""
# Get forward kinematics for all links
q_tensor = torch.tensor(joint_angles, device=self.device, dtype=torch.float32).unsqueeze(0)
fk_results = self.chain.forward_kinematics(q_tensor, end_only=False)
# Initialize total repulsive force in joint space
total_repulsive_forces = np.zeros(7)
collision_detected = False
collision_info_list = []
# Check each collision pair
for link1, link2 in self.collision_pairs:
if link1 not in fk_results or link2 not in fk_results:
continue
# Get link transforms
transform1 = fk_results[link1].get_matrix().squeeze(0).cpu().numpy()
transform2 = fk_results[link2].get_matrix().squeeze(0).cpu().numpy()
# Get link center positions
link1_info, link2_info = self.link_geometries[link1], self.link_geometries[link2]
offset1, offset2 = link1_info['offset'], link2_info['offset']
radius1, radius2 = link1_info['radius'], link2_info['radius']

# Transform offsets to world coordinates
pos1 = transform1[:3, 3] + transform1[:3, :3] @ offset1
pos2 = transform2[:3, 3] + transform2[:3, :3] @ offset2
# Calculate distance between link centers
distance = np.linalg.norm(pos1 - pos2)
# Check if within influence range
min_safe_distance = radius1 + radius2 + self.min_link_distance
influence_distance = min_safe_distance + self.collision_influence_distance
if distance < influence_distance and distance > 1e-6:
collision_detected = True
# Calculate repulsive force magnitude
if distance < min_safe_distance:
force_magnitude = self.collision_force_gain * (1.0/max(distance, 0.01) - 1.0/min_safe_distance)
else:
force_magnitude = 0
# Direction of repulsive force
if distance > 1e-6:
force_direction = (pos1 - pos2) / distance
else:
force_direction = np.array([1, 0, 0])
# Convert Cartesian repulsive force to joint space forces
link1_forces = self.cartesian_to_joint_forces(link1, pos1, force_direction * force_magnitude, joint_angles)
link2_forces = self.cartesian_to_joint_forces(link2, pos2, -force_direction * force_magnitude, joint_angles)
total_repulsive_forces += link1_forces + link2_forces
# Store collision info
collision_info_list.append(f"{link1}<->{link2}: {distance:.2f}m")
collision_info = "; ".join(collision_info_list) if collision_info_list else "No collision"
return total_repulsive_forces, collision_detected, collision_info
def cartesian_to_joint_forces(self, link_name, link_position, cartesian_force, joint_angles):
"""Convert Cartesian force at a link to joint space forces."""
# Get numerical Jacobian for this link position
jacobian = self.get_link_jacobian_simplified(link_name, joint_angles, link_position)
# Convert Cartesian force to joint torques using Jacobian transpose
joint_forces = jacobian.T @ cartesian_force[:3]
return joint_forces
def get_link_jacobian_simplified(self, link_name, joint_angles, link_position):
"""Get link Jacobian using numerical differentiation."""
epsilon = 1e-6
jacobian = np.zeros((3, 7))
# Calculate numerical Jacobian
for i in range(7):
# Perturb joint i
perturbed_joints = joint_angles.copy()
perturbed_joints[i] += epsilon
# Get perturbed link position
q_perturbed = torch.tensor(perturbed_joints, device=self.device, dtype=torch.float32).unsqueeze(0)
fk_perturbed = self.chain.forward_kinematics(q_perturbed, end_only=False)
if link_name in fk_perturbed:
transform_perturbed = fk_perturbed[link_name].get_matrix().squeeze(0).cpu().numpy()
offset = self.link_geometries[link_name]['offset']
perturbed_pos = transform_perturbed[:3, 3] + transform_perturbed[:3, :3] @ offset
# Numerical derivative
jacobian[:, i] = (perturbed_pos - link_position) / epsilon
return jacobian
def apply_deadzone(self, value):
"""Apply deadzone to joystick input."""
return 0.0 if abs(value) < self.deadzone else value
def get_current_ee_pose(self, joint_angles):
"""Get current end-effector pose."""
q = torch.tensor(joint_angles, device=self.device, dtype=torch.float32).unsqueeze(0)
tf = self.chain.forward_kinematics(q)
transform_matrix = tf.get_matrix().squeeze(0).cpu().numpy()
position = transform_matrix[:3, 3]
rotation_matrix = transform_matrix[:3, :3]
return position, rotation_matrix
def compute_jacobian_pytorch(self, joint_angles):
"""Compute Jacobian using PyTorch Kinematics."""
q = torch.tensor(joint_angles, device=self.device, dtype=torch.float32).unsqueeze(0)
J = self.chain.jacobian(q)
return J.squeeze(0).cpu().numpy()
def process_gamepad_input(self, raw_control):
"""Process gamepad input."""
axes = raw_control["axes"]
buttons = raw_control["buttons"]

# Reset position with START button
if len(buttons) > 9 and buttons[9]:
if self.current_joint_pos is not None:
home_ee_pos, _ = self.get_current_ee_pose(self.home_pos)
self.target_pos = home_ee_pos.copy()
print("Reset target to home position")
# Gripper control with X and Y buttons
if len(buttons) > 0 and buttons[0]:
self.gripper_state = 0.0
print("Gripper: CLOSED")
elif len(buttons) > 3 and buttons[3]:
self.gripper_state = 1.0
print("Gripper: OPEN")
# Speed scaling with bumpers (LB/RB)
if len(buttons) > 4 and buttons[4]:
self.speed_scale = max(0.1, self.speed_scale - 0.1)
print(f"Speed: {self.speed_scale:.1f}")
if len(buttons) > 5 and buttons[5]:
self.speed_scale = min(1.0, self.speed_scale + 0.1)
print(f"Speed: {self.speed_scale:.1f}")
# Get cartesian control inputs with deadzone
dx = self.apply_deadzone(axes[0]) * self.speed_scale * 0.1
dy = -self.apply_deadzone(axes[1]) * self.speed_scale * 0.1
dz = -self.apply_deadzone(axes[3]) * self.speed_scale * 0.1
# Update target position incrementally
if abs(dx) > 0 or abs(dy) > 0 or abs(dz) > 0:
self.target_pos += np.array([dx, dy, dz])

def get_target_rotation_matrix(self):
"""Convert RPY to rotation matrix."""
roll_rad, pitch_rad, yaw_rad = np.radians(self.target_rpy)
desired_rot = Rotation.from_euler('ZYX', [yaw_rad, pitch_rad, roll_rad])
return desired_rot.as_matrix()
def apply_cartesian_control(self, current_joints):
"""Apply Cartesian control with collision avoidance using repulsive forces."""
self.current_joint_pos = current_joints[:7]
# Get current end-effector pose
current_ee_pos, current_ee_rot = self.get_current_ee_pose(self.current_joint_pos)
# Calculate position error
pos_error = self.target_pos - current_ee_pos

# Construct 6D twist
twist = np.zeros(6)
twist[:3] = self.Kpos * pos_error / self.integration_dt
# Orientation control
current_rot = Rotation.from_matrix(current_ee_rot)
desired_rot = Rotation.from_matrix(self.get_target_rotation_matrix())
rot_error = (desired_rot * current_rot.inv()).as_rotvec()
twist[3:] = self.Kori * rot_error / self.integration_dt
# Get Jacobian
jac = self.compute_jacobian_pytorch(self.current_joint_pos)
# Damped least squares inverse kinematics
diag = self.damping * np.eye(6)
dq = jac.T @ np.linalg.solve(jac @ jac.T + diag, twist)
# Nullspace control
jac_pinv = np.linalg.pinv(jac)
N = np.eye(7) - jac_pinv @ jac
dq_null = self.Kn * (self.home_pos - self.current_joint_pos)
dq += N @ dq_null
# Collision avoidance
collision_forces, collision_detected, collision_info = self.calculate_collision_repulsive_forces(self.current_joint_pos)
if collision_detected:
dq += collision_forces * self.integration_dt
self.is_in_collision_risk = True
self.current_collision_info = collision_info
# print(f"Active collisions: {collision_info}")
else:
if self.is_in_collision_risk:
print("Collision forces cleared - normal motion resumed")
self.is_in_collision_risk = False
self.current_collision_info = "Safe"
# Limit joint velocities
dq_abs_max = np.abs(dq).max()
if dq_abs_max > self.max_angvel:
dq *= self.max_angvel / dq_abs_max
# Integrate to get new joint positions
new_joints = self.current_joint_pos + dq * self.integration_dt
# Apply joint limits
joint_limits = np.array([[-2.8973, 2.8973], [-1.7628, 1.7628], [-2.8973, 2.8973], [-3.0718, -0.0698], [-2.8973, 2.8973], [-0.0175, 3.7525], [-2.8973, 2.8973]])
for i in range(7):
new_joints[i] = np.clip(new_joints[i], joint_limits[i][0], joint_limits[i][1])
# Return commands
full_commands = np.zeros(8)
full_commands[:7] = new_joints
full_commands[7] = self.gripper_range[0] if self.gripper_state < 0.5 else self.gripper_range[1]
return full_commands


def main():
"""Main controller loop."""
node = Node("franka_collision_controller")
controller = FrankaCollisionController()
print("Franka Collision Controller Node Started")
for event in node:
if event["type"] == "INPUT":
if event["id"] == "joint_positions":
joint_positions = event["value"].to_numpy()
full_commands = controller.apply_cartesian_control(joint_positions)
node.send_output(
"joint_commands",
pa.array(full_commands, type=pa.float64()),
metadata={"timestamp": time.time()}
)
if controller.current_joint_pos is not None:
current_ee_pos, _ = controller.get_current_ee_pose(controller.current_joint_pos)
node.send_output(
"ee_position",
pa.array(current_ee_pos, type=pa.float64()),
metadata={"timestamp": time.time()}
)
# Send collision status
collision_status = {
"collision_risk": controller.is_in_collision_risk,
"collision_info": controller.current_collision_info,
"force_applied_count": controller.collision_force_applied_count
}
node.send_output(
"collision_status",
pa.array([1.0 if controller.is_in_collision_risk else 0.0], type=pa.float64()),
metadata={"collision_status": json.dumps(collision_status)}
)
elif event["id"] == "raw_control":
raw_control = json.loads(event["value"].to_pylist()[0])
controller.process_gamepad_input(raw_control)

if __name__ == "__main__":
main()

+ 0
- 46
examples/franka-mujoco-control/README.md View File

@@ -1,46 +0,0 @@
# Franka MuJoCo Control Tutorial

This comprehensive tutorial demonstrates how to build a modular robot control system using Dora's dataflow architecture with the [`dora-mujoco`](../../node-hub/dora-mujoco) simulation node and robot-specific control logic.

## Tutorial Structure

### [01. Basic Simulation](01_basic_simulation/)
Load and run a Franka Panda simulation using the `dora-mujoco` node.
- Learn the fundamentals of MuJoCo simulation in Dora
- Understand the simulation node architecture
- See how robot descriptions are loaded automatically

### [02. Target Pose Control](02_target_pose_control/)
Add robot-specific control logic with programmatic pose commands.
- Implement Cartesian space control with inverse kinematics
- Create a dedicated Franka controller node
- Send target poses programmatically

### [03. Gamepad Control](03_gamepad_control/)
Connect a gamepad for real-time interactive control.
- Integrate with the existing `gamepad` node
- Process raw gamepad input into robot commands
- Demonstrate real-time teleoperation


## Quick Start

```bash
cd examples/franka-mujoco-control

# 1. Basic simulation
cd 01_basic_simulation
dora build basic.yml
dora run basic.yml

# 2. Target pose control
cd ../02_target_pose_control
dora build target_pose_control.yml
dora run target_pose_control.yml

# 3. Gamepad control
cd ../03_gamepad_control
dora build gamepad_control.yml
dora run gamepad_control.yml
```


+ 21
- 0
examples/mujoco-sim/README.md View File

@@ -0,0 +1,21 @@
# Franka MuJoCo Control Tutorial

This comprehensive tutorial demonstrates how to build a robot control system using Dora with the `dora-mujoco` simulation node and control logic.

## Tutorial Structure

### [01. Basic Simulation](basic_simulation/)
Load and run a Franka Panda simulation using the `dora-mujoco` node.
- Learn the fundamentals of MuJoCo simulation in Dora
- Understand the simulation node architecture
- See how robot descriptions are loaded automatically

### [02. Target Pose Control](target_pose_control/)
Add control logic with pose commands as target.
- Implement Cartesian space control.
- Create generic controller node that is able to control any robotic arm by using `dora-pytorch-kinematics`

### [03. Gamepad Control](gamepad_control/)
Connect a gamepad for real-time interactive control.
- Integrate with dora's `gamepad` node
- Demonstrate real-time teleoperation

examples/franka-mujoco-control/franka_panda/panda.urdf → examples/mujoco-sim/URDF/franka_panda/panda.urdf View File


examples/franka-mujoco-control/kuka_iiwa/model.urdf → examples/mujoco-sim/URDF/kuka_iiwa/model.urdf View File


examples/franka-mujoco-control/01_basic_simulation/README.md → examples/mujoco-sim/basic_simulation/README.md View File

@@ -6,11 +6,6 @@ This example demonstrates the simplest possible setup: loading and running a Fra
- See how robot models are loaded from `robot-descriptions`
- Learn the basic dataflow for physics simulation

## Architecture

```
[Timer] → [MuJoCo Sim] → [Joint Positions, Velocities, Sensor Data]
```

The simulation runs at 500Hz and outputs:
- Joint positions for all robot joints
@@ -27,13 +22,12 @@ dora run basic.yml
```

You should see:
1. MuJoCo viewer window opens with Franka Panda robot
1. MuJoCo viewer window opens with GO2 robot
2. Robot is effected by gravity (enabled by default)
3. Console output showing node activity

## What's Happening

1. **Model Loading**: The `dora-mujoco` node loads the Franka model using `load_robot_description("panda_mj_description")`
1. **Model Loading**: The `dora-mujoco` node loads the Franka model using `load_robot_description("go2_mj_description")`
2. **Physics Loop**: Timer triggers simulation steps at 500Hz (This is default step time for Mujoco)
3. **Data Output**: Joint states are published
4. **Visualization**: MuJoCo viewer shows real-time simulation
@@ -41,6 +35,6 @@ You should see:
## Configuration Details

The `basic.yml` configures:
- Model name: `"panda"` (resolved to `panda_mj_description`)
- Update rate: 2ms (500Hz)
- Model name: `"go2"` you change this to other robots name
- Update rate: 2ms (500Hz)
- Outputs: Joint positions, velocities, and sensor data

examples/franka-mujoco-control/01_basic_simulation/basic.yml → examples/mujoco-sim/basic_simulation/basic.yml View File

@@ -9,4 +9,4 @@ nodes:
- joint_velocities
- sensor_data
env:
MODEL_NAME: "panda" # Load Franka Panda from robot-descriptions
MODEL_NAME: "go2" # Load GO2

+ 75
- 0
examples/mujoco-sim/gamepad_control/README.md View File

@@ -0,0 +1,75 @@
# 03. Gamepad Control

This example demonstrates real-time interactive control by connecting a gamepad to the controller. It builds upon the target pose control example by adding gamepad input processing. Shows how to integrate and implement teleoperation.

### Gamepad Controls:
- **D-pad Vertical**: Move along X-axis (forward/backward)
- **D-pad Horizontal**: Move along Y-axis (left/right)
- **Right Stick Y**: Move along Z-axis (up/down)
- **LB/RB**: Decrease/Increase movement speed (0.1-1.0x scale)
- **START**: Reset to home position

## Running the Example

1. **Connect a gamepad** (Xbox/PlayStation controller via USB or Bluetooth)
2. **Run the simulation**:
```bash
cd 03_gamepad_control
dora build gamepad_control.yml
dora run gamepad_control.yml
```

You should see:
1. Robot responds to gamepad input in real-time
2. Smooth incremental movement based on D-pad/stick input
3. Speed control with bumper buttons
4. Reset functionality with START button
5. GPU-accelerated kinematics computation (if CUDA available)

#### **Gamepad Node** (`gamepad`)
Built-in Dora node that interfaces with system gamepad drivers.
- **Outputs**:
- `cmd_vel`: 6DOF velocity commands `[linear_x, linear_y, linear_z, angular_x, angular_y, angular_z]`
- `raw_control`: Gamepad Input in Json format

#### **GamePad Controller** (`gamepad_controller.py`)
This node processes gamepad input and translates it into target positions for the robot end-effector.

- Receives continuous movement commands (`cmd_vel`) from D-pad and analog sticks
- Processes discrete button presses (`raw_control`) for special functions

**Controlling the End-Effector with Gamepad**:

The controller takes the first 3 values (X, Y, Z movement) from the gamepad `cmd_vel`, updates Target Position continuously

**Button Commands**:
- **START button**: resets end-effector to home position [0.4, 0.0, 0.3]
- **Left Bumper (LB)**: Decreases movement speed
- **Right Bumper (RB)**: Increases movement speed

The end-effector moves smoothly as you hold the controls, with position updates sent to the inverse kinematics solver to calculate required joint angles.

## Troubleshooting

**"Gamepad not responding"**
```bash
# Check if gamepad is connected
ls /dev/input/js*
# Test gamepad input
jstest /dev/input/js0
```

**"Robot doesn't move with D-pad"**
- Check `cmd_vel` output: should show non-zero values when D-pad pressed
- Verify controller processes `cmd_vel` events
- Ensure your gamepad has correct mapping.


## Real Robot Deployment

For actual robot control:
1. **Replace MuJoCo**: Use real robot drivers
2. **Safety Limits**: Add emergency stops and workspace bounds
3. **Force Control**: Integrate force/torque feedback
4. **Network Latency**: Consider wireless controller delay
5. **Deadman Switch**: Require constant button hold for safety

examples/franka-mujoco-control/03_gamepad_control/gamepad_control.yml → examples/mujoco-sim/gamepad_control/gamepad_control.yml View File

@@ -13,7 +13,7 @@ nodes:
path: dora-mujoco
inputs:
tick: dora/timer/millis/2 # 500 Hz simulation
control_input: franka_gamepad_controller/joint_commands
control_input: gamepad_controller/joint_commands
outputs:
- joint_positions
- joint_velocities
@@ -22,8 +22,8 @@ nodes:
env:
MODEL_NAME: "kuka_iiwa14"

- id: franka_gamepad_controller
path: nodes/franka_gamepad_controller.py
- id: gamepad_controller
path: nodes/gamepad_controller.py
inputs:
joint_positions: mujoco_sim/joint_positions
joint_velocities: mujoco_sim/joint_velocities
@@ -40,12 +40,12 @@ nodes:
build: pip install -e ../../../node-hub/dora-pytorch-kinematics
path: dora-pytorch-kinematics
inputs:
fk_request: franka_gamepad_controller/fk_request
jacobian_request: franka_gamepad_controller/jacobian_request
fk_request: gamepad_controller/fk_request
jacobian_request: gamepad_controller/jacobian_request
outputs:
- fk_request
- jacobian_request
env:
URDF_PATH: "/home/shashwatgpatil/Dora-rs/dora/examples/franka-mujoco-control/kuka_iiwa/model.urdf"
URDF_PATH: "../URDF/kuka_iiwa/model.urdf"
END_EFFECTOR_LINK: "lbr_iiwa_link_7"
TRANSFORM: "0. 0. 0. 1. 0. 0. 0."

examples/franka-mujoco-control/03_gamepad_control/nodes/franka_gamepad_controller.py → examples/mujoco-sim/gamepad_control/nodes/gamepad_controller.py View File

@@ -12,10 +12,10 @@ class GamepadFrankaController:
self.damping = 1e-4
self.Kpos = 0.95 # Position gain
self.Kori = 0.95 # Orientation gain
self.max_angvel = 1.57 # Max joint velocity (rad/s)

# Gamepad control parameters
self.speed_scale = 0.5
self.max_angvel = 3.14 * self.speed_scale # Max joint velocity (rad/s)
# Robot state variables
self.dof = None
@@ -39,14 +39,13 @@ class GamepadFrankaController:
self.home_pos = np.zeros(self.dof)
def process_cmd_vel(self, cmd_vel):
print(f"Processing cmd_vel: {cmd_vel}")
dx = cmd_vel[0] * self.speed_scale * 0.1
dy = cmd_vel[1] * self.speed_scale * 0.1
dz = cmd_vel[2] * self.speed_scale * 0.1
# print(f"Processing cmd_vel: {cmd_vel}")
delta = cmd_vel[:3] * 0.03
dx, dy, dz = delta
# Update target position incrementally
if abs(dx) > 0 or abs(dy) > 0 or abs(dz) > 0:
self.target_pos += np.array([dx, dy, dz])
self.target_pos += np.array([dx, -dy, dz])
def process_gamepad_input(self, raw_control):
buttons = raw_control["buttons"]

+ 99
- 0
examples/mujoco-sim/target_pose_control/README.md View File

@@ -0,0 +1,99 @@
# 02. Target Pose Control

This example demonstrates Cartesian space control by creating a Controller node that processes target pose commands and outputs joint commands using inverse kinematics using **dora-pytorch-kinematics**.

## Running the Example

```bash
cd 02_target_pose_control
dora build target_pose_control_pytorch.yml
dora run target_pose_control_pytorch.yml
```

You should see:
1. Robot moves to predefined target poses automatically
2. Smooth Cartesian space motion with differential inverse kinematics
3. End-effector following target positions accurately


### Nodes

#### 1. **Pose Publisher Script** (`pose_publisher.py`)
```python
class PosePublisher:
def __init__(self):
# Predefined sequence of target poses [x, y, z, roll, pitch, yaw]
self.target_poses = [
[0.5, 0.5, 0.3, 180.0, 0.0, 90.0], # Position + RPY orientation
[0.6, 0.2, 0.5, 180.0, 0.0, 45.0], # Different orientation
# ... more poses
]
```

- Sends target poses every 10 seconds
- Cycles through predefined positions and orientations
- Can be replaced with path planning, user input, or any pose generation logic
- Outputs `target_pose` array `[x, y, z, roll, pitch, yaw]`

#### 2. **Controller Script** (`controller.py`)
```yaml
- id: controller
path: nodes/controller.py
inputs:
joint_positions: mujoco_sim/joint_positions
joint_velocities: mujoco_sim/joint_velocities
target_pose: pose_publisher/target_pose
fk_result: pytorch_kinematics/fk_request
jacobian_result: pytorch_kinematics/jacobian_request
outputs:
- joint_commands
- fk_request
- jacobian_request
```
- **Input**: Takes target poses `[x, y, z, roll, pitch, yaw]` and current joint positions from `dora-mujoco`
- **Processing**: Differential inverse kinematics using `dora-pytorch-kinematics` to calculate actuator commands
- **Output**: Control/Actuator commands

#### 3. **PyTorch Kinematics Node** (`dora-pytorch-kinematics`)


```yaml
- id: pytorch_kinematics
build: pip install -e ../../../node-hub/dora-pytorch-kinematics
path: dora-pytorch-kinematics
inputs:
fk_request: controller/fk_request
jacobian_request: controller/jacobian_request
outputs:
- fk_request
- jacobian_request
env:
URDF_PATH: "path to the robot urdf" # URDF is used to create the kinematics model for the robot
END_EFFECTOR_LINK: "name of the end effector"
TRANSFORM: "0. 0. 0. 1. 0. 0. 0." # Pytorch kinematics uses wxyz format. Robot transform from world frame

```

Joint states performs Forward Kinematics, and returns End-effector pose along with jacobian matrix

#### 4. **MuJoCo Simulation Node** (`dora-mujoco`)

```yaml

- id: mujoco_sim
build: pip install -e ../../../node-hub/dora-mujoco
path: dora-mujoco
inputs:
tick: dora/timer/millis/2
control_input: controller/joint_commands
outputs:
- joint_positions
- joint_velocities
- actuator_controls
- sensor_data
env:
MODEL_NAME: "panda" # Name of the robot you want to load
```
- **Input**: Joint commands from controller
- **Processing**: Physics simulation, rendering
- **Output**: Joint positions, velocities, sensor data

examples/franka-mujoco-control/02_target_pose_control/nodes/franka_controller_pytorch.py → examples/mujoco-sim/target_pose_control/nodes/controller.py View File


examples/franka-mujoco-control/02_target_pose_control/nodes/pose_publisher.py → examples/mujoco-sim/target_pose_control/nodes/pose_publisher.py View File


examples/franka-mujoco-control/02_target_pose_control/target_pose_control_pytorch.yml → examples/mujoco-sim/target_pose_control/target_pose_control.yml View File

@@ -4,7 +4,7 @@ nodes:
path: dora-mujoco
inputs:
tick: dora/timer/millis/2
control_input: kuka_controller_pytorch/joint_commands
control_input: controller/joint_commands
outputs:
- joint_positions
- joint_velocities
@@ -13,8 +13,8 @@ nodes:
env:
MODEL_NAME: "panda"

- id: kuka_controller_pytorch
path: nodes/franka_controller_pytorch.py
- id: controller
path: nodes/controller.py
inputs:
joint_positions: mujoco_sim/joint_positions
joint_velocities: mujoco_sim/joint_velocities
@@ -30,19 +30,19 @@ nodes:
build: pip install -e ../../../node-hub/dora-pytorch-kinematics
path: dora-pytorch-kinematics
inputs:
fk_request: kuka_controller_pytorch/fk_request
jacobian_request: kuka_controller_pytorch/jacobian_request
fk_request: controller/fk_request
jacobian_request: controller/jacobian_request
outputs:
- fk_request
- jacobian_request
env:
URDF_PATH: "/home/shashwatgpatil/Dora-rs/dora/examples/franka-mujoco-control/franka_panda/panda.urdf"
URDF_PATH: "../URDF/franka_panda/panda.urdf"
END_EFFECTOR_LINK: "panda_hand"
TRANSFORM: "0. 0. 0. 1. 0. 0. 0." # Pytorch kinematics uses wxyz format for quaternion

- id: pose_publisher
path: nodes/pose_publisher.py
inputs:
tick: dora/timer/millis/5000
tick: dora/timer/millis/10000
outputs:
- target_pose

+ 90
- 12
node-hub/gamepad/README.md View File

@@ -9,6 +9,7 @@ A Dora framework node that provides universal gamepad control functionality. Whi
- Configurable axis mapping for different controllers
- Dead zone handling for precise control
- Real-time velocity command output
- Raw gamepad state output for custom processing
- Clean shutdown with automatic zero velocity
- Customizable speed limits
- Default configuration optimized for Logitech F710
@@ -20,6 +21,7 @@ A Dora framework node that provides universal gamepad control functionality. Whi
- Python 3.8 or higher
- Dora Framework 0.3.6 or higher
- Any compatible USB/Wireless gamepad
- Pygame library for joystick handling

### Installation

@@ -54,10 +56,18 @@ sudo chmod +x /dev/input/js*

### Controls

- **Left Stick Y-Axis**: Forward/Backward movement (±2.0 m/s)
- **Left Stick X-Axis**: Left/Right turning (±1.5 rad/s)
#### Default Control Mapping (cmd_vel output):
- **D-pad Vertical**: Linear X movement (forward/backward)
- **D-pad Horizontal**: Linear Y movement (left/right)
- **Right Stick Y-Axis**: Linear Z movement (up/down, ±0.1 m/s)
- **Right Stick X-Axis**: Angular Z rotation (turning, ±0.8 rad/s)
- **Left Stick Y-Axis**: Angular X rotation (±0.8 rad/s)
- **Left Stick X-Axis**: Angular Y rotation (±0.8 rad/s)
- **Deadzone**: 5% to prevent drift

#### Raw Control Output:
The node also outputs complete gamepad state including all buttons, axes, and D-pad for custom processing.

### YAML Specification

```yaml
@@ -66,20 +76,55 @@ nodes:
build: pip install -e .
path: gamepad
outputs:
- cmd_vel # Velocity commands [vx, vy, vz, wx, wy, wz]
- cmd_vel # Velocity commands [vx, vy, vz, wx, wy, wz]
- raw_control # Complete gamepad state (JSON)
inputs:
tick: dora/timer/millis/10 # Update rate
tick: dora/timer/millis/10 # Update rate (100Hz)
```

### Outputs

The node outputs `cmd_vel` as a 6-element array:
- `[0]`: Linear X velocity (forward/backward)
- `[1]`: Linear Y velocity (always 0)
- `[2]`: Linear Z velocity (always 0)
- `[3]`: Angular X velocity (always 0)
- `[4]`: Angular Y velocity (always 0)
- `[5]`: Angular Z velocity (turning)
#### 1. `cmd_vel` - 6-element velocity array:
- `[0]`: Linear X velocity (D-pad vertical)
- `[1]`: Linear Y velocity (D-pad horizontal)
- `[2]`: Linear Z velocity (right stick Y)
- `[3]`: Angular X velocity (left stick Y)
- `[4]`: Angular Y velocity (left stick X)
- `[5]`: Angular Z velocity (right stick X)

#### 2. `raw_control` - Complete gamepad state (JSON):
```json
{
"axes": [left_x, left_y, right_x, right_y, ...],
"buttons": [X, A, B, Y, LB, RB, LT, RT, BACK, START, LEFT_STICK, RIGHT_STICK],
"hats": [[dpad_x, dpad_y]],
"mapping": {
"axes": {"LEFT-X": 0, "LEFT-Y": 1, "RIGHT-X": 2, "RIGHT-Y": 3},
"buttons": {"X": 0, "A": 1, "B": 2, "Y": 3, "LB": 4, "RB": 5, ...}
}
}
```

## Controller Configuration

To use a different controller, modify the `Controller` class mapping:

```python
class Controller:
def __init__(self):
# Customize for your controller
self.axisNames = {
'LEFT-X': 0,
'LEFT-Y': 1,
'RIGHT-X': 2,
'RIGHT-Y': 3,
}
self.buttonNames = {
'X': 0,
'A': 1,
# ... add your button mapping
}
```

## Development

@@ -97,6 +142,28 @@ Run tests with pytest:
pytest .
```

## Integration Examples

### Simple Velocity Control:
```python
for event in node:
if event["id"] == "cmd_vel":
velocity = event["value"].to_numpy()
# Apply velocity to your robot/system
print(f"Velocity: {velocity}")
```

### Custom Button Processing:
```python
import json

for event in node:
if event["id"] == "raw_control":
control_state = json.loads(event["value"].to_pylist()[0])
buttons = control_state["buttons"]
axis = control_state[axis]
```

## Troubleshooting

**No gamepad detected**:
@@ -104,6 +171,16 @@ pytest .
- Ensure controller is powered on
- Verify mode switch position


**Raw control output empty**:
- Verify gamepad is responding with `jstest /dev/input/js0`
- Check pygame initialization

**Incorrect button mapping**:
- Different controllers have different mappings
- Use `jstest` to identify your controller's button/axis numbers
- Update the `Controller` class accordingly

**If port is not executable then run:**
```bash
sudo chmod +x /dev/input/js*
@@ -117,4 +194,5 @@ Released under the MIT License. See LICENSE file for details.

- Uses Pygame for joystick handling
- Built with the Dora framework
- Designed for Logitech F710 gamepad
- Designed for Logitech F710 gamepad
- Supports any standard USB/Bluetooth gamepad

Loading…
Cancel
Save