Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add safety limits on relative action target #373

Merged
merged 16 commits into from
Aug 26, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions examples/7_get_started_with_real_robot.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,9 +316,12 @@ robot = KochRobot(
leader_arms={"main": leader_arm},
follower_arms={"main": follower_arm},
calibration_path=".cache/calibration/koch.pkl",
max_relative_target=[10, 10, 10, 10, 10, 15],
)
```

Notice the parameter `max_relative_target`. This is a safety measure that prevents someone from providing a positional target that is too far from the current robot position (which would then cause the robot to move too quickly, potentially burning out the motors or making violent impact with another object).

**Calibrate and Connect the KochRobot**

Next, you'll need to calibrate your robot to ensure that the leader and follower arms have the same position values when they are in the same physical position. This calibration is essential because it allows a neural network trained on one Koch robot to work on another.
Expand Down Expand Up @@ -615,6 +618,7 @@ robot = KochRobot(
"laptop": OpenCVCamera(0, fps=30, width=640, height=480),
"phone": OpenCVCamera(1, fps=30, width=640, height=480),
},
max_relative_target=[10, 10, 10, 10, 10, 15],
)
robot.connect()
```
Expand Down
80 changes: 68 additions & 12 deletions lerobot/common/robot_devices/robots/koch.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import logging
import pickle
import time
from dataclasses import dataclass, field, replace
Expand Down Expand Up @@ -164,6 +165,7 @@ class KochRobotConfig:
leader_arms: dict[str, MotorsBus] = field(default_factory=lambda: {})
follower_arms: dict[str, MotorsBus] = field(default_factory=lambda: {})
cameras: dict[str, Camera] = field(default_factory=lambda: {})
max_relative_target: list[float] | None = None
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we choose a sensible default or default to None? @aliberts @Cadene



class KochRobot:
Expand Down Expand Up @@ -206,7 +208,15 @@ class KochRobot:
},
),
}
robot = KochRobot(leader_arms, follower_arms)
robot = KochRobot(
leader_arms=leader_arms,
follower_arms=follower_arms,
max_relative_target=[10, 10, 10, 10, 10, 15],
)

Notice the parameter `max_relative_target`. This is a safety measure that prevents someone from providing
a positional target that is too far from the current robot position (which would then cause the robot to
move too quickly, potentially burning out the motors or making violent impact with another object).

# Connect motors buses and cameras if any (Required)
robot.connect()
Expand All @@ -218,7 +228,11 @@ class KochRobot:
Example of highest frequency data collection without camera:
```python
# Assumes leader and follower arms have been instantiated already (see first example)
robot = KochRobot(leader_arms, follower_arms)
robot = KochRobot(
leader_arms=leader_arms,
follower_arms=follower_arms,
max_relative_target=[10, 10, 10, 10, 10, 15],
)
robot.connect()
while True:
observation, action = robot.teleop_step(record_data=True)
Expand All @@ -236,7 +250,12 @@ class KochRobot:
}

# Assumes leader and follower arms have been instantiated already (see first example)
robot = KochRobot(leader_arms, follower_arms, cameras)
robot = KochRobot(
leader_arms=leader_arms,
follower_arms=follower_arms,
cameras=cameras,
max_relative_target=[10, 10, 10, 10, 10, 15],
)
robot.connect()
while True:
observation, action = robot.teleop_step(record_data=True)
Expand All @@ -245,7 +264,12 @@ class KochRobot:
Example of controlling the robot with a policy (without running multiple policies in parallel to ensure highest frequency):
```python
# Assumes leader and follower arms + cameras have been instantiated already (see previous example)
robot = KochRobot(leader_arms, follower_arms, cameras)
robot = KochRobot(
leader_arms=leader_arms,
follower_arms=follower_arms,
cameras=cameras,
max_relative_target=[10, 10, 10, 10, 10, 15],
)
robot.connect()
while True:
# Uses the follower arms and cameras to capture an observation
Expand Down Expand Up @@ -392,7 +416,7 @@ def teleop_step(
# Send action
for name in self.follower_arms:
before_fwrite_t = time.perf_counter()
self.follower_arms[name].write("Goal_Position", follower_goal_pos[name])
self.send_action(torch.tensor(follower_goal_pos[name]), [name])
self.logs[f"write_follower_{name}_goal_pos_dt_s"] = time.perf_counter() - before_fwrite_t

# Early exit when recording data is not requested
Expand Down Expand Up @@ -474,21 +498,53 @@ def capture_observation(self):
obs_dict[f"observation.images.{name}"] = torch.from_numpy(images[name])
return obs_dict

def send_action(self, action: torch.Tensor):
"""The provided action is expected to be a vector."""
def send_action(self, action: torch.Tensor, follower_names: list[str] | None = None):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aliberts @Cadene I had to make this change (adding follower_arms kwarg) as teleop_step wants to set the follower arms one by one and get a dt for each. We could have also just not gotten a dt for each arm.

"""Command the follower arms to move to a target joint configuration.

The relative action magnitude may be clipped depending on the configuration parameter
`max_relative_target`.

Args:
action: tensor containing the concatenated joint positions for the follower arms.
follower_names: Pass follower arm names to only control a subset of all the follower arms.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Cadene @aliberts pleasee read the logic for this carefully and test if you can, as I don't have multiple arms.

"""
if not self.is_connected:
raise RobotDeviceNotConnectedError(
"KochRobot is not connected. You need to run `robot.connect()`."
)

if follower_names is None:
follower_names = list(self.follower_arms)
elif not set(follower_names).issubset(self.follower_arms):
raise ValueError(
f"You provided {follower_names=} but only the following arms are registered: "
f"{list(self.follower_arms)}"
)

from_idx = 0
to_idx = 0
follower_goal_pos = {}
for name in self.follower_arms:
if name in self.follower_arms:
to_idx += len(self.follower_arms[name].motor_names)
follower_goal_pos[name] = action[from_idx:to_idx].numpy()
from_idx = to_idx
for name in follower_names:
to_idx += len(self.follower_arms[name].motor_names)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Cadene @aliberts do we really expect this to be anything other than 6? The problem is that max_relative_target has a fixed length and refers to the motors of one robot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have resolved this by adding a post_init validation to the config class.

this_action = action[from_idx:to_idx]

if self.config.max_relative_target is not None:
max_relative_target = torch.tensor(self.config.max_relative_target)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @Cadene @aliberts. This is not related to this PR, but see how I need to juggle tensors and numpy arrays here? Another reason we should make this class only interface with numpy arrays.

# Cap relative action target magnitude for safety.
current_pos = torch.tensor(self.follower_arms[name].read("Present_Position"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aliberts @Cadene I'm not a fan of having to read the present position here. It adds time to the loop. This was one advantage of having this logic in the outer loop (not having to read the present position as we already knew what it was).

diff = this_action - current_pos
safe_diff = torch.minimum(diff, max_relative_target)
safe_diff = torch.maximum(safe_diff, -max_relative_target)
safe_action = current_pos + safe_diff
if not torch.allclose(safe_action, action):
logging.warning(
"Relative action magnitude had to be clamped to be safe.\n"
f" requested relative action target: {diff}\n"
f" clamped relative action target: {safe_diff}"
)

follower_goal_pos[name] = safe_action.numpy()
from_idx = to_idx

for name in self.follower_arms:
self.follower_arms[name].write("Goal_Position", follower_goal_pos[name].astype(np.int32))
Expand Down
Loading