handeye_calibration_ros2

This is a ROS2 python handeye calibration package. It is hardware-independent and can be applied to any depth/RGB camera with ROS2 wrappers available.

In summary, it provides functionality includes:

1. Sample and save the robot pose and tracking object pose with a simple keyboard click.
2. Compute and save the result of the eye-in-hand calibration through the Tsai-Lenz method implemented with OpenCV.
3. Publish the hand-eye calibration result and visualize it rviz.

We also provided a simulation demo where you can add a virtual camera to start using the package. Please find the pkg handeye_calibration_ros2_sim

To adapt to your usage, you only need to modify all the information in a single file:

1. The camera image topic
2. The name of the robot link names
3. The camera matrix and distortion coefficient

You can also modify other parameters as per your preference in the information file.

Please feel free to raise an issue as we seek to test the package performance under different hardware software environments. This would help us improve the package and make it truly universal!

Hardware and Software Tested:

Hardware:

1. Robot: KUKA LBR iiwa 7
2. Camera:
   • Intel RealSense D415
   • Microsoft Azure Kinect
3. Chessboard
4. ArUco Marker:
   • ID: 365
   • Side Length: 150mm

Software:

1. Robot Driver: lbr_fri_ros2_stack
2. Camera Drivers:
   • realsense-ros
   • Azure_Kinect_ROS_Driver
3. Operating System: Ubuntu 22.04
4. ROS 2: Humble

Table of Contents

1. Overview
2. Installation
3. Usage
4. Example
5. Contributing
6. Citation

Overview

This package provides a solution for hand-eye calibration between a robot arm and a camera in a ROS2 environment. It is designed to support various robotic and vision systems, offering flexibility for different hardware configurations.

Installation

1. Create your workspace and clone this repository:

   mkdir -p handeye_calibration_ws/src
   cd handeye_calibration_ws/src
   git clone https://github.com/shengyangzhuang/handeye_calibration_ros2.git

2. Install dependencies:

   rosdep install --from-paths src --ignore-src -r -y

3. Build the package

   RealSense: colcon build --packages-select handeye_realsense

   Azure Kinect: colcon build --packages-select handeye_ak

4. Source your ROS2 workspace:

   source install/setup.bash
   

Prerequisits

1. Install the RealSense ROS2 wrapper.

Usage

1. Calibrate the camera intrinsics and record the intrinsic matrix K and distortion coefficient D at realsense_info.yaml. We also provided a package using the chessboard to calibrate the camera intrinsics, see camera_calibration.

2. Change the parameters at config.yaml

3. Start the camera node

   RealSense:

   ros2 run realsense2_camera realsense2_camera_node

4. To start the hand-eye calibration, launch the node as follows and press key q to record marker and robot poses:

   ros2 launch handeye_realsense taking_sample_launch.py

5. To calculate the hand-eye calibration result:

   ros2 run handeye_realsense handeye

6. To visualize the result in rviz, publish the hand-eye transformation by:

   ros2 run handeye_realsense eye2hand

Example

We provided an example usage with RealSense camera. Please refer here.

A video of hand-eye calibration process can be seen here, where we demonstrated ensuring calibration accuracy through simulation and replicating the similar poses in real LBR iiwa 7 robot and RealSense depth camera.

Contributing

We welcome contributions from the community! If you'd like to contribute, please follow these steps:

1. Fork the repository: Click the "Fork" button at the top of this repository to create a copy on your GitHub account.

2. Create a feature branch: Clone the forked repository to your local machine and create a new branch for your feature or bug fix:

   git checkout -b feature/my-feature

3. Make your changes: Implement your changes in the code, and don't forget to write tests if applicable.

4. Commit your changes: Once your changes are ready, commit them with a descriptive commit message:

   git commit -m 'Add some feature'

   Push your changes: Push your changes to the branch on your forked repository:

   git push origin feature/my-feature

   Open a Pull Request: Go to the original repository on GitHub and open a Pull Request (PR). Make sure to describe your changes in detail and explain why they are necessary.

Citation

If you enjoyed using this package, we would really appreciate it if you could leave a ⭐ and / or cite it!

@mastersthesis{zhuang2024multirobot,
  author    = {Zhuang, Shengyang},
  title     = {Multi-Robot System Prototyping for Cooperative Control in Robot-Assisted Spine Surgery},
  school    = {Imperial College London},
  year      = {2024},
}