diff --git a/image-projects/robotic-arm-sorting-arduino-braccio.md b/image-projects/robotic-arm-sorting-arduino-braccio.md
index af98964..e0e00b4 100644
--- a/image-projects/robotic-arm-sorting-arduino-braccio.md
+++ b/image-projects/robotic-arm-sorting-arduino-braccio.md
@@ -9,6 +9,8 @@ Created By: Naveen Kumar
Public Project Link: [https://studio.edgeimpulse.com/public/178900/live](https://studio.edgeimpulse.com/public/178900/live)
+GitHub Repository: [https://github.com/metanav/EI_Pick_n_Place/tree/main/pnp_ws/src/braccio_description/urdf](https://github.com/metanav/EI_Pick_n_Place/tree/main/pnp_ws/src/braccio_description/urdf)
+
## Introduction
@@ -28,17 +30,17 @@ We are using [Arduino Braccio ++](https://www.arduino.cc/education/braccio/) for
-For a depth camera, we will be utilizing [Luxonis OAK-D](https://docs.luxonis.com/projects/hardware/en/latest/pages/BW1098OAK/) which will be doing object recognition and localization. An object detection model trained using the Edge Impulse Studio will be deployed directly on the OAK-D camera.
+For a depth camera, we will be utilizing the [Luxonis OAK-D](https://docs.luxonis.com/projects/hardware/en/latest/pages/BW1098OAK/), which will be doing object recognition and localization. An object detection model trained using the Edge Impulse Studio will be deployed directly on the OAK-D camera.
-A Raspberry Pi 5 will be used as a main controller to host ROS 2 nodes and an interface between the robotic arm and the depth camera.
+A Raspberry Pi 5 will be used as a main controller, to host ROS 2 nodes and an interface between the robotic arm and the depth camera.
Instead of sticking with the same old boring color cubes 🧊 that you see everywhere online for a pick-and-place demo, we’re going to have some fun sorting through these colorful plastic toys, **Penguins** 🐧 and **Pigs** 🐷!
-
+
## Setting up the Development Environment
@@ -46,13 +48,11 @@ We can use the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) to i
-
-
After the installation is completed, we can insert the SD card back into the kit and power it on. Once it boots up, we can log in via ssh.
## Installing ROS 2 Humble
-The Robot Operating System (ROS) is a set of software libraries and tools for building robot applications. We will use ROS 2 Humble for this project since it is stable on the Raspberry Pi OS. The ROS 2 binary packages are not available for Raspberry Pi OS so we need to build it from the source. Please follow the steps below to install it.
+The Robot Operating System (ROS) is a set of software libraries and tools for building robot applications. We will use ROS 2 Humble for this project since it is stable on the Raspberry Pi OS. The ROS 2 binary packages are not available for Raspberry Pi OS, so we need to build it from the source. Please follow the steps below to install it.
### Set locale
@@ -178,13 +178,13 @@ $ MAKEFLAGS="-j1 -l1" colcon build
## micro-ROS
-The **micro-ROS** stack integrates microcontrollers seamlessly with standard ROS 2 and brings all major ROS concepts such as nodes, publishers, subscriptions, parameters, and lifecycle onto embedded systems. We will use the micro-ROS on the **Arduino Nano RP2040 Connect** mounted on the **Braccio Carrier** board. The Arduino Nano RP2040 will publish the joint states and subscribe to the arm manipulation commands. It will communicate to ROS 2 on the Raspberry Pi 5 over serial port transports.
+The **micro-ROS** stack integrates microcontrollers seamlessly with standard ROS 2 and brings all major ROS concepts such as nodes, publishers, subscriptions, parameters, and lifecycle onto embedded systems. We will use micro-ROS on the **Arduino Nano RP2040 Connect** mounted on the **Braccio Carrier** board. The Arduino Nano RP2040 will publish the joint states and subscribe to the arm manipulation commands. It will communicate to ROS 2 on the Raspberry Pi 5 over serial port transports.
### micro-ROS Agent Installation
-The **micro-ROS agent** is a ROS 2 node that receives and sends messages from micro-ROS nodes and keeps track of the micro-ROS nodes exposing them to the ROS 2 network. Execute the following command to install the micro-ROS agent on the Raspberry Pi 5.
+The **micro-ROS agent** is a ROS 2 node that receives and sends messages from micro-ROS nodes and keeps track of the micro-ROS nodes, exposing them to the ROS 2 network. Execute the following command to install the micro-ROS agent on the Raspberry Pi 5.
```
$ mkdir ~/microros_ws && cd microros_ws
@@ -198,7 +198,6 @@ $ ros2 run micro_ros_setup create_agent_ws.sh
$ ros2 run micro_ros_setup build_agent.sh
```
-
## Data Collection
We captured 101 images of the pigs and penguins using the OAK-D camera and uploaded them to Edge Impulse Studio.
@@ -209,14 +208,10 @@ We can see the uploaded images on the **Data Acquisition** page.
-
-
We can now label the data using bounding boxes in the **Labeling Queue** tab, as demonstrated in the GIF below.
-
-
## Model Training
To create an Impulse, follow these steps:
@@ -228,36 +223,27 @@ To create an Impulse, follow these steps:
-####
-
On the **Image** page, choose *RGB* as color depth and click on the **Save parameters** button. The page will be redirected to the **Generate Features** page.
-
Now we can initiate feature generation by clicking on the **Generate features** button. Once the feature generation is completed, the data visualization will be visible in the **Feature Explorer** panel.
-####
-
Go to the **Object Detection** page, then click "Choose a different model" and select the **YOLOv5** model. There are 4 variations of the model size available, and we selected the **Nano** version with 1.9 million parameters. Afterward, click the "Start training" button. The training process will take a few minutes to complete.
-
-
-
+
Once the training is completed we can see the precision score and metrics as shown below.
-
+
## Model Testing
On the **Model testing** page, click on the "Classify All" button which will initiate model testing with the trained float32 model. The testing accuracy is **100%**.
-
-
-
+
## Model Deployment
@@ -307,8 +293,6 @@ We can see the inferencing output on the web browser. Also, we can monitor the
-
-
To allow DepthAI to use our custom-trained model, we need to convert them into a MyriadX blob file format so that they are optimized for the Movidius Myriad X processor on the OAK-D.
@@ -317,8 +301,6 @@ The Edge Impulse Studio helps us save a step by providing the ONNX format for th
-
-
We will utilize the **OpenVINO** model optimizer for conversion on an x86 Linux machine. OpenVINO is an open-source software toolkit for optimizing and deploying deep learning models. Execute the following commands to install all prerequisites for the conversion process.
```
@@ -411,7 +393,7 @@ blob_path = blobconverter.from_openvino(
shutil.move(str(blob_path), blob_dir)
```
-This will create the *ei-pnp_yolov5n_320_openvino_2022.1_6shave.blob* file in the IR directory. We should copy this blob file to the ~/EI_Pick_n_Place/pnp_ws/src/ei_yolov5_detections/resources on the Raspberry Pi 5. We can test the generated model using the depthai-python library.
+This will create the *ei-pnp_yolov5n_320_openvino_2022.1_6shave.blob* file in the IR directory. We should copy this blob file to the `~/EI_Pick_n_Place/pnp_ws/src/ei_yolov5_detections/resources` folder on the Raspberry Pi 5. We can test the generated model using the depthai-python library:
```
$ pip3 install -U pip
@@ -437,7 +419,7 @@ First, we need to define a visual model of the Arduino Braccio ++ using the URDF
-We created a ROS 2 package `moveit_resources_braccio_description` to keep all STL files and URDF for reusability. The robot model URDF can be found in the GitHub repository for this project:
+We created a ROS 2 package `moveit_resources_braccio_description` to keep all STL files and URDF for reusability. The robot model URDF can be found in the GitHub repository for this project:
https://github.com/metanav/EI_Pick_n_Place/tree/main/pnp_ws/src/braccio_description/urdf
@@ -459,14 +441,10 @@ $ colcon build --packages-select moveit_resources_braccio_description
$ ros2 launch moveit_resources_braccio_description display.launch.py
```
-
-
By adjusting the sliders for the joints, we can observe the corresponding changes in the robot model.
-
-
### Generate configuration using the MoveIt Setup Assistant 2.0
The MoveIt Setup Assistant 2.0 is a GUI for configuring the manipulator for use with MoveIt 2. Its primary function is generating a Semantic Robot Description Format (SRDF) file for the manipulator, which specifies additional information required by MoveIt 2 such as planning groups, end effectors, and various kinematic parameters. Additionally, it generates other necessary configuration files for use with the MoveIt 2 pipeline.
@@ -487,8 +465,6 @@ To generate the collision matrix, select the **Self-Collisions** pane on the lef
-
-
We will define a `fixed` virtual joint that attaches the `base_link` of the arm to the `world` frame. This virtual joint signifies that the base of the arm remains stationary in the world frame.
@@ -509,13 +485,13 @@ Now we can designate the `braccio_gripper` group as an end effector. The end e
-## Arduino Braccio++ controller firmware
+## Arduino Braccio++ Controller Firmware
Please follow the instructions [here](https://www.arduino.cc/en/software) to download and install the Arduino IDE. After installation, open the Arduino IDE and install the board package for the **Arduino Mbed OS Nano Boards** by going to **Tools** > **Board** > **Boards Manager**. Search the board package as shown below and install it.
-After completing the board package installation, choose the **Arduino Nano RP2040 Connect** from **Tools** > **Board** > **Arduino Mbed OS Nano boards** menu. We must install [Arduino_Braccio_plusplus](https://github.com/arduino-libraries/Arduino_Braccio_plusplus) (1.3.2) and [micro_ros_arduino]([micro_ros_arduino](https://github.com/micro-ROS/micro_ros_arduino)) (humble) libraries. The firmware sketch can be found in the GitHub repository:
+After completing the board package installation, choose the **Arduino Nano RP2040 Connect** from **Tools** > **Board** > **Arduino Mbed OS Nano boards** menu. We must install **[Arduino_Braccio_plusplus](https://github.com/arduino-libraries/Arduino_Braccio_plusplus) (1.3.2)** and **[micro_ros_arduino]([micro_ros_arduino](https://github.com/micro-ROS/micro_ros_arduino)) (humble)** libraries. The firmware sketch can be found in the GitHub repository:
https://github.com/metanav/EI_Pick_n_Place/blob/main/Arduino/braccio_plus_plus_controller_final_v3.1/braccio_plus_plus_controller_final_v3.1.ino.
@@ -523,11 +499,11 @@ Now we should build and upload the firmware to the Arduino Nano RP2040 connect.
## Launch ROS 2 Nodes
-We should launch the ROS 2 nodes on separate terminals on the Raspberry Pi 5 by executing the following commands step-by-step.
+We should launch the ROS 2 nodes on separate terminals on the Raspberry Pi 5 by executing the following commands step-by-step.
1. ##### Launch micro-ROS agent
- The micro-ROS agent exposes the publishers and action server running on the Braccio ++ MCU to the ROS 2.
+ The micro-ROS agent exposes the publishers and action server running on the Braccio ++ MCU to ROS 2.
```
$ source ~/ros2_humble/install/setup.sh
@@ -543,7 +519,7 @@ We should launch the ROS 2 nodes on separate terminals on the Raspberry Pi 5 by
$ source ~/ros2_humble/install/setup.sh
$ source ~/dai_ws/install/setup.sh
$ source ~/pnp_ws/install/setup.sh
- $ ros2 launch ei_yolov5_detections ei_yolov5_publisher.launch.py
+ $ ros2 launch ei_yolov5_detections ei_yolov5_publisher.launch.py
```
We can check the spatial detection message as follows.
@@ -583,9 +559,7 @@ We should launch the ROS 2 nodes on separate terminals on the Raspberry Pi 5 by
The `pick_n_place` node plans a pick and place operation using [MoveIt Task Constructor](https://github.com/ros-planning/moveit_task_constructor/tree/ros2/). MoveIt Task Constructor provides a way to plan for tasks that consist of multiple different subtasks (known as stages as shown in the image below).
-
-
- 
+
This node subscribes to the `/ei_yolov5/spatial_detections` topic and plans the pick and place operation. While bringing up this node, we need to provide command line parameters for the exact (X, Y, Z) position of the camera in meters from the base of the robot.
@@ -625,6 +599,3 @@ We should launch the ROS 2 nodes on separate terminals on the Raspberry Pi 5 by
This project successfully demonstrates the design and implementation of a sophisticated pick-and-place system using a robot arm equipped with a 3D depth camera. The system's ability to recognize and locate objects in a cluttered and dynamic environment, coupled with its precise grasping and placing actions, showcases its potential for various industrial and domestic applications. This project underscores the complexity and importance of sorting tasks in various sectors, from manufacturing to logistics, and demonstrates how advanced robotic systems can meet these challenges with high efficiency and accuracy.
-Edge Impulse Studio Project Link: https://studio.edgeimpulse.com/public/178900/live
-
-