Computer vision project for the MSOE Senior Design Team
- Set up stream to simultaneously intake RGB images and depth maps from an Intel RealSense D435 Depth Camera
- Identify objects in RGB image using a custom trained YOLOv5 network and Batch Mode Cluster-NMS, and then use OpenCV to determine the state of all platform and goal objects
- Obtain the distance from the camera to each of the identified objects using OpenCV/trig and the D435's depth map
- Use the D435's intrinsic parameters and trig to determine the horizontal/vertical angles of the identified objects relative to the camera
- Make use of the ros_tf2 library to create a static broadcaster ROS node that transforms the object vectors (created from the data in steps 3 and 4) from the coordinate frame of the camera (1) to the coordinate frame of the center of the robot, at ground level (2)
- Create a second ROS node which subscribes to the static broadcaster ROS node which uses the coordinates of the robot (x, y, θ) and the object vectors, which are now in the robot's coordinate frame, to determine the coordinates of each object, and then publish them, along with the state of the platforms and goals
