Get Camera Working

[schromya]

• The bot has a camera on top that we need to interface with. Figure how to get it working and interface with it.

Tips:

• Object Detection with retrainable model like YOLO or other
• Hand-arm-calibration to configure the camera position wrt the robot coordinate
• Coordinate Transformation from camera frame to robot frame

[schromya]

Use Kinect or intel d-435 camera in box.

[schromya]

Yay Rus got docker file to work and to show camera feed (needed EULA agreement for docker). Got C-C cable to work on Panda Computer! Microphone isn't working in Container (but works outside). Could need to tell Docker that there's a mic. Or could be driver issue.

[schromya]

Rus is looking into smaller models that can run locally.

[schromya]

Rus setup meeting with other researcher about lightweight model.

[schromya]

Rus will start with Llama 3.2 and perhaps try Florence 2 as well.