ros2_intel_movidius_ncs

1 Introduction

The Movidius™ Neural Compute Stick (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit (VPU) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.

This project is a ROS2 wrapper for NC API of NCSDK, providing the following features:

• A ROS2 service for object classification and detection of a static image file
• A ROS2 publisher for object classification and detection of a video stream from a RGB camera
• Demo applications to show the capabilities of ROS2 service and publisher
• Support multiple CNN models of Caffe and Tensorflow

2 Prerequisite

• An x86_64 computer running Ubuntu 16.04. OS X and Windows are not supported yet
• ROS2 Bouncy
• Movidius Neural Compute Stick (NCS)
• Movidius Neural Compute (MvNC) SDK
• Movidius Neural Compute Application Zoo
• RGB Camera, e.g. RealSense D400 Series

3 Environment Setup

• Install OpenCV 3.x (guide)
• Build ROS2 Core from source (guide)
• Create a ROS2 overlay workspace

mkdir -p ~/ros2_overlay_ws/src
source ~/ros2_ws/install/local_setup.bash

• Install NCSDK v1.12.00 (github)
• Install NC APP Zoo(github)
• NCSDK should be installed in /opt/movidius by default. Create a symbol link in /opt/movidius to NC APP Zoo.

sudo ln -s <your-workspace>/ncappzoo /opt/movidius/ncappzoo

After that, make sure you can find graph data in /opt/movidius/ncappzoo/caffe or /opt/movidius/ncappzoo/tensorflow and image data in /opt/movidius/ncappzoo/data/images

• Install ROS2 package for different cameras as needed. e.g.

  1. RealSense D400 series camera (github)   Note: Create a symbol link from libusb.a to libusb-1.0.a, otherwise "libusb.a" is probably not to be found by librealsense.
        sudo ln -s /usr/lib/x86_64-linux-gnu/libusb-1.0.a /usr/lib/libusb.a

• Install object_msgs for ROS2 (github)

cd ~/ros2_overlay_ws/src
git clone https://github.com/intel/ros2_object_msgs

• Install ROS2 Message Filters (github)

cd /usr/lib/x86_64-linux-gnu
sudo ln -s libboost_python-py35.so libboost_python3.so
cd ~/ros2_overlay_ws/src
git clone https://github.com/ros2/message_filters

• Install ROS2 vision_opencv (github)

cd ~/ros2_overlay_ws/src
git clone https://github.com/ros-perception/vision_opencv
cd vision_opencv
git checkout ros2

4 Building and Installation

cd ~/ros2_overlay_ws/src
git clone https://github.com/intel/ros2_intel_movidius_ncs
git clone https://github.com/intel/ros2_object_msgs
cd ~/ros2_overlay_ws/
colcon build --symlink-install
source install/local_setup.bash

Copy object label file from this project to NCSDK installation location.

cp ~/ros2_overlay_ws/src/ros2_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/

5 Running the Demo

5.1 Classification

5.1.1 Supported CNN Models

Table1

CNN Model	Framework	Usage
AlexNet	Caffe	Image/Video
GoogLeNet	Caffe	Image/Video
SqueezeNet	Caffe	Image/Video
Inception_v1	Tensorflow	Image/Video
Inception_v2	Tensorflow	Image/Video
Inception_v3	Tensorflow	Image/Video
Inception_v4	Tensorflow	Image/Video
MobileNet	Tensorflow	Image/Video

5.1.2 Classification Result with GoogLeNet

5.1.3 Running the Demo

• Static Image
• Video Streaming

5.2 Detection

5.1.1 Supported CNN Models

CNN Model	Framework	Usage
MobileNetSSD(Recommended)	Caffe	Image/Video
TinyYolo_v1	Caffe	Image/Video

5.1.2 Detection Result with MobileNetSSD

5.1.3 Running the Demo

• Static Image
• Video Streaming

6 Interfaces

6.1 Topic

Classification: /movidius_ncs_nodelet/classified_objects
Detection: /movidius_ncs_nodelet/detected_objects

6.2 Service

Classification: /movidius_ncs_image/classify_object
Detection: /movidius_ncs_image/detect_object

7 Known Issues

• Only absolute path of image file supported in image inference demo
• Only test on RealSense D400 series camera

8 TODO

• Keep synchronized with ROS NCS Package

Any security issue should be reported using process at https://01.org/security