DISCONTINUATION OF PROJECT

This project will no longer be maintained by Intel.

Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.

Intel no longer accepts patches to this project.

If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.

Contact: [email protected]

ros2_intel_movidius_ncs

Warning: This repo is deprecated. For latest ROS2 wrapper for Movidius NCS, please refer to our project ros2_openvino_toolkit.

See: https://github.com/intel/ros2_openvino_toolkit!!

1 Introduction

The Movidius™ Neural Compute Stick (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit (VPU) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.

This project is a ROS2 wrapper for NC API of NCSDK, providing the following features:

• A ROS2 service for object classification and detection of a static image file
• A ROS2 publisher for object classification and detection of a video stream from a RGB camera
• Demo applications to show the capabilities of ROS2 service and publisher
• Support multiple CNN models of Caffe and Tensorflow

2 Prerequisite

• An x86_64 computer running Ubuntu 16.04. OS X and Windows are not supported yet
• ROS2 Bouncy
• Movidius Neural Compute Stick (NCS)
• Movidius Neural Compute (MvNC) SDK
• Movidius Neural Compute Application Zoo
• RGB Camera, e.g. RealSense D400 Series

3 Environment Setup

• Install OpenCV 3.x (guide)
• Install ROS2 Ardent (guide)
• Create a ROS2 workspace

mkdir -p ~/ros2_ws/src

• Install NCSDK v1.12.00 (github)
• Install NC APP Zoo(github)
• NCSDK should be installed in /opt/movidius by default. Create a symbol link in /opt/movidius to NC APP Zoo.

sudo ln -s <your-workspace>/ncappzoo /opt/movidius/ncappzoo

After that, make sure you can find graph data in /opt/movidius/ncappzoo/caffe or /opt/movidius/ncappzoo/tensorflow and image data in /opt/movidius/ncappzoo/data/images

• Install object_msgs for ROS2 (github)
• Install ROS2 package for different cameras as needed. e.g.
  1. RealSense D400 series camera (github)   Note: Create a symbol link from libusb.a to libusb-1.0.a, otherwise "libusb.a" is probably not to be found by librealsense.
        sudo ln -s /usr/lib/x86_64-linux-gnu/libusb-1.0.a /usr/lib/libusb.a
• Install ROS2 Message Filters (github)

cd /usr/lib/x86_64-linux-gnu
sudo ln -s libboost_python-py35.so libboost_python3.so
cd ~/ros2_ws/src
git clone https://github.com/ros2/message_filters

• Install ROS2 vision_opencv (github)

cd ~/ros2_ws/src
git clone https://github.com/ros-perception/vision_opencv
cd vision_opencv
git checkout ros2

4 Building and Installation

cd ~/ros2_ws/src
git clone https://github.com/intel/ros2_intel_movidius_ncs
git clone https://github.com/intel/ros2_object_msgs
cd ~/ros2_ws
ament build
source install/local_setup.bash

Copy object label file from this project to NCSDK installation location.

cp ~/ros2_ws/src/ros2_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/

5 Running the Demo

5.1 Classification

5.1.1 Supported CNN Models

Table1

CNN Model	Framework	Usage
AlexNet	Caffe	Image/Video
GoogLeNet	Caffe	Image/Video
SqueezeNet	Caffe	Image/Video
Inception_v1	Tensorflow	Image/Video
Inception_v2	Tensorflow	Image/Video
Inception_v3	Tensorflow	Image/Video
Inception_v4	Tensorflow	Image/Video
MobileNet	Tensorflow	Image/Video

5.1.2 Classification Result with GoogLeNet

5.1.3 Running the Demo

• Static Image
• Video Streaming

5.2 Detection

5.1.1 Supported CNN Models

CNN Model	Framework	Usage
MobileNetSSD(Recommended)	Caffe	Image/Video
TinyYolo_v1	Caffe	Image/Video

5.1.2 Detection Result with MobileNetSSD

5.1.3 Running the Demo

• Static Image
• Video Streaming

6 Interfaces

6.1 Topic

Classification: /movidius_ncs_nodelet/classified_objects
Detection: /movidius_ncs_nodelet/detected_objects

6.2 Service

Classification: /movidius_ncs_image/classify_object
Detection: /movidius_ncs_image/detect_object

7 Known Issues

• Only absolute path of image file supported in image inference demo
• Only test on RealSense D400 series camera

8 TODO

• Keep synchronized with ROS NCS Package

Any security issue should be reported using process at https://01.org/security