Merge branch 'devel' of https://github.com/intel/ros2_openvino_toolkit

README.md

@@ -58,6 +58,7 @@ Currently, the inference feature list is supported:
 |Object Detection| object detection based on SSD-based trained models.|
 |Vehicle Detection| Vehicle and passenger detection based on Intel models.|
 |Object Segmentation| object detection and segmentation.|
+|Person Reidentification| Person Reidentification based on object detection.|
 
 ## ROS interfaces and outputs
 ### Topic
@@ -77,6 +78,8 @@ Currently, the inference feature list is supported:
 ```/ros2_openvino_toolkit/detected_objects```([object_msgs::msg::ObjectsInBoxes](https://github.com/intel/ros2_object_msgs/blob/master/msg/ObjectsInBoxes.msg))
 - Object Segmentation:
 ```/ros2_openvino_toolkit/segmented_obejcts```([people_msgs::msg::ObjectsInMasks](https://github.com/intel/ros2_openvino_toolkit/blob/devel/people_msgs/msg/ObjectsInMasks.msg))
+- Person Reidentification:
+```/ros2_openvino_toolkit/reidentified_persons```([people_msgs::msg::ReidentificationStamped](https://github.com/intel/ros2_openvino_toolkit/blob/devel/people_msgs/msg/ReidentificationStamped.msg))
 - Rviz Output:
 ```/ros2_openvino_toolkit/image_rviz```([sensor_msgs::msg::Image](https://github.com/ros2/common_interfaces/blob/master/sensor_msgs/msg/Image.msg))
 
@@ -112,9 +115,57 @@ See below pictures for the demo result snapshots.
 * object segmentation input from video
 ![object_segmentation_demo_video](https://github.com/intel/ros2_openvino_toolkit/blob/devel/data/images/object_segmentation.gif "object segmentation demo video")
 
+* Person Reidentification input from standard camera
+![person_reidentification_demo_video](https://github.com/intel/ros2_openvino_toolkit/blob/devel/data/images/person-reidentification.gif "person reidentification demo video")
+
 # Installation & Launching
 **NOTE:** Intel releases 2 different series of OpenVINO Toolkit, we call them as [OpenSource Version](https://github.com/opencv/dldt/) and [Tarball Version](https://software.intel.com/en-us/openvino-toolkit). This guidelie uses OpenSource Version as the installation and launching example. **If you want to use Tarball version, please follow [the guide for Tarball Version](https://github.com/intel/ros2_openvino_toolkit/blob/devel/doc/BINARY_VERSION_README.md).**
 
+## Enable Intel® Neural Compute Stick 2 (Intel® NCS 2) under the OpenVINO Open Source version (Optional) </br>
+1. Intel Distribution of OpenVINO toolkit </br>
+	* Download OpenVINO toolkit by following the [guide](https://software.intel.com/en-us/openvino-toolkit/choose-download)</br>
+	```bash
+	cd ~/Downloads
+	wget -c http://registrationcenter-download.intel.com/akdlm/irc_nas/15078/l_openvino_toolkit_p_2018.5.455.tgz
+	```
+	* Install OpenVINO toolkit by following the [guide](https://software.intel.com/en-us/articles/OpenVINO-Install-Linux) </br>
+	```bash
+	cd ~/Downloads
+	tar -xvf l_openvino_toolkit_p_2018.5.455.tgz
+	cd l_openvino_toolkit_p_2018.5.455
+	# root is required instead of sudo
+	sudo -E ./install_cv_sdk_dependencies.sh
+	sudo ./install_GUI.sh
+	# build sample code under OpenVINO toolkit
+	source /opt/intel/computer_vision_sdk/bin/setupvars.sh
+ 	cd /opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/
+ 	mkdir build
+	cd build
+ 	cmake ..
+ 	make
+	```
+	* Configure the Neural Compute Stick USB Driver
+	```bash
+	cd ~/Downloads
+	cat <<EOF > 97-usbboot.rules
+	SUBSYSTEM=="usb", ATTRS{idProduct}=="2150", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
+	SUBSYSTEM=="usb", ATTRS{idProduct}=="2485", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
+	SUBSYSTEM=="usb", ATTRS{idProduct}=="f63b", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
+	EOF
+	sudo cp 97-usbboot.rules /etc/udev/rules.d/
+	sudo udevadm control --reload-rules
+	sudo udevadm trigger
+	sudo ldconfig
+	rm 97-usbboot.rules
+	```
+
+2. Configure the environment (you can write the configuration to your ~/.basrch file)</br>
+	**Note**: If you used root privileges to install the OpenVINO binary package, it installs the Intel Distribution of OpenVINO toolkit in this directory: */opt/intel/openvino_<version>/*
+	```bash
+	export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib
+	source /opt/intel/computer_vision_sdk/bin/setupvars.sh
+	```
+
 ## Dependencies Installation
 One-step installation scripts are provided for the dependencies' installation. Please see [the guide](https://github.com/intel/ros2_openvino_toolkit/blob/devel/doc/OPEN_SOURCE_CODE_README.md) for details.
 
@@ -144,6 +195,8 @@ One-step installation scripts are provided for the dependencies' installation. P
 		python3 downloader.py --name age-gender-recognition-retail-0013
 		python3 downloader.py --name emotions-recognition-retail-0003
 		python3 downloader.py --name head-pose-estimation-adas-0001
+		python3 downloader.py --name person-detection-retail-0013
+		python3 downloader.py --name person-reidentification-retail-0076
 		```
 	* copy label files (excute _once_)<br>
 		```bash
@@ -180,6 +233,10 @@ One-step installation scripts are provided for the dependencies' installation. P
 	```bash
 	ros2 launch dynamic_vino_sample pipeline_video.launch.py
 	```
+* run person reidentification sample code input from StandardCamera.
+	```bash
+	ros2 launch dynamic_vino_sample pipeline_reidentification_oss.launch.py
+	```
 * run object detection service sample code input from Image  
   Run image processing service:
 	```bash
@@ -189,6 +246,15 @@ One-step installation scripts are provided for the dependencies' installation. P
 	```bash
 	ros2 run dynamic_vino_sample image_object_client ~/Pictures/car.png
 	```
+* run face detection service sample code input from Image  
+  Run image processing service:
+	```bash
+	ros2 launch dynamic_vino_sample image_people_server_oss.launch.py
+	```
+  Run example application with an absolute path of an image on another console:
+	```bash
+	ros2 run dynamic_vino_sample image_people_client ~/Pictures/face.png
+	```
 
 # TODO Features
 * Support **result filtering** for inference process, so that the inference results can be filtered to different subsidiary inference. For example, given an image, firstly we do Object Detection on it, secondly we pass cars to vehicle brand recognition and pass license plate to license number recognition.
@@ -198,3 +264,4 @@ One-step installation scripts are provided for the dependencies' installation. P
 # More Information
 * ROS2 OpenVINO discription writen in Chinese: https://mp.weixin.qq.com/s/BgG3RGauv5pmHzV_hkVAdw 
 
+

doc/BINARY_VERSION_README.md

@@ -15,11 +15,11 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 * Demo application to show above detection and recognitions
 
 ## 2. Prerequisite
-- An x86_64 computer running Ubuntu 16.04. Below processors are supported:
+- An x86_64 computer running Ubuntu 18.04. Below processors are supported:
 	* 6th-8th Generation Intel® Core™
 	* Intel® Xeon® v5 family
 	* Intel®  Xeon® v6 family
-- ROS2 [Bouncy](https://github.com/ros2/ros2/wiki)
+- ROS2 [Crystal](https://github.com/ros2/ros2/wiki)
 - [OpenVINO™ Toolkit](https://software.intel.com/en-us/openvino-toolkit)
 - RGB Camera, e.g. RealSense D400 Series or standard USB camera or Video/Image File
 - Graphics are required only if you use a GPU. The official system requirements for GPU are:
@@ -38,7 +38,7 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 ./environment_setup_binary.sh
 ```
 **Note**:You can also choose to follow the steps below to build the environment step by step.
-* Install ROS2 [Bouncy](https://github.com/ros2/ros2/wiki) ([guide](https://github.com/ros2/ros2/wiki/Linux-Development-Setup))<br>
+* Install ROS2 [Crystal](https://github.com/ros2/ros2/wiki) ([guide](https://index.ros.org/doc/ros2/Installation/Linux-Development-Setup/))<br>
 * Install [OpenVINO™ Toolkit](https://software.intel.com/en-us/openvino-toolkit) ([guide](https://software.intel.com/en-us/articles/OpenVINO-Install-Linux))<br>
     	**Note**: Please use  *root privileges* to run the installer when installing the core components.
 * Install OpenCL Driver for GPU
@@ -178,6 +178,10 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 	```bash
 	ros2 launch dynamic_vino_sample pipeline_video.launch.py
 	```
+* run person reidentification sample code input from StandardCamera.
+	```bash
+	ros2 launch dynamic_vino_sample pipeline_reidentification.launch.py
+	```
 * run object detection service sample code input from Image  
   Run image processing service:
 	```bash
@@ -187,6 +191,16 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 	```bash
 	ros2 run dynamic_vino_sample image_object_client ~/Pictures/car.png
 	```
+* run people detection service sample code input from Image  
+  Run image processing service:
+	```bash
+	ros2 launch dynamic_vino_sample image_people_server.launch.py
+	```
+  Run example application with an absolute path of an image on another console:
+	```bash
+	ros2 run dynamic_vino_sample image_people_client ~/Pictures/face.png
+	```
+
 
 ## 6.Known Issues
 * Possible problems
@@ -196,6 +210,6 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 		# or
 		E: [ncAPI] [         0] ncDeviceCreate:324      global mutex initialization failed
 		```
-	> solution - Please reboot while connecting Intel® Neural Compute Stick 2.
+	> solution - Please refer to the [guide](https://software.intel.com/en-us/neural-compute-stick/get-started) to set up the environment.
 
 

doc/OPEN_SOURCE_CODE_README.md

@@ -15,11 +15,11 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 * Demo application to show above detection and recognitions
 
 ## 2. Prerequisite
-- An x86_64 computer running Ubuntu 16.04 and Ubuntu 18.04. Below processors are supported:
+- An x86_64 computer running Ubuntu 18.04. Below processors are supported:
 	* 6th-8th Generation Intel® Core™
 	* Intel® Xeon® v5 family
 	* Intel®  Xeon® v6 family
-- ROS2 [Bouncy](https://github.com/ros2/ros2/wiki)
+- ROS2 [Crystal](https://github.com/ros2/ros2/wiki)
 
 - OpenVINO™ Toolkit Open Source<br>
   	* The [Deep Learning Deployment Toolkit](https://github.com/opencv/dldt) that helps to enable fast, heterogeneous deep learning inferencing for Intel® processors (CPU and GPU/Intel® Processor Graphics), and supports more than 100 public and custom models.<br>
@@ -41,7 +41,7 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 ./environment_setup.sh
 ```
 **Note**:You can also choose to follow the steps below to build the environment step by step.
-* Install ROS2 [Bouncy](https://github.com/ros2/ros2/wiki) ([guide](https://github.com/ros2/ros2/wiki/Linux-Development-Setup))<br>
+* Install ROS2 [Crystal](https://github.com/ros2/ros2/wiki) ([guide](https://index.ros.org/doc/ros2/Installation/Linux-Development-Setup/))<br>
 * Install OpenVINO™ Toolkit Open Source<br>
 	* Install [OpenCV 3.3 or later](https://docs.opencv.org/master/d9/df8/tutorial_root.html)([guide](https://docs.opencv.org/master/d7/d9f/tutorial_linux_install.html))
 		```bash
@@ -84,7 +84,7 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 		mkdir ~/code && cd ~/code
 		git clone https://github.com/opencv/dldt.git
 		cd dldt/inference-engine/
-		git checkout 2018_R4
+		git checkout 2018_R5
 		git submodule init
 		git submodule update --recursive
 		./install_dependencies.sh
@@ -99,7 +99,7 @@ This project is a ROS2 wrapper for CV API of [OpenVINO™](https://software.inte
 		cd ~/code
 		git clone https://github.com/opencv/open_model_zoo.git
 		cd open_model_zoo/demos/
-		git checkout 2018_R4
+		git checkout 2018_R5
 		mkdir build && cd build
 		cmake -DCMAKE_BUILD_TYPE=Release /opt/openvino_toolkit/dldt/inference-engine
 		make -j8

dynamic_vino_lib/CMakeLists.txt

@@ -189,6 +189,8 @@ add_library(${PROJECT_NAME} SHARED
         src/inferences/object_detection.cpp
         src/inferences/head_pose_detection.cpp
         src/inferences/object_segmentation.cpp
+        src/inferences/person_reidentification.cpp
+        src/inferences/person_attribs_detection.cpp
         src/inputs/realsense_camera.cpp
         src/inputs/realsense_camera_topic.cpp
         src/inputs/standard_camera.cpp
@@ -201,6 +203,8 @@ add_library(${PROJECT_NAME} SHARED
         src/models/object_detection_model.cpp
         src/models/head_pose_detection_model.cpp
         src/models/object_segmentation_model.cpp
+        src/models/person_reidentification_model.cpp
+        src/models/person_attribs_detection_model.cpp
         src/outputs/image_window_output.cpp
         src/outputs/ros_topic_output.cpp
         src/outputs/rviz_output.cpp

dynamic_vino_lib/include/dynamic_vino_lib/common.hpp

@@ -209,7 +209,7 @@ static UNUSED std::ostream & operator<<(std::ostream & os, const PluginVersion &
 
 inline void printPluginVersion(InferenceEngine::InferenceEnginePluginPtr ptr, std::ostream & stream)
 {
-  const PluginVersion * pluginVersion;
+  const PluginVersion * pluginVersion = nullptr;
   ptr->GetVersion((const InferenceEngine::Version * &)pluginVersion);
   stream << pluginVersion << std::endl;
 }
@@ -308,7 +308,7 @@ static UNUSED void writeOutputBmp(
 {
   unsigned int seed = (unsigned int)time(NULL);
   // Known colors for training classes from Cityscape dataset
-  static std::vector<Color> colors = { {128, 64, 128}, {232, 35, 244}, {70, 70, 70},
+  static std::vector<Color> colors = {{128, 64, 128}, {232, 35, 244}, {70, 70, 70},
     {156, 102, 102}, {153, 153, 190}, {153, 153, 153},
     {30, 170, 250}, {0, 220, 220}, {35, 142, 107},
     {152, 251, 152}, {180, 130, 70}, {60, 20, 220},
@@ -497,7 +497,7 @@ static UNUSED void addRectangles(
   unsigned char * data, size_t height, size_t width,
   std::vector<int> rectangles, std::vector<int> classes)
 {
-  std::vector<Color> colors = { {128, 64, 128}, {232, 35, 244}, {70, 70, 70},
+  std::vector<Color> colors = {{128, 64, 128}, {232, 35, 244}, {70, 70, 70},
     {156, 102, 102}, {153, 153, 190}, {153, 153, 153},
     {30, 170, 250}, {0, 220, 220}, {35, 142, 107},
     {152, 251, 152}, {180, 130, 70}, {60, 20, 220},
@@ -1071,7 +1071,7 @@ static UNUSED void addRectangles(
   unsigned char * data, size_t height, size_t width,
   std::vector<DetectedObject> detectedObjects)
 {
-  std::vector<Color> colors = { {128, 64, 128}, {232, 35, 244}, {70, 70, 70},
+  std::vector<Color> colors = {{128, 64, 128}, {232, 35, 244}, {70, 70, 70},
     {156, 102, 102}, {153, 153, 190}, {153, 153, 153},
     {30, 170, 250}, {0, 220, 220}, {35, 142, 107},
     {152, 251, 152}, {180, 130, 70}, {60, 20, 220},

dynamic_vino_lib/include/dynamic_vino_lib/inferences/base_inference.hpp

@@ -156,6 +156,13 @@ class BaseInference
    * @return The name of the Inference instance.
    */
   virtual const std::string getName() const = 0;
+  /**
+   * @brief Get the max batch size of one inference.
+   */
+  inline int getMaxBatchSize()
+  {
+    return max_batch_size_;
+  }
 
 protected:
   /**
@@ -185,6 +192,7 @@ class BaseInference
   {
     max_batch_size_ = max_batch_size;
   }
+
   std::vector<Result> results_;
 
 private:

dynamic_vino_lib/include/dynamic_vino_lib/inferences/face_detection.hpp

@@ -54,7 +54,7 @@ class FaceDetectionResult : public ObjectDetectionResult
 class FaceDetection : public ObjectDetection
 {
 public:
-  explicit FaceDetection(double);
+  explicit FaceDetection(bool, double);
 };
 }  // namespace dynamic_vino_lib
 #endif  // DYNAMIC_VINO_LIB__INFERENCES__FACE_DETECTION_HPP_

dynamic_vino_lib/include/dynamic_vino_lib/inferences/object_detection.hpp

@@ -68,7 +68,7 @@ class ObjectDetection : public BaseInference
 {
 public:
   using Result = dynamic_vino_lib::ObjectDetectionResult;
-  explicit ObjectDetection(double);
+  explicit ObjectDetection(bool, double);
   ~ObjectDetection() override;
   /**
    * @brief Load the face detection model.
@@ -125,6 +125,7 @@ class ObjectDetection : public BaseInference
   int max_proposal_count_;
   int object_size_;
   double show_output_thresh_ = 0;
+  bool enable_roi_constraint_ = false;
 };
 }  // namespace dynamic_vino_lib
 #endif  // DYNAMIC_VINO_LIB__INFERENCES__OBJECT_DETECTION_HPP_