Merge pull request #1360 from fspindle/feat_hsv_optimization

Optimize hsv related conversion functions

.vscode/settings.json

@@ -138,7 +138,30 @@
     "xtr1common": "cpp",
     "xtree": "cpp",
     "xutility": "cpp",
-    "execution": "cpp"
+    "execution": "cpp",
+    "bufferobject": "cpp",
+    "image": "cpp",
+    "texture": "cpp",
+    "vertexarraystate": "cpp",
+    "buffered_value": "cpp",
+    "fast_back_stack": "cpp",
+    "graphicscontext": "cpp",
+    "program": "cpp",
+    "shader": "cpp",
+    "shape": "cpp",
+    "camera": "cpp",
+    "viewport": "cpp",
+    "types": "cpp",
+    "callback": "cpp",
+    "graphicsthread": "cpp",
+    "operationthread": "cpp",
+    "stateset": "cpp",
+    "export": "cpp",
+    "observer_ptr": "cpp",
+    "primitiveset": "cpp",
+    "stateattribute": "cpp",
+    "uniform": "cpp",
+    "polytope": "cpp"
   },
   "C_Cpp.vcFormat.indent.namespaceContents": false,
   "editor.formatOnSave": true,

ChangeLog.txt

@@ -16,6 +16,7 @@ ViSP 3.x.x (Version in development)
     . vpStatisticalTestAbstract, vpStatisticalTestEWMA, vpStatisticalTestHinkley, vpStatisticalTestMeanAdjustedCUSUM
       vpStatisticalTestShewhart and vpStatisticalTestSigma: classes implementing Statistical Control Process methods to
       detect mean drift / jump of a signal
+    . vpDisplayPCL to display a point cloud using PCL 3rdparty
   - Deprecated
     . vpPlanarObjectDetector, vpFernClassifier deprecated classes are removed
     . End of supporting c++98 standard. As a consequence, ViSP is no more compatible with Ubuntu 12.04
@@ -31,6 +32,10 @@ ViSP 3.x.x (Version in development)
     . Speed up build by including only opencv2/opencv_modules.hpp instead of opencv2/opencv.hpp header in vpConfig.h
     . In imgproc module, implementation of automatic gamma factor computation methods for gamma correction.
     . Eliminate the use of pthread in favour of std::thread
+    . RGB or RGBa to/from HSV conversion optimization in vpImageConvert class
+    . New vpImageTools::inRange() functions to ease binary mask computation by thresholding HSV channels
+    . New tutorials in tutorial/segmentation/color folder to show how to use HSV color segmentation to
+      extract the corresponding point cloud
   - Applications
     . Migrate eye-to-hand tutorials in apps
   - Tutorials
@@ -43,6 +48,14 @@ ViSP 3.x.x (Version in development)
       https://visp-doc.inria.fr/doxygen/visp-daily/tutorial-spc.html
     . New tutorial: Installing ViSP Python bindings
       https://visp-doc.inria.fr/doxygen/visp-daily/tutorial-install-python-bindings.html
+    . New tutorial: Introduction to color segmentation using HSV color scale
+      https://visp-doc.inria.fr/doxygen/visp-daily/tutorial-hsv-segmentation-intro.html
+    . New tutorial: HSV low/high range tuner tool
+      https://visp-doc.inria.fr/doxygen/visp-daily/tutorial-hsv-range-tuner.html
+    . New tutorial: Live color segmentation using HSV color scale
+      https://visp-doc.inria.fr/doxygen/visp-daily/tutorial-hsv-segmentation-live.html
+    . New tutorial: Point cloud segmentation using HSV color scale
+      https://visp-doc.inria.fr/doxygen/visp-daily/tutorial-hsv-segmentation-pcl.html
   - Bug fixed
     . [#1251] Bug in vpDisplay::displayFrame()
     . [#1270] Build issue around std::clamp and optional header which are not found with cxx17

doc/tutorial/segmentation/color/tutorial-hsv-range-tuner.dox

@@ -0,0 +1,61 @@
+/**
+  \page tutorial-hsv-range-tuner Tutorial: HSV low/high range tuner tool
+  \tableofcontents
+
+\section hsv_range_tuner_intro Introduction
+
+This tutorial follows \ref tutorial-hsv-segmentation-intro.
+
+Note that all the material (source code and images) described in this tutorial is part of ViSP source code
+(in `tutorial/segmentation/color` folder) and could be found in
+https://github.com/lagadic/visp/tree/master/tutorial/segmentation/color.
+
+\section hsv_range_tuner HSV Range tuner tool
+
+In the previous tutorial (see \ref tutorial-hsv-segmentation-intro), we used the following lines to determine
+HSV low/high range values in `hsv_range` vector:
+
+\snippet tutorial-hsv-segmentation-basic.cpp Set HSV range
+
+Then this vector was used to determine the pixels that were in the HSV low/high ranges in oder to compute a `mask`:
+
+\snippet tutorial-hsv-segmentation-basic.cpp Create HSV mask
+
+In tutorial-hsv-range-tuner.cpp we propose a tool that allows to set these HSV low/high values either using a
+trackbar or by clicking on a pixel in the image before fine tuning the range values with the trackbar.
+
+- Once build, you can either use this tool on a single image like
+\verbatim
+$ cd $VISP_WS/visp-build/tutorial/segmentation/color
+$ ./tutorial-hsv-range-tuner --image ballons.jpg
+\endverbatim
+\image html ballons-hsv-tuner.jpg
+- or if you have a Realsense camera, proceed with the live stream simply by running
+\verbatim
+$ ./tutorial-hsv-range-tuner
+\endverbatim
+
+To use this tool, in the "Current frame" window:
+- Left click allows to set the HSV low/high values in the trackbar
+- Middle click allows to print the RGB and HSV values corresponding to the clicked point
+- Right mouse click allows to save the HSV low/high range values in a yaml file. By default this file is saved in
+  `calib/hsv-thresholds.yml`. Hereafter we provide a possible content:
+\verbatim
+$ cat calib/hsv-thresholds.yml
+# File created 2024/04/03 17:28:10
+rows: 6
+cols: 1
+data:
+  - [0]
+  - [64]
+  - [222]
+  - [255]
+  - [128]
+  - [242]
+\endverbatim
+
+\section hsv_range_tuner_next Next tutorial
+
+You are now ready to see how to continue with \ref tutorial-hsv-segmentation-live.
+
+*/

doc/tutorial/segmentation/color/tutorial-hsv-segmentation-intro.dox

@@ -0,0 +1,128 @@
+/**
+  \page tutorial-hsv-segmentation-intro Tutorial: Introduction to color segmentation using HSV color scale
+  \tableofcontents
+
+\section hsv_intro Introduction
+
+The HSV scale, which stands for Hue Saturation and Value, provides a numerical readout of your color image that
+corresponds to the color names contained therein. Hue is measured in degrees from 0 to 360, while Saturation and Value
+of a color are both analyzed on a scale of 0 to 100 percent.
+
+Hue, Saturation, and Value are the main color properties that allow us to distinguish between different colors.
+In this tutorial, you will learn how to use HSV color scale to segment a specific color in an image.
+
+Note that all the material (source code and images) described in this tutorial is part of ViSP source code
+(in `tutorial/segmentation/color` folder) and could be found in
+https://github.com/lagadic/visp/tree/master/tutorial/segmentation/color.
+
+\section hsv_converter RGB to HSV color scale conversion
+
+In ViSP, color images can be read and converted to the RGB color scale. The RGB color scale is based on the color
+theory that all visible colors can be obtained from the additive primary colors red, green and blue. In ViSP,
+we introduce an additional Alpha channel to add color transparency. The RGB + Alpha channels are therefore
+implemented in the vpRGBa class. The following snippet shows how to load a color image in ViSP:
+\code
+#include <visp3/io/vpImageIo.h>
+
+int main()
+{
+  vpImage<vpRGBa> I;
+  vpImageIo::read(I, "ballons.jpg");
+}
+\endcode
+
+The color conversion from RGB to HSV or from RGBa to HSV color scale is performed in ViSP using one of the following
+functions:
+- vpImageConvert::RGBToHSV()
+- vpImageConvert::RGBaToHSV()
+
+The following snippet shows how to convert to HSV color scale:
+\code
+#include <visp3/io/vpImageIo.h>
+#include <visp3/core/vpImageConvert.h>
+
+int main()
+{
+  vpImage<vpRGBa> I;
+  vpImageIo::read(I, "ballons.jpg");
+
+  unsigned int width = I.getWidth();
+  unsigned int height = I.getHeight();
+
+  vpImage<unsigned char> H(height, width);
+  vpImage<unsigned char> S(height, width);
+  vpImage<unsigned char> V(height, width);
+
+  vpImageConvert::RGBaToHSV(reinterpret_cast<unsigned char *>(I.bitmap),
+                            reinterpret_cast<unsigned char *>(H.bitmap),
+                            reinterpret_cast<unsigned char *>(S.bitmap),
+                            reinterpret_cast<unsigned char *>(V.bitmap), I.getSize());
+}
+\endcode
+In the previous example, we obtained for each pixel:
+- Hue in `H` image where values are scaled from 0 to 255. here 255 stands for 360 degrees.
+- Saturation in `S` image where values are scaled from 0 to 255. Here 255 stands for 100%.
+- Value in `V` image where values are scaled from 0 to 255. Here 255 stands for 100%.
+
+\section hsv_segmentation HSV color segmentation
+
+It's easy to segment a given color if we select the range of hue, saturation and value we're interested in.
+
+In the image `ballons.jpg`, the pixel at coordinates [93][164] has an RGB value (209, 72, 0) which corresponds to
+an HSV value (14, 255, 209). We can use these HSV values and an additional offset to determine the low and high
+values of the HSV ranges used to create a mask corresponding to the segmented color.
+
+\code
+#include <visp3/io/vpImageIo.h>
+#include <visp3/core/vpImageConvert.h>
+#include <visp3/core/vpImageTools.h>
+
+int main()
+{
+  vpImage<vpRGBa> I;
+  vpImageIo::read(I, "ballons.jpg");
+
+  unsigned int width = I.getWidth();
+  unsigned int height = I.getHeight();
+
+  vpImage<unsigned char> H(height, width);
+  vpImage<unsigned char> S(height, width);
+  vpImage<unsigned char> V(height, width);
+
+  vpImageConvert::RGBaToHSV(reinterpret_cast<unsigned char *>(I.bitmap),
+                            reinterpret_cast<unsigned char *>(H.bitmap),
+                            reinterpret_cast<unsigned char *>(S.bitmap),
+                            reinterpret_cast<unsigned char *>(V.bitmap), I.getSize());
+
+  int h = 14, s = 255, v = 209;
+  int offset = 30;
+  int h_low = std::max<int>(0, h - offset), h_high = std::min<int>(h + offset, 255);
+  int s_low = std::max<int>(0, s - offset), s_high = std::min<int>(s + offset, 255);
+  int v_low = std::max<int>(0, v - offset), v_high = std::min<int>(v + offset, 255);
+  std::vector<int> hsv_range({ h_low, h_high, s_low, s_high, v_low, v_high });
+
+  vpImage<unsigned char> mask(height, width);
+  vpImageTools::inRange(reinterpret_cast<unsigned char *>(H.bitmap),
+                        reinterpret_cast<unsigned char *>(S.bitmap),
+                        reinterpret_cast<unsigned char *>(V.bitmap),
+                        hsv_range,
+                        reinterpret_cast<unsigned char *>(mask.bitmap),
+                        mask.getSize());
+}
+\endcode
+
+Using the mask we can create a segmented color image. The following snippet shows how to combine the mask and the
+color image using vpImageTools::inMask() to create the segmented image as given in the next snippet also available in
+tutorial-hsv-segmentation-basic.cpp
+
+\include tutorial-hsv-segmentation-basic.cpp
+
+The end of the previous snippet shows also how to display the following images.
+
+\image html ballons-segmented.jpg
+
+\section hsv_next Next tutorial
+
+You are now ready to see how to continue with \ref tutorial-hsv-range-tuner.
+
+*/

doc/tutorial/segmentation/color/tutorial-hsv-segmentation-live.dox

@@ -0,0 +1,68 @@
+/**
+  \page tutorial-hsv-segmentation-live Tutorial: Live color segmentation using HSV color scale
+  \tableofcontents
+
+\section hsv_video_intro Introduction
+
+This tutorial follows \ref tutorial-hsv-range-tuner.
+
+To run this tutorial you will need:
+- a Realsense camera like a D435 device
+- ViSP build with librealsense and PCL libraries as 3rd parties
+
+We suppose here that you already set the HSV low/high ranges using the range tuner tool explained in
+\ref tutorial-hsv-range-tuner.
+
+Note that all the material (source code and images) described in this tutorial is part of ViSP source code
+(in `tutorial/segmentation/color` folder) and could be found in
+https://github.com/lagadic/visp/tree/master/tutorial/segmentation/color.
+
+\section hsv_video_recorded Color segmentation on a recorded video
+
+- If not already done, the first step is to record a video as described in \ref tutorial-grabber.
+  Let us consider the case of a Realsense camera. To record a video you may run:
+\verbatim
+$ cd $VISP_WS/visp-build/tutorial/grabber
+$ ./tutorial-grabber-realsense --seqname /tmp/seq/image-%04d.png --record 0
+\endverbatim
+  - Use left click to start recording, and then right click to stop and quit.
+  - The sequence of successive images is recorded in `/tmp/seq/` folder.
+
+- Use the HSV range tuner tool explained in \ref tutorial-hsv-range-tuner to select the color you want to segment.
+  We select the HSV low/high range on the first image of the sequence with
+\verbatim
+$ cd $VISP_WS/visp-build/tutorial/segmentation/color
+$ ./tutorial-hsv-range-tuner --image /tmp/seq/image-0001.png
+\endverbatim
+  - As a result you will find the learned HSV low/high ranges in `calib/hsv-thresholds.yml`
+
+- Now you are ready to process all the recorded video in order to perform color segmentation based on the
+  content of `calib/hsv-thresholds.yml`
+\verbatim
+$ ./tutorial-hsv-segmentation --video /tmp/seq/image-%04d.png --hsv-thresholds calib/hsv-thresholds.yml
+\endverbatim
+
+\section hsv_video_live Color segmentation on a live stream
+
+- We suppose here that you have a Realsense camera and that you install librealsense before building ViSP with
+  librealsense 3rdparty support. If you are not familiar with these steps, follow one of the \ref tutorial_install_src
+  tutorials.
+- Plug your Realsense camera and use the HSV range tuner tool explained in \ref tutorial-hsv-range-tuner to select
+  the color you want to segment.
+\verbatim
+$ cd $VISP_WS/visp-build/tutorial/segmentation/color
+$ ./tutorial-hsv-range-tuner --hsv-thresholds calib/hsv-thresholds.yml
+\endverbatim
+  - As a result you will find the learned HSV low/high ranges in `calib/hsv-thresholds.yml`
+
+- Now you are ready to process the Realsense live stream in order to perform color segmentation based on the
+  content of `calib/hsv-thresholds.yml`
+\verbatim
+$ ./tutorial-hsv-segmentation --hsv-thresholds calib/hsv-thresholds.yml
+\endverbatim
+
+\section hsv_video_next Next tutorial
+
+You are now ready to see how to continue with \ref tutorial-hsv-segmentation-pcl.
+
+*/