Uses depth subtraction and RGB subtraction on inputs from a realsense camera to make a binary image of objects on a surface. This code/documentation is the product of Jacob Fiola's CU Boulder senior thesis - "Segmentation of RGB-D Data Using RGB-Based Background Subtractors: Two Proposed Methods for Robust Segmentation of Camouflaged Objects"
This thesis addresses the problem of segmenting camouflaged objects of interest using background subtraction techniques. If this problem can be resolved, less post-processing will be necessary for the optimal segmentation of camouflaged objects of interest.
If a camouflaged object enters the scene, most traditional RGB based background subtractors will fail to create an accurate segmentation of the camouflaged object. This is because there are little to no RGB differences detected. Likewise, if a very flat object is placed on a surface in a scene, most depth based background subtractors will fail to create an accurate segmentation of the flat object. This is because there are little to no depth differences detected. This thesis proposes and evaluates RGB Union + Simple Depth Subtraction (RUSDS), a method for modeling the location of objects using a combination of RGB-based and depth-based background subtractions.
This thesis also proposes evaluates a method called RGB-based Background Subtraction Using Depth Colormap Input (RBSUDCI), which allows any existing RGB-based background subtraction algorithm to use depth value colormaps as an input instead of a traditional RGB image. Both qualitative and quantitative results suggest that RBSUDCI is superior to any RGB-only background subtraction algorithm, and that RUSDS can compete with the performance of the algorithms which utilized the RBSUDCI technique. All algorithms are quantitatively evaluated by calculating their respective IOU's to the ground truth.
For the full thesis, please see documentation/Segmentation_Thesis.pdf in this GitHub repo.