Hello @clalancette ,

could you please take a look at this PR?

I attached a small video showing a comparison of the current state of using the minimum value in each column (orange) and my proposed approach of using quantiles/percentiles (purple). In this test a scan_height of 180 pixels was used and for my version a quantile_value of 0.35.
In my opinion using a quantile instead of the minimum of a row greatly reduces noise and more accurately reflects the position of obstacles/walls.
It is also easily adjustable. If you have very little noise or want to detect small objects that might only take up a couple pixels in each row, you can set the quantile_value parameter to a small numer (or even to 0.0 to retain the current behavior). If you want smooth and accurate laserscans and only care about larger objects you can set the value to something bigger.
depthimage_to_laserscan_quantile_comparison.webm

Hello @clalancette ,
as some time passed, I just wanted to check in if you had any chance to take a look at this PR? If not, are there maybe other maintainers that you could refer this to?
Or is this functionality just not wanted?
In any case, I would be happy for a response.
Kind regards, Yannic

Hey @chadrockey :)
as you are listed as the maintainer in the package.xml, could you please take a look at this PR or tag someone else that might want to review it? Thanks!

Ah, sorry that you've been making contributions to this repo and they haven't been getting merged. It takes a lot of effort to document and improve software and I hope we can find a way to incorporate your work.

Your scans do look a lot cleaner with these changes and could be very beginner friendly as this package was created for low performance computation and to ease users into sensor data via lighter weight laser scan messages vs full on image processing.

My largest concern with this package has always been ease of use and documentation. A lot of things in ROS have existed and been improved, but you have to come across the change in the source to understand how it could help you. Especially as this package intended for newer users and students, the settings should be clearly stated. I'm not sure if there's an auto documentation step now or if there's still manual processes like the wiki.

A quick glance suggest the code looks good and the video should support that it works. My only questions are how does this relate to other statistical filtering techniques such as the median filter or even the bilateral filter as used in image processing. It may be worth considering the more difficult approaches depending on the kind of output fidelity you hope for this feature. For instance, bilateral filters are typically better at preserving edges which could be important for obstacle detection or creating sharper output masks.

I believe you're only using each column, but having an ROI approach could be closer to how some brands of scanning laser rangefinders actually work. For instance, SICK typically have large overlapping beam widths so that thin objects cannot slip between measurements and go undetected. SICK also sometimes samples at a far higher resolution than is output in their data formats. See this image for an example of both the angular filtering and how the circles of the beam for each laser are both overlapped, have a circular 2d profile, and increase with range:

Is the 0.35 value the 35th percentile of measurements closest to the sensor? What were your findings from other settings like 50th percentile or even further? Something interesting here is that algorithms like AMCL uses the Probabilistic Robotics model that laser scanners have multi modal probability distributions where there's uniform random error reads, random clutter noise (will always be shorter than walls/fixed structures), and a distribution of measurement noise around the actual fixed structure. Their model heavily favors using further measurements rather than closer so it's fun to see you have something different and/or the properties of the depth camera vary enough from true laser scanners to change these assumptions.

Lastly you've made several PRs and contributions for this package and seem interested in it. I'm sure ROS still has a formal maintainer takeover process if that's something you'd be interested in.

Another option to consider would be if you released essentially a fork or more complicated version as a different package if you plan on adding a lot of sophistication and pursuing a high level of performance. I do consider this package a kind of sample or tutorial. I had always imagined the simplicity to be a guide into robotics and sensor data. Learning how to diagnose when sensor data at fault is a skill and developing algorithms that are tolerant to non-ideal data is a necessary lesson. The original Kinect's large disparity gap -> limited depth output resolution was a great case of all the walls in your map being quite thick but the localization and map shape still working well. This was always balanced with the Turtlebot's need to be an excellent demo robot vs a learning platform though.

I could see a place for a more sophisticated depthimage to laserscan "pro", either in this package or another, if you wanted more creative control or less legacy to contend with. Next steps will largely depend on your vision for your involvement and what this package could be.

Thanks for the detailled answer!

Regarding the low performance computation, I would have to run some tests to see how it is affected by the more complex calculation.
Maybe it does make sense to keep this package as simple and computationally lightweight if it is aimed at newer users and students and instead create a new more powerful package for those who need it.

how does this relate to other statistical filtering techniques such as the median filter or even the bilateral filter

To get a median filter, we just have to set the quantile value to 0.5. This worked well in my testing for large obstacles or walls, however you have to be cautious if you want to detect obstacles that only occupy a couples of pixel rows (e.g. a horizontal beam or a table top). If you had a scan_height of 100 pixels and your object only occupies 49 pixels, the filter would remove the obstacle when a quantile value of 0.5 or larger is used. For my usecase I found a quantile value of about 0.2 to be a good balance between filtering out a lot of noise while still beeing able to detect small obstacles (theoretically up to 20% of the scan_height).

I never thought about using a bilateral filter for this usecase, but as long as each column gets filtered individually (meaning we would apply 1D bilateral filter to each column) I don't think there would be any benefit, as we don't have to preserve vertical edges (along each column) for the laserscan message. But it could make sense to apply a 2D bilateral filter to the complete depthimage before performing the depthimage_to_laserscan step (with using the current column-minimum or the proposed column-quantile approach). However in that case maybe it would make sense to perform the bilateral filtering of the depthimage in a prior separate step outside of this package.

One can still filter in the horizontal direction, over multiple columns/scan-ranges. I currently do this using laser_filters after the depthimage_to_laserscan step. I maily use a spatial median filter see this PR for the laserscan message. Here a 1D bilateral filter (this time in the horizontal direction) would make sense in my opinion.

having an ROI approach

This is also an interesting idea. We could use a moving window approach to use a ROI for each column that includes the neighboring columns around the current one (this way we would have overlapping ROIs). Another option would be to split the set of columns into non-overlapping chunks, which would lead to a lower output-resolution of the laserscan (number of columns divided by the chunksize). For both options the size of the set of pixels for which the quantile has to be calculated would increase (by a factor of the number of columns in each row). This will lead to increased computational load because a larger set of pixels has to be sorted each time.

In my opinion using a ROI here would (probably) not have any benefits over using two separate steps (1. depthimage_to_laserscan, 2. laser_filters) while leading to increased computational load and making this package more complex.

Is the 0.35 value the 35th percentile of measurements closest to the sensor? What were your findings from other settings like 50th percentile or even further?

Yes. A value of 0.0 is the 0th percentile of measurements closest to the sensor. This has the same effect as using the minimum value of each column (the current behavior and default value). A value of 0.5 will use the median distance in each column. A value of 1.0 will use the max distance (this probably won't make much sense most of the time).
Using 0.5 worked well in my tests, but you can't detect objects that occupy less than 50% of the scan_height (as described above).
If your sensors noise is centered around the true distance, a quantile value of 0.5 will get you the closest result to the true distance (still keep in mind small obstacles) though. Using values above 0.5 would only make sense if your sensors distribution is centered around a value smaller than the true distance (which was not the case with the 3D-cameras I used) and you don't care about (vertically) small obstacles.

algorithms like AMCL uses the Probabilistic Robotics model [...] Their model heavily favors using further measurements rather than closer

This is something I did not know of at all. So even if the filtered scans do look cleaner, it might actually be decremental to the performance of things like AMCL because they expect data with a certain kind of noise distribution?

formal maintainer takeover process

I this does not mean that I would be the only maintainer, I would be interested in that. I am not even close to beeing an expert in ROS by any means but I do plan to get more into it and keep working with it in the future. This package (or a new "depthimage to laserscan pro" package) could be a good start into getting more in touch with the ecosystem (e.g. things like the build farm which I did not touch at all until now).

Options for next steps

Because you said this package is intended to be a simple and low performance computation solution, I do agree that it should not be bloated with more complex stuff that many new uses don't need or can't run on low-end hardware.
From a user/maintainer complexity standpoint I think this PR could still be added to the package. However if this has to run fast on low-end hardware we should probably also compare the computational load of this PR compared to the current minimum-calculation.
A "depthimage to laserscan pro" seems like a good idea, but I am not sure if there are enough additions/improvements possible to justify the time to invest into a new package. For me this partly depends on whether a new package could be implemented in Python instead of C++. I am just more experienced in Python and maybe some other users are also more willing to contribute to a package if it is written in Python. Is there something against Python for a new package?