Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Center circle detection for camera calibration #1057

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

tuxbotix
Copy link
Contributor

@tuxbotix tuxbotix commented Jun 14, 2024

Why? What?

Provides 3 features, could be splitted but makes it harder to test all together.

  1. Center Circle detection with a RANSAC implementation.
    • Canny edge detection or Segments used to get points
    • A variant is provided to handle transforming from Pixel (ellipse) to ground (circle) prior to fitting.
  2. Integration into calibration measurement provider.
  3. Twix panel to view the detections, a parameter in the circle detection cycler tuning_mode triggers detection on every circle regardless of CalibrationCommand.

This was originally developed on top of work on branches/ PRs (#695)

How it works: from RC2023 Bordeaux
center_circle

Differences from the line-RANSAC:

  1. Uses 3-point circle formula to get a sample circle
  2. Early-rejection of candidates -> uses radius, minimm distance between points, etc.
  3. Performing the fit only on a sample of the population (if large enough) to make the iterations reasonably fast.

Testing via #1508

image

Fixes #

ToDo / Known Issues

  • Avoid fitting to small arcs - bottom image of the above example. (3 points selected randomly should be sufficiently spaced out)
  • [ ] Alternatively, construct the circle from a minimum-sized arc (i.e. 100 degree continuous arc with radius R)
    • This can be more robust as we can check all points fitting in this region to be continuous or at least mostly continous.

Ideas for Next Iterations (Not This PR)

How to Test

You can subscribe to:

  • calibration_command to see the state changes
  • calibration_inner.measurements additional from calibration controller to see the measurements collected
  • Output last_calibration_corrections

Tuning mode (vision parameters)

The parameters are tuned to a decent state already, but if needed it can be tuned further.

Parameters

  • tuning_mode - keeps running the cycler even when no calibration command is recieving from the calibration controller.
  • ransac_circle_minimum_circumference_percentage helps to reduce false positive.hits on T junctions, etc with very few points.
  • get_edges_from_segments - uses segments instead of Canny to generate points for RANSAC Overall speed is quite decent but not as good as Canny based approach
  • gaussian_sigma controlls the Gaussian blur strength. Best left untouched.
  • canny_* - controlls canny edge detection. Can be tuned for lighting conditions, but these values should work already.
  • Other parameters are already tuned and does not need changing.

How-to

Twix:

  • Open an image panel, there is an overlay for calibration center circle (not fully tested with latest Twix changes!!)
  • subscribe to:
    • main output calibration_center_circles of the given cycler to see raw.output
    • calibration_center_circle_detection.circles_points_pixel_scores additional to see the score for each circle.
  • open a parameters panel
    • Look for calibration_center_circle_detection
    • Enable tuning_mode for top or bottom (not both as this would slow things down.
    • if no circles are detected, try to reduce ransac_circle_minimum_circumference_percentage to 0.1 or something, it should

@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch 3 times, most recently from 4d47a7e to dba2eff Compare June 24, 2024 01:25
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch from dba2eff to 83d1ebe Compare June 28, 2024 00:57
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch from 83d1ebe to d617886 Compare July 6, 2024 22:51
knoellle
knoellle previously approved these changes Jul 7, 2024
Copy link
Contributor

@knoellle knoellle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@knoellle knoellle dismissed their stale review July 7, 2024 12:49

Whoops, wrong PR

@tuxbotix
Copy link
Contributor Author

tuxbotix commented Jul 7, 2024

I'm cleaning up this branch to remove the testing stuff, hopefully can get it ready today.

@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch from d617886 to 0b37ea7 Compare July 8, 2024 15:02
@schmidma
Copy link
Member

Is this still on track for RC24? If we cannot manage for RC24, I'd suggest to remove the label 👍

@schluis schluis removed the RC24 label Jul 15, 2024
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch from 0b37ea7 to 9d1f987 Compare July 15, 2024 16:20
@tuxbotix tuxbotix changed the title Circle detection with RANSAC Center circle detection for camera calibration Jul 15, 2024
@tuxbotix
Copy link
Contributor Author

Is this still on track for RC24? If we cannot manage for RC24, I'd suggest to remove the label 👍

Sorry, I didn't see this. I'm afraid there won't be enough time to properly review and merge it to main. But it is ready for "beta" testing now ;) I'll update the how-to later today.

I tested the workflow and detection seperately and with initial integration a week ago, it worked but needed cleaning up.

@tuxbotix tuxbotix marked this pull request as ready for review July 16, 2024 08:35
@tuxbotix tuxbotix enabled auto-merge July 16, 2024 08:35
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch 6 times, most recently from d93d5ac to 54f6409 Compare July 19, 2024 09:28
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch from 54f6409 to df71f00 Compare August 1, 2024 16:56
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch 2 times, most recently from b2c4c90 to 9f244b6 Compare September 15, 2024 10:38
Minimum distance between three points of circle ransac is enforced

WIP filtering

Remove unused components and have a more "functional" interface.

Additional outputs are ready
Add cycle instance to the additional outputs

Initial work to get line detection debug images into twix

Can visualise the intermediate stages of line detection via twix.

Get some crude timing

Resize image for easier debug
Fixed the resizing

Parameters for fast debugging

Faster processing setup

View filtered calibration lines
Filter points by horizon
Make the cycler to work

WIP tuning
Cleaning up & clippifying

Refactored the cycler

Render edge points

Use image segments as an alternative source of edge points

updated score calculation scheme and random generator change to ChaCha

Sub-sampling for faster search (= more iterations per search can be done)
Edge detection crate
Downgrade stuff & fix transitive dependency issue
Rename parameters and struct members.
Update the twix panel to work properly

Clippy fixes
@tuxbotix tuxbotix force-pushed the 2024-06_extracted_ransac_circle_detection branch from 9f244b6 to afcf572 Compare November 19, 2024 12:52
@pejotejo
Copy link
Contributor

@tuxbotix what's the state with this pr

@tuxbotix
Copy link
Contributor Author

tuxbotix commented Nov 27, 2024

Currently being tested under #1508 . The scope of this PR works as advertised and any refinement steps should be in a separate PR IMHO

I discussed with @oleflb last week and while it is easier to test calibration with an all in one branch, reviewing and merging would be annoying 😬.

This branch could be revised to match the scope of the title and merged. Then #1508 will have the integration part to the calibration pipeline and possible refinements.

I'll finish cleaning this up today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

5 participants