Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Precision evaluation using measured distances #2

Open
xoox opened this issue Sep 18, 2018 · 1 comment
Open

Precision evaluation using measured distances #2

xoox opened this issue Sep 18, 2018 · 1 comment

Comments

@xoox
Copy link
Owner

xoox commented Sep 18, 2018

The precision evaluation was conducted with dataset1, dataset2 and
dataset3 of calibrel_testdata.
The three dataset were sampled from the same dataset actually which was
split into three parts. So camera parameters calculated from them should
be consistent to each other.

The calibration board has the following information. Pattern width and
height was 16 and 10 respectively. It's square size was 15mm ideally.
The chessboard pattern was printed with an ordinary office laser
printer and then pasted to a glass panel which was not so perfect planar
structure.

The following results were gotten with test_calibrel with arguments of
-d=224.14 --winSize=7

Camera parameters found with OpenCV's standard method

RMS reprojection errors

dataset1 dataset2 dataset3
0.449666 0.451857 0.461138
Parameter dataset1 dataset2 dataset3
fx 3025.42461 3027.95253 3026.34759
fy 3035.50074 3038.12042 3035.99243
cx 659.54461 655.741128 660.265037
cy 482.46653 482.198968 484.282884
k1 -0.1072574 -0.1083618 -0.1083125
k2 0.799394 0.85499888 0.85331468
p1 -0.0031994 -0.0032028 -0.0027532
p2 0.00523079 0.00468779 0.00508313

With k2 = p1 = p2 = 0 were set. We got the following.

RMS reprojection errors

dataset1 dataset2 dataset3
0.476395 0.476218 0.484569
Parameter dataset1 dataset2 dataset3
fx 3022.20878 3024.75515 3022.96343
fy 3032.68985 3034.98629 3033.06211
cx 609.398155 610.493838 611.186506
cy 505.539925 504.981172 503.674785
k1 -0.0697226 -0.0683047 -0.0683070

Parameters such as fx, fy and k1 are so much different between the above
two circumstances (with and without k2 = p1 = p2 = 0).

Camera parameters found with refining 3D object points

RMS reprojection errors

dataset1 dataset2 dataset3
0.0866284 0.089332 0.0860607
Parameter dataset1 dataset2 dataset3
fx 3041.09080 3041.08956 3040.54346
fy 3041.13957 3040.33822 3038.93562
cx 617.358690 615.113893 613.587902
cy 535.139389 531.679761 535.998730
k1 -0.0862624 -0.0842604 -0.0860289
k2 0.07004163 0.04037006 0.04790104
p1 -0.0002223 -0.0006121 -0.0003499
p2 0.00063364 0.00060760 0.00043702

With k2 = p1 = p2 = 0 were set. We got the following results.

RMS reprojection errors

dataset1 dataset2 dataset3
0.0867244 0.0895241 0.0861504
Parameter dataset1 dataset2 dataset3
fx 3041.01665 3040.99082 3040.37711
fy 3041.10761 3040.19260 3038.72519
cx 608.810470 607.321286 607.935824
cy 537.219851 537.245663 538.764664
k1 -0.0829604 -0.0823206 -0.0837447

We can see parameters (e.g. fx, fy and k1) found with this method are
more consistent and robust than OpenCV's standard method.

Refined 3D grid points

For the above 3 datasets, with and without k2 = p1 = p2 = 0, we got 6
list of refined 3D target grid points. Firstly, Situations with and
without k2 = p1 = p2 = 0 were compared. The following table shows the
coordinates delta between these two situations. And we can see the
coordinates are very consistent.

Delta dataset1 dataset2 dataset3
mean 0.00416 0.00570 0.00362
min 0.00000821 0.00000249 0.00000800
max 0.0206 0.0251 0.0147
Std Dev 0.00445 0.00728 0.00412

The delta between refined positions of three datasets were also
computed. In the following table, d1 - d2 means coordinates difference
between refined positions generated with dataset1 and dataset2. d1 - d3
and d2 - d3 keep the similar meaning.

Delta d1 - d2 d1 - d3 d2 - d3
mean 0.0132 0.0193 0.0137
min 0.0000686 0.0000152 0.00000311
max 0.0422 0.0953 0.0624
Std Dev 0.00863 0.0186 0.0131

The refined coordinates agree to each other very much.

Refined 3D grid corner distances compared to measured distances

We chose 4 corners p1, p2, p3 and p4 as shown below to evaluate the
accuracy of the refining process.

cornerpoints

In board frame the coordinates of the 4 points are: p1(0, 0, 0), p2(x2,
0, 0), p3(x3, y3, z3) and p4(x4, y4, 0).

When Refining 3D grid points, p1, p2 and z4 of p4 are fixed according to
Strobl's paper. x2 was measured as the distance between p1 and p2. For
the board in dataset1, dataset2 and dataset3, x2 = 224.14.

The refined 3D grid corners are shown in the following table.
(x2 = 224.14).

coordinate dataset1 dataset2 dataset3
x3 -0.391143 -0.36592 -0.384941
y3 135.211 135.249 135.296
z3 0.0832837 0.0737153 0.0588442
x4 224.334 224.355 224.377
y4 135.151 135.173 135.189

We measured the distances between combinations of p1, p2, p3 and p4 with
a vernier caliper of 0.02mm resolution (The uncertainties of the
measurements are probably larger than 0.1mm I think). Then the computed
distances from above data were compared to measured one as the following
table.

distance hypothesis measured dataset1 dataset2 dataset3
|p1-p2| 225 224.14 224.14 224.14 224.14
|p1-p3| 135 135.35 135.212 135.25 135.297
|p1-p4| 262.393 261.85 261.9 261.929 261.956
|p2-p3| 262.393 262 262.1 262.098 262.138
|p2-p4| 135 135.13 135.151 135.173 135.189
|p3-p4| 225 224.73 224.725 224.721 224.762

The differences between hypothesis and measured values indicate that the
printed paper is not accurate at all. The computed distances agree very
well with measured values. The mean error is 0.0687mm with standard
derivation of 0.0435mm. The maximum error is 0.138mm.

In conclusion, this new extended camera calibration method is very
helpful when accurate calibration board is not available which is a very
common case for most people.

@5trobl
Copy link

5trobl commented Nov 6, 2018

Good validation results indeed.

Still, they could get even better if the calibration plate was more tilted to obtain even more perspectively distorted images. Perspective distortion is needed to tell range from focal length after all, see Strobl et al.'s "On the Issue of Camera Calibration with Narrow Angular Field of View."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants