SBA (Poser)

Sparse Bundle Adjustment Solver

Bundle adjustment is, loosely speaking, an approach that adjusts the position of the object or the lighthouse and simulates where it expects the resulting angle_x, angle_y measurement to be. Its goal is to adjust the pose of the moving piece to minimize the difference between observed data and the simulated data.

The cannonical use of SBA in computer vision is take a bunch of pictures, and a bunch of correspondences from the same object in all the pictures, and recreate both the 3d position of objects and the pose of the cameras.

An important point to SBA is that it needs an OK solution to adjust for good results in most applications. It solves for the local minima of the error function, so to find the global minima it needs to start near it.

Since SBA models the whole system, the details for calibration and for tracked object position are different. For calibration, you declare the tracked object fixed at origin, and adjust for the position of the two lighthouses. This looks like a N 3d point, 2 camera setup, where N is the number of 3d points on the tracked object.

For tracking, you want to leverage the idea that you know the rigid structure of the tracked object, and the exact position of the lighthouses, and you just want the pose of the tracked object. The way this is modeled here is that there is one pose for the tracked object, and effectively two cameras for each sensor location, all at known positions that aren't to be adjusted.

Both approaches use the same reprojection function which is part of libsurvive.

SBA buys us a few things in this domain. Primarilly, it incorporates data from all correspondences and all lighthouses; something that PNP doesn't really do out of the box. But more importantly, it lets us model the actual parametrics of the lighthouse. The lighthouse is not a pinhole camera, but treating it as one gets you arbitrarilly close. SBA typically will optimize 'arbitrailly close' into the optimal least squares solution. This is going to be important in the long run, paticurally since each lighthouse has factory calibration parameters that are almost certainly easier to reproject than any other approach.

The SBA library used here is the one here. Performance wise, since the measurement sizes are so small -- a maximum of ~50 measurements whereas typical uses of SBA have 50 measurements for each of dozens or hundres of images -- I haven't implemented a jacobian for the reprojection function -- it runs as fast as it can get light data. If it needs to run on a much slower platform, a jacobian would speed it up substantially.