Find correspondences between a set of 3D gaussian distributions (quadrics or ellipsoids) and a set of 2d gaussian distributions (conics or ellipses)

[ttsesm]

Hello,

I don't have a formal background in OT, therefore with this issue I am also trying to understand a bit better whether OT could be an option for my task or not.

So briefly, I am having the following problem. Imagine that I have set of 3D gaussian distributions, i.e quadrics or Ellipsoids, in the 3d space and a set of 2D gaussian distributions, i.e. conics or ellipses, projected on an plane. This number of 3D/2D gaussians in each set could be couple of thousands. Moreover, for each of the sets I have all the information that describe these two sets as gaussian distributions respectively. Thus, the mean, μ, and the covariance matrices Σ.

My question is, based on this information is there an approach that I could possibly obtain some good correspondences so that I can match an ellipsoid with an ellipse. Each distribution in each of the sets can be considered as individual distributions or combined. Moreover, it is not necessary that all distributions in one set match with all distributions on the other set. It can be that only few ellipsoids match with some of the ellipses in the other set. I am trying to figure out whether geometrically or statistically is possible to get something based on OT.

The inspiration comes from these works: Generalized Wasserstein barycenters between
probability measures living on different subspaces and this thesis Generalised Wasserstein Barycentres
where some of the authors (@eloitanguy, @rflamary) from what I've noticed are also contributors of the POT library. The idea as described there is that based on some probability measures, marginals or projections (≥2) using OT it possible to recover the 3d probability or the reconstructed generalized barycenter which apparently describes the geometry of the distribution. In my case though I have only one projection, which of course this projection contains multiple 2D gaussians.

To my understanding the Blind Generalised Wasserstein Barycentre Problem could be an approach to follow but I am not quite sure since the whole concept is totally new to me. Thus, any feedback would be appreciated.

[eloitanguy]

Hi,

With BGWB (which is not implemented in POT), you need discrete distributions. A possible avenue would be to sample from your 3D and 2D gaussians, compute a projection P: R^3 -> R^2 which maps your samples, as well as a transport plan $\pi$ which will tell you which 3d sample matches which 2d sample.

The minimisation problem in question would be $\min_{P}W_2^2(P$# $\mu, \nu)$, where $\mu$ is your 3D samples and $\nu$ is your 2D samples. You could solve this using BCD or SGD (similarly to Generalised Wasserstein Barycentres ), but this could yield poor results numerically.

To use Gaussians and not samples, you could see each Gaussian as a point on the Bures-Wasserstein manifold. To compute an assignment between your 3D gaussians and 2D gaussians, you could then consider the Gromov-Wasserstein distance, which can correspond elements of different metric spaces.

In general, your problem also makes me think of Domain Adaptation, which has been tackled with OT successfully.

Thanks for the interesting question, I don't think we can give an immediate answer with accessible tools :D
Let us know how your thoughts go along!

[eloitanguy]

The example you cite deals with a different scenario which does not apply here, it boils down to finding a linear map matching two measures on (possibly different) Euclidean spaces. In you case, you want to match ellispoids, which are points on the Bures-Wasserstein space.
The idea that i suggest requires you to code it: back-propagation of the gradient with respect to $P$ of the 2-Wasserstein distance (ot.emd2) between the 2D-projected gaussians $P\#\mu$ and the 2D gaussians $\mu$, using the 2D Bures-Wasserstein distance to build your cost matrix (ot.gaussian.bures_wasserstein_distance at each entry).

[ttsesm]

I see, then what about the ot.gaussian.bures_wasserstein_mapping() can't I directly use this function to estimate the mapping 🤔

[eloitanguy]

ot.gaussian.bures_wasserstein_mapping() computes the linear map which maps a Gaussian to another. This is not the same problem.

[ttsesm]

ok got it, I will try to test the 3D->2D mapping with the ot.emd2.

You also mentioned that, for applying GW directly further experimentation could be possibly applied. Could you elaborate a bit what other approaches I could possibly test there. For example I've tried to play with the function parameters but it didn't really change much. Then is there anything else that could worth trying.

[eloitanguy]

You also mentioned that, for applying GW directly further experimentation could be possibly applied. Could you elaborate a bit what other approaches I could possibly test there. For example I've tried to play with the function parameters but it didn't really change much. Then is there anything else that could worth trying.

No specific ideas right now, and no time to spend on it, sorry. I feel like I have gone above and beyond to give you some starting points on the subject, and this is the most I can reasonably do.