+
+
+ 
+
+
+ + The content of the query image is searched inside the semantic panorama rendering via cross-domain image-to-panorama matching. First, both inputs are encoded with CNN backbones, then the feature tensors are interwoven via depth-wise correlations and multiple tensor operations. This matching is inspired by DTOID, a generic 2D object instance detection approach. +
++ The correlated features R* are fed into multiple network heads, which estimate the binary mask corresponding to the viewport of the camera in the panorama, the 2D bounding box enclosing the estimated viewport mask, and the relative 6D pose offset between the camera image and the panorama. The key point is that each match is given a score via the bounding box estimation, which can be used to select the best match from a large number of reference panoramas. The absolute pose is determined from the estimated relative pose between this panorama and the camera image. +
+
+