Question about the Evaluation-Time Pose Alignment

[fangzhou2000]

Excellent work!

I have two questions about the Evaluation-Time Pose Alignment mentioned in the paper:

1. Why is Evaluation-Time Pose Alignment necessary for evaluation? Is it because the reconstructed scene might have a different scale, either proportionally smaller or larger, compared to the real scene?

2. In Table 1 & 2, do other pose-free methods (DUSt3R, MASt3R and CoPoNeRF) adopt the same Evaluation-Time Pose Alignment strategy ?

[botaoye]

Hi,

1. you can refer to Sec. 3.5 of our paper for why it is needed:

3D scene reconstruction with just two input views is inherently ambiguous as many different scenes can produce the same two images. As a result, though the scene generated by our method successfully explains the input views, it might not be exactly the same as the ground truth scene in the validation dataset

This has nothing to do with scale, but rather the existence of multiple possible 3D scenes that can explain the same input image.

2. For DUSt3R and MASt3R, both methods only output point clouds, and we render novel views by projecting the point cloud to the desired pose, so it is not possible to apply this ‘evaluation-temporal pose alignment’ as it requires gradients. As for CoPoNeRF, we are using its official code and there is no ‘evaluation-time pose alignment’ as they use NeRF as representation, it is not easy to apply this and we outperform it to such a large extent that even applying the alignment step does not have much impact. Some recent other methods also employ the evaluation-temporal pose alignment, such as PF3plat and instantSplat, you can check them if interested.