Long term perspectives

[ElliotMaitre]

No particular order in the list

1. Standardizing the code structure accross MegaPose and Cosypose, in particuliar:

• API Detector (Done)
• [API Coarse
• API Refiner
  —> Standardizing the two approaches would then make it possible to create a more modular architecture for integrating other methods.

2. Decoupling detection / pose estimation. Currently prediction_runner and pose_estimator handle this... Decoupling would allow greater flexibility.

3. in megapose/model/pose_rigid, move PosePredictorOutputCosypose elsewhere, but where?

4. More generally, some code is duplicated between MegaPose and CosyPose (pose_predictor.py, ...). A refactoring of the code to make a general API for HappyPose would be needed. For now, it is similar in both repositories but some work is still needed to make something really clean.

5. Currently, there are pointers between pose_estimators. For example, CosyPose use the evaluation of MegaPose, which is stored in pose_estimators.megapose. This should be looked at, as a pose_estimator should probably import only from itself and the toolbox part.

6. If re-training CosyPose (if necessary?) Adapt object names to choices made in MegPose (change from obj-000001 to ycbv_obj-000001, for example, in the config of each model).

7. For CosyPose training, pose_training.py is working on a single GPU but it seems that it doesn't work on Jean Zay. More investigation on why is needed. detector_training.py shouldn't be working yet, as no one worked on it.

8. Rewriting of the repo using more recent technologies (e.g. PytorchLightning, Weight&Biases)

9. Update of the Neural Backbone (Yolo for detection ?, PoseEstimator ?)

10. A good look over the code to remove potentially useless elements (many (unused?) scripts ...) -> Compare with entry points and uses cases?

11. Docker/singularity/apptainer for Happypose ?