New Multimodal Fusion Features and New Ionosphere Dataset Support

[DrHoisu]

Thank you so much for the wonderful PredRNN. I am a research fellow of Kyoto University. Japan.

This week, I publish a new journal paper: https://doi.org/10.1029/2024SW004121 on ionosphere electron density prediction.

We add the multimodal fusion prediction feature Github rather than the graphic video prediction of your origin PredRNN, because ionosphere is mainly influenced by sun activity and geomagnetic field so the modeling of these factors improves performance significantly. We found that the action-conditioned PredRNN/PredRNNv2 may have a worse performance compared to that only inputs image channel. We think it is not a good choice to multiply the convolution results of different channels in action-conditioned PredRNN model, which is = torch.conv_2d(RGB)*torch.conv_2d(robot_position).

Meanwhile, we add the ionosphere dataset support in the repository. We will be appreciative if you can utilize this dataset and adopt these changes in the future.