coding details for a problem similar to DREAM challenge

[TiaHao]

Dear developer,

Thanks for sharing this amazing repository about the Neural Granger Causality project. I am dealing with a dataset containing 126 duplicates of small time series datasets representing data from 126 different cities, each of them having around 30 features and 100+ time steps. I am unsure how to apply cMLP or cLSTM for the whole dataset. I am thinking of separately training 126 models, but this would be too time-consuming. So I would appreciate it if there would be any suggestion or demo for how to implement the model for a challenge similar to DREAM datasets in the paper. Thank you very much!

Best,

[iancovert]

Thanks for checking out the package!

For this situation, would you prefer to train a single model with all of the 126 time series? If so, the training routines for each model (cMLP, cLSTM, etc) could treat them as independent time series, like different samples in a minibatch. You can see here that the forward function has a leading batch dimension, and you can see in this notebook that our demos treat the data like a batch of one long time series.

Let me know how this sounds to you.

[TiaHao]

Thanks for your reply! If I understand correctly, in this way, a batch is one city's data right? One batch has dimension of (30, 100), 30: feature number, 100: time steps. in total, 126 batches.

[iancovert]

Yes, that's right. And it would mean that the temporal dynamics and connectivity between time series are shared across all cities. Does that sound like what you're looking for?

[TiaHao]

Yes! thanks a lot, that is what I am looking for! btw maybe one last question: would you know if I do not have GPU acceleration ( I am using MacBook pro 2017, no M chip), how much time could it take to train the cMLP or cLSTM?

[iancovert]

I'm not exactly sure how long training will take, that depends on the dataset (how long each gradient step takes, how many steps are required to converge). cMLP is typically faster than cLSTM, so you might start with that. But training will definitely be slower on CPU than GPU, so it could be worth finding one, especially because you'll want to train with different lambda values to compare the results.

[TiaHao]

Thank you very much! I am going to try and test on my data!