This is an unofficial PyTorch implementation of Powerformer: A Transformer with Weighted Causal Attention for Time-Series Forecasting.
🚧 Work in Progress 🚧
To train the model using the default example dataset, run:
python src/train_powerformer.py
Model configurations can be modified in either trainer.yaml
or config.yaml
. Alternatively, parameters can be overridden directly via the command line:
python train_powerformer.py model.num_layers=4
Hyperparameter search spaces can be defined in trainer.yaml
and config.yaml
:
trainer.yaml
configures tuning fortransformers.TrainingArguments
parameters.config.yaml
allows customization of model-specific parameters.
To enable hyperparameter optimization, run:
python train_powerformer.py do_hyperoptim=true
A Jupyter notebook (data/data.ipynb
) is provided for dataset preparation and splitting. The formatted dataset follows a structure similar to the Chronos dataset.
The initial dataset, stored as a Parquet file, should include:
- A time or timestamp column.
- Any number of numerical feature columns for forecasting.