Research Papers

[leb9212]

Discussion thread to share AI research papers.

[leb9212]

Some Papers on Multidimensional Timeseries Models Using Attention Mechanism.

Multidimensional Timeseries Classification with Multiple Attention Mechanism

Multidimensional Dynamic Attention for Multivariate Time Series Forecasting

Revisiting Attention For Multivariate Time Series Forecasting

[thorwhalen]

Tried to extract some model specs from it, but didn't do a great job (or simply, the information is not in it):

Summary of Model Specifications

Model Name	Model Size (Params)	Data Dimensions	Data Records	Error Stats (MSE, MAE, etc.)	Training Hardware Specs	Training Time	Key Features
FSatten (Frequency Spectrum Attention)		Multivariate time series	Six real-world datasets (ECL, ETT, Exchange, Traffic, Weather, Solar-Energy)	Outperforms SOTA by 8.1%-21.8% in MSE			Fourier Transform-based attention, Multi-head Spectrum Scaling (MSS)
SOatten (Scaled Orthogonal Attention)		Multivariate time series	Same datasets as FSatten	Improved over FSatten on general datasets			Learnable orthogonal transformation, Head Coupling Convolution (HCC)
PatchTST		Sub-series patches for MTS	L = 96, T = {96, 192, 336, 720}	MSE and MAE provided in benchmarks			Temporal attention applied to patches of each variate
iTransformer		Multivariate time series		Compared to FSatten and SOatten			Variate Transformer for explicit multivariate correlations
TFT (Temporal Fusion Transformer)		Multivariate time series		Feature importance ranking included	GPU-based	Long	Self-attention mechanism
MDA (Multidimensional Dynamic Attention)		Heterogeneous MTS	Synthetic & real-world datasets	Up to 33% improvement over baselines			Dynamic representation learner, multiple attention calculations
DA-RNN (Dual-Stage Attention RNN)		Multivariate time series					Dual-stage attention (input and temporal)
LSTNet		Multivariate time series					Temporal attention for long-term dependencies
DSANet		Multivariate time series					Autoregressive components for time series
IMV-LSTM		Multivariate time series		Attention-based feature selection			Assigns attention weights to feature importance
STAM (Spatiotemporal Attention Model)		Multivariate time series					Spatiotemporal context learning
MrCAN		Multivariate time series					Temporal convolution + spatial attention
ALSTM-FCN (Attention LSTM-FCN)		Multivariate time series					Attention-based temporal dependencies

This table provides a structured overview of various models used for multivariate time series forecasting and classification. If more detailed specifications are needed, such as exact parameter counts or dataset sizes, further investigation into the provided research articles may be required.

[thorwhalen]

Dataset Links

Dataset Name	Description	Download Link
ECL (Electricity Consumed Load)	Contains fifteen-minute electricity consumption (KWh) data for 321 customers from 2012 to 2014.	ECL Dataset
ETT (Electricity Transformer Temperature)	Monitors an electricity transformer from a region of a province of China, including oil temperature and variants of load from July 2016 to July 2018.	ETT Dataset
Exchange	Collection of daily exchange rates of eight countries relative to the US dollar from 1990 to 2016.	Exchange Dataset
Traffic	Reports road hourly occupancy rates of 862 sensors, from January 2015 to December 2016, collected by the California Department of Transportation.	Traffic Dataset
Weather	Contains meteorological measurements recorded every 10 minutes from the Weather Station of the Max Planck Biogeochemistry Institute in Jena, Germany, for the year 2020.	Weather Dataset
Solar-Energy	Provides synthetic solar photovoltaic (PV) power plant data points for the United States representing the year 2006.	Solar Power Data for Integration Studies