Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing

Peihao Wang, Ruisi Cai, Yuehao Wang, Jiajun Zhu, Pragya Srivastava, Zhangyang Wang, Pan Li

Reproduce

Code coming soon...

Citation

If you find this work or our work helpful for your own research, please cite our paper.

@article{wang2024ssmbottleneck,
  title={Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing},
  author={Wang, Peihao and Cai, Ruisi and Wang, Yuehao and Zhu, Jiajun and Srivastava, Pragya and Wang, Zhangyang and Li, Pan},
  journal={arXiv preprint: 2501.00658},
  year={2024}
}