[fp8] support fp8 communication and fp8 training for Colossalai#6012

Merged

ver217 merged 75 commits intomainfrom feature/fp8_comm

Aug 27, 2024

+2,856-267

Commits on Jul 1, 2024

fp8 operators for compressed communication
BurkeHulk
authored

Commits on Jul 4, 2024

Commits on Jul 10, 2024

fix typo
GuangyaoZhang
committed

Commits on Jul 16, 2024

Commits on Jul 17, 2024

Commits on Jul 18, 2024

Commits on Jul 24, 2024

[fp8] support all-gather flat tensor (#5932 )
ver217
authored

Commits on Aug 2, 2024

Commits on Aug 5, 2024

[Feature] llama shardformer fp8 support (#5938 )
GuangyaoZhang
authored

Commits on Aug 6, 2024

Commits on Aug 7, 2024

Commits on Aug 8, 2024

Commits on Aug 12, 2024

[fp8] support hybrid parallel plugin (#5982 )
wangbluo
authored

Commits on Aug 13, 2024

[fp8] refactor fp8 linear with compile (#5993 )
ver217
authored

Commits on Aug 14, 2024

[fp8] support asynchronous FP8 communication (#5997 )

flybird11111
and
pre-commit-ci[bot]
authored

Commits on Aug 15, 2024

Commits on Aug 16, 2024

Commits on Aug 17, 2024

Commits on Aug 20, 2024

Commits on Aug 21, 2024

Commits on Aug 26, 2024