Code

Torchao float8 training#3348

Code

Open

muellerzr wants to merge 9 commits intomainhuggingface/accelerate:mainfrom torchao-float8huggingface/accelerate:torchao-float8Copy head branch name to clipboard

+1,269-111

Commits

Commits on Jan 9, 2025

• Bookmark

  muellerzr

  committed
  0b5a9c1

  Copy full SHA for 0b5a9c1

Commits on Jan 10, 2025

• bookmark

  muellerzr

  committed
  b2cce71

  Copy full SHA for b2cce71

Commits on Jan 16, 2025

• Add torchao base example

  muellerzr

  committed
  e1a1304

  Copy full SHA for e1a1304

• Currently broken

  muellerzr

  committed
  be210db

  Copy full SHA for be210db

• Clean

  muellerzr

  committed
  fbeb5a7

  Copy full SHA for fbeb5a7

Commits on Jan 17, 2025

• DDP varient working

  muellerzr

  committed
  3820c40

  Copy full SHA for 3820c40

• FSDP as well

  muellerzr

  committed
  4dfa816

  Copy full SHA for 4dfa816

• Works for all but zero3

  muellerzr

  committed
  04c9f56

  Copy full SHA for 04c9f56

• Bookmark: currently zero3 is underperforming

  muellerzr

  committed
  f805876

  Copy full SHA for f805876