Code

Small fixes when resuming training#245

Code

Merged

NouamaneTazi merged 6 commits intomainhuggingface/nanotron:mainfrom nouamane/fix-optim-states-resuminghuggingface/nanotron:nouamane/fix-optim-states-resumingCopy head branch name to clipboardNov 21, 2024

+77-19

Commits

Commits on Nov 19, 2024

• fix optim states dtype + fix lr scheduler initial value when resuming

  NouamaneTazi

  committed
  4cc62ae

  Copy full SHA for 4cc62ae

• fix ".module" issue in load_optimizer by passing unwrapped_model

  NouamaneTazi

  committed
  8e3c5e6

  Copy full SHA for 8e3c5e6

Commits on Nov 20, 2024

• .

  NouamaneTazi

  committed
  e967e78

  Copy full SHA for e967e78

Commits on Nov 21, 2024

• fix non sharded optim states when loading checkpoint

  NouamaneTazi

  committed
  013a153

  Copy full SHA for 013a153

• added test to highlight previous bug

  NouamaneTazi

  committed
  9aedd84

  Copy full SHA for 9aedd84

• add small warning for PP case

  NouamaneTazi

  committed
  ab8c145

  Copy full SHA for ab8c145