Optimizer and trainer states for OLMo-7B (Feb. 2024)

[rahuln]

❓ The question

I'd like to be able to do continued pretraining with the intermediate checkpoints of the original OLMo-7B model (from February 2024), including initializing from the intermediate optimizer and trainer states. I found an older version of the README that includes instructions on how to access these intermediate checkpoints as well as a list of paths pointing to those files, but they don't seem to work with the OLMo training code (i.e., files like model.pt / model.safetensors and optim.pt / optim.safetensors can't be found at those links). Is there somewhere I can go to find links to the original OLMo checkpoints with optimizer and trainer state files?

[aman-17]

Hey @rahuln, those files were deleted by mistake. I'm in the process of restoring them. I will update you once they are back.