Inference Code and trained Model

[J-M-pixel]

Hi,
Thanks a lot for your great work.
Is it possible that you could provide trained model (the parameters), and inference code of your model which can be directly compared to Llama 2-7b or Llama 3-8b opensource models ?

[ridgerchu]

Hi,
Sorry for the late reply! You can use lm-evaluation-harness (lm-harness) to evaluate HuggingFace models, including ours, in a straightforward manner.

Regarding your comparison question, it's important to note that our model was trained on 100B tokens, which is optimized for academic research and designed with more lightweight training in mind. In contrast, LLaMA 2-7B was trained on 2T tokens, and LLaMA 3-8B on 15T tokens, making them more industrial-scale models.