Skip to content

Commit

Permalink
Update Llama table with new evals (huggingface#1895)
Browse files Browse the repository at this point in the history
* Update llama2.md

* Update llama2.md
  • Loading branch information
osanseviero authored Mar 11, 2024
1 parent 29d1153 commit e54a38b
Showing 1 changed file with 13 additions and 14 deletions.
27 changes: 13 additions & 14 deletions llama2.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,20 +50,19 @@ If you’ve been waiting for an open alternative to closed-source chatbots, Llam

| Model | License | Commercial use? | Pretraining length [tokens] | Leaderboard score |
| --- | --- | --- | --- | --- |
| [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) | Apache 2.0 || 1,500B | 47.01 |
| [MPT-7B](https://huggingface.co/mosaicml/mpt-7b) | Apache 2.0 || 1,000B | 48.7 |
| Llama-7B | Llama license || 1,000B | 49.71 |
| [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) | Llama 2 license || 2,000B | 54.32 |
| Llama-33B | Llama license || 1,500B | * |
| [Llama-2-13B](https://huggingface.co/meta-llama/Llama-2-13b-hf) | Llama 2 license || 2,000B | 58.67 |
| [mpt-30B](https://huggingface.co/mosaicml/mpt-30b) | Apache 2.0 || 1,000B | 55.7 |
| [Falcon-40B](https://huggingface.co/tiiuae/falcon-40b) | Apache 2.0 || 1,000B | 61.5 |
| Llama-65B | Llama license || 1,500B | 62.1 |
| [Llama-2-70B](https://huggingface.co/meta-llama/Llama-2-70b-hf) | Llama 2 license || 2,000B | * |
| [Llama-2-70B-chat](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf)* | Llama 2 license || 2,000B | 66.8 |

*we’re currently running evaluation of the Llama 2 70B (non chatty version). This table will be updated with the results.

| [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) | Apache 2.0 || 1,500B | 44.17 |
| [MPT-7B](https://huggingface.co/mosaicml/mpt-7b) | Apache 2.0 || 1,000B | 47.24 |
| Llama-7B | Llama license || 1,000B | 45.65 |
| [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) | Llama 2 license || 2,000B | 50.97 |
| Llama-33B | Llama license || 1,500B | - |
| [Llama-2-13B](https://huggingface.co/meta-llama/Llama-2-13b-hf) | Llama 2 license || 2,000B | 55.69 |
| [mpt-30B](https://huggingface.co/mosaicml/mpt-30b) | Apache 2.0 || 1,000B | 52.77 |
| [Falcon-40B](https://huggingface.co/tiiuae/falcon-40b) | Apache 2.0 || 1,000B | 58.07 |
| Llama-65B | Llama license || 1,500B | 61.19 |
| [Llama-2-70B](https://huggingface.co/meta-llama/Llama-2-70b-hf) | Llama 2 license || 2,000B | 67.87 |
| [Llama-2-70B-chat](https://huggingface.co/meta-llama/Llama-2-70b-chat-hf) | Llama 2 license || 2,000B | 62.4 |

*Note: the performance scores shown in the table below have been updated to account for the new methodology introduced in November 2023, which added new benchmarks. More details in [this post](https://huggingface.co/blog/open-llm-leaderboard-drop)*.

## Demo

Expand Down

0 comments on commit e54a38b

Please sign in to comment.