Skip to content

Commit

Permalink
refactor(README): use Meta-Llama-3-8B-Instruct as the RoPE scaling ex…
Browse files Browse the repository at this point in the history
…ample
  • Loading branch information
ganler committed May 19, 2024
1 parent 5700a58 commit 1de7de5
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,10 @@ repoqa.search_needle_function --model "Qwen/CodeQwen1.5-7B-Chat" --backend vllm

> [!Tip]
>
> You can unlock the model's context using dynamic RoPE scaling.
> You can unlock the model's context using [dynamic RoPE scaling](https://blog.eleuther.ai/yarn/#dynamic-scaling).
> For example, `Meta-Llama-3-8B-Instruct` has 8k context but running the default 16k test needs more (approx. 20k).
>
> To extend the context to 32k, edit its config file (`hub/models--meta-llama--Meta-Llama-3-8B-Instruct/snapshots/[hash]/config.json`) by setting:
> To extend the context to 32k, in its config file (`hub/models--meta-llama--Meta-Llama-3-8B-Instruct/snapshots/[hash]/config.json`) set:
>
> `"rope_scaling": {"type": "dynamic", "factor": 4.0}`
>
Expand Down

0 comments on commit 1de7de5

Please sign in to comment.