Skip to content

Commit

Permalink
Enhance README with specific warning for Qwen2.5 models: clarify that…
Browse files Browse the repository at this point in the history
… using these models without flash-attn may cause the corrector to get stuck in the beam search process.
  • Loading branch information
Jacob-Zhou committed Jan 3, 2025
1 parent 37c2e83 commit 1e269be
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ pip install flash-attn --no-build-isolation
<!-- Add a warning about Qwen2.5 -->

> [!WARNING]
> Reported by a user, using Qwen2 or Qwen2.5 family models without flash-attn will lead unexpected errors.
> Reported by a user, using Qwen2 or Qwen2.5 family models without flash-attn will lead unexpected errors. **Specifically, the corrector will stuck in the beam search process.**
>
> Please install flash-attn to avoid this issue. Or you can set `torch_dtype=torch.bfloat16` in the `LMCorrector` class to avoid this issue.
>
Expand Down

0 comments on commit 1e269be

Please sign in to comment.