Skip to content

Commit

Permalink
update README
Browse files Browse the repository at this point in the history
  • Loading branch information
SimJeg committed Nov 21, 2024
1 parent 90ca972 commit 58ed978
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,11 @@ We support KV cache quantization through the transformers `QuantizedCache` class

```python
from transformers import QuantizedCacheConfig, QuantoQuantizedCache

config = QuantizedCacheConfig(nbits=4)
cache = QuantoQuantizedCache(config)

pipe(...., cache=cache)
pipe(..., cache=cache)
```

By default, the `DynamicCache` is used (no quantization).
Expand Down

0 comments on commit 58ed978

Please sign in to comment.