Transformers past_key_values deprecated

[hudson-ai]

With transformers version 4.44.0 (latest at time of writing), I get a warning regarding the use of past_key_values in the TransformersEngine.

We detected that you are passing past_key_values as a tuple and this is deprecated and will be removed in v4.43. Please use an appropriate Cache class (https://huggingface.co/docs/transformers/v4.41.3/en/internal/generation_utils#transformers.Cache)
You are not running the flash-attention implementation, expect numerical differences.

The warning is clearly wrong (as 4.44.0 > 4.43), but if this usage is truly going to be removed, we need to fix our caching logic.

[usholanb]

agree, also T5 from

https://github.com/huggingface/transformers/blob/746104ba6f0514159d58dc2fb09c887d0e9d4863/src/transformers/models/t5/modeling_t5.py#L1650

expects past_key_values as Tuple

[Harsha-Nori]

@usholanb, interesting, thanks for sharing! If I'm reading this right, does that mean that we can't continue to support T5 if we move onto the new transformers.Cache API?

[usholanb]

@Harsha-Nori I don't know. I haven't switched to transformers.Cache myself. Just sharing my concern since this message is making me worried for no clear reason

[hudson-ai]

Getting a new error in CI: "HybridCache not subscriptable" -- may very well be due to them finally making some breaking changes with the Cache API.

https://github.com/guidance-ai/guidance/actions/runs/11050523063