caikit-nlp tests fail with transformers 4.44.0

[markstur]

Describe the bug

Our unit tests fail with the new transformers 4.44.0 (currently latest).

Platform

CI and reproducible on Mac M3 with PYTORCH_ENABLE_MPS_FALLBACK=1 tox -e py

Sample Code

Output shows errors like this:

================================================================================================ short test summary info ================================================================================================
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_model - RuntimeError: The expanded size of the tensor (14) must match the existing size (22) at non-singleton dimension 2.  Target sizes: [4...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_stream_out_model - RuntimeError: The expanded size of the tensor (14) must match the existing size (22) at non-singleton dimension 2.  Targe...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_train_model - RuntimeError: The expanded size of the tensor (23) must match the existing size (31) at non-singleton dimension 2.  Target sizes: ...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_gen_trained_mpt - RuntimeError: The expanded size of the tensor (23) must match the existing size (31) at non-singleton dimension 2.  Target siz...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_train_model_classification_record - RuntimeError: The expanded size of the tensor (23) must match the existing size (31) at non-singleton dimens...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_repetition_penalty_0_works - RuntimeError: The expanded size of the tensor (16) must match the existing size (24) at non-singleton dimension...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_truncate_tokens_0 - RuntimeError: The expanded size of the tensor (16) must match the existing size (24) at non-singleton dimension 2.  Targ...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_with_preserve_input_text - RuntimeError: The expanded size of the tensor (16) must match the existing size (24) at non-singleton dimension 2...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_sampling_param_ignored_greedy_decoding - RuntimeError: The expanded size of the tensor (16) must match the existing size (24) at non-singlet...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_with_custom_stop_criteria - RuntimeError: The expanded size of the tensor (16) must match the existing size (24) at non-singleton dimension ...
FAILED tests/modules/text_generation/test_peft_prompt_tuning.py::test_run_exponential_decay_len_penatly_object - RuntimeError: The expanded size of the tensor (16) must match the existing size (24) at non-singleton...
FAILED tests/toolkit/text_generation/test_model_run_utils.py::test_generate_text_func_serialization_json[to_dict-dict-causal_lm_dummy_model] - RuntimeError: The expanded size of the tensor (22) must match the exist...
FAILED tests/toolkit/text_generation/test_model_run_utils.py::test_generate_text_func_serialization_json[to_json-str-causal_lm_dummy_model] - RuntimeError: The expanded size of the tensor (22) must match the existi...
FAILED tests/toolkit/text_generation/test_model_run_utils.py::test_generate_text_func_serialization_json[to_proto-GeneratedTextResult-causal_lm_dummy_model] - RuntimeError: The expanded size of the tensor (22) must...
FAILED tests/toolkit/text_generation/test_model_run_utils.py::test_generate_text_func_preserve_input_causal_lm[causal_lm_dummy_model] - RuntimeError: The expanded size of the tensor (22) must match the existing siz...
=========================================================================== 15 failed, 316 passed, 5 skipped, 23 warnings in 63.65s (0:01:03) ===========================================================================

Expected behavior

We did not expect 4.44 to be a breaking change for this. It's a 6 day old release, so TBD issues/fixes/workarounds.

Observed behavior

See above.

Additional context

Happened to show up on a seemingly unrelated PR and then was reproduced on main.