Unable to run NLLB-200-1.3B T2T Translation

[kopaka7]

• OS: macOS 15.2
• When I selected the 1.3B model, doing a Japanese to English translation, I got an error in the UI and the following error logged to console in Docker:
  2025-01-03 23:04:50 app-1 | Initializing NLLB Model.. 2025-01-03 23:04:50 app-1 | 2025-01-03 23:04:50 app-1 | Error translating file: 'NoneType' object has no attribute 'endswith' 2025-01-03 23:04:50 app-1 | Traceback (most recent call last): 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/queueing.py", line 625, in process_events 2025-01-03 23:04:50 app-1 | response = await route_utils.call_process_api( 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/route_utils.py", line 322, in call_process_api 2025-01-03 23:04:50 app-1 | output = await app.get_blocks().process_api( 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 2047, in process_api 2025-01-03 23:04:50 app-1 | result = await self.call_function( 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1594, in call_function 2025-01-03 23:04:50 app-1 | prediction = await anyio.to_thread.run_sync( # type: ignore 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync 2025-01-03 23:04:50 app-1 | return await get_async_backend().run_sync_in_worker_thread( 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread 2025-01-03 23:04:50 app-1 | return await future 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1005, in run 2025-01-03 23:04:50 app-1 | result = context.run(func, *args) 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/utils.py", line 869, in wrapper 2025-01-03 23:04:50 app-1 | response = f(*args, **kwargs) 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/modules/translation/translation_base.py", line 91, in translate_file 2025-01-03 23:04:50 app-1 | self.update_model(model_size=model_size, 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/modules/translation/nllb_inference.py", line 55, in update_model 2025-01-03 23:04:50 app-1 | self.model = AutoModelForSeq2SeqLM.from_pretrained(pretrained_model_name_or_path=model_size, 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained 2025-01-03 23:04:50 app-1 | return model_class.from_pretrained( 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4036, in from_pretrained 2025-01-03 23:04:50 app-1 | state_dict = load_state_dict(resolved_archive_file, weights_only=weights_only) 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 502, in load_state_dict 2025-01-03 23:04:50 app-1 | if checkpoint_file.endswith(".safetensors") and is_safetensors_available(): 2025-01-03 23:04:50 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:04:50 app-1 | AttributeError: 'NoneType' object has no attribute 'endswith' 2025-01-03 23:05:15 app-1 | 2025-01-03 23:05:15 app-1 | Initializing NLLB Model.. 2025-01-03 23:05:15 app-1 | 2025-01-03 23:05:15 app-1 | Error translating file: 'NoneType' object has no attribute 'endswith' 2025-01-03 23:05:15 app-1 | Traceback (most recent call last): 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/queueing.py", line 625, in process_events 2025-01-03 23:05:15 app-1 | response = await route_utils.call_process_api( 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/route_utils.py", line 322, in call_process_api 2025-01-03 23:05:15 app-1 | output = await app.get_blocks().process_api( 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 2047, in process_api 2025-01-03 23:05:15 app-1 | result = await self.call_function( 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/blocks.py", line 1594, in call_function 2025-01-03 23:05:15 app-1 | prediction = await anyio.to_thread.run_sync( # type: ignore 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync 2025-01-03 23:05:15 app-1 | return await get_async_backend().run_sync_in_worker_thread( 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread 2025-01-03 23:05:15 app-1 | return await future 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1005, in run 2025-01-03 23:05:15 app-1 | result = context.run(func, *args) 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/gradio/utils.py", line 869, in wrapper 2025-01-03 23:05:15 app-1 | response = f(*args, **kwargs) 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/modules/translation/translation_base.py", line 91, in translate_file 2025-01-03 23:05:15 app-1 | self.update_model(model_size=model_size, 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/modules/translation/nllb_inference.py", line 55, in update_model 2025-01-03 23:05:15 app-1 | self.model = AutoModelForSeq2SeqLM.from_pretrained(pretrained_model_name_or_path=model_size, 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained 2025-01-03 23:05:15 app-1 | return model_class.from_pretrained( 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4036, in from_pretrained 2025-01-03 23:05:15 app-1 | state_dict = load_state_dict(resolved_archive_file, weights_only=weights_only) 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | File "/Whisper-WebUI/venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 502, in load_state_dict 2025-01-03 23:05:15 app-1 | if checkpoint_file.endswith(".safetensors") and is_safetensors_available(): 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | AttributeError: 'NoneType' object has no attribute 'endswith'

[jhj0517]

Hi.

if checkpoint_file.endswith(".safetensors") and is_safetensors_available(): 2025-01-03 23:05:15 app-1 | ^^^^^^^^^^^^^^^^^^^^^^^^ 2025-01-03 23:05:15 app-1 | AttributeError: 'NoneType' object has no attribute 'endswith'

With this error message, I suspect that this happens because the model is not completely downloaded in the path, probably the model download is somehow paused / corrupted and caused this.

Would you delete the model directory (it's likely in the "/Whisper-WebUI/models/NLLB/models--facebook--nllb-200-1.3B" directory) and try again?

The model files will be there if you didn't edit the default mount path of the VOLUMES in docker-compose.yaml.