default chat template

[pelikhan]

Feature request

When an text-generation model configuration does not have a tokernizer.chat_template (like https://huggingface.co/onnx-community/moondream2.text_model-ONNX/blob/main/tokenizer_config.json), provide a default implementation. Otherwise, trying to send a Chat in the pipeline fails with

Cannot use apply_chat_template() because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating

Motivation

The text-generation pipeline supports a "chat interface" but most published models are lacking the chat_template entry.

See https://huggingface.co/onnx-community/Qwen2.5-Coder-0.5B-Instruct/blob/main/tokenizer_config.json#L198

Your contribution

I could probably produce a PR but I'd like to know whether I'm missing something here.

[xenova]

Hi there 👋 Default chat templates in transformers (and transformers.js) have been deprecated (see huggingface/transformers#30346). Since then, we require users to specify the chat template in their models. For the model you are referring to, I do think it's best to make a PR adding the correct chat template.

[pelikhan]

Is there an api in the pipeline instance to query whether a chat template is available? many transformers.js do not have it.

[xenova]

There are a few ways you could do this:

1. Use pipeline

   import { pipeline } from "@huggingface/transformers";
   const generator = await pipeline("text-generation", "hf-internal-testing/tiny-random-LlamaForCausalLM");
   console.log(generator.tokenizer.chat_template)

2. Use AutoTokenizer

   import { AutoTokenizer } from "@huggingface/transformers";
   const tokenizer = await AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-LlamaForCausalLM");
   console.log(tokenizer.chat_template);

3. Use the Hugging Face Hub API directly: e.g., https://huggingface.co/api/models/hf-internal-testing/tiny-random-LlamaForCausalLM

[pelikhan]

Thanks!🙏

[xenova]

No worries :) Feel free to close the issue if that solves your use-case 👍