Add more settings to CompletionOptions

[sestinj]

Every LLM completion is passed a set of parameters in the CompletionOptions object.

We currently support common settings like max_tokens, temperature, top_p, top_k, frequency_penalty, and presence_penalty, but are missing things like tail-free sampling or certain mirostat parameters.

Some model providers, like llama.cpp will accept these, and so it is only a matter of allowing the parameter to be passed in.

1. Update CompletionOptions to have the parameter
2. Many model providers (all are in the core/llm/llms folder) have a function called _convertArgs that turns the CompletionOptions object into the request body expected by their API. For the providers taht support this parameter, make sure that it gets passed to the request. For other providers, take a look to make sure that this extraneous parameter doesn't get sent in the request.

[ffshreyansh]

Hey @TyDunn can I move forward with this one. please let me know.

[TyDunn]

@ffshreyansh We'd love for you to contribute! Let's check with @sestinj about how to scope this one if it is still makes sense now

[sestinj]

@ffshreyansh This is definitely still open for contribution! I think one of the last parameters we'll want to add is tfs_z, for tail-free sampling. It is supported only in Ollama and Llama.cpp as far as I know, as shown in their docs:

• Ollama: https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter
• Llama.cpp: https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md#post-completion-given-a-prompt-it-returns-the-predicted-completion

[gordonwu66]

Hi @TyDunn, @sestinj I'm interested in contributing and picking this up if its still open. If it's already worked on then I'd love to take a look at any other good first issues you may have!