You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm new to LLMs, so I'm sorry if I'm missing something obvious. However, when I run this simple code: from guidance import models, gen, select, user, assistant, system, substring model_path = "Llama-3.2-3B-Instruct-Q4_K_M.gguf" llm = models.LlamaCpp(model_path, n_gpu_layers=-1) llm + f'Pick a random number: ' + select(['One',"Two","Three","Four",'Five',"Six","Seven","Eight","Nine","Ten"])
I always get the same output, which is Eight.
If I change the prompt a tiny bit, for example if I remove the colon and the space, it would always get Nine.
What's wrong?
I also tried running the example code on guidance's main github page: llama2 + f'Do you want a joke or a poem? A ' + select(['joke', 'poem'])
Same thing here, it always picks the same option.
I tried changing top_k and temperature parameters when loading the model, but nothing seems to work.
The text was updated successfully, but these errors were encountered:
I'm new to LLMs, so I'm sorry if I'm missing something obvious. However, when I run this simple code:
from guidance import models, gen, select, user, assistant, system, substring
model_path = "Llama-3.2-3B-Instruct-Q4_K_M.gguf"
llm = models.LlamaCpp(model_path, n_gpu_layers=-1)
llm + f'Pick a random number: ' + select(['One',"Two","Three","Four",'Five',"Six","Seven","Eight","Nine","Ten"])
I always get the same output, which is Eight.
If I change the prompt a tiny bit, for example if I remove the colon and the space, it would always get Nine.
What's wrong?
I also tried running the example code on guidance's main github page:
llama2 + f'Do you want a joke or a poem? A ' + select(['joke', 'poem'])
Same thing here, it always picks the same option.
I tried changing top_k and temperature parameters when loading the model, but nothing seems to work.
The text was updated successfully, but these errors were encountered: