You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running ramalama on a computer with podman installed, if you try to setup a server through the ramalama serve. The server will be inaccesible from the host computer.
As it can be seen, both run the llama-server command without the --host parameter, this defaults llama-server to listen to 127.0.0.1, which makes sense on the first case, but makes it so that it fails to listen from requests from the host environment on the second case.
I was able to verify this by entering the container through podman exec and being successfully able to run the python script from inside the container.
The text was updated successfully, but these errors were encountered:
I believe that this problem was a byproduct of 8ed6f48
Before this commit, ramalama ran inside the container and processed the aruments from the inside, so it was able to propperly evaluate if the llama-server command was going to run in a container since it was running inside the container
What is the problem:
When running ramalama on a computer with podman installed, if you try to setup a server through the
ramalama
serve. The server will be inaccesible from the host computer.How to reproduce
What is happening?
First, here is the difference between running ramalama with and without podman installed.
Without podman installed
With podman installed
ramalama --dryrun serve tiny > podman run --rm -i --label RAMALAMA ... quay.io/ramalama/ramalama:latest llama-server --port 8080 -m /path/to/mod
As it can be seen, both run the
llama-server
command without the --host parameter, this defaultsllama-server
to listen to127.0.0.1
, which makes sense on the first case, but makes it so that it fails to listen from requests from the host environment on the second case.I was able to verify this by entering the container through
podman exec
and being successfully able to run the python script from inside the container.The text was updated successfully, but these errors were encountered: