-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump LangChain4j version and make Jlama to work in dev mode #45
Conversation
step-10/src/main/java/dev/langchain4j/quarkus/workshop/NumericOutputSanitizerGuard.java
Show resolved
Hide resolved
step-10/src/main/java/dev/langchain4j/quarkus/workshop/NumericOutputSanitizerGuard.java
Show resolved
Hide resolved
@mariofusco do you think it would make sense to only change the jlama step to use the langchain4j RC version and leave the other ones to use the latest stable version? Otherwise I think the workshop attendees might think the entire lab only works with RC version. Either way we should probably mention in the docs why we're using the RC version. |
We probably also want to add a mention of the output guardrail addition in the tutorial |
Ok for me to bump only step 10, we will bump other steps when there will be a new stable release of langchain4j. |
I was not sure if we wanted to explicit discuss that guardrail (which is maybe a bit of an hack 😄 ) during the workshop, but since I believe that it is also a nice example of how to sanitize LLM responses with an output guardrail I will add it. |
@kdubois Done. |
@mariofusco Looks good. I'm wondering if we should disable the observability settings for dev mode since the dev services take a while to start up and it could seem like things start up slower than they should? e.g.
|
That's ok for me, also because with Jlama all LLM inference is local to the JVM so there isn't much to observe. It could be easily reenabled if necessary anyway. Will do so. |
@kdubois Done. |
With Quarkus 3.18.1 and Langchain4j 0.24.0.CR1 it is possible to run Jlama at a reasonable speed even in dev mode, so this pull request adds back into step 10 all the features that were removed because they couldn't work without the dev services.