Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump LangChain4j version and make Jlama to work in dev mode #45

Merged
merged 3 commits into from
Jan 31, 2025

Conversation

mariofusco
Copy link
Collaborator

With Quarkus 3.18.1 and Langchain4j 0.24.0.CR1 it is possible to run Jlama at a reasonable speed even in dev mode, so this pull request adds back into step 10 all the features that were removed because they couldn't work without the dev services.

@mariofusco mariofusco requested a review from kdubois January 30, 2025 15:32
@kdubois
Copy link
Collaborator

kdubois commented Jan 31, 2025

@mariofusco do you think it would make sense to only change the jlama step to use the langchain4j RC version and leave the other ones to use the latest stable version? Otherwise I think the workshop attendees might think the entire lab only works with RC version.

Either way we should probably mention in the docs why we're using the RC version.

@kdubois
Copy link
Collaborator

kdubois commented Jan 31, 2025

We probably also want to add a mention of the output guardrail addition in the tutorial

@mariofusco
Copy link
Collaborator Author

@mariofusco do you think it would make sense to only change the jlama step to use the langchain4j RC version and leave the other ones to use the latest stable version? Otherwise I think the workshop attendees might think the entire lab only works with RC version.

Either way we should probably mention in the docs why we're using the RC version.

Ok for me to bump only step 10, we will bump other steps when there will be a new stable release of langchain4j.

@mariofusco
Copy link
Collaborator Author

We probably also want to add a mention of the output guardrail addition in the tutorial

I was not sure if we wanted to explicit discuss that guardrail (which is maybe a bit of an hack 😄 ) during the workshop, but since I believe that it is also a nice example of how to sanitize LLM responses with an output guardrail I will add it.

@mariofusco
Copy link
Collaborator Author

@kdubois Done.

@kdubois
Copy link
Collaborator

kdubois commented Jan 31, 2025

@mariofusco Looks good. I'm wondering if we should disable the observability settings for dev mode since the dev services take a while to start up and it could seem like things start up slower than they should?

e.g.

quarkus.observability.enabled=false
quarkus.micrometer.enabled=false

@mariofusco
Copy link
Collaborator Author

@mariofusco Looks good. I'm wondering if we should disable the observability settings for dev mode since the dev services take a while to start up and it could seem like things start up slower than they should?

That's ok for me, also because with Jlama all LLM inference is local to the JVM so there isn't much to observe. It could be easily reenabled if necessary anyway. Will do so.

@mariofusco
Copy link
Collaborator Author

@kdubois Done.

@kdubois kdubois merged commit 01f3670 into quarkusio:main Jan 31, 2025
11 checks passed
@mariofusco mariofusco deleted the jlama-dev branch January 31, 2025 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants