[FEATURE] Add Xinference implementation #1179

[alvinlee518]

Issue

Contributes to langchain4j/langchain4j#1179

Change

Support for Xorbits Inference.

• Added a XinferenceChatModel
• Added a XinferenceEmbeddingModel
• Added a XinferenceImageModel
• Added a XinferenceLanguageModel
• Added a XinferenceScoringModel
• Added a XinferenceStreamingChatModel
• Added a XinferenceStreamingLanguageModel

General checklist

• There are no breaking changes
• I have added unit and integration tests for my change
• I have manually run all the unit tests in all modules, and they are all green
• I have manually run all integration tests in the module I have added/changed, and they are all green

Checklist for adding new maven module

• I have added my new module in the root pom.xml and langchain4j-community-bom/pom.xml

[Martin7-1]

@alvinlee518 Hi! Thank you for your contribution. Will review it asap.

BTW, it looks like XInference support docker container. Would it possible to use testcontainers in IT?

[Martin7-1]

BTW, could you make all static method call by using import static? (e.g. InternalXinferenceHelper.toTools -> toTools). Just for consistency with other modules.

And, could you please add tests about ChatModelListener like OllamaChatModelListenerIT?

[alvinlee518]

BTW, could you make all static method call by using import static? (e.g. InternalXinferenceHelper.toTools -> toTools). Just for consistency with other modules.

And, could you please add tests about ChatModelListener like OllamaChatModelListenerIT?

Okay, got it.

[alvinlee518]

@alvinlee518 Hi! Thank you for your contribution. Will review it asap.

BTW, it looks like XInference support docker container. Would it possible to use testcontainers in IT?

It's a bit tricky for me to test the docker container locally, but I can try adding it.

[alvinlee518]

@Martin7-1 The Xinferenrece testcontainers require GPU support. How can I configure it in GitHub Actions to ensure the unit tests pass?

[Martin7-1]

Since we're using the free tier, Github Actions do not support GPU... Are there any ways that Xinference can run on CPU (such as latest-cpu version). BTW, could we use smaller models (I don't know how much time you test with GPU locally, but if we change it to CPU, it may cost much time, so we need to make the model as small as possible) to reduce IT cost?

If the test takes too long to run with CPU, maybe I'll consider disabling Xinference's IT in Github Actions and just running it locally.

Xinference's ITs are like Ollama's ITs in that they both take a long time to execute tests, and I'm also thinking about and trying to figure out how to optimise them and get them working perfectly in Github Actions. You can try the ideas I suggested above, but if it doesn't work that's fine, I'd run those test cases locally. I think I will review it next week :)

[alvinlee518]

@Martin7-1 I have modified the image to the CPU version. When executed in GitHub Actions, the entire process takes approximately 10 minutes.
ImageModel, VisionModel, and tool call streaming functions rely on GPUs and can be validated locally.

[Martin7-1]

@Martin7-1 I have modified the image to the CPU version. When executed in GitHub Actions, the entire process takes approximately 10 minutes. ImageModel, VisionModel, and tool call streaming functions rely on GPUs and can be validated locally.

Thank you! Will try to review it this week.

[Martin7-1]

@alvinlee518 Thank you!

[Martin7-1]

@alvinlee518 Thank you! About IT: maybe we should give up comitting new image, because so much images will exceed Github Actions disk space limit. WDYT?

[Martin7-1]

Thank you! Would you mind removing all final in method level if it's not really needed?

[alvinlee518]

@alvinlee518 Thank you! About IT: maybe we should give up comitting new image, because so much images will exceed Github Actions disk space limit. WDYT?
@Martin7-1
The new image is intended to avoid re-downloading the model. If you're concerned about the disk space limit, I can remove it.

[Martin7-1]

@alvinlee518 Thank you! LGTM. Could you please add docs in langchain4j main repo?

[alvinlee518]

@alvinlee518 Thank you! LGTM. Could you please add docs in langchain4j main repo?

Okay, I'll add it later.