Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GenAIComps microservices refactor #1072

Open
wants to merge 134 commits into
base: main
Choose a base branch
from
Open

GenAIComps microservices refactor #1072

wants to merge 134 commits into from
+3,834 −5,069

Conversation

chensuyue
Copy link
Collaborator

@chensuyue chensuyue commented Dec 24, 2024 •
edited
Loading

Description

GenAIComps microservices refactor

Issues

List the issue or RFC link this PR is working on. If there is no such link, please mark it as n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

Dependencies

List the newly introduced 3rd party dependency if exists.

Tests

Describe the tests that you ran to verify your changes.

XinyaoWa and others added 30 commits December 10, 2024 15:01
@XinyaoWa @pre-commit-ci
[TEST] Refactor helmchart (#964)
bdf4b21 
* refactor example

Signed-off-by: Xinyao Wang <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Xinyao Wang <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@XinyaoWa
add deployment for 3rd party
6682318 
Signed-off-by: Xinyao Wang <[email protected]>
@XinyaoWa
refactor src & development
bec8360 
Signed-off-by: Xinyao Wang <[email protected]>
@XinyaoWa
fix
14dea41 
Signed-off-by: Xinyao Wang <[email protected]>
@XinyaoWa
fix
29ce89d 
Signed-off-by: Xinyao Wang <[email protected]>
@lvliang-intel
refactor embedding
36d90f4 
Signed-off-by: lvliang-intel <[email protected]>
@XinyaoWa
refactor for TGI LLM
67db564 
Signed-off-by: Xinyao Wang <[email protected]>
@letonghan
refactor dataprep redis
fb57cb5 
Signed-off-by: letonghan <[email protected]>
@XinyaoWa
fix for tgi llm
93a0d24 
Signed-off-by: Xinyao Wang <[email protected]>
@XinyaoWa
support vllm LLM
e0edef9 
Signed-off-by: Xinyao Wang <[email protected]>
@wangkl2 @xiguiw @pre-commit-ci
Refactor for Reranking with TEI (#1029)
aeb0ec4 
* reranks folder reorg

Signed-off-by: Wang, Kai Lawrence <[email protected]>

* update opea_reranking_microservice

Signed-off-by: Wang, Kai Lawrence <[email protected]>

* add each concrete reranking class

Signed-off-by: Wang, Kai Lawrence <[email protected]>

* update concrete reranking classes

Signed-off-by: Wang, Kai Lawrence <[email protected]>

* Fix some issues

* Fix issue reranking Micro Service in component controller

Signed-off-by: Wang, Xigui <[email protected]>

* Enabel check_health

Signed-off-by: Wang, Xigui <[email protected]>

* Put data format conversion in Micro-Service

Move data format conversion from reranking to Micro-Service

Signed-off-by: Wang, Xigui <[email protected]>

* Add comments for code

Clean the code

Signed-off-by: Wang, Xigui <[email protected]>

* refine

Signed-off-by: Wang, Kai Lawrence <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Wang, Kai Lawrence <[email protected]>
Signed-off-by: Wang, Xigui <[email protected]>
Co-authored-by: Wang, Xigui <[email protected]>
Co-authored-by: xiguiw <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@letonghan
support milvus dataprep
20b91d1 
Signed-off-by: letonghan <[email protected]>
@letonghan
Merge branch 'refactor_comps' of https://github.com/opea-project/GenA…
b359328 
…IComps into refactor_comps
@XinyaoWa
add docker compose
4e3e228 
Signed-off-by: Xinyao Wang <[email protected]>
@lvliang-intel
update embedding code
423a950 
Signed-off-by: lvliang-intel <[email protected]>
@lvliang-intel
Merge branch 'refactor_comps' of https://github.com/opea-project/GenA…
ea4913f 
…IComps into refactor_comps

Signed-off-by: lvliang-intel <[email protected]>
@lvliang-intel
Merge branch 'main' of https://github.com/opea-project/GenAIComps int…
b852e04 
…o refactor_comps

Signed-off-by: lvliang-intel <[email protected]>
@lvliang-intel
update docker compose and test scripts
6945af4 
Signed-off-by: lvliang-intel <[email protected]>
@letonghan
modify dataprep variable names
6da4f71 
Signed-off-by: letonghan <[email protected]>
@letonghan
refactor retrievers redis
7b6c8f6 
Signed-off-by: letonghan <[email protected]>
@letonghan
Merge branch 'refactor_comps' of https://github.com/opea-project/GenA…
bb8ff6a 
…IComps into refactor_comps
@letonghan
fix merge conflict
843f1c0 
Signed-off-by: letonghan <[email protected]>
@lvliang-intel
Fix embedding issue
ceb4869 
Signed-off-by: lvliang-intel <[email protected]>
@lvliang-intel
Merge branch 'refactor_comps' of https://github.com/opea-project/GenA…
4ec50b0 
…IComps into refactor_comps

Signed-off-by: lvliang-intel <[email protected]>
@letonghan
delete redundant comments
cb5527e 
Signed-off-by: letonghan <[email protected]>
@letonghan
modify test scripts
3494b7b 
Signed-off-by: letonghan <[email protected]>
@letonghan
update milvus dataprep docker compose
191e58e 
Signed-off-by: letonghan <[email protected]>
@letonghan
fix spell issue
3cfb65d 
Signed-off-by: letonghan <[email protected]>
@letonghan
fix bugs
4f6677f 
Signed-off-by: letonghan <[email protected]>
@letonghan
fix multi components issue and refine ci scripts
9c6f50c 
Signed-off-by: letonghan <[email protected]>
XinyaoWa and others added 12 commits December 24, 2024 10:05
@XinyaoWa
refine 3rd party folder (#1063)
b6eaeb4 
Signed-off-by: Xinyao Wang <[email protected]>
@XinyaoWa
remove openvino ut to fix pr, will add later
354c480 
Signed-off-by: Xinyao Wang <[email protected]>
@chensuyue
Update CI Infra (#1060)
1250d42 
Signed-off-by: chensuyue <[email protected]>
@letonghan
fix retrievers pinecone (#1064)
d72f05f 
Signed-off-by: letonghan <[email protected]>
@letonghan
fix dataprep pinecone
d9eac3c 
Signed-off-by: letonghan <[email protected]>
@chensuyue
update comps ci (#1066)
dba68e2 
Signed-off-by: chensuyue <[email protected]>
@letonghan
fix dataprep pinecone (#1067)
58813c2 
Signed-off-by: letonghan <[email protected]>
@letonghan
fix dataprep redis llama_index (#1068)
a6098f0 
Signed-off-by: letonghan <[email protected]>
@letonghan
fix redis langchain ray ci issue (#1069)
b83f246 
Signed-off-by: letonghan <[email protected]>
Temporarily comment out the rerank test script.
be48b85 
Signed-off-by: WenjiaoYue <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>
Temporarily comment out the rerank test script
0750c64 
Signed-off-by: WenjiaoYue <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>
@lvliang-intel
Remove MOSEC dependency microservice (#1071)
5f881ef 
Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: chensuyue <[email protected]>
@chensuyue
Copy link
Collaborator Author

Related GenAIExamples update. https://github.com/opea-project/GenAIExamples/pull/1285/files

@codecov Codecov
Copy link

codecov bot commented Dec 24, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
comps/cores/common/component.py 77.77% 2 Missing ⚠️
Files with missing lines Coverage Δ
comps/cores/common/component.py 88.88% <77.77%> (+0.79%) ⬆️

letonghan and others added 6 commits December 25, 2024 10:38
WenjiaoYue and others added 3 commits December 27, 2024 12:20
@WenjiaoYue
Refactor rerank tei (#1082)
64c547e 
Signed-off-by: WenjiaoYue <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>
@letonghan
Merge branch 'main' into refactor_comps
9d9c919
@letonghan
modify dataprep utils path
c0f794c 
Signed-off-by: letonghan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lvliang-intel lvliang-intel Awaiting requested review from lvliang-intel lvliang-intel is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@chensuyue @xiguiw @XinyaoWa @lvliang-intel @letonghan @wangkl2 @WenjiaoYue