Skip to content

Commit

Permalink
Update LLM logic for pure text input (opea-project#40)
Browse files Browse the repository at this point in the history
* fix requirement issue

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update llm codes

Signed-off-by: letonghan <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: letonghan <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
letonghan and pre-commit-ci[bot] authored May 10, 2024
1 parent 592ce60 commit 2867c05
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion comps/llms/langchain/llm_tgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,19 +36,22 @@ def llm_generate(input: Union[TextDoc, RerankedDoc]) -> GeneratedDoc:
repetition_penalty=params.repetition_penalty,
streaming=params.streaming,
)
final_prompt = None
if isinstance(input, RerankedDoc):
template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)
chain = prompt | llm | StrOutputParser()
final_prompt = input.query
response = chain.invoke({"question": input.query, "context": input.doc.text})
elif isinstance(input, TextDoc):
final_prompt = input.text
response = llm.invoke(input.text)
else:
raise TypeError("Invalid input type. Expected TextDoc or RerankedDoc.")
res = GeneratedDoc(text=response, prompt=input.query)
res = GeneratedDoc(text=response, prompt=final_prompt)
return res


Expand Down

0 comments on commit 2867c05

Please sign in to comment.