Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline: Add pipeline subsystem #60

Merged
merged 23 commits into from
Feb 21, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b18b16e
Add pipeline base class and a simple pipeline
kaancayli Feb 15, 2024
46c09a5
Run black formatter
kaancayli Feb 15, 2024
8cde996
Rename base pipeline
kaancayli Feb 15, 2024
95e767a
Address feedbacks
kaancayli Feb 15, 2024
c24439c
Merge branch 'main' into feature/pipeline-subsystem-v1
kaancayli Feb 19, 2024
dbb189f
Fix syntax errors
kaancayli Feb 19, 2024
04566a4
Add tutor chat and summary pipelines
kaancayli Feb 19, 2024
93bc858
Remove unused imports
kaancayli Feb 19, 2024
d7355b4
Generalize pipeline superclass
MichaelOwenDyer Feb 19, 2024
741b14f
Merge commit
MichaelOwenDyer Feb 19, 2024
44ac136
Fix import errors and filepaths
kaancayli Feb 19, 2024
1feadd1
Merge branch 'main' of github.com:ls1intum/Pyris into feature/pipelin…
kaancayli Feb 19, 2024
6e6839f
Merge branch 'feature/pipeline-subsystem-v1' of github.com:ls1intum/P…
kaancayli Feb 19, 2024
47c49e5
Format file
kaancayli Feb 19, 2024
aba2d25
Naming changes and import bug fixes
kaancayli Feb 19, 2024
1a921e3
Create a singleton abstract metaclass and make pipelines singleton
kaancayli Feb 21, 2024
5b4e55c
Add caching to summary pipeline
kaancayli Feb 21, 2024
bfa3da5
Add repr and str methods
kaancayli Feb 21, 2024
9d82946
Minor adjustments
kaancayli Feb 21, 2024
1c2f107
Remove singleton abstract metaclass for now, since pipelines can use …
kaancayli Feb 21, 2024
588a44b
Address feedbacks
kaancayli Feb 21, 2024
578f9d8
Revert __str__ implementation
kaancayli Feb 21, 2024
12a709b
Uncomment
kaancayli Feb 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/pipeline/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from pipeline.pipeline import SimplePipeline
39 changes: 39 additions & 0 deletions app/pipeline/pipeline.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
from abc import ABCMeta, abstractmethod
from operator import itemgetter

from langchain_core.output_parsers import StrOutputParser
from domain import IrisMessage, IrisMessageRole


class AbstractPipeline(metaclass=ABCMeta):
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
"""Abstract class for all pipelines"""

def __init__(self, name=None):
self.name = name

def __repr__(self):
return f"{self.__class__.__name__} {self.name if self.name is not None else id(self)}"
kaancayli marked this conversation as resolved.
Show resolved Hide resolved

def __str__(self):
return f"{self.__class__.__name__} {self.name if self.name is not None else id(self)}"
kaancayli marked this conversation as resolved.
Show resolved Hide resolved

@abstractmethod
def run(self, *args, **kwargs) -> IrisMessage:
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
"""Runs the pipeline"""
raise NotImplementedError


class SimplePipeline(AbstractPipeline):
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
"""A simple pipeline that does not have any memory etc."""

def __init__(self, llm, name=None):
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
super().__init__(name=name)
self.llm = llm
self.pipeline = {"query": itemgetter("query")} | llm | StrOutputParser()

def run(self, *args, query: IrisMessage, **kwargs) -> IrisMessage:
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
if query is None:
raise ValueError("IrisMessage must not be None")
message = query.text
response = self.pipeline.invoke({"query": message})
return IrisMessage(role=IrisMessageRole.ASSISTANT, text=response)
Loading