Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FastAPI: Add DTOs and pipeline run endpoint #70

Merged
merged 60 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
b18b16e
Add pipeline base class and a simple pipeline
kaancayli Feb 15, 2024
46c09a5
Run black formatter
kaancayli Feb 15, 2024
8cde996
Rename base pipeline
kaancayli Feb 15, 2024
95e767a
Address feedbacks
kaancayli Feb 15, 2024
c24439c
Merge branch 'main' into feature/pipeline-subsystem-v1
kaancayli Feb 19, 2024
dbb189f
Fix syntax errors
kaancayli Feb 19, 2024
04566a4
Add tutor chat and summary pipelines
kaancayli Feb 19, 2024
93bc858
Remove unused imports
kaancayli Feb 19, 2024
d7355b4
Generalize pipeline superclass
MichaelOwenDyer Feb 19, 2024
741b14f
Merge commit
MichaelOwenDyer Feb 19, 2024
44ac136
Fix import errors and filepaths
kaancayli Feb 19, 2024
1feadd1
Merge branch 'main' of github.com:ls1intum/Pyris into feature/pipelin…
kaancayli Feb 19, 2024
6e6839f
Merge branch 'feature/pipeline-subsystem-v1' of github.com:ls1intum/P…
kaancayli Feb 19, 2024
47c49e5
Format file
kaancayli Feb 19, 2024
aba2d25
Naming changes and import bug fixes
kaancayli Feb 19, 2024
1a921e3
Create a singleton abstract metaclass and make pipelines singleton
kaancayli Feb 21, 2024
5b4e55c
Add caching to summary pipeline
kaancayli Feb 21, 2024
bfa3da5
Add repr and str methods
kaancayli Feb 21, 2024
9d82946
Minor adjustments
kaancayli Feb 21, 2024
1c2f107
Remove singleton abstract metaclass for now, since pipelines can use …
kaancayli Feb 21, 2024
588a44b
Address feedbacks
kaancayli Feb 21, 2024
578f9d8
Revert __str__ implementation
kaancayli Feb 21, 2024
12a709b
Uncomment
kaancayli Feb 21, 2024
3634583
Merge branch 'main' of github.com:ls1intum/Pyris into feature/pipelin…
kaancayli Feb 21, 2024
b734cad
Integrating DTOs
kaancayli Feb 21, 2024
236ae85
Add file selector pipeline
kaancayli Feb 21, 2024
84b33f3
Start integrating fast-api
kaancayli Feb 26, 2024
17943fa
Data loading and status callbacks
MichaelOwenDyer Feb 26, 2024
cb6f6da
Implement dtos
kaancayli Feb 27, 2024
f19e0d4
Implement all dtos :D
kaancayli Feb 27, 2024
a2a2f3a
Adjust pipelines
kaancayli Feb 27, 2024
ac8dcbe
Merge main
kaancayli Feb 28, 2024
cda7905
Adjust file selector pipeline
kaancayli Feb 28, 2024
4ef9022
Remove single abstract class
kaancayli Feb 28, 2024
dcf2b2a
Add build logs to file selector pipeline
kaancayli Feb 28, 2024
7554653
Send response back to Artemis
kaancayli Feb 28, 2024
62bbcc9
Fix import errors
kaancayli Feb 28, 2024
69d6601
Bug fixing
kaancayli Feb 28, 2024
0edd466
Address flake8 warnings
kaancayli Feb 28, 2024
84d89e8
Add authentication
kaancayli Feb 28, 2024
f961ebc
Address flake8 warnings
kaancayli Feb 28, 2024
cc8d56a
Fix fields in dtos
kaancayli Mar 1, 2024
c24d5e5
Fix 2 small issues
Hialus Mar 1, 2024
dfc75ea
Address issues
kaancayli Mar 5, 2024
81516d2
Remove trailing white space
kaancayli Mar 5, 2024
55ba15f
Merge main from origin
kaancayli Mar 5, 2024
ad04c47
Write README.MD & Fix import errors
kaancayli Mar 6, 2024
f8afaf7
Fix nullable fields
kaancayli Mar 6, 2024
da7880d
Replicate guidance behaviour in tutor chat pipeline
kaancayli Mar 6, 2024
01d38fe
Add guard prompt to the chain
kaancayli Mar 6, 2024
c3b11ba
Capabilities are not optional
kaancayli Mar 6, 2024
10d10ee
Adjust prompts
kaancayli Mar 6, 2024
05cac0a
Address flake warnings
kaancayli Mar 6, 2024
27c088f
Remove unused import
kaancayli Mar 6, 2024
8b13adf
Add missing status updates
kaancayli Mar 6, 2024
bb32775
Add try catch blocks for proper status updates
kaancayli Mar 6, 2024
f683132
Adapt config. Replace None unions with Optional. Remove unnecessary f…
kaancayli Mar 11, 2024
257ea62
Remove unused function
kaancayli Mar 11, 2024
b1991e6
Overhaul status update logic
kaancayli Mar 11, 2024
34b0932
Linter error and status update fixes
kaancayli Mar 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,4 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/
/app/pipeline_playground.py
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
rev: v2.0.0
hooks:
- id: flake8
language_version: python3.12
language_version: python3.12
21 changes: 20 additions & 1 deletion app/common/message_converters.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
from datetime import datetime

from langchain_core.messages import BaseMessage

from domain import IrisMessage, IrisMessageRole
from domain.iris_message import IrisMessage, IrisMessageRole
from domain.data.message_dto import MessageDTO, MessageContentDTO, IrisMessageSender


def convert_iris_message_to_message_dto(iris_message: IrisMessage) -> MessageDTO:
match iris_message.role:
case "user":
sender = IrisMessageSender.USER
case "assistant":
sender = IrisMessageSender.LLM
case _:
raise ValueError(f"Unknown message role: {iris_message.role}")

return MessageDTO(
sent_at=datetime.now(),
sender=sender,
contents=[MessageContentDTO(textContent=iris_message.text)],
)


def convert_iris_message_to_langchain_message(iris_message: IrisMessage) -> BaseMessage:
Expand Down
12 changes: 7 additions & 5 deletions app/domain/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
from domain.message import IrisMessage, IrisMessageRole
from domain.course import Course
from domain.exercise import ProgrammingExercise
from domain.submission import ProgrammingSubmission
from domain.codehint import CodeHint
from error_response_dto import IrisErrorResponseDTO
from pipeline_execution_dto import PipelineExecutionDTO
from pipeline_execution_settings_dto import PipelineExecutionSettingsDTO
from domain.tutor_chat.tutor_chat_pipeline_execution_dto import (
TutorChatPipelineExecutionDTO,
)
from result_dto import PyrisResultDTO
28 changes: 0 additions & 28 deletions app/domain/codehint.py

This file was deleted.

9 changes: 0 additions & 9 deletions app/domain/course.py

This file was deleted.

Empty file added app/domain/data/__init__.py
Empty file.
10 changes: 10 additions & 0 deletions app/domain/data/build_log_entry.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from datetime import datetime
from pydantic import BaseModel


class BuildLogEntryDTO(BaseModel):
timestamp: datetime
message: str

def __str__(self):
return f"{self.timestamp}: {self.message}"
7 changes: 7 additions & 0 deletions app/domain/data/course_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from pydantic import BaseModel


class CourseDTO(BaseModel):
id: int
name: str
description: str
7 changes: 7 additions & 0 deletions app/domain/data/feedback_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from pydantic import BaseModel, Field


class FeedbackDTO(BaseModel):
text: str
test_case_name: str = Field(alias="testCaseName")
credits: float
5 changes: 5 additions & 0 deletions app/domain/data/image_message_content_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel, Field


class ImageMessageContentDTO(BaseModel):
image_data: str = Field(alias="imageData")
6 changes: 6 additions & 0 deletions app/domain/data/json_message_content_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from pydantic import BaseModel, Field, Json
from typing import Any


class JsonMessageContentDTO(BaseModel):
json_content: Json[Any] = Field(alias="jsonContent")
11 changes: 11 additions & 0 deletions app/domain/data/lecture_unit_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from datetime import datetime

from pydantic import BaseModel, Field


class LectureUnitDTO(BaseModel):
id: int
lecture_id: int = Field(alias="lectureId")
release_date: datetime = Field(alias="releaseDate")
name: str
attachment_version: int = Field(alias="attachmentVersion")
5 changes: 5 additions & 0 deletions app/domain/data/message_content_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel


class MessageContentDTO(BaseModel):
pass
40 changes: 40 additions & 0 deletions app/domain/data/message_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
from datetime import datetime
from enum import Enum
from typing import List, Literal

from domain.iris_message import IrisMessage
from message_content_dto import MessageContentDTO

from pydantic import BaseModel, Field


class IrisMessageSender(str, Enum):
USER = "USER"
LLM = "LLM"


class MessageDTO(BaseModel):
sent_at: datetime = Field(alias="sentAt")
sender: Literal[IrisMessageSender.USER, IrisMessageSender.LLM]
contents: List[MessageContentDTO]

def __str__(self):
match self.sender:
case IrisMessageSender.USER:
sender = "user"
case IrisMessageSender.LLM:
sender = "ai"
case _:
raise ValueError(f"Unknown message sender: {self.sender}")
return f"{sender}: {self.contents[0].textContent}"

def convert_to_iris_message(self):
match self.sender:
case IrisMessageSender.USER:
sender = "user"
case IrisMessageSender.LLM:
sender = "assistant"
case _:
raise ValueError(f"Unknown message sender: {self.sender}")

return IrisMessage(text=self.contents[0].textContent, role=sender)
30 changes: 30 additions & 0 deletions app/domain/data/programming_exercise_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
from typing import Dict

from pydantic import BaseModel, Field
from datetime import datetime
from enum import Enum


class ProgrammingLanguage(str, Enum):
JAVA = "JAVA"
PYTHON = "PYTHON"
C = "C"
HASKELL = "HASKELL"
KOTLIN = "KOTLIN"
VHDL = "VHDL"
ASSEMBLER = "ASSEMBLER"
SWIFT = "SWIFT"
OCAML = "OCAML"
EMPTY = "EMPTY"


class ProgrammingExerciseDTO(BaseModel):
id: int
name: str
programming_language: ProgrammingLanguage = Field(alias="programmingLanguage")
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
template_repository: Dict[str, str] = Field(alias="templateRepository")
solution_repository: Dict[str, str] = Field(alias="solutionRepository")
test_repository: Dict[str, str] = Field(alias="testRepository")
problem_statement: str = Field(alias="problemStatement")
start_date: datetime = Field(alias="startDate")
end_date: datetime = Field(alias="endDate")
12 changes: 12 additions & 0 deletions app/domain/data/result_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
from typing import List

from pydantic import BaseModel, Field
from datetime import datetime

from domain.data.feedback_dto import FeedbackDTO


class ResultDTO(BaseModel):
completion_date: datetime = Field(alias="completionDate")
successful: bool
feedbacks: List[FeedbackDTO]
17 changes: 17 additions & 0 deletions app/domain/data/submission_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
from typing import List, Dict

from pydantic import BaseModel, Field

from datetime import datetime
from domain.data.build_log_entry import BuildLogEntryDTO
from domain.data.result_dto import ResultDTO


class SubmissionDTO(BaseModel):
id: int
date: datetime
repository: Dict[str, str]
is_practice: bool = Field(alias="isPractice")
build_failed: bool = Field(alias="buildFailed")
build_log_entries: List[BuildLogEntryDTO] = Field(alias="buildLogEntries")
latest_result: ResultDTO = Field(alias="latestResult")
5 changes: 5 additions & 0 deletions app/domain/data/text_message_content_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel, Field


class TextMessageContentDTO(BaseModel):
text_content: str = Field(alias="textContent")
7 changes: 7 additions & 0 deletions app/domain/data/user_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
from pydantic import BaseModel, Field


class UserDTO(BaseModel):
id: int
first_name: str = Field(alias="firstName")
last_name: str = Field(alias="lastName")
60 changes: 0 additions & 60 deletions app/domain/dtos.py

This file was deleted.

5 changes: 5 additions & 0 deletions app/domain/error_response_dto.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel, Field


class IrisErrorResponseDTO(BaseModel):
error_message: str = Field(alias="errorMessage")
9 changes: 0 additions & 9 deletions app/domain/exercise.py

This file was deleted.

13 changes: 13 additions & 0 deletions app/domain/health_status_dto.py
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from pydantic import BaseModel
from enum import Enum


class ModelStatus(str, Enum):
UP = "UP"
DOWN = "DOWN"
NOT_AVAILABLE = "NOT_AVAILABLE"


class HealthStatusDTO(BaseModel):
model: str
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
status: ModelStatus
20 changes: 20 additions & 0 deletions app/domain/iris_message.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
from enum import Enum
from typing import Literal

from pydantic import BaseModel, Field


class IrisMessageRole(str, Enum):
kaancayli marked this conversation as resolved.
Show resolved Hide resolved
USER = "USER"
ASSISTANT = "ASSISTANT"
SYSTEM = "SYSTEM"


class IrisMessage(BaseModel):
text: str
role: Literal[
IrisMessageRole.USER, IrisMessageRole.ASSISTANT, IrisMessageRole.SYSTEM
]

def __str__(self):
return f"{self.role.lower()}: {self.text}"
20 changes: 0 additions & 20 deletions app/domain/message.py

This file was deleted.

Loading
Loading