Add ChatGPT wrapper pipeline

README.MD

@@ -5,16 +5,24 @@ Pyris is an intermediary system that connects the [Artemis](https://github.com/l
 Currently, Pyris powers [Iris](https://artemis.cit.tum.de/about-iris), a virtual AI tutor that assists students with their programming exercises on Artemis in a pedagogically meaningful way.
 
 ## Table of Contents
-- [Features](#features)
-- [Setup](#setup)
-  - [Prerequisites](#prerequisites)
-  - [Local Development Setup](#local-development-setup)
-  - [Docker Setup](#docker-setup)
-    - [Development Environment](#development-environment)
-    - [Production Environment](#production-environment)
-- [Customizing Configuration](#customizing-configuration)
-- [Troubleshooting](#troubleshooting)
-- [Additional Notes](#additional-notes)
+- [Pyris V2](#pyris-v2)
+  - [Table of Contents](#table-of-contents)
+  - [Features](#features)
+  - [Setup](#setup)
+    - [Prerequisites](#prerequisites)
+    - [Local Development Setup](#local-development-setup)
+      - [Steps](#steps)
+    - [Docker Setup](#docker-setup)
+      - [Prerequisites](#prerequisites-1)
+      - [Docker Compose Files](#docker-compose-files)
+      - [Running the Containers](#running-the-containers)
+        - [**Development Environment**](#development-environment)
+        - [**Production Environment**](#production-environment)
+          - [**Option 1: With Nginx**](#option-1-with-nginx)
+          - [**Option 2: Without Nginx**](#option-2-without-nginx)
+      - [Managing the Containers](#managing-the-containers)
+      - [Customizing Configuration](#customizing-configuration)
+  - [Troubleshooting](#troubleshooting)
 
 ## Features
 
@@ -87,10 +95,10 @@ Currently, Pyris powers [Iris](https://artemis.cit.tum.de/about-iris), a virtual
 
    - **Create an LLM Config File**
 
-     Create an `llm-config.local.yml` file in the root directory. You can use the provided `llm-config.example.yml` as a base.
+     Create an `llm_config.local.yml` file in the root directory. You can use the provided `llm_config.example.yml` as a base.
 
      ```bash
-     cp llm-config.example.yml llm-config.local.yml
+     cp llm_config.example.yml llm_config.local.yml
      ```
 
      **Example OpenAI Configuration:**
@@ -176,15 +184,15 @@ Currently, Pyris powers [Iris](https://artemis.cit.tum.de/about-iris), a virtual
      >- GPT-4 equivalent: 4
      >- GPT-3.5 Turbo equivalent: 3.5
 
-     > **Warning:** Most existing pipelines in Pyris require a model with a `gpt_version_equivalent` of **4.5 or higher**. It is advised to define models in the `llm-config.local.yml` file with a `gpt_version_equivalent` of 4.5 or higher.
+     > **Warning:** Most existing pipelines in Pyris require a model with a `gpt_version_equivalent` of **4.5 or higher**. It is advised to define models in the `llm_config.local.yml` file with a `gpt_version_equivalent` of 4.5 or higher.
 
 4. **Run the Server**
 
    Start the Pyris server:
 
    ```bash
    APPLICATION_YML_PATH=./application.local.yml \
-   LLM_CONFIG_PATH=./llm-config.local.yml \
+   LLM_CONFIG_PATH=./llm_config.local.yml \
    uvicorn app.main:app --reload
    ```
 
@@ -203,7 +211,7 @@ Deploying Pyris using Docker ensures a consistent environment and simplifies the
 - **Docker**: Install Docker from the [official website](https://www.docker.com/get-started).
 - **Docker Compose**: Comes bundled with Docker Desktop or install separately on Linux.
 - **Clone the Pyris Repository**: If not already done, clone the repository.
-- **Create Configuration Files**: Create the `application.local.yml` and `llm-config.local.yml` files as described in the [Local Development Setup](#local-development-setup) section.
+- **Create Configuration Files**: Create the `application.local.yml` and `llm_config.local.yml` files as described in the [Local Development Setup](#local-development-setup) section.
 
   ```bash
   git clone https://github.com/ls1intum/Pyris.git Pyris
@@ -312,7 +320,7 @@ Deploying Pyris using Docker ensures a consistent environment and simplifies the
 
   - `PYRIS_DOCKER_TAG`: Specifies the Pyris Docker image tag.
   - `PYRIS_APPLICATION_YML_FILE`: Path to your `application.yml` file.
-  - `PYRIS_LLM_CONFIG_YML_FILE`: Path to your `llm-config.yml` file.
+  - `PYRIS_LLM_CONFIG_YML_FILE`: Path to your `llm_config.yml` file.
   - `PYRIS_PORT`: Host port for Pyris application (default is `8000`).
   - `WEAVIATE_PORT`: Host port for Weaviate REST API (default is `8001`).
   - `WEAVIATE_GRPC_PORT`: Host port for Weaviate gRPC interface (default is `50051`).
@@ -321,7 +329,7 @@ Deploying Pyris using Docker ensures a consistent environment and simplifies the
 
   Modify configuration files as needed:
 
-  - **Pyris Configuration**: Update `application.yml` and `llm-config.yml`.
+  - **Pyris Configuration**: Update `application.yml` and `llm_config.yml`.
   - **Weaviate Configuration**: Adjust settings in `weaviate.yml`.
   - **Nginx Configuration**: Modify Nginx settings in `nginx.yml` and related config files.
 

app/domain/status/text_exercise_chat_status_update_dto.py

@@ -1,5 +1,7 @@
+from typing import Optional
+
 from app.domain.status.status_update_dto import StatusUpdateDTO
 
 
 class TextExerciseChatStatusUpdateDTO(StatusUpdateDTO):
-    result: str
+    result: Optional[str]

app/main.py

@@ -21,6 +21,24 @@
 app = FastAPI(default_response_class=ORJSONResponse)
 
 
+def custom_openapi():
+    if not app.openapi_schema:
+        openapi_schema = FastAPI.openapi(app)
+        # Add security scheme
+        openapi_schema["components"]["securitySchemes"] = {
+            "bearerAuth": {"type": "apiKey", "in": "header", "name": "Authorization"}
+        }
+        # Apply the security globally
+        for path in openapi_schema["paths"].values():
+            for method in path.values():
+                method.setdefault("security", []).append({"bearerAuth": []})
+        app.openapi_schema = openapi_schema
+    return app.openapi_schema
+
+
+app.openapi = custom_openapi
+
+
 @app.exception_handler(RequestValidationError)
 async def validation_exception_handler(request: Request, exc: RequestValidationError):
     exc_str = f"{exc}".replace("\n", " ").replace("   ", " ")

app/pipeline/chat/course_chat_pipeline.py

@@ -100,14 +100,16 @@ def __init__(
                 requirements=RequirementList(
                     gpt_version_equivalent=4.5,
                 )
-            ), completion_args=completion_args
+            ),
+            completion_args=completion_args,
         )
         self.llm_small = IrisLangchainChatModel(
             request_handler=CapabilityRequestHandler(
                 requirements=RequirementList(
                     gpt_version_equivalent=4.25,
                 )
-            ), completion_args=completion_args
+            ),
+            completion_args=completion_args,
         )
         self.callback = callback
 

app/pipeline/chat/exercise_chat_agent_pipeline.py

@@ -533,7 +533,9 @@ def lecture_content_retrieval() -> str:
                     ]
                 )
 
-                guide_response = (self.prompt | self.llm_small | StrOutputParser()).invoke(
+                guide_response = (
+                    self.prompt | self.llm_small | StrOutputParser()
+                ).invoke(
                     {
                         "response": out,
                     }

app/pipeline/chat_gpt_wrapper_pipeline.py

@@ -0,0 +1,102 @@
+import logging
+from typing import List, Optional
+
+from langchain_core.prompts import (
+    ChatPromptTemplate,
+)
+from app.common.pyris_message import IrisMessageRole, PyrisMessage
+from app.domain.chat.exercise_chat.exercise_chat_pipeline_execution_dto import (
+    ExerciseChatPipelineExecutionDTO,
+)
+from langchain_core.output_parsers import StrOutputParser
+from app.llm.langchain.iris_langchain_chat_model import IrisLangchainChatModel
+from app.pipeline.prompts.chat_gpt_wrapper_prompts import chat_gpt_initial_system_prompt
+from langchain_core.messages import SystemMessage, HumanMessage
+from langchain_core.runnables import Runnable
+
+from app.llm import CapabilityRequestHandler, RequirementList, CompletionArguments
+from app.pipeline import Pipeline
+from app.web.status.status_update import ExerciseChatStatusCallback
+from ..common.message_converters import convert_iris_message_to_langchain_human_message
+
+logger = logging.getLogger(__name__)
+
+
+def convert_chat_history_to_str(chat_history: List[PyrisMessage]) -> str:
+    """
+    Converts the chat history to a string
+    :param chat_history: The chat history
+    :return: The chat history as a string
+    """
+
+    def map_message_role(role: IrisMessageRole) -> str:
+        if role == IrisMessageRole.SYSTEM:
+            return "System"
+        elif role == IrisMessageRole.ASSISTANT:
+            return "AI Tutor"
+        elif role == IrisMessageRole.USER:
+            return "Student"
+        else:
+            return "Unknown"
+
+    return "\n\n".join(
+        [
+            f"{map_message_role(message.sender)} {"" if not message.sent_at else f"at {message.sent_at.strftime(
+                "%Y-%m-%d %H:%M:%S")}"}: {message.contents[0].text_content}"
+            for message in chat_history
+        ]
+    )
+
+
+class ChatGPTWrapperPipeline(Pipeline):
+    callback: ExerciseChatStatusCallback
+    llm: IrisLangchainChatModel
+    pipeline: Runnable
+
+    def __init__(self, callback: Optional[ExerciseChatStatusCallback] = None):
+        super().__init__(implementation_id="chat_gpt_wrapper_pipeline_reference_impl")
+        self.callback = callback
+        request_handler = CapabilityRequestHandler(
+            requirements=RequirementList(
+                gpt_version_equivalent=4.5,
+                context_length=16385,
+            )
+        )
+        completion_args = CompletionArguments(temperature=0.5, max_tokens=2000)
+        self.llm = IrisLangchainChatModel(
+            request_handler=request_handler, completion_args=completion_args
+        )
+        self.pipeline = self.llm | StrOutputParser()
+        self.tokens = []
+
+    def __call__(
+        self,
+        dto: ExerciseChatPipelineExecutionDTO,
+        prompt: Optional[ChatPromptTemplate] = None,
+        **kwargs,
+    ):
+        """
+        Run the ChatGPT wrapper pipeline.
+        This consists of a single response generation step.
+        """
+        query = dto.chat_history[-1] if dto.chat_history else None
+        if query and query.sender != IrisMessageRole.USER:
+            query = None
+
+        chat_history = (
+            dto.chat_history[-5:] if query is None else dto.chat_history[-6:-1]
+        )
+
+        chat_history_messages = convert_chat_history_to_str(chat_history)
+
+        prompt = ChatPromptTemplate.from_messages(
+            [
+                SystemMessage(chat_gpt_initial_system_prompt),
+                HumanMessage(chat_history_messages),
+                convert_iris_message_to_langchain_human_message(query),
+            ]
+        )
+
+        response = (prompt | self.pipeline).invoke({})
+        self.callback.done()
+        self.callback.done(final_result=response)

app/pipeline/prompts/chat_gpt_wrapper_prompts.py

@@ -0,0 +1,4 @@
+chat_gpt_initial_system_prompt = """
+You are a helpful, smart, kind, and efficient AI assistant.
+You always fulfill the user's requests to the best of your ability.
+"""

app/pipeline/shared/citation_pipeline.py

@@ -57,7 +57,8 @@ def create_formatted_string(self, paragraphs):
                 paragraph.get(LectureSchema.LECTURE_NAME.value),
                 paragraph.get(LectureSchema.LECTURE_UNIT_NAME.value),
                 paragraph.get(LectureSchema.PAGE_NUMBER.value),
-                paragraph.get(LectureSchema.LECTURE_UNIT_LINK.value) or "No link available",
+                paragraph.get(LectureSchema.LECTURE_UNIT_LINK.value)
+                or "No link available",
                 paragraph.get(LectureSchema.PAGE_TEXT_CONTENT.value),
             )
             formatted_string += lct

app/web/routers/pipelines.py

@@ -31,6 +31,7 @@
 )
 from app.pipeline.text_exercise_chat_pipeline import TextExerciseChatPipeline
 from app.web.status.status_update import TextExerciseChatCallback
+from app.pipeline.chat_gpt_wrapper_pipeline import ChatGPTWrapperPipeline
 
 router = APIRouter(prefix="/api/v1/pipelines", tags=["pipelines"])
 logger = logging.getLogger(__name__)
@@ -74,9 +75,12 @@ def run_exercise_chat_pipeline(
         description="Exercise Chat Pipeline Execution DTO"
     ),
 ):
-    thread = Thread(
-        target=run_exercise_chat_pipeline_worker, args=(dto, variant, event)
-    )
+    if variant == "chat-gpt-wrapper":
+        thread = Thread(target=run_chatgpt_wrapper_pipeline_worker, args=(dto, variant))
+    else:
+        thread = Thread(
+            target=run_exercise_chat_pipeline_worker, args=(dto, variant, event)
+        )
     thread.start()
 
 
@@ -232,6 +236,30 @@ def run_competency_extraction_pipeline(
     thread.start()
 
 
+def run_chatgpt_wrapper_pipeline_worker(
+    dto: ExerciseChatPipelineExecutionDTO, _variant: str
+):
+    try:
+        callback = ExerciseChatStatusCallback(
+            run_id=dto.settings.authentication_token,
+            base_url=dto.settings.artemis_base_url,
+            initial_stages=dto.initial_stages,
+        )
+        pipeline = ChatGPTWrapperPipeline(callback=callback)
+    except Exception as e:
+        logger.error(f"Error preparing ChatGPT wrapper pipeline: {e}")
+        logger.error(traceback.format_exc())
+        capture_exception(e)
+        return
+
+    try:
+        pipeline(dto=dto)
+    except Exception as e:
+        logger.error(f"Error running ChatGPT wrapper pipeline: {e}")
+        logger.error(traceback.format_exc())
+        callback.error("Fatal error.", exception=e)
+
+
 @router.get("/{feature}/variants")
 def get_pipeline(feature: str):
     """
@@ -294,5 +322,13 @@ def get_pipeline(feature: str):
                     description="Default lecture chat variant.",
                 )
             ]
+        case "CHAT_GPT_WRAPPER":
+            return [
+                FeatureDTO(
+                    id="default",
+                    name="Default Variant",
+                    description="Default ChatGPT wrapper variant.",
+                )
+            ]
         case _:
             return Response(status_code=status.HTTP_400_BAD_REQUEST)

app/web/status/status_update.py

@@ -1,5 +1,6 @@
 from typing import Optional, List
 
+
 from sentry_sdk import capture_exception, capture_message
 
 import requests

application.example.yml

@@ -4,4 +4,7 @@ api_keys:
 weaviate:
   host: "localhost"
   port: "8001"
-  grpc_port: "50051"
+  grpc_port: "50051"
+
+env_vars:
+  SOME: 'value'