Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement structured output by using outlines #745

Open
wants to merge 85 commits into
base: master
Choose a base branch
from

Conversation

Appointat
Copy link
Member

This PR introduces structured output capabilities for both open-source and closed-source language models (LLMs) in our system:

  1. Open-source LLMs:

    • Implemented a schema model to integrate open-source, self-hostable LLMs supported by Outlines
    • This enables structured output for open-source models
    • Currently supports platforms: Transformers, LlamaCpp, and vLLM
  2. Closed-source LLMs with function calling:

    • Utilized Pydantic to pass expected output structures
    • This approach enables structured output for closed-source models that support function calling

Key changes:

  • Added SchemaModel class for open-source LLM integration
  • Implemented platform-specific adapters for Transformers, LlamaCpp, and vLLM
  • Created PydanticOutputParser for closed-source LLMs with function calling support

Testing:

  • Added unit tests for SchemaModel and PydanticOutputParser

Next steps:

  • Expand support for additional open-source LLM platforms
  • For the closed llms, we will optimize the issue of multi-functions calling at the same time.
  • Better ci/cd for the self-hosted models

Copy link
Collaborator

@WHALEEYE WHALEEYE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Appointat! The core codebase LGTM. There are just some changes needed on the dependencies management though.
Besides, there are some errors raised when checking with mypy on my side, maybe need some double checking on that,

camel/models/schema_model.py Outdated Show resolved Hide resolved
camel/models/schema_model.py Show resolved Hide resolved
camel/models/schema_model.py Show resolved Hide resolved
camel/models/schema_model.py Show resolved Hide resolved
@Wendong-Fan Wendong-Fan modified the milestones: Sprint 7, Sprint 17 Nov 17, 2024
@Wendong-Fan Wendong-Fan added call for contribution P0 Task with high level priority labels Nov 18, 2024
def run(
self,
messages: List[OpenAIMessage],
output_schema: Optional[Type[T]] = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this is T not BaseModel, because our schema of output parse is baseModel type

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request P0 Task with high level priority
Projects
Status: No status
5 participants