Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring suggestions #59

Merged
merged 8 commits into from
Feb 16, 2024
Merged

Refactoring suggestions #59

merged 8 commits into from
Feb 16, 2024

Conversation

MichaelOwenDyer
Copy link
Contributor

I have some suggestions for the LLM subsystem code. @Hialus, not trying to step on your toes, every change in this PR is a personal opinion. Could be that I am missing the bigger picture with some of these things, let me know your thoughts. Here's a summary of what I did:

  • Renamed classes to reflect their conceptual purpose rather than implementation-oriented. In my opinion, conceptual class names make it easier to reason about the code using those classes.
  • Took out some code in basic_request_handler that I thought was redundant.
  • In LLMManager, I replaced the linear LLM search with a dictionary lookup by ID. Even though O(n) would not be a performance bottleneck with the low amount of LLMs we have, let's do it in O(1).
  • Renamed the methods completion(), chat_completion(), create_embedding() to complete(), chat(), and embed() for consistency
  • Added one requirement for PyYAML since it is being used in LLMManager, and changed the requirements in requirements.txt to use ~= instead of ==.

# Conflicts:
#	app/llm/basic_request_handler.py
#	app/llm/request_handler_interface.py
#	requirements.txt
Copy link
Member

@Hialus Hialus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it stands right now you did a lot of changes in the internals of the LLM subsystem. I very clearly stated that the entire internals of the subsystem are subject to change and further improvements. The current implementation is merely a simple first version to use in the development of the pipeline subsystem.

The changes to the RequestHandler are sensible and I'd suggest we implement them, as we already discussed that.
The changes to the LlmManager are entirely unnecessary, as the current implementation will effectively be completely replaced. As it is not part of the public API of the subsystem this is easily doable. So any changes here do not really make much sense.
For the changes about the AbstractLlmWrapper I'd need to spent some additional time thinking about the best possible name, but as they are internal classes this is not really high priority and can be done at any time.

Also your code fails the linter checks. I'd suggest to install the pre commit hooks.

app/llm/wrapper/openai_chat.py Outdated Show resolved Hide resolved
app/llm/wrapper/openai_chat.py Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
@Hialus Hialus merged commit 5fc091b into main Feb 16, 2024
4 checks passed
@Hialus Hialus deleted the refactor/llm-code branch February 16, 2024 17:05
@Hialus Hialus added this to the 1.0.0-Prototype milestone Feb 21, 2024
isabellagessl pushed a commit that referenced this pull request Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants