You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
User should be able to add to their pyproject.toml a dict mapping LLM types to factory functions that create LLM of that type:
[tool.ragbits.core.default_llm_factories]
text = "my_project.llm.small_local"
vision = "my_project.llm.gpt4o"
structured_output = "my_project.llm.gpt4o"
Ragbits should define an enum with all possible LLM types. The has_default_llm and get_default_llm should be updated so that they take the value of this enum as an argument (default: "text") and returns LLM of the correct type.
Using an LLM type in pyproject.toml that is not present in the enum should result in an error
UnstructuredImageProvider should be modified to use configured default LLM of type vision, instead of having a hard-coded default LLM.
The text was updated successfully, but these errors were encountered:
PR #121 introduced a default LLM object for UnstructuredImageProvider / UnstructuredPdfProvider which is used whenever the user does not provide an LLM object explicitly.
This is very similar to what we wanted to achieve with user-configurable default_llm_factory but the configuration option is not used here - instead the default is not user configurable and is hard-coded to be a LiteLLM client, using gpt-4o-mini and assumes that the API key is present in an environment variable.
The problem with using default_llm_factory here is that default_llm_factory is not guaranteed to be vision-enabled, and UnstructuredImageProvider / UnstructuredPdfProvider require a vision-enabled model.
When we first talked about default_llm_factory there was an idea of being able to configure a map with factories for different types of LLMs. For example:
[tool.ragbits.core.default_llm_factories]
text = "my_project.llm.small_local"
vision = "my_project.llm.gpt4o"
structured_output = "my_project.llm.gpt4o"
ludwiktrammer
changed the title
feat(core): add default visual llm
feat(core): user should be able to configure default types of default LLMs
Oct 23, 2024
User should be able to add to their
pyproject.toml
a dict mapping LLM types to factory functions that create LLM of that type:Ragbits should define an enum with all possible LLM types. The
has_default_llm
andget_default_llm
should be updated so that they take the value of this enum as an argument (default: "text") and returns LLM of the correct type.Using an LLM type in
pyproject.toml
that is not present in the enum should result in an errorUnstructuredImageProvider
should be modified to use configured default LLM of type vision, instead of having a hard-coded default LLM.The text was updated successfully, but these errors were encountered: