Separate Model Configuration for Context Window Limitations

[monilpat]

Is your feature request related to a problem? Please describe.

Currently, model configurations are set based on the model provider, which does not account for individual model limitations, particularly context windows. This discrepancy is preventing the merging of PR #1617, which has a context window of 8192, while the model provider's configuration is set to 128000, leading to potential issues.

Describe the solution you'd like

Implement a separation of model configurations to allow settings that are specific to each model rather than being generalized across model providers. This would involve creating a structure where each model can have its own context window and other relevant settings.

Describe alternatives you've considered

Leaving the current configuration as is, but this would lead to inconsistencies and issues with model performance and compatibility.

Additional context

This change is necessary to ensure that models can be utilized effectively without running into configuration conflicts, thereby enhancing the overall functionality and reliability of the system.

Related Issues / PRs

• PR #1617

[tcm390]

Thanks for pointing this out! I’ll look into it and clean up the code.