You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, dstack's OpenAI-compatible endpoint performs strict validation of all requests and responses against a hardcoded schema. However, some model launchers don't adhere to the official OpenAI schema, and the official schema can also evolve rapidly. As a result, some models or launchers can't be used with dstack because validation fails for them. Examples: #1777, #2228.
Solution
Do not perform strict validation for format: openai models. Only extract the fields that dstack needs, such as the model field, and pass the rest of the fields in requests and responses as is.
Note: strict validation may still be necessary for format: tgi models in order to perform format conversion.
The text was updated successfully, but these errors were encountered:
Problem
Currently,
dstack
's OpenAI-compatible endpoint performs strict validation of all requests and responses against a hardcoded schema. However, some model launchers don't adhere to the official OpenAI schema, and the official schema can also evolve rapidly. As a result, some models or launchers can't be used withdstack
because validation fails for them. Examples: #1777, #2228.Solution
Do not perform strict validation for
format: openai
models. Only extract the fields thatdstack
needs, such as themodel
field, and pass the rest of the fields in requests and responses as is.Note: strict validation may still be necessary for
format: tgi
models in order to perform format conversion.The text was updated successfully, but these errors were encountered: