You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some Application frameworks can't handle special characters very well and may not follow the chunk size strictly. In some cases, the size of embedding or reranking may exceed the setting and cause error based on model configuration. As it's a bug at the application level and may or may not be fixed, we can't guarantee that using a model with a bigger context size and setting a bigger ubatch size can 100% resolve the problem. Some users are willing to bear some accuracy loss and embed the document without error. We can add a parameter to handle this as both ollama and some other platforms support this. It's a setting not recommended, but users can turn it on at their own risk.
Some Application frameworks can't handle special characters very well and may not follow the chunk size strictly. In some cases, the size of embedding or reranking may exceed the setting and cause error based on model configuration. As it's a bug at the application level and may or may not be fixed, we can't guarantee that using a model with a bigger context size and setting a bigger ubatch size can 100% resolve the problem. Some users are willing to bear some accuracy loss and embed the document without error. We can add a parameter to handle this as both ollama and some other platforms support this. It's a setting not recommended, but users can turn it on at their own risk.
Refer to gpustack/gpustack#950
The text was updated successfully, but these errors were encountered: