You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
We've found that in some situations we'd prefer the model provider (e.g. Bedrock) to return with an error message if the prompt is too long rather than truncate the end of the prompt which can lead to non-sensical answers depending on how the prompt template is constructed. So I propose adding a parameter to toggle this behavior with the default behavior being unchanged (i.e. truncation=True is the same as the previous default behavior).
Describe the solution you'd like
Similar solution as was implemented in this PR for Haystack v1.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
We've found that in some situations we'd prefer the model provider (e.g. Bedrock) to return with an error message if the prompt is too long rather than truncate the end of the prompt which can lead to non-sensical answers depending on how the prompt template is constructed. So I propose adding a parameter to toggle this behavior with the default behavior being unchanged (i.e. truncation=True is the same as the previous default behavior).
Describe the solution you'd like
Similar solution as was implemented in this PR for Haystack v1.
The text was updated successfully, but these errors were encountered: