fix: forcing response format to be JSON valid (#7692)

* forcing response format to be JSON valid * adding release notes * cleaning up * Update haystack/components/evaluators/llm_evaluator.py Co-authored-by: Madeesh Kannan <[email protected]> --------- Co-authored-by: Madeesh Kannan <[email protected]>
deepset-ai · May 14, 2024 · 75cf35c · 75cf35c
1 parent a2be90b
commit 75cf35c
Show file tree

Hide file tree

Showing 2 changed files with 9 additions and 1 deletion.
diff --git a/haystack/components/evaluators/llm_evaluator.py b/haystack/components/evaluators/llm_evaluator.py
@@ -87,7 +87,9 @@ def __init__(
         self.api_key = api_key
 
         if api == "openai":
-            self.generator = OpenAIGenerator(api_key=api_key)
+            self.generator = OpenAIGenerator(
+                api_key=api_key, generation_kwargs={"response_format": {"type": "json_object"}}
+            )
         else:
             raise ValueError(f"Unsupported API: {api}")
 

diff --git a/releasenotes/notes/force-valid-JSON-OpeanAI-LLM-based-evaluators-64816e68f137739b.yaml b/releasenotes/notes/force-valid-JSON-OpeanAI-LLM-based-evaluators-64816e68f137739b.yaml
@@ -0,0 +1,6 @@
+---
+
+enhancements:
+  - |
+    Enforce JSON mode on OpenAI LLM-based evaluators so that the they always return valid JSON output.
+    This is to ensure that the output is always in a consistent format, regardless of the input.