Instructor.echo_response/1 no function clause matching

[kevinschweikert]

We are using Version 0.0.5 in our application. We recently got this error:

** (FunctionClauseError) no function clause matching in Instructor.echo_response/1
(instructor 0.0.5) lib/instructor.ex:510: Instructor.echo_response(%{"choices" => [%{"finish_reason" => "stop", "index" => 0, "message" => %{"content" => "[OUR_CONTENT]", "role" => "assistant"}}], "created" => 1709884788, "id" => "chatcmpl-90PQq8TYhzCvxtCz2w2xneKudwU0O", "model" => "gpt-4-1106-vision-preview", "object" => "chat.completion", "usage" => %{"completion_tokens" => 76, "prompt_tokens" => 1408, "total_tokens" => 1484}})
(instructor 0.0.5) lib/instructor.ex:456: anonymous fn/3 in Instructor.do_chat_completion/3

I couldn't find time to look at the code yet. Maybe I could provide a fix for this later.

[thmsmlr]

Does your code also use validations and max_retries? Could be a weird interaction between the vision api response and the instructor validation / reasking feature

[kevinschweikert]

Yes, this is the code we're using:

 Instructor.chat_completion(
  model: "gpt-4-vision-preview",
  response_model: __MODULE__,
  max_tokens: 800,
  max_retries: 3,
  mode: :md_json,
  messages: [
    %{
      role: "user",
      content: [
        %{type: "text", text: data.image_to_text_prompt},
        %{type: "image_url", image_url: %{url: data.original_image, detail: "high"}}
      ]
    },
    %{
      role: "system",
      content: "Straightforward and direct answers."
    }
  ]
)

And we're doing validation with Ecto.Changesets

[dankentfield]

I had the same issue here too

** (FunctionClauseError) no function clause matching in Instructor.echo_response/1    
    
    The following arguments were given to Instructor.echo_response/1:
    
        # 1
        %{
          "choices" => [
            %{
              "finish_reason" => "stop",
              "index" => 0,
              "logprobs" => nil,
              "message" => %{
                "content" => "blah blah blah",
                "role" => "assistant"
              }
            }
          ],
          "created" => 1715288065,
          "id" => "chatcmpl-9N54Tf0WXPiYPCkrWq5PbiKLvGAa2",
          "model" => "gpt-4-turbo-2024-04-09",
          "object" => "chat.completion",
          "system_fingerprint" => "fp_0737e0dfd9",
          "usage" => %{
            "completion_tokens" => 419,
            "prompt_tokens" => 1420,
            "total_tokens" => 1839
          }
        }
    
    Attempted function clauses (showing 1 out of 1):
    
        defp echo_response(%{
      "choices" => [
        %{
          "message" =>
            %{
              "tool_calls" => [
                %{"id" => tool_call_id, "function" => %{"name" => name, "arguments" => args}} =
                  function
              ]
            } = message
        }
      ]
    })
    
    (instructor 0.0.5) lib/instructor.ex:510: Instructor.echo_response/1
    (instructor 0.0.5) lib/instructor.ex:456: anonymous fn/3 in Instructor.do_chat_completion/3
    (elixir 1.16.2) lib/keyword.ex:1149: Keyword.update_guarded/4
    (elixir 1.16.2) lib/keyword.ex:1153: Keyword.update_guarded/4
    (instructor 0.0.5) lib/instructor.ex:454: Instructor.do_chat_completion/3
    iex:14: (file)
iex(14)> 

Looks like tool_calls is for function calling, so not entirely sure why it would be expected in a vision response. Not an expert here so take it with a pinch of salt.

[srcrip]

I am getting this issue as well. I'm not using vision though. Just sending regular text. I seem to be getting this with both the chat gpt and ollama adapters. And I'm using the json mode, while this seems related to tool calling so I don't really get what's happening

[harmonicss]

We have hit this problem too, and appreciate the fix. Is there any chance this can be merged back to main?