HTTPS 400 Error Bad Request when trying to connect Azure ML

[mmentese21]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

llm = AzureMLOnlineEndpoint(
    endpoint_url="https://Meta-Llama-3-1-8B-Instruct-zovhh.swedencentral.models.ai.azure.com/v1/chat/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key=os.getenv("LLAMA3_API_KEY"),
    content_formatter=CustomOpenAIContentFormatter(),
    model_kwargs={"temperature": 0.8, "max_new_tokens": 400},
)
response = llm.invoke("Write me a song about sparkling water:")
print(response)

Description

I am trying to invoke an llm thats deployed on azure ai. It gives a bad request error.

System Info

python 3

[dosubot[bot]]

You are encountering an HTTPS 400 Error (Bad Request) because the endpoint_url might not be correctly formatted. For endpoint_api_type='serverless', the endpoint_url should include /v1/completions or /v1/chat/completions.

Ensure your endpoint_url is complete and correctly formatted. Here is the relevant code snippet that validates the endpoint_url:

@validator("endpoint_url")
def validate_endpoint_url(cls, field_value: Any) -> str:
    """Validate that endpoint url is complete."""
    if field_value.endswith("/"):
        field_value = field_value[:-1]
    if field_value.endswith("inference.ml.azure.com"):
        raise ValueError(
            "`endpoint_url` should contain the full invocation URL including "
            "`/score` for `endpoint_api_type='dedicated'` or `/v1/completions` "
            "or `/v1/chat/completions` for `endpoint_api_type='serverless'`"
        )
    return field_value

Additionally, ensure that your request payload is correctly formatted. An invalid request format can also lead to a 400 Error. For example, if you are using a custom content formatter, make sure the request payload structure matches the expected format by the endpoint.

Here is an example of a test case for an invalid request format:

def test_invalid_request_format() -> None:
    """Test invalid request format."""

    class CustomContentFormatter(ContentFormatterBase):
        content_type = "application/json"
        accepts = "application/json"

        def format_request_payload(self, prompt: str, model_kwargs: Dict) -> bytes:  # type: ignore[override]
            input_str = json.dumps(
                {
                    "incorrect_input": {"input_string": [prompt]},
                    "parameters": model_kwargs,
                }
            )
            return str.encode(input_str)

        def format_response_payload(self, output: bytes) -> str:  # type: ignore[override]
            response_json = json.loads(output)
            return response_json[0]["0"]

    with pytest.raises(HTTPError):
        llm = AzureMLOnlineEndpoint(
            endpoint_api_key=os.getenv("OSS_ENDPOINT_API_KEY"),  # type: ignore[arg-type]
            endpoint_url=os.getenv("OSS_ENDPOINT_URL"),  # type: ignore[arg-type]
            deployment_name=os.getenv("OSS_DEPLOYMENT_NAME"),  # type: ignore[arg-type]
            content_formatter=CustomContentFormatter(),
        )
        llm.invoke("Foo")

Make sure your endpoint_url and request payload follow the correct format to avoid the 400 Error ^[1][2].

To continue talking to Dosu, mention @dosu.