diff --git a/docs/proxy/azure_openai.mdx b/docs/proxy/azure_openai.mdx new file mode 100644 index 00000000..305a3119 --- /dev/null +++ b/docs/proxy/azure_openai.mdx @@ -0,0 +1,81 @@ +--- +sidebar_label: Use LangSmith Proxy with Azure OpenAI +sidebar_position: 2 +--- + +# Configure LangSmith Proxy to talk to your Azure OpenAI Endpoint + +## Prerequisites + +1. Docker installed on your local machine + - Instructions for installing Docker can be found [here](https://docs.docker.com/get-docker/) +2. An Azure OpenAI API Key +3. An Azure OpenAI endpoint + +## 1. Deploy the LangSmith Proxy + +The LangSmith Proxy is available as a Docker container. You can run it in your environment by running the following command: + +```bash +docker pull docker.io/langchain/langsmith-proxy:latest # Force pull the latest version of the LangSmith Proxy +docker run -e AZURE_OPENAI_ENDPOINT -p 8080:8080 docker.io/langchain/langsmith-proxy:latest # Run the LangSmith Proxy on port 8080 and publish it to the host +``` + +You should see the following output: + +```bash +2024-03-06 12:59:57,458 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. +2024-03-06 12:59:57,467 INFO supervisord started with pid 1 +2024-03-06 12:59:58,503 INFO spawned: 'nginx' with pid 8 +2024-03-06 12:59:58,552 INFO spawned: 'trace-processor' with pid 10 +2024-03-06 12:59:59,562 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) +2024-03-06 12:59:59,563 INFO success: trace-processor entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) +Couldn't create langsmith client: API key must be provided when using hosted LangSmith API, will skip creating runs +Listening for traces at 0.0.0.0:9999 +Connection from ('127.0.0.1', 47370) +``` + +Ignore the `Couldn't create langsmith client` message if you are not configuring tracing. + +## 2. Update your app to make requests to the LangSmith Proxy + +For this example, we'll be using your local proxy running on `localhost:8080`. You can replace this with the address of your proxy if it's running on a different machine. + +You may need to install some packages for this example: + +```bash +pip install langchain_openai +``` + +Now, you can use the LangSmith Proxy to make requests to Azure OpenAI. Here's an example of how you can do this in Python: +Let's create a file called `azure_openai_test.py` and add the following code: + +```python +from langchain_openai import AzureChatOpenAI +import os + +os.environ["OPENAI_API_VERSION"] = "2023-06-01-preview" +os.environ["OPENAI_API_KEY"] = "YOUR API KEY" +os.environ["AZURE_OPENAI_ENDPOINT"] = "http://localhost:8080/proxy/azure-openai" + +llm = AzureChatOpenAI(deployment_name="gpt-35-turbo") +print(llm.invoke("Hello, world!")) +``` + +Run the script: + +```bash +python azure_openai_test.py +``` + +You should see some output like this + +``` +content='Hello! How can I assist you today?' +response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 11, 'total_tokens': 20}, 'model_name': 'gpt-35-turbo', 'system_fingerprint': 'fp_2f57f81c11', +'prompt_filter_results': [{'prompt_index': 0, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, 'self_harm': {'filtered': False, 'severity': 'safe'}, +'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}], 'finish_reason': 'stop', 'logprobs': None, 'content_filter_results': {'hate': {'filtered': False, 'severity': 'safe'}, +'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}} id='run-0884458f-bccd-4444-8357-3f8becc7ea2c-0' +``` + +Nice! You have successfully configured the LangSmith Proxy to talk to your Azure OpenAI endpoint. You can now use your proxy endpoint as a drop-in replacement for the Azure OpenAI endpoint in your applications. diff --git a/docs/proxy/quickstart.mdx b/docs/proxy/quickstart.mdx index fb451f77..25eeae00 100644 --- a/docs/proxy/quickstart.mdx +++ b/docs/proxy/quickstart.mdx @@ -7,7 +7,7 @@ sidebar_position: 1 ## What is the LangSmith Proxy -The LangSmith Proxy is intended to be a drop-in replacement for your LLM apis with some additional features. It is designed to be a simple, easy-to-use, and easy-to-configure tool that adds minimal overhead to your existing LLM API usage. +The LangSmith Proxy(Alpha) is intended to be a drop-in replacement for your LLM apis with some additional features. It is designed to be a simple, easy-to-use, and easy-to-configure tool that adds minimal overhead to your existing LLM API usage. - **Easy to use**: The LangSmith LLM Proxy is designed to be easy to use. You can run it as a sidecar to your existing app and start using it immediately. Besides changing the URL of your LLM API, you don't need to make any changes to your app to start using the LangSmith Proxy. - **Cache support**: The LangSmith LLM Proxy supports caching of requests/responses(with streaming support) from the LLM API. This allows you to cache responses for a configurable amount of time, reducing the number of requests to the LLM API and improving response times. This can be especially useful if you are using the LLM API in a high-traffic environment or running CI(in cases like evals)