diff --git a/docs/deployment/agents/index.md b/docs/deployment/agents/index.md index c79a5ecd53..912ab8613c 100644 --- a/docs/deployment/agents/index.md +++ b/docs/deployment/agents/index.md @@ -29,6 +29,8 @@ If you are using a managed deployment of Flyte, you will need to contact your de - Configuring your Flyte deployment for the sensor agent. * - {ref}`SageMaker Inference ` - Deploy models and create, as well as trigger inference endpoints on SageMaker. +* - {ref}`OpenAI Batch ` + - Submit requests to OpenAI GPT models for asynchronous batch processing. ``` ```{toctree} @@ -44,4 +46,5 @@ mmcloud sagemaker_inference sensor snowflake +openai_batch ``` diff --git a/docs/deployment/agents/openai_batch.rst b/docs/deployment/agents/openai_batch.rst new file mode 100644 index 0000000000..7aff9d262e --- /dev/null +++ b/docs/deployment/agents/openai_batch.rst @@ -0,0 +1,134 @@ +.. _deployment-agent-setup-openai-batch: + +OpenAI Batch Agent +================== + +This guide provides an overview of how to set up the OpenAI Batch agent in your Flyte deployment. + +Specify agent configuration +--------------------------- + +.. tabs:: + + .. group-tab:: Flyte binary + + Edit the relevant YAML file to specify the agent. + + .. code-block:: bash + + kubectl edit configmap flyte-sandbox-config -n flyte + + .. code-block:: yaml + :emphasize-lines: 7,11,15 + + tasks: + task-plugins: + enabled-plugins: + - container + - sidecar + - k8s-array + - agent-service + default-for-task-types: + - container: container + - container_array: k8s-array + - openai-batch: agent-service + plugins: + agent-service: + supportedTaskTypes: + - openai-batch + + .. group-tab:: Flyte core + + Create a file named ``values-override.yaml`` and add the following configuration to it: + + .. code-block:: yaml + :emphasize-lines: 9,14,18 + + configmap: + enabled_plugins: + tasks: + task-plugins: + enabled-plugins: + - container + - sidecar + - k8s-array + - agent-service + default-for-task-types: + container: container + sidecar: sidecar + container_array: k8s-array + openai-batch: agent-service + plugins: + agent-service: + supportedTaskTypes: + - openai-batch + +Add the OpenAI API token +------------------------ + +1. Install flyteagent pod using helm: + +.. code-block:: + + helm repo add flyteorg https://flyteorg.github.io/flyte + helm install flyteagent flyteorg/flyteagent --namespace flyte + +2. Get the base64 value of your OpenAI API token: + +.. code-block:: + + echo -n "" | base64 + +3. Edit the flyteagent secret: + + .. code-block:: bash + + kubectl edit secret flyteagent -n flyte + + .. code-block:: yaml + :emphasize-lines: 3 + + apiVersion: v1 + data: + FLYTE_OPENAI_API_KEY: + kind: Secret + metadata: + annotations: + meta.helm.sh/release-name: flyteagent + meta.helm.sh/release-namespace: flyte + creationTimestamp: "2023-10-04T04:09:03Z" + labels: + app.kubernetes.io/managed-by: Helm + name: flyteagent + namespace: flyte + resourceVersion: "753" + uid: 5ac1e1b6-2a4c-4e26-9001-d4ba72c39e54 + type: Opaque + + +Upgrade the Flyte Helm release +------------------------------ + +.. tabs:: + + .. group-tab:: Flyte binary + + .. code-block:: bash + + helm upgrade flyteorg/flyte-binary -n --values + + Replace ```` with the name of your release (e.g., ``flyte-backend``), + ```` with the name of your namespace (e.g., ``flyte``), + and ```` with the name of your YAML file. + + .. group-tab:: Flyte core + + .. code-block:: bash + + helm upgrade flyte/flyte-core -n --values values-override.yaml + + Replace ```` with the name of your release (e.g., ``flyte``) + and ```` with the name of your namespace (e.g., ``flyte``). + +You can refer to the `documentation `__ +to run the agent on your Flyte cluster.