Skip to content

Commit

Permalink
rearrange _wait inbetween api calls_ section
Browse files Browse the repository at this point in the history
  • Loading branch information
Sheepsta300 committed Sep 11, 2024
1 parent a10e583 commit 0bab309
Showing 1 changed file with 20 additions and 31 deletions.
51 changes: 20 additions & 31 deletions getting_started.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,26 @@
"- Other parameters may be available in different SDK's."
]
},
{
"cell_type": "markdown",
"id": "b9edabc3",
"metadata": {},
"source": [
"### Wait in between API calls"
]
},
{
"cell_type": "markdown",
"id": "239cdb0b",
"metadata": {},
"source": [
"The number of API requests a user can make depends on their Azure plan and account settings. If too many requests are sent in a short period, an error may occur, prompting the user to wait for **x** amount of time before sending another request.\n",
"\n",
"When creating a model, one of the key parameters is the `max_retries` setting. The underlying Python OpenAI library will automatically wait and retry the call on your behalf at least 2 times by default before raising a `RateLimitError`. This behavior can be adjusted by setting a different value for `max_retries`.\n",
"\n",
"Visit the [**quotas and limits**](https://learn.microsoft.com/azure/ai-services/openai/quotas-limits) page to view detailed information related to account limits and restrictions."
]
},
{
"cell_type": "markdown",
"id": "28a397b3-27b7-49f0-bccf-108f39196ee6",
Expand Down Expand Up @@ -928,37 +948,6 @@
" print(chunk.content, end=\"\", flush=True)"
]
},
{
"cell_type": "markdown",
"id": "3f186179-13e7-4056-9b62-9b4b1ac8de60",
"metadata": {},
"source": [
"### Wait in between API calls"
]
},
{
"cell_type": "markdown",
"id": "9f747f98-7b78-436f-a889-e3cff3f19bf9",
"metadata": {},
"source": [
"Depending on the users Azure plan and account will determine how many API requests can be made within a given interval. If too many requests are sent frequently an error may occur where the user will be told to wait **x** amount of time before sending another request.\n",
"\n",
"When creating a model, one of the parameters briefly mentioned earlier was the `max_retries` parameter. When calling the underlying Python OpenAI library, the library will wait and retry the call on your behalf at least 2 times (the default) before raising a RateLimiteError, or whatever `max retries` is set as.\n",
"\n",
"Otherwise, to ensure the user is not overwhelming the Azure services, make sure to wait between API calls when possible - this can be enforced by using the `time` package."
]
},
{
"cell_type": "code",
"execution_count": 65,
"id": "99e45cb7-6e35-4420-bd41-651157e5596f",
"metadata": {},
"outputs": [],
"source": [
"import time\n",
"time.sleep(5)"
]
},
{
"cell_type": "markdown",
"id": "c4a85fe6-ee18-4fc8-88c2-b67297ac8ef8",
Expand Down

0 comments on commit 0bab309

Please sign in to comment.