Exponential backoff when customers hit rate limits

[ferristocrat]

Describe the feature or problem you'd like to solve

Gateway-mt should use exponentially increasing backoff times on 429 responses. Using 120 as a default kills performance for those that hit it where as otherwise they would have only needed to slow down a little.

Propose a Solution

Implement exponential backoff so that clients can slow their rate of requests, but not completely stop, so they can maintain good performance while staying within rate limits.

Additional context

[NiaStorj]

1/7/25:

From sprint planning we figured out that it would be best to validate the status code we are returning for rate limiting responses. Change the return status code to 503 to be compatible with existing SDK's

[storj-gerrit]

Change miniogw: change 429s to 503s mentions this issue.

[storj-gerrit]

Change pkg/minio/api-router.go: change 429s to 503s mentions this issue.