Changing retry arguments

[lunden23]

Hi there,

Thank you for this wonderful package. I am wondering if it is possible to change the arguments of the decorated method for each retry? The reason why I need to do this is best explained by my actual use case. I have class that encapsulates HTTPX requests. I have a method that looks like this:

    @stamina.retry(on=httpx.HTTPError, attempts=3, wait_initial=1, wait_max=16, wait_exp_base=2)
    async def fetch(self, url: str, **kwargs) -> httpx.Response:
        self.url = url
        self.kwargs = kwargs

        try:
            for attempt in stamina.retry_context(on=httpx.HTTPError, attempts=4, wait_initial=1, wait_exp_base=2):
                with attempt:
                    response = await self.client.get(self.url, **self.kwargs)
                    response.raise_for_status()

        except httpx.HTTPStatusError:
            if response.status_code == 401:
                await self._handle_401()
                # more handling of status codes
            else:
                await self._handle_unknown_error()
            raise

        except httpx.HTTPError:
            # some error handling
            raise

        return response
    
    async def _handle_401(self):
        raise NotImplementedError("This method should be overridden in a subclass.")

The HTTP status error handlers in the child classes sometimes fetch new API keys and similar and update the self.kwargs with the new API key by changing the HTTPX paremeter argument value. I've read through the documentation but have been unable to figure out if stamina exposes the possibility of dynamically changing the retry argument at runtime. In this case I would use it to change the kwargs argument to reflect the new API key.

I know I can work around this in some ways, perhaps by refactoring the method itself from scratch. Another way is to perhaps use a new instance property called self.new_kwargs and then checking for them on each retry thereby changing the argument passed to the method indirectly via the instance property. I guess I could also make a new decorator for this particular method, although I am too much of a beginner to envision how it would look. I am therefore wondering if I could change the callback arguments for the retry decorator itself?

And if I haven't been clear enough, this is not a feature request. I am trying to solve the problem in as an efficient way as possible. Any suggestions to that effect are also very much welcome.

[hynek]

I'm a bit lost at your class's design. You have an instance of a class (let's call it Fetcher so we have a name), but Fetcher is somehow only single-use, because you set its self.url and self.kwargs based on a method argument?

What I think is happening is that you're coupling the state of a request with to the whole httpx pool self.client?

Also is the double-retry (via decorator plus via retry_context) on purpose or a remnant of experimentation?

From squinting at this, the best way forward is, I think, that you stop the self.X = X and make _handle_401 return the new kwargs: kwargs = await self._handle_401(response, kwargs). This way your instance doesn't explode when more than one places call it.

(and no, stamina doesn't support changing arguments 😅)