swh.core.retry module#

swh.core.retry.is_throttling_exception(e: Exception) → bool[source]#: Checks if an exception is a requests.exception.HTTPError for a response with status code 429 (too many requests).

swh.core.retry.is_retryable_exception(e: Exception) → bool[source]#: Checks if an exception is worth retrying (connection, throttling or a server error).

swh.core.retry.retry_if_exception(retry_state, predicate: Callable[[Exception], bool]) → bool[source]#: Custom tenacity retry predicate for handling exceptions with the given predicate.

swh.core.retry.retry_policy_generic(retry_state) → bool[source]#

Custom tenacity retry predicate for handling failed requests:

ConnectionError
Server errors (status >= 500)
Throttling errors (status == 429)

This does not handle 403, 404 or other status codes.

swh.core.retry.http_retry(retry=<function retry_policy_generic>, wait=<tenacity.wait.wait_exponential object>, stop=<tenacity.stop.stop_after_attempt object>, **retry_args)[source]#

Decorator based on tenacity for retrying a function possibly raising requests.exception.HTTPError for status code 429 (too many requests) or >= 500.

It provides a default configuration that should work properly in most cases but all tenacity.retry() parameters can also be overridden in client code.

When the maximum of attempts is reached, the HTTPError exception will then be reraised.

Parameters:

retry – function defining request retry condition (default to 429 status code) https://tenacity.readthedocs.io/en/latest/#whether-to-retry
wait – function defining wait strategy before retrying (default to exponential backoff) https://tenacity.readthedocs.io/en/latest/#waiting-before-retrying
stop – function defining when to stop retrying (default after 5 attempts) https://tenacity.readthedocs.io/en/latest/#stopping