swh.objstorage.backends.azure module#

swh.objstorage.backends.azure.get_container_url(account_name: str, account_key: str, container_name: str, access_policy: str = 'read_only', expiry: timedelta = datetime.timedelta(days=365), container_url_template: str = 'https://{account_name}.blob.core.windows.net/{container_name}?{signature}', **kwargs) → str[source]#

Get the full url, for the given container on the given account, with a Shared Access Signature granting the specified access policy.

Parameters:

  • account_name – name of the storage account for which to generate the URL

  • account_key – shared account key of the storage account used to generate the SAS

  • container_name – name of the container for which to grant access in the storage account

  • access_policy – one of read_only, append_only, full

  • expiry – the interval in the future with which the signature will expire

Returns:

the full URL of the container, with the shared access signature.

class swh.objstorage.backends.azure.AzureCloudObjStorage(*, container_url: str | None = None, account_name: str | None = None, api_secret_key: str | None = None, container_name: str | None = None, connection_string: str | None = None, compression: Literal['bz2', 'lzma', 'gzip', 'zlib', 'none'] | None = None, connection_limit: int = 30, **kwargs)[source]#

Bases: _BaseAzureCloudObjStorage

ObjStorage backend for Azure blob storage accounts.

Parameters:

  • container_url – the URL of the container in which the objects are stored.

  • account_name – (deprecated) the name of the storage account under which objects are stored

  • api_secret_key – (deprecated) the shared account key

  • container_name – (deprecated) the name of the container under which objects are stored

  • compression – the compression algorithm used to compress objects in storage

  • connection_limit – maximum number of HTTP connections to Azure

  • use_secondary_endpoint_for_downloads – if True, use the secondary endpoint url to generate download URLs. To configure the secondary endpoint, use the BlobSecondaryEndpoint entry of the connection string.

Notes

The container url should contain the credentials via a “Shared Access Signature”. The get_container_url() helper can be used to generate such a URL from the account’s access keys. The account_name, api_secret_key and container_name arguments are deprecated.

name: str = 'azure'#

Default objstorage name; can be overloaded at instantiation time giving a ‘name’ argument to the constructor

get_async_container_clients() → Iterable[ContainerClient][source]#

Returns a collection of container clients, to be passed to get_async_blob_client.

Each container may not be used in more than one asyncio loop.

get_async_blob_client(hex_obj_id) → BlobClient[source]#

Get the azure blob client for the given hex obj id and a collection yielded by get_async_container_clients.

class swh.objstorage.backends.azure.PrefixedAzureCloudObjStorage(accounts: Mapping[str, str | Dict[str, str]], name: str = 'azure-prefixed', compression: Literal['bz2', 'lzma', 'gzip', 'zlib', 'none'] | None = None, connection_limit_per_container: int = 6, **kwargs)[source]#

Bases: _BaseAzureCloudObjStorage

ObjStorage with azure capabilities, striped by prefix.

Parameters:

connection_limit_per_container – maximum number of HTTP connections to each Azure container.

accounts is a dict containing entries of the form:

<prefix>: <container_url_for_prefix>

get_async_container_clients() → Iterable[ContainerClient][source]#

Returns a collection of container clients, to be passed to get_async_blob_client.

Each container may not be used in more than one asyncio loop.

get_async_blob_client(hex_obj_id) → BlobClient[source]#

Get the azure blob client for the given hex obj id and a collection yielded by get_async_container_clients.