swh.objstorage.backends.azure module#
- swh.objstorage.backends.azure.get_container_url(account_name: str, account_key: str, container_name: str, access_policy: str = 'read_only', expiry: timedelta = datetime.timedelta(days=365), container_url_template: str = 'https://{account_name}.blob.core.windows.net/{container_name}?{signature}', **kwargs) str [source]#
Get the full url, for the given container on the given account, with a Shared Access Signature granting the specified access policy.
- Parameters:
account_name – name of the storage account for which to generate the URL
account_key – shared account key of the storage account used to generate the SAS
container_name – name of the container for which to grant access in the storage account
access_policy – one of
read_only
,append_only
,full
expiry – the interval in the future with which the signature will expire
- Returns:
the full URL of the container, with the shared access signature.
- class swh.objstorage.backends.azure.AzureCloudObjStorage(container_url: Optional[str] = None, account_name: Optional[str] = None, api_secret_key: Optional[str] = None, container_name: Optional[str] = None, connection_string: Optional[str] = None, compression='gzip', **kwargs)[source]#
Bases:
ObjStorage
ObjStorage backend for Azure blob storage accounts.
- Parameters:
container_url – the URL of the container in which the objects are stored.
account_name – (deprecated) the name of the storage account under which objects are stored
api_secret_key – (deprecated) the shared account key
container_name – (deprecated) the name of the container under which objects are stored
compression – the compression algorithm used to compress objects in storage
Notes
The container url should contain the credentials via a “Shared Access Signature”. The
get_container_url()
helper can be used to generate such a URL from the account’s access keys. Theaccount_name
,api_secret_key
andcontainer_name
arguments are deprecated.- PRIMARY_HASH: typing_extensions.Literal[sha1] = 'sha1'#
- get_container_client(hex_obj_id)[source]#
Get the container client for the container that contains the object with internal id hex_obj_id
This is used to allow the PrefixedAzureCloudObjStorage to dispatch the client according to the prefix of the object id.
- get_async_container_clients()[source]#
Returns a collection of container clients, to be passed to
get_async_blob_client
.Each container may not be used in more than one asyncio loop.
- get_async_blob_client(hex_obj_id, container_clients)[source]#
Get the azure blob client for the given hex obj id and a collection yielded by
get_async_container_clients
.
- add(content: bytes, obj_id: Union[bytes, CompositeObjId], check_presence: bool = True) None [source]#
Add an obj in storage if it’s not there already.
- get_batch(obj_ids: Iterable[Union[bytes, CompositeObjId]]) Iterator[Optional[bytes]] [source]#
Retrieve objects’ raw content in bulk from storage, concurrently.
- delete(obj_id: Union[bytes, CompositeObjId])[source]#
Delete an object.
- class swh.objstorage.backends.azure.PrefixedAzureCloudObjStorage(accounts: Mapping[str, Union[str, Mapping[str, str]]], compression='gzip', **kwargs)[source]#
Bases:
AzureCloudObjStorage
ObjStorage with azure capabilities, striped by prefix.
- accounts is a dict containing entries of the form:
<prefix>: <container_url_for_prefix>
- get_container_client(hex_obj_id)[source]#
Get the block_blob_service and container that contains the object with internal id hex_obj_id
- get_async_container_clients()[source]#
Returns a collection of container clients, to be passed to
get_async_blob_client
.Each container may not be used in more than one asyncio loop.