swh.objstorage.backends.libcloud module#

swh.objstorage.backends.libcloud.patch_libcloud_s3_urlencode()[source]#

Patches libcloud’s S3 backend to properly sign queries.

Recent versions of libcloud are not affected (they use signature V4), but 1.5.0 (the one in Debian 9) is.

class swh.objstorage.backends.libcloud.CloudObjStorage(container_name: str, compression: str = 'gzip', path_prefix: Optional[str] = None, **kwargs)[source]#

Bases: ObjStorage

Abstract ObjStorage that connect to a cloud using Libcloud

Implementations of this class must redefine the _get_provider method to make it return a driver provider (i.e. object that supports get_driver method) which return a LibCloud driver (see https://libcloud.readthedocs.io/en/latest/storage/api.html).

Parameters:

  • container_name – Name of the base container

  • path_prefix – prefix to prepend to object paths in the container, separated with a slash

  • compression – compression algorithm to use for objects

  • kwargs – extra arguments are passed through to the LibCloud driver

PRIMARY_HASH: typing_extensions.Literal[sha1] = 'sha1'#
check_config(*, check_write)[source]#

Check the configuration for this object storage

add(content: bytes, obj_id: Union[bytes, CompositeObjId], check_presence: bool = True) → None[source]#
restore(content: bytes, obj_id: Union[bytes, CompositeObjId]) → None[source]#
get(obj_id: Union[bytes, CompositeObjId]) → bytes[source]#
download_url(obj_id: Union[bytes, CompositeObjId], content_disposition: Optional[str] = None, expiry: Optional[timedelta] = None) → Optional[str][source]#
check(obj_id: Union[bytes, CompositeObjId]) → None[source]#
delete(obj_id: Union[bytes, CompositeObjId])[source]#
class swh.objstorage.backends.libcloud.AwsCloudObjStorage(container_name: str, compression: str = 'gzip', path_prefix: Optional[str] = None, **kwargs)[source]#

Bases: CloudObjStorage

Amazon’s S3 Cloud-based object storage

download_url(obj_id: Union[bytes, CompositeObjId], content_disposition: Optional[str] = None, expiry: Optional[timedelta] = None) → Optional[str][source]#
class swh.objstorage.backends.libcloud.OpenStackCloudObjStorage(container_name: str, compression: str = 'gzip', path_prefix: Optional[str] = None, **kwargs)[source]#

Bases: CloudObjStorage

OpenStack Swift Cloud based object storage