swh.objstorage.api.client module#

class swh.objstorage.api.client.RemoteObjStorage(url, api_exception=None, timeout=None, chunk_size=4096, reraise_exceptions=None, **kwargs)[source]#

Bases: RPCClient

Proxy to a remote object storage.

This class allows to connect to an object storage server via http protocol.


The url of the server to connect. Must end with a ‘/’




The session to send requests.


alias of ObjStorageAPIError

reraise_exceptions: ClassVar[List[Type[Exception]]] = [<class 'swh.objstorage.exc.ObjNotFoundError'>, <class 'swh.objstorage.exc.Error'>]#

On server errors, if any of the exception classes in this list has the same name as the error name, then the exception will be instantiated and raised instead of a generic RemoteException.


alias of ObjStorageInterface

restore(content: bytes, obj_id: Union[bytes, CompositeObjId]) None[source]#
list_content(last_obj_id: Optional[Union[bytes, CompositeObjId]] = None, limit: Optional[int] = 10000) Iterator[CompositeObjId][source]#
add(content: bytes, obj_id: Union[bytes, CompositeObjId], check_presence: bool = True) None#

Add a new object to the object storage.

  • content – object’s raw content to add in storage.

  • obj_id – either dict of checksums, or single checksum of [bytes] using [ID_HASH_ALGO] algorithm. It is trusted to match the bytes.

  • check_presence (bool) – indicate if the presence of the content should be verified before adding the file.


the id (bytes) of the object into the storage.

add_batch(contents: Union[Mapping[bytes, bytes], Iterable[Tuple[Union[bytes, CompositeObjId], bytes]]], check_presence: bool = True) Dict#

Add a batch of new objects to the object storage.


contents – either mapping from [ID_HASH_ALGO] checksums to object contents, or list of pairs of dict hashes and object contents


the summary of objects added to the storage (count of object, count of bytes object)

check(obj_id: Union[bytes, CompositeObjId]) None#

Perform an integrity check for a given object.

Verify that the file object is in place and that the content matches the object id.


obj_id – object identifier.

check_config(*, check_write)#

Check whether the object storage is properly configured.

  • check_write (bool) – if True, check if writes to the object storage

  • succeed. (can) –


True if the configuration check worked, an exception if it didn’t.

delete(obj_id: Union[bytes, CompositeObjId])#

Delete an object.


obj_id – object identifier.


ObjNotFoundError – if the requested object is missing.

get(obj_id: Union[bytes, CompositeObjId]) bytes#

Retrieve the content of a given object.


obj_id – object id.


the content of the requested object as bytes.


ObjNotFoundError – if the requested object is missing.

get_batch(obj_ids: Iterable[Union[bytes, CompositeObjId]]) Iterator[Optional[bytes]]#

Retrieve objects’ raw content in bulk from storage.

Note: This function does have a default implementation in ObjStorage that is suitable for most cases.

For object storages that needs to do the minimal number of requests possible (ex: remote object storages), that method can be overridden to perform a more efficient operation.


obj_ids – list of object ids.


list of resulting contents, or None if the content could not be retrieved. Do not raise any exception as a fail for one content will not cancel the whole request.