swh.objstorage.backends.mosaic module

Implementation of the ObjStorage API based on a single MOSAIC file.

This is suitable for small objects collections (less than 100 million objects) that can benefit from the efficiency of swh.mosaic without setting up a complete Winery back-end.

class swh.objstorage.backends.mosaic.MosaicObjStorage(path: str, primary_hash: Literal['sha1', 'sha1_git', 'sha256'] = 'sha1_git', **kwargs)

Bases: ObjStorage

Readonly objstorage backed by a single MOSAIC file.

name: str = 'mosaic'

Default objstorage name; can be overloaded at instantiation time giving a ‘name’ argument to the constructor

check_config(*, check_write)

Check whether the object storage is properly configured.

Parameters:

• check_write (bool) – if True, check if writes to the object storage

• succeed. (can)

Returns:

True if the configuration check worked, False if ‘check_write’ is True and the object storage is actually read only, and an exception if the check failed.

add(content: bytes, obj_id: HashDict, check_presence: bool = True) → None

Add a new object to the object storage.

Parameters:

• content – object’s raw content to add in storage.

• obj_id – dict of checksums.

• check_presence (bool) – indicate if the presence of the content should be verified before adding the file.

Returns:

the id (bytes) of the object into the storage.

get(obj_id: HashDict) → bytes

Retrieve the content of a given object.

Parameters:

obj_id – object id.

Returns:

the content of the requested object as bytes.

Raises:

ObjNotFoundError – if the requested object is missing.

get_batch(obj_ids: Iterable[HashDict]) → Iterator[bytes | None]

Retrieve objects’ raw content in bulk from storage.

Note: This function does have a default implementation in ObjStorage that is suitable for most cases.

For object storages that needs to do the minimal number of requests possible (ex: remote object storages), that method can be overridden to perform a more efficient operation.

Parameters:

obj_ids – list of object ids.

Returns:

list of resulting contents, or None if the content could not be retrieved. Do not raise any exception as a fail for one content will not cancel the whole request.

delete(obj_id: HashDict)

Delete an object.

Parameters:

obj_id – object identifier.

Raises:

ObjNotFoundError – if the requested object is missing.