swh.objstorage.multiplexer.striping_objstorage module

class swh.objstorage.multiplexer.striping_objstorage.StripingObjStorage(storages, **kwargs)[source]

Bases: swh.objstorage.multiplexer.multiplexer_objstorage.MultiplexerObjStorage

Stripes objects across multiple objstorages

This objstorage implementation will write objects to objstorages in a predictable way: it takes the modulo of the last 8 bytes of the object identifier with the number of object storages passed, which will yield an (almost) even distribution.

Objects are read from all storages in turn until it succeeds.

MOD_BYTES = 8
get_storage_index(obj_id)[source]
get_write_threads(obj_id)[source]
get_read_threads(obj_id=None)[source]
add_batch(contents, check_presence=True)[source]

Add a batch of new objects to the object storage.