swh.storage package#

Subpackages:

Submodules:

Module contents:

swh.storage.get_storage(cls: str, **kwargs) → StorageInterface[source]#

Get a storage object of class storage_class with arguments storage_args.

Parameters:

cls (str) –
storage’s class, can be:
- local to use a postgresql database
- cassandra to use a cassandra database
- remote to connect to a swh-storage server
- memory for an in-memory storage, useful for fast tests
- filter, buffer, … to use specific storage “proxies”, see their respective documentations
args (dict) – dictionary with keys

Returns:

an instance of swh.storage.Storage or compatible class

Raises:

ValueError if passed an unknown storage class. –

swh.storage.get_datastore(cls: str, **kwargs) → StorageInterface#

Get a storage object of class storage_class with arguments storage_args.

Parameters:

cls (str) –
storage’s class, can be:
- local to use a postgresql database
- cassandra to use a cassandra database
- remote to connect to a swh-storage server
- memory for an in-memory storage, useful for fast tests
- filter, buffer, … to use specific storage “proxies”, see their respective documentations
args (dict) – dictionary with keys

Returns:

an instance of swh.storage.Storage or compatible class

Raises:

ValueError if passed an unknown storage class. –

swh.storage.get_storage_pipeline(steps: List[Dict[str, Any]], check_config=None) → StorageInterface[source]#

Recursively get a storage object that may use other storage objects as backends.

Parameters:: steps (List[dict]) – List of dicts that may be used as kwargs for get_storage.
Returns:: an instance of swh.storage.Storage or compatible class
Raises:: ValueError if passed an unknown storage class. –