swh.storage package

Subpackages:

Submodules:

Module contents:

swh.storage.get_storage(cls: str, **kwargs) StorageInterface[source]

Get a storage object of class storage_class with arguments storage_args.

Parameters
  • cls (str) –

    storage’s class, can be:
    • local to use a postgresql database

    • cassandra to use a cassandra database

    • remote to connect to a swh-storage server

    • memory for an in-memory storage, useful for fast tests

    • filter, buffer, … to use specific storage “proxies”, see their respective documentations

  • args (dict) – dictionary with keys

Returns

an instance of swh.storage.Storage or compatible class

Raises

ValueError if passed an unknown storage class.

swh.storage.get_storage_pipeline(steps: List[Dict[str, Any]], check_config=None) StorageInterface[source]

Recursively get a storage object that may use other storage objects as backends.

Parameters

steps (List[dict]) – List of dicts that may be used as kwargs for get_storage.

Returns

an instance of swh.storage.Storage or compatible class

Raises

ValueError if passed an unknown storage class.