swh.vault.cookers.base module

exception swh.vault.cookers.base.PolicyError

Bases: Exception

Raised when the bundle violates the cooking policy.

exception swh.vault.cookers.base.BundleTooLargeError

Bases: PolicyError

Raised when the bundle is too large to be cooked.

class swh.vault.cookers.base.BytesIOBundleSizeLimit(*args, size_limit=None, **kwargs)

Bases: BytesIO

write(chunk)

Write bytes to file.

Return the number of bytes written.

class swh.vault.cookers.base.BaseVaultCooker(swhid: CoreSWHID, backend, storage: StorageInterface, graph=None, objstorage=None, max_bundle_size: int = 536870912, thread_pool_size: int = 10)

Bases: object

Abstract base class for the vault’s bundle creators

This class describes a common API for the cookers.

To define a new cooker, inherit from this class and override: - CACHE_TYPE_KEY: key to use for the bundle to reference in cache - def cook(): cook the object into a bundle

Initialize the cooker.

The type of the object represented by the id depends on the concrete class. Very likely, each type of bundle will have its own cooker class.

Parameters:

• swhid – id of the object to be cooked into a bundle.

• backend – the vault backend (swh.vault.backend.VaultBackend).

SUPPORTED_OBJECT_TYPES: ClassVar[Set[ObjectType]]

BUNDLE_TYPE: ClassVar[str]

classmethod check_object_type(object_type: ObjectType) → None

abstract check_exists()

Checks that the requested object exists and can be cooked.

Override this in the cooker implementation.

abstract prepare_bundle()

Implementation of the cooker. Yields chunks of the bundle bytes.

Override this with the cooker implementation.

cache_type_key() → str

write(chunk)

cook()

Cook the requested object into a bundle