swh.objstorage.backends.winery.roshard module

exception swh.objstorage.backends.winery.roshard.ShardNotMapped

Bases: Exception

class swh.objstorage.backends.winery.roshard.Pool(shard_max_size: int, rbd_use_sudo: bool = True, rbd_pool_name: str = 'shards', rbd_data_pool_name: str | None = None, rbd_image_features_unsupported: Tuple[str, ...] = ())

Bases: object

Manage a Ceph RBD pool for Winery shards.

Parameters:

• shard_max_size – max size of shard contents

• rbd_use_sudo – whether to use sudo for rbd commands

• rbd_pool_name – name of the pool used for RBD images (metadata)

• rbd_data_pool_name – name of the pool used for RBD images (data)

• rbd_image_features_unsupported – features not supported by the kernel mounting the rbd images

POOL_CONFIG: Tuple[str, ...] = ('shard_max_size', 'rbd_use_sudo', 'rbd_pool_name', 'rbd_data_pool_name', 'rbd_image_features_unsupported')

classmethod from_kwargs(**kwargs) → Pool

Create a Pool from a set of arbitrary keyword arguments

run(*cmd: str) → Iterable[str]

Run the given command, and return its output as lines.

Return: the standard output of the run command

Raises: CalledProcessError if the command doesn’t exit with exit code 0.

rbd(*arguments: str) → Iterable[str]

Run rbd with the given arguments

image_exists(image: str)

image_mapped(image: str) → Literal['ro', 'rw'] | None

Check whether the image is already mapped, read-only or read-write

image_list()

image_path(image: str) → str

image_create(image: str)

image_map(image: str, options: str)

image_remap_ro(image: str)

image_unmap(image: str)

manage_images(base_dsn: str, manage_rw_images: bool, wait_for_image: Callable[[int], None], stop_running: Callable[[], bool], application_name: str | None = None) → None

Manage RBD image creation and mapping automatically.

Parameters:

• base_dsn – the DSN of the connection to the SharedBase

• manage_rw_images – whether RW images should be created and mapped

• wait_for_image – function which is called at each loop iteration, with an attempt number, if no images had to be mapped recently

• stop_running – callback that returns True when the manager should stop running

• application_name – the application name sent to PostgreSQL

class swh.objstorage.backends.winery.roshard.ROShard(name, **kwargs)

Bases: object

open()

get(key)

close()

static delete(pool, shard_name, obj_id)

class swh.objstorage.backends.winery.roshard.ROShardCreator(name: str, count: int, rbd_create_images: bool = True, rbd_wait_for_image: ~typing.Callable[[int], None] = <function sleep_exponential.<locals>.sleep>, **kwargs)

Bases: object

Helper for Read-Only shard creation.

Parameters:

• name – Name of the shard to be initialized

• count – Number of objects to provision in the shard

• rbd_create_images – whether the ROShardCreator should create the rbd image, or delegate to the rbd_shard_manager

• rbd_wait_for_image – function called when waiting for a shard to be mapped

• shard_max_size – the size of the shard, passed to Pool

• rbd_* – other RBD-related Pool arguments

• throttle_* – Throttler arguments

add(content, obj_id)