swh.objstorage.backends.winery.sharedbase module

class swh.objstorage.backends.winery.sharedbase.ShardState(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

STANDBY = 'standby'

WRITING = 'writing'

FULL = 'full'

PACKING = 'packing'

PACKED = 'packed'

CLEANING = 'cleaning'

READONLY = 'readonly'

property locked

property image_available

property readonly

class swh.objstorage.backends.winery.sharedbase.SignatureState(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

INFLIGHT = 'inflight'

PRESENT = 'present'

DELETED = 'deleted'

class swh.objstorage.backends.winery.sharedbase.TemporaryShardLocker(base: SharedBase, current_state: ShardState, new_state: ShardState, min_mapped_hosts: int = 0)

Bases: object

Opportunistically lock a shard, and provide a context manager to unlock the shard if an operation fails.

Use this through the SharedBase.maybe_lock_one_shard() method.

class swh.objstorage.backends.winery.sharedbase.SharedBase(base_dsn: str, application_name: str = 'SWH Winery SharedBase', **kwargs)

Bases: Database

current_version: int = 1

property locked_shard: str

property locked_shard_id: int

set_locked_shard() → None

lock_one_shard(current_state: ShardState, new_state: ShardState, min_mapped_hosts: int = 0) → Tuple[str, int] | None

Lock one shard in current_state, putting it into new_state. Only lock a shard if it has more than min_mapped_hosts hosts that have registered as having mapped the shard.

maybe_lock_one_shard(current_state: ShardState, new_state: ShardState, min_mapped_hosts: int = 0) → TemporaryShardLocker

Opportunistically lock a shard, and, if a shard was locked, provide a context manager to rollback the locking on failure.

Example:

locked = base.maybe_lock_one_shard(
    current_state=ShardState.FULL,
    new_state=ShardState.PACKING,
)
if not locked:
    wait_a_minute()
    return

with locked:
    do_something_with_locked_shard(locked.name)

If do_something_with_locked_shard fails, the shard will be moved back to the current_state on exit.

set_shard_state(new_state: ShardState, set_locker: bool = False, check_locker: bool = False, name: str | None = None, db: Connection | None = None)

create_shard(new_state: ShardState) → Tuple[str, int]

shard_packing_starts(name: str)

shard_packing_ends(name)

get_shard_info(id: int) → Tuple[str, ShardState] | None

get_shard_state(name: str) → ShardState | None

list_shards() → Iterator[Tuple[str, ShardState]]

count_objects(name: str | None = None) → int | None

record_shard_mapped(host: str, name: str) → Set[str]

contains(obj_id) → int | None

get(obj_id) → Tuple[str, ShardState] | None

record_new_obj_id(db, obj_id) → int | None

delete(obj_id)

deleted_objects() → Iterator[Tuple[bytes, str, ShardState]]

clean_deleted_object(obj_id) → None