swh.storage.noop module

class swh.storage.noop.NoopTable(row_class: Type[TRow])

Bases: Table[TRow], Generic[TRow]

insert(row: TRow) → None

delete(predicate: Callable[[TRow], bool]) → None

class swh.storage.noop.NoopCqlRunner

Bases: InMemoryCqlRunner

content_index_add_concurrent(algo: str, contents: List[Tuple[int, Content]]) → None

content_get_tokens_from_single_algo(algo: str, hashes: List[bytes]) → Iterable[int]

skipped_content_index_add_one(algo: str, content: SkippedContent, token: int) → None

skipped_content_get_tokens_from_single_hash(algo: str, hash_: bytes) → Iterable[int]

class swh.storage.noop.NoopStorage(journal_writer: Dict[str, Any] | None = None, objstorage: Dict[str, Any] | None = None)

Bases: InMemoryStorage

A backend of swh-storage backed by Cassandra

Parameters:

• hosts – Seed Cassandra nodes, to start connecting to the cluster

• keyspace – Name of the Cassandra database to use

• objstorage – Passed as argument to ObjStorage; if unset, use a NoopObjStorage

• port – Cassandra port

• journal_writer – Passed as argument to JournalWriter

• allow_overwrite – Whether *_add functions will check if an object already exists in the database before sending it in an INSERT. False is the default as it is more efficient when there is a moderately high probability the object is already known, but True can be useful to overwrite existing objects (eg. when applying a schema update), or when the database is known to be mostly empty. Note that a False value does not guarantee there won’t be any overwrite.

• consistency_level – The default read/write consistency to use

• directory_entries_insert_algo – Must be one of: * one-by-one: naive, one INSERT per directory entry, serialized * concurrent: one INSERT per directory entry, concurrent * batch: using UNLOGGED BATCH to insert many entries in a few statements

• auth_provider –

  An optional dict describing the authentication provider to use. Must contain at least a cls entry and the parameters to pass to the constructor. For example:

  auth_provider:
      cls: cassandra.auth.PlainTextAuthProvider
      username: myusername
      password: mypassword
  

• table_options – An optional dict mapping each table name (or the literal object_references_*) to CQL table options

reset()

origin_visit_add(visits: List[OriginVisit]) → Iterable[OriginVisit]

origin_visit_status_add(visit_statuses: List[OriginVisitStatus]) → Dict[str, int]

raw_extrinsic_metadata_add(metadata: List[RawExtrinsicMetadata]) → Dict[str, int]

Add extrinsic metadata on objects (contents, directories, …).