swh.alter.inventory module#

This module implements the inventory stage of the removal algorithm.

exception swh.alter.inventory.RootsNotFound(swhids: Iterable[ExtendedSWHID])[source]#

Bases: Exception

get_labels(requested: Collection[Origin | ExtendedSWHID]) → List[str][source]#

Returns a list of either an origin URL if it can be found in requested and the SWHID otherwise.

exception swh.alter.inventory.StuckInventoryException(swhids: List[ExtendedSWHID])[source]#

Bases: Exception

class swh.alter.inventory.InventorySubgraph(*args, **kwargs)[source]#

Bases: Subgraph

A subgraph holding an inventory of all candidates for removal

When all references from a given node have been accounted for, the complete attribute is set to True.

See Subgraph

default_vertex_attributes: Dict[str, Any] = {'complete': False}#

Vertex will get the following attributes on creation unless specified otherwise.

select_incomplete() → List[Vertex][source]#

Return vertices known to be incomplete ordered by object type from origins to contents.

class swh.alter.inventory.ProgressBarItem(origin, total, remaining)[source]#

Bases: NamedTuple

Create new instance of ProgressBarItem(origin, total, remaining)

origin: ExtendedSWHID | None#

Alias for field number 0

total: int#

Alias for field number 1

remaining: int#

Alias for field number 2

swh.alter.inventory.make_inventory(storage, graph_client, swhids: List[ExtendedSWHID], known_missing: Set[ExtendedSWHID] | None = None, progressbar: ProgressBarInit | None = None) → InventorySubgraph[source]#

Inventory candidates for removal from the given set of SWHID.

By querying the given storage and graph_client, create a subgraph with all objects belonging to the given set of SWHIDs (typically origins). The result should then used to verify which candidate can safely be removed.

swh.alter.inventory.get_raw_extrinsic_metadata(storage: StorageInterface, referencing_swhids: List[ExtendedSWHID], progressbar: ProgressBarInit | None = None) → Iterator[ExtendedSWHID][source]#

Find RawExtrinsicMetadata referencing the given SWHIDs.

This will recursively find RawExtrinsicMetadata referencing the found RawExtrinsicMetadata objects. The output is sorted in the order of the iterations: the objects coming first are referenced by objects latter in the list.