swh.scanner.policy module#
- swh.scanner.policy.source_size(source_tree: Directory)[source]#
return the size of a source tree as the number of nodes it contains
- class swh.scanner.policy.Policy(source_tree: Directory, data: MerkleNodeInfo)[source]#
Bases:
object
- data: MerkleNodeInfo#
information about contents and directories of the merkle tree
- class swh.scanner.policy.LazyBFS(source_tree: Directory, data: MerkleNodeInfo)[source]#
Bases:
Policy
Read nodes in the merkle tree using the BFS algorithm. Lookup only directories that are unknown otherwise set all the downstream contents to known.
- data: MerkleNodeInfo#
information about contents and directories of the merkle tree
- class swh.scanner.policy.GreedyBFS(source_tree: Directory, data: MerkleNodeInfo)[source]#
Bases:
Policy
Query graph nodes in chunks (to maximize the Web API rate limit use) and set the downstream contents of known directories to known.
- async get_nodes_chunks(client: Client, ssize: int)[source]#
Query chunks of QUERY_LIMIT nodes at once in order to fill the Web API rate limit. It query all the nodes in the case the source code contains less than QUERY_LIMIT nodes.
- data: MerkleNodeInfo#
information about contents and directories of the merkle tree
- class swh.scanner.policy.FilePriority(source_tree: Directory, data: MerkleNodeInfo)[source]#
Bases:
Policy
Check the Merkle tree querying all the file contents and set all the upstream directories to unknown in the case a file content is unknown. Finally check all the directories which status is still unknown and set all the sub-directories of known directories to known.
- data: MerkleNodeInfo#
information about contents and directories of the merkle tree
- class swh.scanner.policy.DirectoryPriority(source_tree: Directory, data: MerkleNodeInfo)[source]#
Bases:
Policy
Check the Merkle tree querying all the directories that have at least one file content and set all the upstream directories to unknown in the case a directory is unknown otherwise set all the downstream contents to known. Finally check the status of empty directories and all the remaining file contents.
- data: MerkleNodeInfo#
information about contents and directories of the merkle tree
- class swh.scanner.policy.QueryAll(source_tree: Directory, data: MerkleNodeInfo)[source]#
Bases:
Policy
Check the status of every node in the Merkle tree.
- data: MerkleNodeInfo#
information about contents and directories of the merkle tree