swh.graph.libs.luigi package

Submodules:

• swh.graph.libs.luigi.topology module
  • Luigi tasks to analyze, and produce datasets related to, graph topology
    • File layout
  • inverse_direction()
  • TopoSort
    • TopoSort.local_graph_path
    • TopoSort.topology_dir
    • TopoSort.graph_name
    • TopoSort.object_types
    • TopoSort.direction
    • TopoSort.algorithm
    • TopoSort.requires()
    • TopoSort.output()
    • TopoSort.run()
  • ComputeGenerations
    • ComputeGenerations.local_graph_path
    • ComputeGenerations.topology_dir
    • ComputeGenerations.graph_name
    • ComputeGenerations.object_types
    • ComputeGenerations.direction
    • ComputeGenerations.requires()
    • ComputeGenerations.output()
    • ComputeGenerations.run()
  • UploadGenerationsToS3
    • UploadGenerationsToS3.local_graph_path
    • UploadGenerationsToS3.topology_dir
    • UploadGenerationsToS3.dataset_name
    • UploadGenerationsToS3.graph_name
    • UploadGenerationsToS3.object_types
    • UploadGenerationsToS3.direction
    • UploadGenerationsToS3.requires()
    • UploadGenerationsToS3.output()
    • UploadGenerationsToS3.run()
  • CountPaths
    • CountPaths.local_graph_path
    • CountPaths.topology_dir
    • CountPaths.graph_name
    • CountPaths.write_parquet
    • CountPaths.object_types
    • CountPaths.direction
    • CountPaths.resources
    • CountPaths.requires()
    • CountPaths.output()
    • CountPaths.run()
  • CountDescendants
    • CountDescendants.local_graph_path
    • CountDescendants.topology_dir
    • CountDescendants.graph_name
    • CountDescendants.object_types
    • CountDescendants.write_parquet
    • CountDescendants.direction
    • CountDescendants.rsd
    • CountDescendants.seed
    • CountDescendants.resources
    • CountDescendants.requires()
    • CountDescendants.output()
    • CountDescendants.run()
  • PathCountsParquetToS3
    • PathCountsParquetToS3.topology_dir
    • PathCountsParquetToS3.object_types
    • PathCountsParquetToS3.direction
    • PathCountsParquetToS3.dataset_name
    • PathCountsParquetToS3.s3_athena_output_location
    • PathCountsParquetToS3.requires()

Module contents:

Luigi tasks

This package contains Luigi tasks. These come in two kinds: