swh.graph.libs.luigi package

Submodules

• swh.graph.libs.luigi.topology module
  • Luigi tasks to analyze, and produce datasets related to, graph topology
    • File layout
  • TopoSort
    • TopoSort.local_graph_path
    • TopoSort.topological_order_dir
    • TopoSort.graph_name
    • TopoSort.object_types
    • TopoSort.direction
    • TopoSort.algorithm
    • TopoSort.requires()
    • TopoSort.output()
    • TopoSort.run()
  • ComputeGenerations
    • ComputeGenerations.local_graph_path
    • ComputeGenerations.topological_order_dir
    • ComputeGenerations.graph_name
    • ComputeGenerations.object_types
    • ComputeGenerations.direction
    • ComputeGenerations.requires()
    • ComputeGenerations.output()
    • ComputeGenerations.run()
  • UploadGenerationsToS3
    • UploadGenerationsToS3.local_graph_path
    • UploadGenerationsToS3.topological_order_dir
    • UploadGenerationsToS3.dataset_name
    • UploadGenerationsToS3.graph_name
    • UploadGenerationsToS3.object_types
    • UploadGenerationsToS3.direction
    • UploadGenerationsToS3.requires()
    • UploadGenerationsToS3.output()
    • UploadGenerationsToS3.run()
  • CountPaths
    • CountPaths.local_graph_path
    • CountPaths.topological_order_dir
    • CountPaths.graph_name
    • CountPaths.object_types
    • CountPaths.direction
    • CountPaths.resources
    • CountPaths.requires()
    • CountPaths.output()
    • CountPaths.nb_lines()
    • CountPaths.run()
  • PathCountsParquetToS3
    • PathCountsParquetToS3.topological_order_dir
    • PathCountsParquetToS3.object_types
    • PathCountsParquetToS3.direction
    • PathCountsParquetToS3.dataset_name
    • PathCountsParquetToS3.s3_athena_output_location
    • PathCountsParquetToS3.requires()

Module contents

Luigi tasks

This package contains Luigi tasks. These come in two kinds: