swh.graph package

Subpackages:

• swh.graph.example_dataset package
  • swh.graph.example_dataset.generate_dataset module
    • main()
  • PERSONS
  • CONTENTS
  • SKIPPED_CONTENTS
  • DIRECTORIES
  • REVISIONS
  • RELEASES
  • SNAPSHOTS
  • ORIGIN_VISITS
  • ORIGIN_VISIT_STATUSES
  • INITIAL_ORIGIN
  • FORKED_ORIGIN
  • ORIGINS
  • DATASET
  • DATASET_DIR
• swh.graph.grpc namespace
  • swh.graph.grpc.swhgraph_pb2 module
    • ContentData
    • CountResponse
    • EdgeLabel
    • FindPathBetweenRequest
    • FindPathToRequest
    • GetNodeRequest
    • Node
    • NodeFilter
    • OriginData
    • Path
    • ReleaseData
    • RevisionData
    • StatsRequest
    • StatsResponse
    • Successor
    • TraversalRequest
  • swh.graph.grpc.swhgraph_pb2_grpc module
    • TraversalServiceStub
    • TraversalServiceServicer
    • add_TraversalServiceServicer_to_server()
    • TraversalService
• swh.graph.luigi package
  • swh.graph.luigi.blobs_datasets module
    • Luigi tasks for blob-centric datasets
    • atomic_zstd_writer()
    • atomic_csv_zstd_writer()
    • check_csv()
    • SelectBlobs
    • DownloadBlobs
    • MakeBlobTarball
    • MakeSampleBlobTarball
    • ComputeBlobFileinfo
    • BlobScancode
    • FindBlobOrigins
    • CountBlobOrigins
    • FindEarliestRevisions
    • RunBlobDataset
  • swh.graph.luigi.compressed_graph module
    • Luigi tasks for compression
    • ObjectTypesParameter
    • ExtractNodes
    • Mph
    • ConvertMph
    • Bv
    • BvOffsets
    • BvEf
    • Bfs
    • PermuteAndSimplifyBfs
    • BfsEf
    • BfsDcf
    • Llp
    • PermuteLlp
    • LlpOffsets
    • Obl
    • ComposeOrders
    • Stats
    • Transpose
    • TransposeObl
    • Maps
    • ExtractPersons
    • MphPersons
    • NodeProperties
    • MphLabels
    • FclLabels
    • EdgeLabels
    • EdgeLabelsObl
    • EdgeLabelsTransposeObl
    • CompressGraph
    • UploadGraphToS3
    • DownloadGraphFromS3
    • LocalGraph
  • swh.graph.luigi.file_names module
    • Luigi tasks for producing the most common names of every content and datasets based on file names
    • PopularContentNames
    • PopularContentPaths
    • PopularContentNamesOrcToS3
    • ListFilesByName
  • swh.graph.luigi.origin_contributors module
    • Luigi tasks for contribution graph
    • ListOriginContributors
    • ExportDeanonymizationTable
    • DeanonymizeOriginContributors
    • RunOriginContributors
  • swh.graph.luigi.provenance module
    • Luigi tasks to help compute the provenance of content blobs
    • ListProvenanceNodes
    • ComputeEarliestTimestamps
    • ListDirectoryMaxLeafTimestamp
    • ComputeDirectoryFrontier
    • ListFrontierDirectoriesInRevisions
    • ListContentsInRevisionsWithoutFrontier
    • ListContentsInFrontierDirectories
    • RunProvenance
  • swh.graph.luigi.topology module
    • Luigi tasks to analyze, and produce datasets related to, graph topology
    • TopoSort
    • CountPaths
    • PathCountsOrcToS3
  • swh.graph.luigi.utils module
    • count_nodes()
  • Luigi tasks
  • RunExportCompressUpload
    • RunExportCompressUpload.requires()
    • RunExportCompressUpload.complete()

Submodules:

• swh.graph.cli module
  • StepOption
    • StepOption.name
    • StepOption.convert()
  • PathlibPath
    • PathlibPath.convert()
  • get_all_subclasses()
  • main()
• swh.graph.client module
• swh.graph.config module
  • find_graph_jar()
  • check_config()
  • check_config_compress()
• swh.graph.grpc_server module
  • build_java_grpc_server_cmdline()
  • build_rust_grpc_server_cmdline()
  • spawn_java_grpc_server()
  • spawn_rust_grpc_server()
  • stop_java_grpc_server()
• swh.graph.http_client module
  • GraphAPIError
  • GraphArgumentException
  • RemoteGraphClient
    • RemoteGraphClient.raw_verb_lines()
    • RemoteGraphClient.get_lines()
    • RemoteGraphClient.raise_for_status()
    • RemoteGraphClient.stats()
    • RemoteGraphClient.leaves()
    • RemoteGraphClient.neighbors()
    • RemoteGraphClient.visit_nodes()
    • RemoteGraphClient.visit_edges()
    • RemoteGraphClient.visit_paths()
    • RemoteGraphClient.walk()
    • RemoteGraphClient.random_walk()
    • RemoteGraphClient.count_leaves()
    • RemoteGraphClient.count_neighbors()
    • RemoteGraphClient.count_visit_nodes()
• swh.graph.http_naive_client module
  • check_arguments()
  • filter_node_types()
  • NaiveClient
    • NaiveClient.stats()
    • NaiveClient.leaves()
    • NaiveClient.neighbors()
    • NaiveClient.visit_nodes()
    • NaiveClient.visit_edges()
    • NaiveClient.visit_paths()
    • NaiveClient.walk()
    • NaiveClient.random_walk()
    • NaiveClient.count_leaves()
    • NaiveClient.count_neighbors()
    • NaiveClient.count_visit_nodes()
  • Graph
    • Graph.get_filtered_neighbors()
    • Graph.get_subgraph()
    • Graph.iter_paths_dfs()
    • Graph.iter_edges_dfs()
  • SubgraphIterator
    • SubgraphIterator.more_work()
    • SubgraphIterator.pop()
    • SubgraphIterator.push()
  • DfsSubgraphIterator
    • DfsSubgraphIterator.more_work()
    • DfsSubgraphIterator.pop()
    • DfsSubgraphIterator.push()
• swh.graph.http_rpc_server module
  • GraphServerApp
  • index()
  • GraphView
    • GraphView.get_direction()
    • GraphView.get_edges()
    • GraphView.get_return_types()
    • GraphView.get_max_matching_nodes()
    • GraphView.get_max_edges()
    • GraphView.check_swhid()
  • StreamingGraphView
    • StreamingGraphView.content_type
    • StreamingGraphView.response_streamer()
    • StreamingGraphView.get()
    • StreamingGraphView.prepare_response()
    • StreamingGraphView.stream_response()
    • StreamingGraphView.stream_line()
  • StatsView
    • StatsView.get()
  • SimpleTraversalView
    • SimpleTraversalView.prepare_response()
    • SimpleTraversalView.configure_request()
    • SimpleTraversalView.stream_response()
  • LeavesView
    • LeavesView.configure_request()
  • NeighborsView
    • NeighborsView.configure_request()
  • VisitNodesView
  • VisitEdgesView
    • VisitEdgesView.configure_request()
    • VisitEdgesView.stream_response()
  • CountView
    • CountView.count_type
    • CountView.get()
    • CountView.configure_request()
  • CountNeighborsView
    • CountNeighborsView.configure_request()
  • CountLeavesView
    • CountLeavesView.configure_request()
  • CountVisitNodesView
  • make_app()
  • make_app_from_configfile()
• swh.graph.naive_client module
• swh.graph.pytest_plugin module
  • GraphServerProcess
    • GraphServerProcess.run()
    • GraphServerProcess.start()
  • graph_grpc_backend_implementation()
  • graph_grpc_server_process()
  • graph_grpc_server_started()
  • graph_grpc_stub()
  • graph_grpc_server()
  • remote_graph_client_url()
  • remote_graph_client()
  • naive_graph_client()
  • graph_client()
• swh.graph.shell module
  • LOGBACK_CONF
  • CommandException
  • base_cgroup()
  • create_cgroup()
  • move_to_cgroup()
  • Command
    • Command.run()
  • Java
  • Rust
  • Pipe
    • Pipe.run()
  • wc()
  • Sink
    • Sink.run()
  • AtomicFileSink
    • AtomicFileSink.run()
  • RunResult
    • RunResult.cgroup
    • RunResult.command
    • RunResult.cgroup_stats
• swh.graph.webgraph module
  • CompressionSubprocessError
  • CompressionStep
    • CompressionStep.EXTRACT_NODES
    • CompressionStep.MPH
    • CompressionStep.CONVERT_MPH
    • CompressionStep.BV
    • CompressionStep.BV_OFFSETS
    • CompressionStep.BV_EF
    • CompressionStep.BFS
    • CompressionStep.PERMUTE_AND_SIMPLIFY_BFS
    • CompressionStep.BFS_EF
    • CompressionStep.BFS_DCF
    • CompressionStep.LLP
    • CompressionStep.COMPOSE_ORDERS
    • CompressionStep.PERMUTE_LLP
    • CompressionStep.LLP_OFFSETS
    • CompressionStep.OBL
    • CompressionStep.STATS
    • CompressionStep.TRANSPOSE
    • CompressionStep.TRANSPOSE_OBL
    • CompressionStep.MAPS
    • CompressionStep.EXTRACT_PERSONS
    • CompressionStep.MPH_PERSONS
    • CompressionStep.NODE_PROPERTIES
    • CompressionStep.MPH_LABELS
    • CompressionStep.FCL_LABELS
    • CompressionStep.EDGE_LABELS
    • CompressionStep.EDGE_LABELS_OBL
    • CompressionStep.EDGE_LABELS_TRANSPOSE_OBL
    • CompressionStep.CLEAN_TMP
  • do_step()
  • compress()

Module contents: