swh.datasets package# Subpackages: swh.datasets.luigi package swh.datasets.luigi.aggregate_datasets module Luigi tasks for producing the aggregated derived datasets ExportNodesTable AggregateContentDatasets UploadNodesTable UploadAggregatedContentDataset RunAggregatedDatasets swh.datasets.luigi.blobs_datasets module Luigi tasks for blob-centric datasets atomic_zstd_writer() atomic_csv_zstd_writer() check_csv() SelectBlobs DownloadBlobs MakeBlobTarball MakeSampleBlobTarball ComputeBlobFileinfo BlobScancode FindBlobOrigins CountBlobOrigins FindEarliestRevisions RunBlobDataset swh.datasets.luigi.file_names module Luigi tasks for producing the most common names of every content and datasets based on file names PopularContentNames PopularContentPaths PopularContentNamesOrcToS3 ListFilesByName swh.datasets.luigi.impact module Luigi tasks to measure institutional impact ComputeRawImpact ComputeIndexedImpact swh.datasets.luigi.origin_contributors module Luigi tasks for contribution graph ListOriginContributors ExportDeanonymizationTable DeanonymizeContributors DeanonymizeOriginContributors RunOriginContributors Luigi tasks UploadExportAndCompressedGraphToS3 UploadExportAndCompressedGraphToS3.requires() RunNewGraph RunNewGraph.requires() RunNewGraph.complete() Submodules: swh.datasets.cli module PathlibPath PathlibPath.convert() get_all_subclasses() main() swh.datasets.download module DatasetDownloader DatasetDownloader.filter_objects() DatasetDownloader.post_downloads() swh.datasets.shell module Rust Module contents:
