datasets.json export#
The datasets.json file contains a machine readable list of all the datasets published on the website.
Specifications#
- id:
A unique internal ID.
- title:
This dataset’s title.
- absolute_url
Permalink to the details page.
- anchor_url
Anchor link to access the dataset in its category page.
- annex_url
URL to the dataset folder in the Annex.
- category:
The category (compressed, columnar, etc.).
- citations:
A list of BibTeX file to use when referencing the dataset.
- description:
Generic information about this dataset.
- comments:
Extra information related to this dataset.
- tables:
List of tables contained in this dataset.
- date:
The export date of the archive this dataset is based on, format: Y-m-d.
- deprecated:
True when the dataset is deprecated (the
commentsfield should explain why).- derived_of:
The ID of the dataset this dataset is derived from, if applicable.
- teaser_of:
The ID of the dataset this dataset is a teaser of, if applicable.
- group:
Indicates if this graph is a complete one or a teaser/derivation.
- hosted_on_annex:
True if the dataset is available in the Annex
- license:
SPDX short ID of this dataset license.
- hosted_on_s3:
True if the dataset is available in S3.
- s3_url:
S3 download URL.
- s3_download_command:
How to download this dataset using
awscli.- swh_download_command:
How to download this dataset using
swh-datasets.- size:
Dataset size (unit included).
- slug:
Dataset slug in the website.
- tags:
A list of tags used to classify the dataset.
- type:
The type (orc, parquet, csv, …).
- url_safe_id:
A safe dataset ID used to build URLs and pages.
- variant:
Specific information about the graph variants.