datasets.json export
====================

The `datasets.json <export>`_ file contains a *machine readable* list of all the
datasets published on the website.

.. _export: https://datasets.softwareheritage.org/datasets.json


Specifications
--------------

id:
   A unique internal ID.

title:
   This dataset's title.

absolute_url
   Permalink to the details page.

anchor_url
   Anchor link to access the dataset in its category page.

annex_url
   URL to the dataset folder in the Annex.

category:
   The category (compressed, columnar, etc.).

citations:
   A list of BibTeX file to use when referencing the dataset.

description:
   Generic information about this dataset.

comments:
   Extra information related to this dataset.

tables:
   List of tables contained in this dataset.

date:
   The export date of the archive this dataset is based on, format: Y-m-d.

deprecated:
   True when the dataset is deprecated (the ``comments`` field should explain why).

derived_of:
   The ID of the dataset this dataset is derived from, if applicable.

teaser_of:
   The ID of the dataset this dataset is a teaser of, if applicable.

group:
   Indicates if this graph is a complete one or a teaser/derivation.

hosted_on_annex:
   True if the dataset is available in the Annex

license:
   SPDX short ID of this dataset license.

hosted_on_s3:
   True if the dataset is available in S3.

s3_url:
   S3 download URL.

s3_download_command:
   How to download this dataset using ``awscli``.

swh_download_command:
   How to download this dataset using ``swh-datasets``.

size:
   Dataset size (unit included).

slug:
   Dataset slug in the website.

tags:
   A list of tags used to classify the dataset.

type:
   The type (orc, parquet, csv, ...).

url_safe_id:
   A safe dataset ID used to build URLs and pages.

variant:
   Specific information about the graph variants.