Data used to generate the website
=================================

The data used to build the website are all coming from a single source of
truth, which is ``swh/dataportal/data/data.yml``.

Each entry in this file is the name of the dataset it relates to. Additional fields can be added, such as statistics, specific title or comments.

As such, it is this file that is to be modified when adding a new dataset, see :doc:`../howto/update`.


Specifications
--------------

.. code-block:: yaml

   dataset-unique-id (YYYY-MM-DD-{optional slug}-type):
     title: foo
     size: 2 TiB
     ...

The only required element is the ID of the dataset.

dataset-unique-id:
   an identifier composed of the export date, an optional slug (for example
   popular-10-shell) an the dataset type (MUST BE one of orc, parquet, csv, compressed,
   contents, popular_contents, license, citation, provenance/all, provenance/heads, topology, path_counts, descendent_counts, vulnerabilities, digestmap), as defined in
   ``DatasetType``

title:
   overwrite the default dataset title (see :py:module:swh.dataportal.models)

description:
   overwrite the default dataset description (see :py:module:swh.dataportal.models)

comments:
   markdown comment about the dataset

teaser_of:
   The ID of the dataset this dataset is a teaser of, if applicable.

deprecated:
   True when the dataset is deprecated (the ``comments`` field should explain why).

derived_of:
   The ID of the dataset this dataset is derived from, if applicable.

size:
   Dataset size (unit included).