How to update datasets
======================

To add a new dataset or update an existing one you need to modify
``swh/dataportal/data/data.yml``, increment the version number of the package and make
a merge request with your changes.


Modify data.yml
---------------

Please refer to :doc:`../references/data` to understand the meaning of each dataset properties, the
only required element is the **unique** dataset ID, composed of its export date and its
type.


Increment version number
------------------------

swh-dataportal uses the `Calendar Versioning system <calver>`_ to number its releases.
The version number is stored in ``swh/dataportal/__init__.py`` and is managed by
`hatch-calver`_.

If it's the 4th of August 2026 and the previous version number was ``2026.07.21``

.. code-block:: console

   hatch version patch
   # __version__ is now 2026.07.21.1
   hatch version release
   # __version__ is now 2026.08.04


See `Hatch versioning documentation <hatch-version>` for more details about the
``version`` command.

.. _calver: https://calver.org/
.. _hatch-version: https://hatch.pypa.io/latest/version/#updating
.. _hatch-calver: https://github.com/akx/hatch-calver


Make a merge request
--------------------

Create a `Merge request <dataportal-mr>` with your changes, check if the pipeline succeeds and request a review from one of our team member.

Once merged, a new git tag needs to be created to trigger a new package upload to pypi
and a website update.

.. _dataportal-mr: https://gitlab.softwareheritage.org/swh/devel/swh-dataportal/-/merge_requests