How to update datasets

To add a new dataset or update an existing one you need to modify swh/dataportal/data/data.yml, increment the version number of the package and make a merge request with your changes.

Modify data.yml

Please refer to Data used to generate the website to understand the meaning of each dataset properties, the only required element is the unique dataset ID, composed of its export date and its type.

Increment version number

swh-dataportal uses the Calendar Versioning system to number its releases. The version number is stored in swh/dataportal/__init__.py and is managed by hatch-calver.

If it’s the 4th of August 2026 and the previous version number was 2026.07.21

hatch version patch
# __version__ is now 2026.07.21.1
hatch version release
# __version__ is now 2026.08.04

See Hatch versioning documentation <hatch-version> for more details about the version command.

Make a merge request

Create a Merge request <dataportal-mr> with your changes, check if the pipeline succeeds and request a review from one of our team member.

Once merged, a new git tag needs to be created to trigger a new package upload to pypi and a website update.