Source code for swh.lister.dlang

# Copyright (C) 2023  The Software Heritage developers
# See the AUTHORS file at the top-level directory of this distribution
# License: GNU General Public License version 3, or any later version
# See top-level LICENSE file for more information


"""
Dlang lister
=============

D is a general-purpose programming language with static typing, systems-level access,
and C-like syntax.

The `Dlang`_ lister list origins from its packages manager registry `DUB`_.

The registry provides an `http api endpoint`_ that helps in getting the packages index
with name, url, versions and dates.

As of July 2023 `DUB`_ list 2364 package names.

Origins retrieving strategy
---------------------------

To build a list of origins we make a GET request to an `http api endpoint`_ that returns
a JSON-formatted array of objects.
The origin url for each package is constructed with the information of corresponding
`repository` entry which represents Git based projects hosted on Github, GitLab or
Bitbucket.

Page listing
------------

There is only one page listing all origins url.

Origins from page
-----------------

The lister is stateless and yields all origins url from one page. It is a list of package
url with last update information.

Running tests
-------------

Activate the virtualenv and run from within swh-lister directory::

   pytest -s -vv --log-cli-level=DEBUG swh/lister/dlang/tests

Testing with Docker
-------------------

Change directory to swh/docker then launch the docker environment::

   docker compose up -d

Then schedule a dlang listing task::

   docker compose exec swh-scheduler swh scheduler task add -p oneshot list-dlang

You can follow lister execution by displaying logs of swh-lister service::

   docker compose logs -f swh-lister

.. _Dlang: https://dlang.org/
.. _DUB: https://code.dlang.org/
.. _http api endpoint: https://code.dlang.org/api/packages/dump"
"""



[docs]
def register():
    from .lister import DlangLister

    return {
        "lister": DlangLister,
        "task_modules": ["%s.tasks" % __name__],
    }