swh.lister.gnu.lister module

class swh.lister.gnu.lister.GNULister(*args, **kwargs)[source]

Bases: swh.lister.core.simple_lister.SimpleLister

MODEL

alias of swh.lister.gnu.models.GNUModel

LISTER_NAME = 'gnu'
instance = 'gnu'
task_dict(origin_type, origin_url, **kwargs)[source]

Return task format dict

This is overridden from the lister_base as more information is needed for the ingestion task creation.

This creates tasks with args and kwargs set, for example:

args:
kwargs: {
    'url': 'https://ftp.gnu.org/gnu/3dldf/',
    'artifacts': [{
        'url': 'https://...',
        'time': '2003-12-09T21:43:20+00:00',
        'length': 128,
        'version': '1.0.1',
        'filename': 'something-1.0.1.tar.gz',
    },
    ...
    ]
}
safely_issue_request(identifier)[source]

Bypass the implementation. It’s now the GNUTree which deals with querying the gnu mirror.

As an implementation detail, we cannot change simply the base SimpleLister as other implementation still uses it. This shall be part of another refactoring pass.

list_packages(response)[source]
List the actual gnu origins (package name) with their name, url and

associated tarballs.

Parameters

response – Unused

Returns

List of packages name, url, last modification time:

[
    {
        'name': '3dldf',
        'url': 'https://ftp.gnu.org/gnu/3dldf/',
        'time_modified': '2003-12-09T20:43:20+00:00'
    },
    {
        'name': '8sync',
        'url': 'https://ftp.gnu.org/gnu/8sync/',
        'time_modified': '2016-12-06T02:37:10+00:00'
    },
    ...
]

get_model_from_repo(repo)[source]

Transform from repository representation to model