swh.lister.core.simple_lister module

class swh.lister.core.simple_lister.SimpleLister(override_config=None)

Bases: swh.lister.core.lister_base.ListerBase

Lister* intermediate class for any service that follows the simple,

‘list in oneshot information’ pattern.

• Client sends a request to list repositories in oneshot

• Client receives structured (json/xml/etc) response with information and stores those in db

flush_packet_db = 2

Number of iterations in-between write flushes of lister repositories to db (see fn:ingest_data).

list_packages(response: Any) → List[Any]

Listing packages method.

ingest_data(identifier, checks=False)

Rework the base ingest_data.

Request server endpoint which gives all in one go.

Simplify and filter response list of repositories. Inject repo information into local db. Queue loader tasks for linked repositories.

Parameters

• identifier – Resource identifier (unused)

• checks (bool) – Additional checks required (unused)

transport_response_simplified(response)

Transform response to list for model manipulation

run()

Query the server which answers in one query. Stores the

information, dropping actual redundant information we already have.

Returns

nothing