swh.lister.core.simple_lister module¶
-
class
swh.lister.core.simple_lister.
SimpleLister
(override_config=None)[source]¶ Bases:
swh.lister.core.lister_base.ListerBase
- Lister* intermediate class for any service that follows the simple,
‘list in oneshot information’ pattern.
Client sends a request to list repositories in oneshot
Client receives structured (json/xml/etc) response with information and stores those in db
-
flush_packet_db
= 2¶ Number of iterations in-between write flushes of lister repositories to db (see fn:ingest_data).
-
ingest_data
(identifier, checks=False)[source]¶ - Rework the base ingest_data.
Request server endpoint which gives all in one go.
Simplify and filter response list of repositories. Inject repo information into local db. Queue loader tasks for linked repositories.
- Parameters
identifier – Resource identifier (unused)
checks (bool) – Additional checks required (unused)