swh.lister package

Subpackages

Submodules

swh.lister.cli module

swh.lister.cli.get_lister(lister_name, db_url=None, **conf)[source]

Instantiate a lister given its name.

Parameters:
  • lister_name (str) – Lister’s name
  • conf (dict) – Configuration dict (lister db cnx, policy, priority…)
Returns:

Tuple (instantiated lister, drop_tables function, init schema function, insert minimum data function)

swh.lister.cli.ensure_task_type(task_type, swhtask, task_config, scheduler)[source]

Ensure a task-type is known by the scheduler

Parameters:
  • task_type (str) – the type of the task to check/insert (correspond to the ‘type’ field in the db)
  • swhtask (SWHTask) – the SWHTask instance the task-type correspond to
  • task_config (dict) – a dict with specific/overloaded values for the task-type to be created
  • scheduler – the scheduler object used to access the scheduler db

swh.lister.utils module

swh.lister.utils.split_range(total_pages, nb_pages)[source]

Module contents