swh.scheduler.model module#

swh.scheduler.model.check_timestamptz(value) None[source]#

Checks the date has a timezone.

class swh.scheduler.model.BaseSchedulerModel[source]#

Bases: object

Base class for database-backed objects.

These database-backed objects are defined through attrs-based attributes that match the columns of the database 1:1. This is a (very) lightweight ORM.

These attrs-based attributes have metadata specific to the functionality expected from these fields in the database:

  • primary_key: the column is a primary key; it should be filtered out when doing an update of the object

  • auto_primary_key: the column is a primary key, which is automatically handled by the database. It will not be inserted to. This must be matched with a database-side default value.

  • auto_now_add: the column is a timestamp that is set to the current time when the object is inserted, and never updated afterwards. This must be matched with a database-side default value.

  • auto_now: the column is a timestamp that is set to the current time when the object is inserted or updated.

Method generated by attrs for class BaseSchedulerModel.

classmethod primary_key_columns() Tuple[str, ...][source]#

Get the primary key columns for this object type

classmethod select_columns() Tuple[str, ...][source]#

Get all the database columns needed for a select on this object type

classmethod insert_columns_and_metavars() Tuple[Tuple[str, ...], Tuple[str, ...]][source]#
Get the database columns and metavars needed for an insert or update on

this object type.

This implements support for the auto_* field metadata attributes.

class swh.scheduler.model.Lister(name: str, instance_name: str, id: UUID | None = None, current_state: Dict[str, Any] = _Nothing.NOTHING, created: datetime | None = None, updated: datetime | None = None)[source]#

Bases: BaseSchedulerModel

Method generated by attrs for class Lister.

class swh.scheduler.model.ListedOrigin(lister_id: UUID, url: str, visit_type: str, extra_loader_arguments: Dict[str, Any] = _Nothing.NOTHING, last_update: datetime | None = None, enabled: bool = True, first_seen: datetime | None = None, last_seen: datetime | None = None)[source]#

Bases: BaseSchedulerModel

Basic information about a listed origin, output by a lister

Method generated by attrs for class ListedOrigin.

class swh.scheduler.model.LastVisitStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#

Bases: Enum

successful = 'successful'#
failed = 'failed'#
not_found = 'not_found'#
swh.scheduler.model.convert_last_visit_status(s: None | str | LastVisitStatus) LastVisitStatus | None[source]#
class swh.scheduler.model.OriginVisitStats(url: str, visit_type: str, last_successful: datetime | None = None, last_visit: datetime | None = None, last_visit_status: None | str | LastVisitStatus = None, last_scheduled: datetime | None = None, last_snapshot: bytes | None = None, next_visit_queue_position: int | None = None, next_position_offset: int = 4, successive_visits: int = 1)[source]#

Bases: BaseSchedulerModel

Represents an aggregated origin visits view.

Method generated by attrs for class OriginVisitStats.

check_last_successful(attribute, value)[source]#
check_last_visit(attribute, value)[source]#
class swh.scheduler.model.SchedulerMetrics(lister_id: UUID, visit_type: str, last_update: datetime | None = None, origins_known: int = 0, origins_enabled: int = 0, origins_never_visited: int = 0, origins_with_pending_changes: int = 0)[source]#

Bases: BaseSchedulerModel

Metrics for the scheduler, aggregated by (lister_id, visit_type)

Method generated by attrs for class SchedulerMetrics.


Number of known (enabled or disabled) origins


Number of origins that were present in the latest listings


Number of enabled origins that have never been visited (according to the visit cache)


Number of enabled origins with known activity (recorded by a lister) since our last visit