swh.scheduler.model module

swh.scheduler.model.check_timestamptz(value) → None

Checks the date has a timezone.

class swh.scheduler.model.BaseSchedulerModel

Bases: object

Base class for database-backed objects.

These database-backed objects are defined through attrs-based attributes that match the columns of the database 1:1. This is a (very) lightweight ORM.

These attrs-based attributes have metadata specific to the functionality expected from these fields in the database:

• primary_key: the column is a primary key; it should be filtered out when doing an update of the object

• auto_primary_key: the column is a primary key, which is automatically handled by the database. It will not be inserted to. This must be matched with a database-side default value.

• auto_now_add: the column is a timestamp that is set to the current time when the object is inserted, and never updated afterwards. This must be matched with a database-side default value.

• auto_now: the column is a timestamp that is set to the current time when the object is inserted or updated.

Method generated by attrs for class BaseSchedulerModel.

classmethod primary_key_columns() → Tuple[str, ...]

Get the primary key columns for this object type

classmethod select_columns() → Tuple[str, ...]

Get all the database columns needed for a select on this object type

classmethod insert_columns_and_metavars() → Tuple[Tuple[str, ...], Tuple[str, ...]]

Get the database columns and metavars needed for an insert or update on

this object type.

This implements support for the auto_* field metadata attributes.

class swh.scheduler.model.Lister(name: str, instance_name: str, id: Optional[uuid.UUID] = None, current_state: Dict[str, Any] = NOTHING, created: Optional[datetime.datetime] = None, updated: Optional[datetime.datetime] = None)

Bases: swh.scheduler.model.BaseSchedulerModel

Method generated by attrs for class Lister.

class swh.scheduler.model.ListedOrigin(lister_id: uuid.UUID, url: str, visit_type: str, extra_loader_arguments: Dict[str, Any] = NOTHING, last_update: Optional[datetime.datetime] = None, enabled: bool = True, first_seen: Optional[datetime.datetime] = None, last_seen: Optional[datetime.datetime] = None)

Bases: swh.scheduler.model.BaseSchedulerModel

Basic information about a listed origin, output by a lister

Method generated by attrs for class ListedOrigin.

as_task_dict()

class swh.scheduler.model.LastVisitStatus(value)

Bases: enum.Enum

An enumeration.

successful = 'successful'

failed = 'failed'

not_found = 'not_found'

swh.scheduler.model.convert_last_visit_status(s: Union[None, str, swh.scheduler.model.LastVisitStatus]) → Optional[swh.scheduler.model.LastVisitStatus]

class swh.scheduler.model.OriginVisitStats(url: str, visit_type: str, last_successful: Optional[datetime.datetime] = None, last_visit: Optional[datetime.datetime] = None, last_visit_status: Union[None, str, swh.scheduler.model.LastVisitStatus] = None, last_scheduled: Optional[datetime.datetime] = None, last_snapshot: Optional[bytes] = None, next_visit_queue_position: Optional[datetime.datetime] = None, next_position_offset: int = 4, successive_visits: int = 1)

Bases: swh.scheduler.model.BaseSchedulerModel

Represents an aggregated origin visits view.

Method generated by attrs for class OriginVisitStats.

url

visit_type

last_successful

last_visit

last_visit_status

last_scheduled

last_snapshot

next_visit_queue_position

next_position_offset

successive_visits

check_last_successful(attribute, value)

check_last_visit(attribute, value)

check_next_visit_queue_position(attribute, value)

class swh.scheduler.model.SchedulerMetrics(lister_id: uuid.UUID, visit_type: str, last_update: Optional[datetime.datetime] = None, origins_known: int = 0, origins_enabled: int = 0, origins_never_visited: int = 0, origins_with_pending_changes: int = 0)

Bases: swh.scheduler.model.BaseSchedulerModel

Metrics for the scheduler, aggregated by (lister_id, visit_type)

Method generated by attrs for class SchedulerMetrics.

lister_id

visit_type

last_update

origins_known

Number of known (enabled or disabled) origins

origins_enabled

Number of origins that were present in the latest listings

origins_never_visited

Number of enabled origins that have never been visited (according to the visit cache)

origins_with_pending_changes

Number of enabled origins with known activity (recorded by a lister) since our last visit