swh.scheduler.celery_backend.config module
- swh.scheduler.celery_backend.config.setup_log_handler(loglevel=None, logfile=None, format=None, colorize=None, log_console=None, log_journal=None, **kwargs)[source]
Setup logging according to Software Heritage preferences.
We use the command-line loglevel for tasks only, as we never really care about the debug messages from celery.
- swh.scheduler.celery_backend.config.setup_queues_and_tasks(sender, instance, **kwargs)[source]
Signal called on worker start.
This automatically registers swh.scheduler.task.Task subclasses as available celery tasks.
This also subscribes the worker to the “implicit” per-task queues defined for these task classes.
- swh.scheduler.celery_backend.config.monotonic(state)[source]
Get the current value for the monotonic clock
- swh.scheduler.celery_backend.config.route_for_task(name, args, kwargs, options, task=None, **kw)[source]
Route tasks according to the task_queue attribute in the task class
- swh.scheduler.celery_backend.config.get_queue_stats(app, queue_name)[source]
Get the statistics regarding a queue on the broker.
- Parameters
queue_name – name of the queue to check
Returns a dictionary raw from the RabbitMQ management API; or None if the current configuration does not use RabbitMQ.
- Interesting keys:
Consumers (number of consumers for the queue)
messages (number of messages in queue)
messages_unacknowledged (number of messages currently being processed)
Documentation: https://www.rabbitmq.com/management.html#http-api
- swh.scheduler.celery_backend.config.get_queue_length(app, queue_name)[source]
Shortcut to get a queue’s length
- swh.scheduler.celery_backend.config.get_available_slots(app, queue_name: str, max_length: Optional[int])[source]
Get the number of tasks that can be sent to queue_name, when the queue is limited to max_length.
- Returns
The number of available slots in the queue. That result should be positive.