swh.scheduler.celery_backend.config module

swh.scheduler.celery_backend.config.setup_log_handler(loglevel=None, logfile=None, format=None, colorize=None, log_console=None, log_journal=None, **kwargs)

Setup logging according to Software Heritage preferences.

We use the command-line loglevel for tasks only, as we never really care about the debug messages from celery.

swh.scheduler.celery_backend.config.setup_queues_and_tasks(sender, instance, **kwargs)

Signal called on worker start.

This automatically registers swh.scheduler.task.Task subclasses as available celery tasks.

This also subscribes the worker to the “implicit” per-task queues defined for these task classes.

swh.scheduler.celery_backend.config.on_worker_init(*args, **kwargs)

swh.scheduler.celery_backend.config.monotonic(state)

Get the current value for the monotonic clock

swh.scheduler.celery_backend.config.route_for_task(name, args, kwargs, options, task=None, **kw)

Route tasks according to the task_queue attribute in the task class

swh.scheduler.celery_backend.config.get_queue_stats(app, queue_name)

Get the statistics regarding a queue on the broker.

Parameters

queue_name – name of the queue to check

Returns a dictionary raw from the RabbitMQ management API; or None if the current configuration does not use RabbitMQ.

Interesting keys:

• Consumers (number of consumers for the queue)

• messages (number of messages in queue)

• messages_unacknowledged (number of messages currently being processed)

Documentation: https://www.rabbitmq.com/management.html#http-api

swh.scheduler.celery_backend.config.get_queue_length(app, queue_name)

Shortcut to get a queue’s length

swh.scheduler.celery_backend.config.register_task_class(app, name, cls)

Register a class-based task under the given name

swh.scheduler.celery_backend.config.build_app(config=None)