swh.lister.aur.lister module#

class swh.lister.aur.lister.AurLister(scheduler: SchedulerInterface, url: str = 'https://aur.archlinux.org', instance: str = 'aur', credentials: Dict[str, Dict[str, List[Dict[str, str]]]] | None = None, max_origins_per_page: int | None = None, max_pages: int | None = None, enable_origins: bool = True)[source]#

Bases: StatelessLister[Dict[str, Any]]

List Arch User Repository (AUR) origins.

Given an url (used as a base url, default is ‘https://aur.archlinux.org’), download a ‘packages-meta-v1.json.gz’ which contains a json file listing all existing packages definitions.

Each entry describes the latest released version of a package. The origin url for a package is built using ‘pkgname’ and corresponds to a git repository.

An rpc api exists but it is recommended to save bandwidth so it’s not used. See https://lists.archlinux.org/pipermail/aur-general/2021-November/036659.html for more on this.

LISTER_NAME: str = 'aur'#
VISIT_TYPE = 'aur'#
INSTANCE = 'aur'#
BASE_URL = 'https://aur.archlinux.org'#
DEFAULT_PACKAGES_INDEX_URL = '{base_url}/packages-meta-v1.json.gz'#
PACKAGE_VCS_URL_PATTERN = '{base_url}/{pkgname}.git'#
PACKAGE_SNAPSHOT_URL_PATTERN = '{base_url}/cgit/aur.git/snapshot/{pkgname}.tar.gz'#
ORIGIN_URL_PATTERN = '{base_url}/packages/{pkgname}'#
download_packages_index() List[Dict[str, Any]][source]#

Build an url based on self.DEFAULT_PACKAGES_INDEX_URL format string, and download the archive to self.DESTINATION_PATH


a directory Path where the archive has been downloaded to.

get_pages() Iterator[Dict[str, Any]][source]#

Yield an iterator which returns ‘page’

Each page corresponds to a package with a ‘version’, an ‘url’ for a Git repository, a ‘project_url’ which represents the upstream project url and a canonical ‘snapshot_url’ from which a tar.gz archive of the package can be downloaded.

get_origins_from_page(origin: Dict[str, Any]) Iterator[ListedOrigin][source]#

Iterate on all pages and yield ListedOrigin instances. It uses the vcs (Git) url as an origin and adds artifacts and aur_metadata entries to ‘extra_loader_arguments’.

artifacts describe the file to download and aur_metadata store some metadata that can be useful for the loader.