swh.loader.package.opam.loader module

class swh.loader.package.opam.loader.OpamPackageInfo(url: str, filename: Optional[str], author: swh.model.model.Person, committer: swh.model.model.Person, version: str, *, directory_extrinsic_metadata: List[swh.loader.package.loader.RawExtrinsicMetadataCore] = [])[source]

Bases: swh.loader.package.loader.BasePackageInfo

Method generated by attrs for class OpamPackageInfo.

swh.loader.package.opam.loader.opam_read(cmd: List[str], init_error_msg_if_any: Optional[str] = None) Optional[str][source]

This executes an opam command and returns the first line of the output.

  • cmd – Opam command to execute as a list of string

  • init_error_msg_if_any – Error message to raise in case a problem occurs during initialization

  • ValueError with the init_error_msg_if_any content in case stdout is not

  • consumable and the variable is provided with non empty value.


the first line of the executed command output

class swh.loader.package.opam.loader.OpamLoader(storage: swh.storage.interface.StorageInterface, url: str, opam_root: str, opam_instance: str, opam_url: str, opam_package: str, max_content_size: Optional[int] = None, initialize_opam_root: bool = False)[source]

Bases: swh.loader.package.loader.PackageLoader[swh.loader.package.opam.loader.OpamPackageInfo]

Load all versions of a given package in a given opam repository.

The state of the opam repository is stored in a directory called an opam root. This folder is a requisite for the opam binary to actually list information on package.

When initialize_opam_root is False (the default for production workers), the opam root must already have been configured outside of the loading process. If not an error is raised, thus failing the loading.

For standalone workers, initialize_opam_root must be set to True, so the ingestion can take care of installing the required opam root properly.

The remaining ingestion uses the opam binary to give the versions of the given package. Then, for each version, the loader uses the opam binary to list the tarball url to fetch and ingest.

Loader’s constructor. This raises exception if the minimal required

configuration is missing (cf. fn:check method).

  • storage – Storage instance

  • url – Origin url to load data from

visit_type: Optional[str] = 'opam'
get_package_dir() str[source]
get_package_name(version: str) str[source]
get_package_file(version: str) str[source]
get_versions() List[str][source]

First initialize the opam root directory if needed then start listing the package versions.

  • ValueError in case the lister is not able to determine the list of

  • versions or if the opam root directory is invalid.

get_default_version() str[source]

Return the most recent version of the package as default.

get_enclosed_single_line_field(field, version) Optional[str][source]
get_package_info(version: str) Iterator[Tuple[str, swh.loader.package.opam.loader.OpamPackageInfo]][source]
Given a release version of a package, retrieve the associated

package information for such version.


version – Package version


(branch name, package metadata)

build_revision(p_info: swh.loader.package.opam.loader.OpamPackageInfo, uncompressed_path: str, directory: bytes) Optional[swh.model.model.Revision][source]

Build the revision from the archive metadata (extrinsic artifact metadata) and the intrinsic metadata.

  • p_info – Package information

  • uncompressed_path – Artifact uncompressed path on disk


Revision object

visit_date: datetime.datetime
origin: Optional[Origin]
origin_metadata: Dict[str, Any]
loaded_snapshot_id: Optional[Sha1Git]