swh.loader.package.crates.loader module#

class swh.loader.package.crates.loader.ExtrinsicPackageMetadata[source]#

Bases: TypedDict

Data structure for package extrinsic metadata pulled from http api endpoint.

We set only the keys we need according to what is available when querying https://crates.io/api/v1/crates/<name>, where name is the name of the crate package (see JSON response example at https://crates.io/api/v1/crates/hg-core).

Usage example:

e_metadata = ExtrinsicPackageMetadata(**self.info())

categories: List[Dict[Any, Any]]#: Related categories

crate: Dict[Any, Any]#: Crate project information

keywords: List[Any]#: Keywords

versions: List[Dict[Any, Any]]#: A list of released versions for a crate

class swh.loader.package.crates.loader.ExtrinsicVersionPackageMetadata[source]#

Bases: TypedDict

Data structure for specific package version extrinsic metadata, pulled from http api endpoint.

Similar to ExtrinsicPackageMetadata in its usage, but we flatten the data related to a specific version.

crate: str#: The package name

crate_size: int#: The package size

created_at: str#: First released at

downloads: str#: Number of downloads

license: str#: Package license

num: str#: Package version

published_by: Dict[Any, Any]#: Publishers information

updated_at: str#: Last update

yanked: bool#: Is that version yanked? (yanked means release-level deprecation)

class swh.loader.package.crates.loader.IntrinsicPackageMetadata[source]#

Bases: TypedDict

Data structure for specific package version intrinsic metadata.

Data is extracted from the crate package’s .toml file. Then the data of the ‘package’ entry is flattened.

Cargo.toml file content example:

[package]
name = "hg-core"
version = "0.0.1"
authors = ["Georges Racinet <georges.racinet@octobus.net>"]
description = "Mercurial pure Rust core library, with no assumption on
Python bindings (FFI)"
homepage = "https://mercurial-scm.org"
license = "GPL-2.0-or-later"
repository = "https://www.mercurial-scm.org/repo/hg"

[lib]
name = "hg"
[dev-dependencies.rand]
version = "~0.6"

[dev-dependencies.rand_pcg]
version = "~0.1"

Parameters:: toml – toml object

name: str#: The package name

version: str#: Package version

authors: List[str]#: Authors

description: str#: Package and release description

homepage: str#: Homepage of the project

license: str#: Package license

repository: str#: Source code repository

class swh.loader.package.crates.loader.CratesPackageInfo(url: str, filename: str | None, name: str, version: str, e_metadata: Dict[str, Any] = NOTHING, e_metadata_version: Dict[str, Any] = NOTHING, i_metadata: Dict[str, Any] = NOTHING, *, directory_extrinsic_metadata: List[RawExtrinsicMetadataCore] = [], checksums: Dict[str, str] = {})[source]#

Bases: BasePackageInfo

Method generated by attrs for class CratesPackageInfo.

name#: Name of the package

version#: Current version

e_metadata: Dict[str, Any]#: Extrinsic package metadata, common to all versions

e_metadata_version: Dict[str, Any]#: Extrinsic package metadata specific to a version

i_metadata: Dict[str, Any]#: Intrinsic metadata of the current package version

swh.loader.package.crates.loader.extract_intrinsic_metadata(dir_path: Path) → Dict[str, Any][source]#

Extract intrinsic metadata from Cargo.toml file at dir_path.

Each crate archive has a Cargo.toml at the root of the archive.

Parameters:: dir_path – A directory on disk where a Cargo.toml must be present
Returns:: A dict mapping from toml parser

swh.loader.package.crates.loader.extract_author(p_info: CratesPackageInfo) → Person[source]#

Extract package author from intrinsic metadata and return it as a Person model.

Parameters:: p_info – CratesPackageInfo that should contains i_metadata entries
Returns:: Only one author (Person) of the package. Currently limited by internal detail of the swh stack (see T3887).

swh.loader.package.crates.loader.extract_description(p_info: CratesPackageInfo) → str[source]#

Extract package description from intrinsic metadata and return it as a string.

Parameters:: p_info – CratesPackageInfo that should contains i_metadata and entries
Returns:: Package description from metadata.

class swh.loader.package.crates.loader.CratesLoader(storage: StorageInterface, url: str, artifacts: List[Dict[str, Any]], **kwargs)[source]#

Bases: PackageLoader[CratesPackageInfo]

Load Crates package origins into swh archive.

Constructor

Parameters:

url – Origin url, (e.g. https://crates.io/api/v1/crates/<package_name>)
artifacts –
A list of dict listing all existing released versions for a package (Usually set with crates lister extra_loader_arguments). Each line is a dict that should have an url (where to download package specific version) and a version entry.

Example:
```
[
    {
        "version": <version>,
        "url": "https://static.crates.io/crates/<package_name>/<package_name>-<version>.crate",
    }
]
```

visit_type: str = 'crates'#

info() → Dict[source]#: Parse http api json response and return the crate metadata information as a Dict.

get_versions() → Sequence[str][source]#

Get all released versions of a crate

Returns:

A sequence of versions

Example:

["0.1.1", "0.10.2"]

get_default_version() → str[source]#

Get the newest release version of a crate

Returns:

A string representing a version

Example:

"0.1.2"

get_package_info(version: str) → Iterator[Tuple[str, CratesPackageInfo]][source]#

Get release name and package information from version

Parameters:: version – crate version (e.g: “0.1.0”)
Returns:: Iterator of tuple (release_name, p_info)

build_release(p_info: CratesPackageInfo, uncompressed_path: str, directory: bytes) → Release | None[source]#

Build the release from the archive metadata (extrinsic artifact metadata) and the intrinsic metadata.

Parameters:

p_info – Package information
uncompressed_path – Artifact uncompressed path on disk