swh.loader.package.pypi.loader module

class swh.loader.package.pypi.loader.PyPIPackageInfo(url: str, filename: Optional[str], raw_info: Dict[str, Any], comment_text: Optional[str], sha256: str, upload_time: str, *, directory_extrinsic_metadata: List[swh.loader.package.loader.RawExtrinsicMetadataCore] = [])[source]

Bases: swh.loader.package.loader.BasePackageInfo

Method generated by attrs for class PyPIPackageInfo.

classmethod from_metadata(metadata: Dict[str, Any]) swh.loader.package.pypi.loader.PyPIPackageInfo[source]
extid() Tuple[str, bytes][source]

Returns a unique intrinsic identifier of this package info, or None if this package info is not ‘deduplicatable’ (meaning that we will always load it, instead of checking the ExtID storage to see if we already did)

class swh.loader.package.pypi.loader.PyPILoader(storage: swh.storage.interface.StorageInterface, url: str, max_content_size: Optional[int] = None)[source]

Bases: swh.loader.package.loader.PackageLoader[swh.loader.package.pypi.loader.PyPIPackageInfo]

Load pypi origin’s artifact releases into swh archive.

Loader’s constructor. This raises exception if the minimal required

configuration is missing (cf. fn:check method).

Parameters
  • storage – Storage instance

  • url – Origin url to load data from

visit_type: Optional[str] = 'pypi'
info() Dict[source]

Return the project metadata information (fetched from pypi registry)

get_versions() Sequence[str][source]

Return the list of all published package versions.

Raises

classswh.loader.exception.NotFound error when failing to read the published package versions.

Returns

Sequence of published versions

get_default_version() str[source]

Retrieve the latest release version if any.

Returns

Latest version

get_metadata_authority()[source]

For package loaders that get extrinsic metadata, returns the authority the metadata are coming from.

get_package_info(version: str) Iterator[Tuple[str, swh.loader.package.pypi.loader.PyPIPackageInfo]][source]
Given a release version of a package, retrieve the associated

package information for such version.

Parameters

version – Package version

Returns

(branch name, package metadata)

build_revision(p_info: swh.loader.package.pypi.loader.PyPIPackageInfo, uncompressed_path: str, directory: bytes) Optional[swh.model.model.Revision][source]

Build the revision from the archive metadata (extrinsic artifact metadata) and the intrinsic metadata.

Parameters
  • p_info – Package information

  • uncompressed_path – Artifact uncompressed path on disk

Returns

Revision object

visit_date: datetime.datetime
origin: Optional[Origin]
origin_metadata: Dict[str, Any]
loaded_snapshot_id: Optional[Sha1Git]
swh.loader.package.pypi.loader.pypi_api_url(url: str) str[source]

Compute api url from a project url

Parameters
  • url (str) – PyPI instance’s url (e.g: https://pypi.org/project/requests)

  • (e.g (This deals with correctly transforming the project's api url) –

  • https – //pypi.org/pypi/requests/json)

Returns

api url

swh.loader.package.pypi.loader.extract_intrinsic_metadata(dir_path: str) Dict[source]
Given an uncompressed path holding the pkginfo file, returns a

pkginfo parsed structure as a dict.

The release artifact contains at their root one folder. For example: $ tar tvf zprint-0.0.6.tar.gz drwxr-xr-x root/root 0 2018-08-22 11:01 zprint-0.0.6/ …

Parameters

dir_path (str) – Path to the uncompressed directory representing a release artifact from pypi.

Returns

the pkginfo parsed structure as a dict if any or None if none was present.

swh.loader.package.pypi.loader.author(data: Dict) swh.model.model.Person[source]
Given a dict of project/release artifact information (coming from

PyPI), returns an author subset.

Parameters

data (dict) – Representing either artifact information or release information.

Returns

swh-model dict representing a person.