swh.loader.package.pypi.loader module

class swh.loader.package.pypi.loader.PyPILoader(url)[source]

Bases: swh.loader.package.loader.PackageLoader

Load pypi origin’s artifact releases into swh archive.

visit_type = 'pypi'
property info

Return the project metadata information (fetched from pypi registry)

get_versions() → Sequence[str][source]

Return the list of all published package versions.

Returns

Sequence of published versions

get_default_version() → str[source]

Retrieve the latest release version if any.

Returns

Latest version

get_package_info(version: str) → Generator[Tuple[str, Mapping[str, Any]], None, None][source]
Given a release version of a package, retrieve the associated

package information for such version.

Parameters

version – Package version

Returns

(branch name, package metadata)

resolve_revision_from(known_artifacts: Dict, artifact_metadata: Dict) → Optional[bytes][source]

Resolve the revision from a snapshot and an artifact metadata dict.

If the artifact has already been downloaded, this will return the existing revision targeting that uncompressed artifact directory. Otherwise, this returns None.

Parameters
  • snapshot – Snapshot

  • artifact_metadata – Information dict

Returns

None or revision identifier

build_revision(a_metadata: Dict, uncompressed_path: str, directory: bytes) → Optional[swh.model.model.Revision][source]

Build the revision from the archive metadata (extrinsic artifact metadata) and the intrinsic metadata.

Parameters
  • a_metadata – Artifact metadata

  • uncompressed_path – Artifact uncompressed path on disk

Returns

SWH data dict

swh.loader.package.pypi.loader.artifact_to_revision_id(known_artifacts: Dict, artifact_metadata: Dict) → Optional[bytes][source]

Given metadata artifact, solves the associated revision id.

The following code allows to deal with 2 metadata formats (column metadata in ‘revision’)

  • old format sample:

    {
        'original_artifact': {
            'sha256': '6975816f2c5ad4046acc676ba112f2fff945b01522d63948531f11f11e0892ec',  # noqa
            ...
        },
        ...
    }
    
  • new format sample:

    {
        'original_artifact': [{
            'checksums': {
                'sha256': '6975816f2c5ad4046acc676ba112f2fff945b01522d63948531f11f11e0892ec',  # noqa
                ...
            },
        }],
        ...
    }
    
swh.loader.package.pypi.loader.pypi_api_url(url: str) → str[source]

Compute api url from a project url

Parameters
  • url (str) – PyPI instance’s url (e.g: https://pypi.org/project/requests)

  • deals with correctly transforming the project's api url (e.g (This) –

  • https – //pypi.org/pypi/requests/json)

Returns

api url

swh.loader.package.pypi.loader.extract_intrinsic_metadata(dir_path: str) → Dict[source]
Given an uncompressed path holding the pkginfo file, returns a

pkginfo parsed structure as a dict.

The release artifact contains at their root one folder. For example: $ tar tvf zprint-0.0.6.tar.gz drwxr-xr-x root/root 0 2018-08-22 11:01 zprint-0.0.6/ …

Parameters

dir_path (str) – Path to the uncompressed directory representing a release artifact from pypi.

Returns

the pkginfo parsed structure as a dict if any or None if none was present.

swh.loader.package.pypi.loader.author(data: Dict)swh.model.model.Person[source]
Given a dict of project/release artifact information (coming from

PyPI), returns an author subset.

Parameters

data (dict) – Representing either artifact information or release information.

Returns

swh-model dict representing a person.