swh.web.common.converters module

swh.web.common.converters.fmap(f, data)[source]

Map f to data at each level.

This must keep the origin data structure type: - map -> map - dict -> dict - list -> list - None -> None

Parameters
  • f – function that expects one argument.

  • data – data to traverse to apply the f function. list, map, dict or bare value.

Returns

The same data-structure with modified values by the f function.

swh.web.common.converters.from_swh(dict_swh, hashess={}, bytess={}, dates={}, blacklist={}, removables_if_empty={}, empty_dict={}, empty_list={}, convert={}, convert_fn=<function <lambda>>)[source]

Convert from a swh dictionary to something reasonably json serializable.

Parameters
  • dict_swh – the origin dictionary needed to be transformed

  • hashess – list/set of keys representing hashes values (sha1, sha256, sha1_git, etc…) as bytes. Those need to be transformed in hexadecimal string

  • bytess – list/set of keys representing bytes values which needs to be decoded

  • blacklist – set of keys to filter out from the conversion

  • convert – set of keys whose associated values need to be converted using convert_fn

  • convert_fn – the conversion function to apply on the value of key in ‘convert’

The remaining keys are copied as is in the output.

Returns

dictionary equivalent as dict_swh only with its keys converted.

swh.web.common.converters.from_origin(origin: Dict[str, Any])swh.web.common.typing.OriginInfo[source]

Convert from a swh origin to an origin dictionary.

swh.web.common.converters.from_release(release)[source]

Convert from a swh release to a json serializable release dictionary.

Parameters
  • release (dict) –

    dictionary with keys:

    • id: identifier of the revision (sha1 in bytes)

    • revision: identifier of the revision the release points to (sha1 in bytes)

  • comment – release’s comment message (bytes)

  • name – release’s name (string)

  • author – release’s author identifier (swh’s id)

  • synthetic – the synthetic property (boolean)

Returns

Release dictionary with the following keys:

  • id: hexadecimal sha1 (string)

  • revision: hexadecimal sha1 (string)

  • comment: release’s comment message (string)

  • name: release’s name (string)

  • author: release’s author identifier (swh’s id)

  • synthetic: the synthetic property (boolean)

Return type

dict

class swh.web.common.converters.SWHMetadataEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

Bases: json.encoder.JSONEncoder

Special json encoder for metadata field which can contain bytes encoded value.

default(obj)[source]

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)
swh.web.common.converters.convert_revision_metadata(metadata)[source]

Convert json specific dict to a json serializable one.

swh.web.common.converters.from_revision(revision)[source]

Convert from a swh revision to a json serializable revision dictionary.

Parameters

revision (dict) –

dict with keys:

  • id: identifier of the revision (sha1 in bytes)

  • directory: identifier of the directory the revision points to (sha1 in bytes)

  • author_name, author_email: author’s revision name and email

  • committer_name, committer_email: committer’s revision name and email

  • message: revision’s message

  • date, date_offset: revision’s author date

  • committer_date, committer_date_offset: revision’s commit date

  • parents: list of parents for such revision

  • synthetic: revision’s property nature

  • type: revision’s type (git, tar or dsc at the moment)

  • metadata: if the revision is synthetic, this can reference dynamic properties.

Returns

Revision dictionary with the same keys as inputs, except:

  • sha1s are in hexadecimal strings (id, directory)

  • bytes are decoded in string (author_name, committer_name, author_email, committer_email)

Remaining keys are left as is

Return type

dict

swh.web.common.converters.from_content(content)[source]

Convert swh content to serializable content dictionary.

swh.web.common.converters.from_person(person)[source]

Convert swh person to serializable person dictionary.

swh.web.common.converters.from_origin_visit(visit: Dict[str, Any])swh.web.common.typing.OriginVisitInfo[source]

Convert swh origin_visit to serializable origin_visit dictionary.

swh.web.common.converters.from_snapshot(snapshot)[source]

Convert swh snapshot to serializable snapshot dictionary.

swh.web.common.converters.from_directory_entry(dir_entry)[source]

Convert swh directory to serializable directory dictionary.

swh.web.common.converters.from_filetype(content_entry)[source]

Convert swh content to serializable dictionary containing keys ‘id’, ‘encoding’, and ‘mimetype’.