swh.web.common.converters module

swh.web.common.converters.fmap(f, data)

Map f to data at each level.

This must keep the origin data structure type: - map -> map - dict -> dict - list -> list - None -> None

Parameters

• f – function that expects one argument.

• data – data to traverse to apply the f function. list, map, dict or bare value.

Returns

The same data-structure with modified values by the f function.

swh.web.common.converters.from_swh(dict_swh, hashess={}, bytess={}, dates={}, blacklist={}, removables_if_empty={}, empty_dict={}, empty_list={}, convert={}, convert_fn=<function <lambda>>)

Convert from a swh dictionary to something reasonably json serializable.

Parameters

• dict_swh – the origin dictionary needed to be transformed

• hashess – list/set of keys representing hashes values (sha1, sha256, sha1_git, etc…) as bytes. Those need to be transformed in hexadecimal string

• bytess – list/set of keys representing bytes values which needs to be decoded

• blacklist – set of keys to filter out from the conversion

• convert – set of keys whose associated values need to be converted using convert_fn

• convert_fn – the conversion function to apply on the value of key in ‘convert’

The remaining keys are copied as is in the output.

Returns

dictionary equivalent as dict_swh only with its keys converted.

swh.web.common.converters.from_origin(origin: Dict[str, Any]) → swh.web.common.typing.OriginInfo

Convert from a swh origin to an origin dictionary.

swh.web.common.converters.from_release(release: swh.model.model.Release) → Dict[str, Any]

Convert from a swh release to a json serializable release dictionary.

Parameters

release – A release model object

Returns

release dictionary with the following keys

• id: hexadecimal sha1 (string)

• revision: hexadecimal sha1 (string)

• comment: release’s comment message (string)

• name: release’s name (string)

• author: release’s author identifier (swh’s id)

• synthetic: the synthetic property (boolean)

class swh.web.common.converters.SWHMetadataEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: json.encoder.JSONEncoder

Special json encoder for metadata field which can contain bytes encoded value.

Constructor for JSONEncoder, with sensible defaults.

If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.

If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.

If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an OverflowError). Otherwise, no such check takes place.

If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.

If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.

If specified, separators should be an (item_separator, key_separator) tuple. The default is (‘, ‘, ‘: ‘) if indent is None and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.

If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a TypeError.

default(obj)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

swh.web.common.converters.convert_revision_metadata(metadata)

Convert json specific dict to a json serializable one.

swh.web.common.converters.from_revision(revision: Union[Dict[str, Any], swh.model.model.Revision]) → Dict[str, Any]

Convert swh revision model object to a json serializable revision dictionary.

Parameters

revision – revision model object

Returns

Revision dictionary with the same keys as inputs, except:

• sha1s are in hexadecimal strings (id, directory)

• bytes are decoded in string (author_name, committer_name, author_email, committer_email)

Remaining keys are left as is

Return type

dict

swh.web.common.converters.from_content(content)

Convert swh content to serializable content dictionary.

swh.web.common.converters.from_person(person)

Convert swh person to serializable person dictionary.

swh.web.common.converters.from_origin_visit(visit: Dict[str, Any]) → swh.web.common.typing.OriginVisitInfo

Convert swh origin_visit to serializable origin_visit dictionary.

swh.web.common.converters.from_snapshot(snapshot)

Convert swh snapshot to serializable (partial) snapshot dictionary.

swh.web.common.converters.from_partial_branches(branches: swh.storage.interface.PartialBranches)

Convert PartialBranches to serializable partial snapshot dictionary

swh.web.common.converters.from_directory_entry(dir_entry)

Convert swh directory to serializable directory dictionary.

swh.web.common.converters.from_filetype(content_entry)

Convert swh content to serializable dictionary containing keys ‘id’, ‘encoding’, and ‘mimetype’.