swh.model.model module

exception swh.model.model.MissingData

Bases: Exception

Raised by Content.with_data when it has no way of fetching the data (but not when fetching the data fails).

swh.model.model.freeze_optional_dict(d: Union[None, Dict[KT, VT], swh.model.collections.ImmutableDict[KT, VT]]) → Optional[swh.model.collections.ImmutableDict[KT, VT]]

swh.model.model.dictify(value)

Helper function used by BaseModel.to_dict()

class swh.model.model.BaseModel

Bases: object

Base class for SWH model classes.

Provides serialization/deserialization to/from Python dictionaries, that are suitable for JSON/msgpack-like formats.

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

anonymize() → Optional[ModelType]

Returns an anonymized version of the object, if needed.

If the object model does not need/support anonymization, returns None.

class swh.model.model.HashableObject

Bases: object

Mixin to automatically compute object identifier hash when the associated model is instantiated.

abstract static compute_hash(object_dict)

Derived model classes must implement this to compute the object hash from its dict representation.

class swh.model.model.Person(fullname: bytes, name: Optional[bytes], email: Optional[bytes])

Bases: swh.model.model.BaseModel

Represents the author/committer of a revision or release.

object_type: typing_extensions.Final = 'person'

classmethod from_fullname(fullname: bytes)

Returns a Person object, by guessing the name and email from the fullname, in the name <email> format.

The fullname is left unchanged.

anonymize() → swh.model.model.Person

Returns an anonymized version of the Person object.

Anonymization is simply a Person which fullname is the hashed, with unset name or email.

class swh.model.model.Timestamp(seconds: int, microseconds: int)

Bases: swh.model.model.BaseModel

Represents a naive timestamp from a VCS.

object_type: typing_extensions.Final = 'timestamp'

check_seconds(attribute, value)

Check that seconds fit in a 64-bits signed integer.

check_microseconds(attribute, value)

Checks that microseconds are positive and < 1000000.

class swh.model.model.TimestampWithTimezone(timestamp: swh.model.model.Timestamp, offset: int, negative_utc: bool)

Bases: swh.model.model.BaseModel

Represents a TZ-aware timestamp from a VCS.

object_type: typing_extensions.Final = 'timestamp_with_timezone'

check_offset(attribute, value)

Checks the offset is a 16-bits signed integer (in theory, it should always be between -14 and +14 hours).

check_negative_utc(attribute, value)

classmethod from_dict(obj: Union[Dict, datetime.datetime, int])

Builds a TimestampWithTimezone from any of the formats accepted by swh.model.normalize_timestamp().

classmethod from_datetime(dt: datetime.datetime)

classmethod from_iso8601(s)

Builds a TimestampWithTimezone from an ISO8601-formatted string.

class swh.model.model.Origin(url: str)

Bases: swh.model.model.BaseModel

Represents a software source: a VCS and an URL.

object_type: typing_extensions.Final = 'origin'

class swh.model.model.OriginVisit(origin: str, date: datetime.datetime, type: str, visit: Optional[int] = None)

Bases: swh.model.model.BaseModel

Represents an origin visit with a given type at a given point in time, by a SWH loader.

object_type: typing_extensions.Final = 'origin_visit'

type

Should not be set before calling ‘origin_visit_add()’.

to_dict()

Serializes the date as a string and omits the visit id if it is None.

class swh.model.model.OriginVisitStatus(origin: str, visit: int, date: datetime.datetime, status: str, snapshot: Optional[bytes], metadata=None)

Bases: swh.model.model.BaseModel

Represents a visit update of an origin at a given point in time.

object_type: typing_extensions.Final = 'origin_visit_status'

class swh.model.model.TargetType(value)

Bases: enum.Enum

The type of content pointed to by a snapshot branch. Usually a revision or an alias.

CONTENT = 'content'

DIRECTORY = 'directory'

REVISION = 'revision'

RELEASE = 'release'

SNAPSHOT = 'snapshot'

ALIAS = 'alias'

class swh.model.model.ObjectType(value)

Bases: enum.Enum

The type of content pointed to by a release. Usually a revision

CONTENT = 'content'

DIRECTORY = 'directory'

REVISION = 'revision'

RELEASE = 'release'

SNAPSHOT = 'snapshot'

class swh.model.model.SnapshotBranch(target: bytes, target_type: swh.model.model.TargetType)

Bases: swh.model.model.BaseModel

Represents one of the branches of a snapshot.

object_type: typing_extensions.Final = 'snapshot_branch'

check_target(attribute, value)

Checks the target type is not an alias, checks the target is a valid sha1_git.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

class swh.model.model.Snapshot(branches, id: bytes = b'')

Bases: swh.model.model.BaseModel, swh.model.model.HashableObject

Represents the full state of an origin at a given point in time.

object_type: typing_extensions.Final = 'snapshot'

static compute_hash(object_dict)

Derived model classes must implement this to compute the object hash from its dict representation.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

class swh.model.model.Release(name: bytes, message: Optional[bytes], target: Optional[bytes], target_type: swh.model.model.ObjectType, synthetic: bool, author: Optional[swh.model.model.Person] = None, date: Optional[swh.model.model.TimestampWithTimezone] = None, metadata=None, id: bytes = b'')

Bases: swh.model.model.BaseModel, swh.model.model.HashableObject

object_type: typing_extensions.Final = 'release'

static compute_hash(object_dict)

Derived model classes must implement this to compute the object hash from its dict representation.

check_author(attribute, value)

If the author is None, checks the date is None too.

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

anonymize() → swh.model.model.Release

Returns an anonymized version of the Release object.

Anonymization consists in replacing the author with an anonymized Person object.

class swh.model.model.RevisionType(value)

Bases: enum.Enum

An enumeration.

GIT = 'git'

TAR = 'tar'

DSC = 'dsc'

SUBVERSION = 'svn'

MERCURIAL = 'hg'

swh.model.model.tuplify_extra_headers(value: Iterable) → Tuple

class swh.model.model.Revision(message: Optional[bytes], author: swh.model.model.Person, committer: swh.model.model.Person, date: Optional[swh.model.model.TimestampWithTimezone], committer_date: Optional[swh.model.model.TimestampWithTimezone], type: swh.model.model.RevisionType, directory: bytes, synthetic: bool, metadata=None, parents: Tuple[bytes, …] = (), id: bytes = b'', extra_headers=())

Bases: swh.model.model.BaseModel, swh.model.model.HashableObject

object_type: typing_extensions.Final = 'revision'

static compute_hash(object_dict)

Derived model classes must implement this to compute the object hash from its dict representation.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

anonymize() → swh.model.model.Revision

Returns an anonymized version of the Revision object.

Anonymization consists in replacing the author and committer with an anonymized Person object.

class swh.model.model.DirectoryEntry(name: bytes, type: str, target: bytes, perms: int)

Bases: swh.model.model.BaseModel

object_type: typing_extensions.Final = 'directory_entry'

perms

Usually one of the values of swh.model.from_disk.DentryPerms.

class swh.model.model.Directory(entries: Tuple[swh.model.model.DirectoryEntry, …], id: bytes = b'')

Bases: swh.model.model.BaseModel, swh.model.model.HashableObject

object_type: typing_extensions.Final = 'directory'

static compute_hash(object_dict)

Derived model classes must implement this to compute the object hash from its dict representation.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

class swh.model.model.BaseContent(status: str)

Bases: swh.model.model.BaseModel

classmethod from_dict(d, use_subclass=True)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

get_hash(hash_name)

hashes() → Dict[str, bytes]

Returns a dictionary {hash_name: hash_value}

class swh.model.model.Content(sha1: bytes, sha1_git: bytes, sha256: bytes, blake2s256: bytes, length: int, status: str = 'visible', data: Optional[bytes] = None, ctime: Optional[datetime.datetime] = None)

Bases: swh.model.model.BaseContent

object_type: typing_extensions.Final = 'content'

check_length(attribute, value)

Checks the length is positive.

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

classmethod from_data(data, status='visible', ctime=None) → swh.model.model.Content

Generate a Content from a given data byte string.

This populates the Content with the hashes and length for the data passed as argument, as well as the data itself.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

with_data() → swh.model.model.Content

Loads the data attribute; meaning that it is guaranteed not to be None after this call.

This call is almost a no-op, but subclasses may overload this method to lazy-load data (eg. from disk or objstorage).

class swh.model.model.SkippedContent(sha1: Optional[bytes], sha1_git: Optional[bytes], sha256: Optional[bytes], blake2s256: Optional[bytes], length: Optional[int], status: str, reason: Optional[str] = None, origin: Optional[str] = None, ctime: Optional[datetime.datetime] = None)

Bases: swh.model.model.BaseContent

object_type: typing_extensions.Final = 'skipped_content'

check_reason(attribute, value)

Checks the reason is full if status != absent.

check_length(attribute, value)

Checks the length is positive or -1.

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

classmethod from_data(data: bytes, reason: str, ctime: Optional[datetime.datetime] = None) → swh.model.model.SkippedContent

Generate a SkippedContent from a given data byte string.

This populates the SkippedContent with the hashes and length for the data passed as argument.

You can use attr.evolve on such a generated content to nullify some of its attributes, e.g. for tests.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

class swh.model.model.MetadataAuthorityType(value)

Bases: enum.Enum

An enumeration.

DEPOSIT = 'deposit'

FORGE = 'forge'

REGISTRY = 'registry'

class swh.model.model.MetadataAuthority(type: swh.model.model.MetadataAuthorityType, url: str, metadata=None)

Bases: swh.model.model.BaseModel

Represents an entity that provides metadata about an origin or software artifact.

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.

class swh.model.model.MetadataFetcher(name: str, version: str, metadata=None)

Bases: swh.model.model.BaseModel

Represents a software component used to fetch metadata from a metadata authority, and ingest them into the Software Heritage archive.

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

class swh.model.model.MetadataTargetType(value)

Bases: enum.Enum

The type of object extrinsic metadata refer to.

CONTENT = 'content'

DIRECTORY = 'directory'

REVISION = 'revision'

RELEASE = 'release'

SNAPSHOT = 'snapshot'

ORIGIN = 'origin'

class swh.model.model.RawExtrinsicMetadata(type: swh.model.model.MetadataTargetType, id: Union[str, swh.model.identifiers.SWHID], discovery_date: datetime.datetime, authority: swh.model.model.MetadataAuthority, fetcher: swh.model.model.MetadataFetcher, format: str, metadata: bytes, origin: Optional[str] = None, visit: Optional[int] = None, snapshot: Optional[swh.model.identifiers.SWHID] = None, release: Optional[swh.model.identifiers.SWHID] = None, revision: Optional[swh.model.identifiers.SWHID] = None, path: Optional[bytes] = None, directory: Optional[swh.model.identifiers.SWHID] = None)

Bases: swh.model.model.BaseModel

id

URL if type=MetadataTargetType.ORIGIN, else core SWHID

check_id(attribute, value)

check_origin(attribute, value)

check_visit(attribute, value)

check_snapshot(attribute, value)

check_release(attribute, value)

check_revision(attribute, value)

check_path(attribute, value)

check_directory(attribute, value)

to_dict()

Wrapper of attr.asdict that can be overridden by subclasses that have special handling of some of the fields.

classmethod from_dict(d)

Takes a dictionary representing a tree of SWH objects, and recursively builds the corresponding objects.