swh.loader.git.base module#

class swh.loader.git.base.BaseGitLoader(*args, **kwargs)[source]#

Bases: BaseLoader

This base class is a pattern for both git loaders

Those loaders are able to load all the data in one go.

cleanup() → None[source]#: Clean up an eventual state installed for computations.

has_contents() → bool[source]#: Checks whether we need to load contents

get_contents() → Iterable[BaseContent][source]#: Get the contents that need to be loaded

has_directories() → bool[source]#: Checks whether we need to load directories

get_directories() → Iterable[Directory][source]#: Get the directories that need to be loaded

has_revisions() → bool[source]#: Checks whether we need to load revisions

get_revisions() → Iterable[Revision][source]#: Get the revisions that need to be loaded

has_releases() → bool[source]#: Checks whether we need to load releases

get_releases() → Iterable[Release][source]#: Get the releases that need to be loaded

get_snapshot() → Snapshot[source]#: Get the snapshot that needs to be loaded

eventful() → bool[source]#: Whether the load was eventful

maybe_log(msg: str, *args, level=20, force=False, **kwargs)[source]#

Only log if LOGGING_INTERVAL has elapsed since the last log line was printed.

Arguments are identical to those of logging.Logger.log, except if the log format arguments are callable, the call only happens if the log is actually being printed.

store_data() → None[source]#

Store fetched and processed data in the storage.

This should call the storage.<object>_add methods, which handle the objects to store in the storage.