swh.storage.api package

Submodules

swh.storage.api.client module

class swh.storage.api.client.RemoteStorage(url, api_exception=None, timeout=None, chunk_size=4096, **kwargs)[source]

Bases: swh.core.api.RPCClient

Proxy to a remote storage API

api_exception

alias of swh.storage.exc.StorageAPIError

check_config(*, check_write)[source]
reset()[source]
content_add(content)[source]
content_add_metadata(content)[source]
content_update(content, keys=[])[source]
content_missing(content, key_hash='sha1')[source]
content_missing_per_sha1(contents)[source]
skipped_content_missing(contents)[source]
content_get(content)[source]
content_get_metadata(content)[source]
content_get_range(start, end, limit=1000)[source]
content_find(content)[source]
directory_add(directories)[source]
directory_missing(directories)[source]
directory_ls(directory, recursive=False)[source]
revision_get(revisions)[source]
revision_log(revisions, limit=None)[source]
revision_shortlog(revisions, limit=None)[source]
revision_add(revisions)[source]
revision_missing(revisions)[source]
release_add(releases)[source]
release_get(releases)[source]
release_missing(releases)[source]
object_find_by_sha1_git(ids)[source]
snapshot_add(snapshots)[source]
snapshot_get(snapshot_id)[source]
snapshot_get_by_origin_visit(origin, visit)[source]
snapshot_get_latest(origin, allowed_statuses=None)[source]
snapshot_count_branches(snapshot_id)[source]
snapshot_get_branches(snapshot_id, branches_from=b'', branches_count=1000, target_types=None)[source]
origin_get(origins=None, *, origin=None)[source]
origin_count(url_pattern, regexp=False, with_visit=False)[source]
origin_get_range(origin_from=1, origin_count=100)[source]
origin_add(origins)[source]
origin_add_one(origin)[source]
origin_visit_add(origin, date, type=None)[source]
origin_visit_update(origin, visit_id, status=None, metadata=None, snapshot=None)[source]
origin_visit_upsert(visits)[source]
origin_visit_get(origin, last_visit=None, limit=None)[source]
origin_visit_find_by_date(origin, visit_date, limit=None)[source]
origin_visit_get_by(origin, visit)[source]
origin_visit_get_latest(origin, allowed_statuses=None, require_snapshot=False)[source]
fetch_history_start(origin_id)[source]
fetch_history_end(fetch_history_id, data)[source]
fetch_history_get(fetch_history_id)[source]
stat_counters()[source]
directory_entry_get_by_path(directory, paths)[source]
tool_add(tools)[source]
tool_get(tool)[source]
origin_metadata_add(origin_id, ts, provider, tool, metadata)[source]
origin_metadata_get_by(origin_id, provider_type=None)[source]
metadata_provider_add(provider_name, provider_type, provider_url, metadata)[source]
metadata_provider_get(provider_id)[source]
metadata_provider_get_by(provider)[source]
diff_directories(from_dir, to_dir, track_renaming=False)[source]
diff_revisions(from_rev, to_rev, track_renaming=False)[source]
diff_revision(revision, track_renaming=False)[source]
__module__ = 'swh.storage.api.client'

swh.storage.api.server module

swh.storage.api.server.timed(f)[source]

Time that function!

swh.storage.api.server.encode(f)[source]
swh.storage.api.server.send_metric(metric, count, method_name)[source]

Send statsd metric with count for method method_name

If count is 0, the metric is discarded. If the metric is not parseable, the metric is discarded with a log message.

Parameters:
  • metric (str) – Metric’s name (e.g content:add, content:add:bytes)
  • count (int) – Associated value for the metric
  • method_name (str) – Method’s name
Returns:

Bool to explicit if metric has been set or not

swh.storage.api.server.process_metrics(f)[source]

Increment object counters for the decorated function.

swh.storage.api.server.my_error_handler(exception)[source]
swh.storage.api.server.get_storage()[source]
swh.storage.api.server.index()[source]
swh.storage.api.server.check_config()[source]
swh.storage.api.server.reset()[source]
swh.storage.api.server.content_missing()[source]
swh.storage.api.server.content_missing_per_sha1()[source]
swh.storage.api.server.skipped_content_missing()[source]
swh.storage.api.server.content_find()[source]
swh.storage.api.server.content_add()[source]
swh.storage.api.server.content_add_metadata()[source]
swh.storage.api.server.content_update()[source]
swh.storage.api.server.content_get()[source]
swh.storage.api.server.content_get_metadata()[source]
swh.storage.api.server.content_get_range()[source]
swh.storage.api.server.directory_missing()[source]
swh.storage.api.server.directory_add()[source]
swh.storage.api.server.directory_entry_get_by_path()[source]
swh.storage.api.server.directory_ls()[source]
swh.storage.api.server.revision_add()[source]
swh.storage.api.server.revision_get()[source]
swh.storage.api.server.revision_log()[source]
swh.storage.api.server.revision_shortlog()[source]
swh.storage.api.server.revision_missing()[source]
swh.storage.api.server.release_add()[source]
swh.storage.api.server.release_get()[source]
swh.storage.api.server.release_missing()[source]
swh.storage.api.server.object_find_by_sha1_git()[source]
swh.storage.api.server.snapshot_add()[source]
swh.storage.api.server.snapshot_get()[source]
swh.storage.api.server.snapshot_get_by_origin_visit()[source]
swh.storage.api.server.snapshot_get_latest()[source]
swh.storage.api.server.snapshot_count_branches()[source]
swh.storage.api.server.snapshot_get_branches()[source]
swh.storage.api.server.origin_get()[source]
swh.storage.api.server.origin_get_range()[source]
swh.storage.api.server.origin_count()[source]
swh.storage.api.server.origin_add()[source]
swh.storage.api.server.origin_add_one()[source]
swh.storage.api.server.origin_visit_get()[source]
swh.storage.api.server.origin_visit_find_by_date()[source]
swh.storage.api.server.origin_visit_get_by()[source]
swh.storage.api.server.origin_visit_get_latest()[source]
swh.storage.api.server.origin_visit_add()[source]
swh.storage.api.server.origin_visit_update()[source]
swh.storage.api.server.origin_visit_upsert()[source]
swh.storage.api.server.fetch_history_get()[source]
swh.storage.api.server.fetch_history_start()[source]
swh.storage.api.server.fetch_history_end()[source]
swh.storage.api.server.tool_get()[source]
swh.storage.api.server.tool_add()[source]
swh.storage.api.server.origin_metadata_add()[source]
swh.storage.api.server.origin_metadata_get_by()[source]
swh.storage.api.server.metadata_provider_add()[source]
swh.storage.api.server.metadata_provider_get()[source]
swh.storage.api.server.metadata_provider_get_by()[source]
swh.storage.api.server.stat_counters()[source]
swh.storage.api.server.diff_directories()[source]
swh.storage.api.server.diff_revisions()[source]
swh.storage.api.server.diff_revision()[source]
swh.storage.api.server.load_and_check_config(config_file, type='local')[source]
Check the minimal configuration is set to run the api or raise an
error explanation.
Parameters:
  • config_file (str) – Path to the configuration file to load
  • type (str) – configuration type. For ‘local’ type, more checks are done.
Raises:

Error if the setup is not as expected

Returns:

configuration as a dict

swh.storage.api.server.make_app_from_configfile()[source]

Run the WSGI app from the webserver, loading the configuration from a configuration file.

SWH_CONFIG_FILENAME environment variable defines the configuration path to load.

Module contents