Skip to main content
Ctrl+K
Software Heritage documentation - Home

Site Navigation

  • Development
  • API reference
  • Usage
  • Infrastructure
  • About this documentation project
  • GitLab
  • PyPI
  • System Status
  • Software Heritage Homepage

Site Navigation

  • Development
  • API reference
  • Usage
  • Infrastructure
  • About this documentation project
  • GitLab
  • PyPI
  • System Status
  • Software Heritage Homepage

Section Navigation

  • Getting started
    • Run your own Software Heritage
    • Developer setup
    • Docker environment
    • Getting Started with the Software Heritage API
  • Software Architecture
    • Software Architecture Overview
    • Metadata workflow and architecture
    • Object Storage Overview
  • Contributing
    • Submitting code to SWH
    • Code review
    • Python style guide
    • Git style guide
    • Sphinx gotchas
    • Tutorial: Best practices when writing SWH docs
  • Tutorials
    • Software testing guide
    • Issue debugging and monitoring guide
    • Add a new package
  • Frequently Asked Questions
  • Roadmap 2023
  • Current and past roadmaps
    • Roadmap 2023
    • Roadmap 2022
    • Roadmap 2021
  • Configuration reference
  • API reference
    • swh.alter
      • Using swh-alter
      • Removal algorithm
      • Recovery bundles
      • swh.alter package
        • swh.alter.bech32 module
        • swh.alter.cli module
        • swh.alter.inventory module
        • swh.alter.operations module
        • swh.alter.recovery_bundle module
        • swh.alter.removable module
        • swh.alter.subgraph module
    • swh.auth
      • Command-line interface
      • Django components
      • swh.auth package
        • swh.auth.django package
        • swh.auth.starlette package
        • swh.auth.cli module
        • swh.auth.keycloak module
        • swh.auth.pytest_plugin module
        • swh.auth.utils module
    • swh.core
      • Command-line interface
      • Common database utilities
      • swh.core package
        • swh.core.api package
        • swh.core.cli package
        • swh.core.db package
        • swh.core.github package
        • swh.core.api_async module
        • swh.core.collections module
        • swh.core.config module
        • swh.core.logger module
        • swh.core.logging module
        • swh.core.pytest_plugin module
        • swh.core.retry module
        • swh.core.sentry module
        • swh.core.statsd module
        • swh.core.tarball module
        • swh.core.utils module
    • swh.counters
      • swh.counters package
        • swh.counters.api package
        • swh.counters.cli module
        • swh.counters.history module
        • swh.counters.in_memory module
        • swh.counters.interface module
        • swh.counters.journal_client module
        • swh.counters.kafka_client module
        • swh.counters.redis module
    • swh.dataset
      • Software Heritage Graph Dataset
        • Dataset
        • Relational schema
        • Setup on Amazon Athena
        • Setup on Azure Databricks
      • Exporting a dataset
      • Exporting a subdataset
      • swh.dataset package
        • swh.dataset.exporters package
        • swh.dataset.test namespace
        • swh.dataset.athena module
        • swh.dataset.cli module
        • swh.dataset.exporter module
        • swh.dataset.journalprocessor module
        • swh.dataset.luigi module
        • swh.dataset.relational module
        • swh.dataset.utils module
    • swh.deposit
      • Deposit API
        • User Manual
        • API Documentation
        • Deposit metadata
        • Use cases
        • Register account
      • Deposit internals
        • Running swh-deposit locally
        • Production deployment
        • Authentication
        • Loading workflow
      • Specifications
        • Loading specification
        • Protocol reference
        • The metadata-only deposit
      • swh.deposit package
        • swh.deposit.api package
        • swh.deposit.cli package
        • swh.deposit.fixtures package
        • swh.deposit.loader package
        • swh.deposit.templates package
        • swh.deposit.tests_migration package
        • swh.deposit.apps module
        • swh.deposit.auth module
        • swh.deposit.client module
        • swh.deposit.config module
        • swh.deposit.errors module
        • swh.deposit.exception module
        • swh.deposit.gunicorn_config module
        • swh.deposit.manage module
        • swh.deposit.models module
        • swh.deposit.parsers module
        • swh.deposit.urls module
        • swh.deposit.utils module
      • Command-line interface
    • swh.fuse
      • Command-line interface
      • Configuration
      • Design notes
      • Tutorial
      • swh.fuse package
        • swh.fuse.fs namespace
        • swh.fuse.cache module
        • swh.fuse.cli module
        • swh.fuse.fuse module
    • swh.graph
      • Quickstart
      • Graph Querying HTTP API
      • Using the GRPC API
      • Using the Java API
      • Memory & Performance tuning
      • Graph compression
      • Luigi workflows
      • Command-line interface
      • Docker environment
      • git2graph
      • Example dataset
      • swh.graph package
        • swh.graph.example_dataset package
        • swh.graph.grpc namespace
        • swh.graph.luigi package
        • swh.graph.cli module
        • swh.graph.client module
        • swh.graph.config module
        • swh.graph.grpc_server module
        • swh.graph.http_client module
        • swh.graph.http_naive_client module
        • swh.graph.http_rpc_server module
        • swh.graph.naive_client module
        • swh.graph.pytest_plugin module
        • swh.graph.webgraph module
    • swh.graphql
      • swh.graphql package
        • swh.graphql.backends package
        • swh.graphql.client package
        • swh.graphql.errors package
        • swh.graphql.middlewares package
        • swh.graphql.resolvers package
        • swh.graphql.utils package
        • swh.graphql.app module
        • swh.graphql.gunicorn_config module
        • swh.graphql.server module
    • swh.indexer
      • Software Heritage - Indexer
      • Hacking on swh-indexer
      • Metadata workflow
      • SwhPkg Vocabulary
      • swh.indexer package
        • swh.indexer.metadata_dictionary package
        • swh.indexer.storage package
        • swh.indexer.bibtex module
        • swh.indexer.cli module
        • swh.indexer.codemeta module
        • swh.indexer.fossology_license module
        • swh.indexer.indexer module
        • swh.indexer.journal_client module
        • swh.indexer.metadata module
        • swh.indexer.metadata_detector module
        • swh.indexer.mimetype module
        • swh.indexer.namespaces module
        • swh.indexer.origin_head module
        • swh.indexer.rehash module
        • swh.indexer.tasks module
      • Command-line interface
    • swh.journal
      • Software Heritage Journal clients
      • swh.journal package
        • swh.journal.writer package
        • swh.journal.client module
        • swh.journal.pytest_plugin module
        • swh.journal.serializers module
    • swh.lister
      • Tutorial: list the content of your favorite forge in just a few steps
      • Tutorial: run a lister within docker-dev in just a few steps
      • Save a forge
      • swh.lister package
        • swh.lister.arch package
        • swh.lister.aur package
        • swh.lister.bioconductor package
        • swh.lister.bitbucket package
        • swh.lister.bower package
        • swh.lister.cgit package
        • swh.lister.conda package
        • swh.lister.cpan package
        • swh.lister.cran package
        • swh.lister.crates package
        • swh.lister.debian package
        • swh.lister.dlang package
        • swh.lister.gitea package
        • swh.lister.github package
        • swh.lister.gitiles package
        • swh.lister.gitlab package
        • swh.lister.gitweb package
        • swh.lister.gnu package
        • swh.lister.gogs package
        • swh.lister.golang package
        • swh.lister.hackage package
        • swh.lister.hex package
        • swh.lister.julia package
        • swh.lister.launchpad package
        • swh.lister.maven package
        • swh.lister.nixguix package
        • swh.lister.npm package
        • swh.lister.nuget package
        • swh.lister.opam package
        • swh.lister.packagist package
        • swh.lister.pagure package
        • swh.lister.phabricator package
        • swh.lister.pubdev package
        • swh.lister.puppet package
        • swh.lister.pypi package
        • swh.lister.rpm package
        • swh.lister.rubygems package
        • swh.lister.sourceforge package
        • swh.lister.stagit package
        • swh.lister.tuleap package
        • swh.lister.cli module
        • swh.lister.pattern module
        • swh.lister.utils module
      • Command-line interface
    • swh.loader
      • swh.loader.core
        • VCS Loader Overview
        • Package Loader Tutorial
        • Package loader specifications
        • swh.loader.core package
        • Command-line interface
      • swh.loader.bzr
        • Software Heritage - How Bazaar/Breezy works
        • swh.loader.bzr package
      • Software Heritage - CVS loader
      • swh.loader.git
        • swh.loader.git package
      • swh.loader.mercurial
        • swh.loader.mercurial package
      • swh.loader.metadata
        • swh.loader.metadata package
      • swh.loader.svn
        • swh.loader.svn package
    • swh.model
      • Data model
      • SoftWare Heritage persistent IDentifiers (SWHIDs)
      • Command-line interface
      • swh.model package
        • swh.model.fields package
        • swh.model.cli module
        • swh.model.collections module
        • swh.model.discovery module
        • swh.model.exceptions module
        • swh.model.from_disk module
        • swh.model.git_objects module
        • swh.model.hashutil module
        • swh.model.hypothesis_strategies module
        • swh.model.identifiers module
        • swh.model.merkle module
        • swh.model.model module
        • swh.model.swhids module
        • swh.model.toposort module
        • swh.model.validators module
    • swh.objstorage
      • Command-line interface
      • Winery backend
      • swh.objstorage package
        • swh.objstorage.api package
        • swh.objstorage.backends package
        • swh.objstorage.multiplexer package
        • swh.objstorage.replayer package
        • swh.objstorage.cli module
        • swh.objstorage.constants module
        • swh.objstorage.exc module
        • swh.objstorage.factory module
        • swh.objstorage.interface module
        • swh.objstorage.objstorage module
        • swh.objstorage.utils module
    • swh.objstorage.replayer
      • Command-line interface
      • swh.objstorage.replayer package
        • swh.objstorage.replayer.cli module
        • swh.objstorage.replayer.replay module
    • swh.perfecthash
      • Benchmarks
      • Read Shard format
      • swh.perfecthash package
        • swh.perfecthash.build module
    • swh.scanner
      • Command-line interface
      • swh.scanner package
        • swh.scanner.dashboard package
        • swh.scanner.backend module
        • swh.scanner.cli module
        • swh.scanner.client module
        • swh.scanner.data module
        • swh.scanner.db module
        • swh.scanner.exceptions module
        • swh.scanner.output module
        • swh.scanner.plot module
        • swh.scanner.policy module
        • swh.scanner.scanner module
    • swh.scheduler
      • Command-line interface
      • Software Heritage Scheduler Simulator
      • swh.scheduler package
        • swh.scheduler.api package
        • swh.scheduler.celery_backend package
        • swh.scheduler.cli package
        • swh.scheduler.simulator package
        • swh.scheduler.sql package
        • swh.scheduler.backend module
        • swh.scheduler.cli_utils module
        • swh.scheduler.exc module
        • swh.scheduler.interface module
        • swh.scheduler.journal_client module
        • swh.scheduler.model module
        • swh.scheduler.pytest_plugin module
        • swh.scheduler.task module
        • swh.scheduler.utils module
    • swh.scrubber
      • Command-line interface
      • swh.scrubber package
        • swh.scrubber.cli module
        • swh.scrubber.db module
        • swh.scrubber.fixer module
        • swh.scrubber.journal_checker module
        • swh.scrubber.origin_locator module
        • swh.scrubber.storage_checker module
        • swh.scrubber.utils module
    • swh.search
      • Command-line interface
      • Search Query Language
      • swh.search package
        • swh.search.api package
        • swh.search.cli module
        • swh.search.elasticsearch module
        • swh.search.exc module
        • swh.search.in_memory module
        • swh.search.interface module
        • swh.search.journal_client module
        • swh.search.metrics module
        • swh.search.translator module
        • swh.search.utils module
    • swh.storage
      • Command-line interface
      • swh.storage package
        • swh.storage.algos package
        • swh.storage.api package
        • swh.storage.cassandra package
        • swh.storage.postgresql package
        • swh.storage.proxies package
        • swh.storage.backfill module
        • swh.storage.cli module
        • swh.storage.common module
        • swh.storage.exc module
        • swh.storage.fixer module
        • swh.storage.in_memory module
        • swh.storage.interface module
        • swh.storage.metrics module
        • swh.storage.migrate_extrinsic_metadata module
        • swh.storage.objstorage module
        • swh.storage.pytest_plugin module
        • swh.storage.replay module
        • swh.storage.utils module
        • swh.storage.writer module
    • swh.vault
      • Getting started
      • Vault API Reference
      • swh.vault package
        • swh.vault.api package
        • swh.vault.cookers package
        • swh.vault.backend module
        • swh.vault.cache module
        • swh.vault.cli module
        • swh.vault.cooking_tasks module
        • swh.vault.exc module
        • swh.vault.in_memory_backend module
        • swh.vault.interface module
        • swh.vault.to_disk module
      • Command-line interface
    • Software Heritage - Web applications
    • swh.web package
      • swh.web.add_forge_now package
        • swh.web.add_forge_now.admin_views module
        • swh.web.add_forge_now.api_views module
        • swh.web.add_forge_now.apps module
        • swh.web.add_forge_now.models module
        • swh.web.add_forge_now.signal_receivers module
        • swh.web.add_forge_now.urls module
        • swh.web.add_forge_now.utils module
        • swh.web.add_forge_now.views module
      • swh.web.api package
        • swh.web.api.views package
        • swh.web.api.apidoc module
        • swh.web.api.apiresponse module
        • swh.web.api.apiurls module
        • swh.web.api.renderers module
        • swh.web.api.throttling module
        • swh.web.api.urls module
        • swh.web.api.utils module
      • swh.web.archive_coverage package
        • swh.web.archive_coverage.urls module
        • swh.web.archive_coverage.views module
      • swh.web.auth package
        • swh.web.auth.apps module
        • swh.web.auth.models module
        • swh.web.auth.urls module
        • swh.web.auth.utils module
        • swh.web.auth.views module
      • swh.web.badges package
        • swh.web.badges.urls module
      • swh.web.banners package
        • swh.web.banners.urls module
        • swh.web.banners.views module
      • swh.web.browse package
        • swh.web.browse.views package
        • swh.web.browse.browseurls module
        • swh.web.browse.identifiers module
        • swh.web.browse.snapshot_context module
        • swh.web.browse.urls module
        • swh.web.browse.utils module
      • swh.web.client package
        • swh.web.client.cli module
        • swh.web.client.client module
      • swh.web.deposit package
        • swh.web.deposit.urls module
      • swh.web.inbound_email package
        • swh.web.inbound_email.management package
        • swh.web.inbound_email.apps module
        • swh.web.inbound_email.signals module
        • swh.web.inbound_email.utils module
      • swh.web.jslicenses package
        • swh.web.jslicenses.urls module
      • swh.web.mailmap package
        • swh.web.mailmap.management package
        • swh.web.mailmap.apps module
        • swh.web.mailmap.models module
        • swh.web.mailmap.urls module
        • swh.web.mailmap.views module
      • swh.web.metrics package
        • swh.web.metrics.prometheus module
        • swh.web.metrics.urls module
        • swh.web.metrics.views module
      • swh.web.save_code_now package
        • swh.web.save_code_now.management package
        • swh.web.save_code_now.admin_views module
        • swh.web.save_code_now.api_views module
        • swh.web.save_code_now.apps module
        • swh.web.save_code_now.models module
        • swh.web.save_code_now.origin_save module
        • swh.web.save_code_now.origin_visit_webhook module
        • swh.web.save_code_now.urls module
        • swh.web.save_code_now.views module
      • swh.web.save_origin_webhooks package
        • swh.web.save_origin_webhooks.bitbucket module
        • swh.web.save_origin_webhooks.generic_receiver module
        • swh.web.save_origin_webhooks.gitea module
        • swh.web.save_origin_webhooks.github module
        • swh.web.save_origin_webhooks.gitlab module
        • swh.web.save_origin_webhooks.sourceforge module
        • swh.web.save_origin_webhooks.urls module
      • swh.web.utils package
        • swh.web.utils.management package
        • swh.web.utils.archive module
        • swh.web.utils.converters module
        • swh.web.utils.exc module
        • swh.web.utils.highlightjs module
        • swh.web.utils.identifiers module
        • swh.web.utils.middlewares module
        • swh.web.utils.origin_visits module
        • swh.web.utils.query module
        • swh.web.utils.swh_templatetags module
        • swh.web.utils.typing module
        • swh.web.utils.urlsindex module
      • swh.web.vault package
        • swh.web.vault.api_views module
        • swh.web.vault.urls module
      • swh.web.webapp package
        • swh.web.webapp.urls module
      • swh.web.config module
      • swh.web.gunicorn_config module
      • swh.web.manage module
      • swh.web.urls module
    • Developers Information
    • swh-web API URLs
    • URI scheme for swh-web Browse application
    • URI scheme for SoftWare Hash IDentifiers (SWHIDs)
    • Miscellaneous URLs
    • swh.web.client
    • swh.webhooks
      • swh.webhooks package
        • swh.webhooks.cli module
        • swh.webhooks.interface module
        • swh.webhooks.journal_client module
        • swh.webhooks.utils module
  • Archive ChangeLog
  • Journal Specification
  • Statsd metrics and Grafana dashboards
  • Development
  • Software...

Software Architecture#

  • Software Architecture Overview
  • Metadata workflow and architecture
  • Object Storage Overview

previous

Getting Started with the Software Heritage API

next

Software Architecture Overview
Edit
Show Source

© Copyright 2015-2023 The Software Heritage developers.

Created using Sphinx 7.2.6.

Built with the PyData Sphinx Theme 0.14.4.