swh.search.interface module

class swh.search.interface.MinimalOriginDict(**kwargs)[source]

Bases: dict

Mandatory keys of an OriginDict

url: str
class swh.search.interface.OriginDict(**kwargs)[source]

Bases: dict

Argument passed to SearchInterface.origin_update().

visit_types: List[str]
has_visits: bool
url: str
class swh.search.interface.SearchInterface[source]

Bases: object

check()[source]

Dedicated method to execute some specific check per implementation.

flush() None[source]

Blocks until all previous calls to _update() are completely applied.

origin_update(documents: Iterable[swh.search.interface.OriginDict]) None[source]

Persist documents to the search backend.

Searches for origins matching the url_pattern.

Parameters
  • url_pattern – Part of the URL to search for

  • metadata_pattern – Keywords to look for

  • intrinsic_metadata) ((across all the fields of) –

  • with_visit – Whether origins with no visit are to be filtered out

  • visit_types – Only origins having any of the provided visit types (e.g. git, svn, pypi) will be returned

  • min_nb_visits – Filter origins that have number of visits >= the provided value

  • min_last_visit_date – Filter origins that have last_visit_date on or after the provided date(ISO format)

  • min_last_eventful_visit_date – Filter origins that have last_eventful_visit_date (eventful = snapshot_id changed) on or after the provided date(ISO format)

  • min_last_revision_date – Filter origins that have last_revision_date on or after the provided date(ISO format)

  • min_last_release_date – Filter origins that have last_release_date on or after the provided date(ISO format)

  • min_date_created – Filter origins that have date_created from intrinsic_metadata on or after the provided date

  • min_date_modified – Filter origins that have date_modified from intrinsic_metadata on or after the provided date

  • min_date_published – Filter origins that have date_published from intrinsic_metadata on or after the provided date

  • programming_languages – Filter origins with programming languages present in the given list (based on instrinsic_metadata)

  • licenses – Filter origins with licenses present in the given list (based on instrinsic_metadata)

  • keywords – Filter origins having description/keywords (extracted from instrinsic_metadata) that match given values

  • sort_by – Sort results based on a list of fields mentioned in SORT_BY_OPTIONS (nb_visits,last_visit_date, last_eventful_visit_date, last_revision_date, last_release_date). Return results in descending order if “-” is present at the beginning otherwise in ascending order.

  • page_token – Opaque value used for pagination

  • limit – number of results to return

Returns

PagedResult of origin dicts matching the search criteria. If next_page_token is None, there is no longer data to retrieve.