swh.indexer.fossology_license module

swh.indexer.fossology_license.compute_license(path) → Dict

Determine license from file at path.

Parameters:

path – filepath to determine the license

Returns:

A dict with the following keys:

• licenses ([str]): associated detected licenses to path

• path (bytes): content filepath

Return type:

dict

class swh.indexer.fossology_license.MixinFossologyLicenseIndexer(*args, **kwargs)

Bases: object

Mixin fossology license indexer.

See FossologyLicenseIndexer

tool: Any

idx_storage: IndexerStorageInterface

index(id: bytes, data: bytes | None = None, **kwargs) → List[ContentLicenseRow]

Index sha1s’ content and store result.

Parameters:

• id (bytes) – content’s identifier

• raw_content (bytes) – associated raw content to content id

Returns:

A dict, representing a content_license, with keys:

• id (bytes): content’s identifier (sha1)

• license (bytes): license in bytes

• path (bytes): path

• indexer_configuration_id (int): tool used to compute the output

Return type:

dict

persist_index_computations(results: List[ContentLicenseRow]) → Dict[str, int]

Persist the results in storage.

Parameters:

results –

list of content_license dict with the following keys:

• id (bytes): content’s identifier (sha1)

• license (bytes): license in bytes

• path (bytes): path

class swh.indexer.fossology_license.FossologyLicenseIndexer(*args, **kwargs)

Bases: MixinFossologyLicenseIndexer, ContentIndexer[ContentLicenseRow]

Indexer in charge of:

• filtering out content already indexed

• reading content from objstorage per the content’s id (sha1)

• computing {license, encoding} from that content

• store result in storage

Prepare and check that the indexer is ready to run.

filter(ids)

Filter out known sha1s and return only missing ones.