swh.core.config module

swh.core.config.exists_accessible(filepath: str) → bool[source]

Check whether a file exists, and is accessible.


True if the file exists and is accessible False if the file does not exist


PermissionError if the file cannot be read.

swh.core.config.config_basepath(config_path: str) → str[source]

Return the base path of a configuration file

swh.core.config.read_raw_config(base_config_path: str) → Dict[str, Any][source]

Read the raw config corresponding to base_config_path.

Can read yml files.


Check whether the given config exists

swh.core.config.read(conf_file: Optional[str] = None, default_conf: Optional[Dict[str, Tuple[str, Any]]] = None) → Dict[str, Any][source]

Read the user’s configuration file.

Fill in the gap using default_conf. default_conf is similar to this:

    'a': ('str', '/tmp/swh-loader-git/log'),
    'b': ('str', 'dbname=swhloadergit')
    'c': ('bool', true)
    'e': ('bool', None)
    'd': ('int', 10)

If conf_file is None, return the default config.

swh.core.config.priority_read(conf_filenames: List[str], default_conf: Optional[Dict[str, Tuple[str, Any]]] = None)[source]

Try reading the configuration files from conf_filenames, in order, and return the configuration from the first one that exists.

default_conf has the same specification as it does in read.

swh.core.config.merge_default_configs(base_config, *other_configs)[source]

Merge several default config dictionaries, from left to right

swh.core.config.merge_configs(base: Optional[Dict[str, Any]], other: Optional[Dict[str, Any]])[source]

Merge two config dictionaries

This does merge config dicts recursively, with the rules, for every value of the dicts (with ‘val’ not being a dict):

  • None + type -> type

  • type + None -> None

  • dict + dict -> dict (merged)

  • val + dict -> TypeError

  • dict + val -> TypeError

  • val + val -> val (other)

for instance:

>>> d1 = {
...   'key1': {
...     'skey1': 'value1',
...     'skey2': {'sskey1': 'value2'},
...   },
...   'key2': 'value3',
... }


>>> d2 = {
...   'key1': {
...     'skey1': 'value4',
...     'skey2': {'sskey2': 'value5'},
...   },
...   'key3': 'value6',
... }

will give:

>>> d3 = {
...   'key1': {
...     'skey1': 'value4',  # <-- note this
...     'skey2': {
...       'sskey1': 'value2',
...       'sskey2': 'value5',
...     },
...   },
...   'key2': 'value3',
...   'key3': 'value6',
... }
>>> assert merge_configs(d1, d2) == d3

Note that no type checking is done for anything but dicts.

swh.core.config.swh_config_paths(base_filename: str) → List[str][source]

Return the Software Heritage specific configuration paths for the given filename.

swh.core.config.prepare_folders(conf, *keys)[source]

Prepare the folder mentioned in config under keys.


Load the global Software Heritage config

swh.core.config.load_named_config(name, default_conf=None, global_conf=True)[source]

Load the config named name from the Software Heritage configuration paths.

If global_conf is True (default), read the global configuration too.

swh.core.config.load_from_envvar(default_config: Optional[Dict[str, Any]] = None) → Dict[str, Any][source]

Load configuration yaml file from the environment variable SWH_CONFIG_FILENAME, eventually enriched with default configuration key/value from the default_config dict if provided.


Configuration dict


AssertionError if SWH_CONFIG_FILENAME is undefined