Configuration#
The configuration class is defined in configuration.py. From CLI, you can load custom configuration yaml with -c
or --config
argument. From Python, you can replace the default configuration with
from pathlib import Path
import sec_certs.configuration as config_module
config_module.config.load_from_yaml("/path/to/your/config.yaml")
# or just set the individual key
config_module.config.log_filepath = Path("/some/path/where/log/will/be/stored.txt")
The configuration yaml is a simple flat dictionary of keys and values. The configuration file can specify only some of the fields. For the content of unspecified fields, environment variable with seccerts_
prefix (case insensitive) will be checked. If such variable is not set, default value will be used. Content in the yaml always beats the environment variable.
For instance, when user provides the following yaml
log_filepath: my_own_log_file.txt
n_threads: 7
and sets SECCERTS_MINIMAL_TOKEN_LENGTH=4
as environment variable, only these 3 keys will be loaded with config.load_from_yaml()
, others will be untouched.
Tip
You can load settings even without providing yaml configuration. Simply set the corresponding environment variables or use .env
file.
Configuration keys, types, default values and descriptions#
log_filepath
type:
<class 'pathlib.Path'>
default:
./cert_processing_log.log
description: Path to the file, relative to working directory, where the log will be stored.
env name:
SECCERTS_LOG_FILEPATH
always_false_positive_fips_cert_id_threshold
type:
<class 'int'>
default:
40
description: During validation we don’t connect certificates with number lower than this to connections due to these numbers being typically false positives.
env name:
SECCERTS_ALWAYS_FALSE_POSITIVE_FIPS_CERT_ID_THRESHOLD
year_difference_between_validations
type:
<class 'int'>
default:
7
description: During validation we don’t connect certificates with validation dates difference higher than this.
env name:
SECCERTS_YEAR_DIFFERENCE_BETWEEN_VALIDATIONS
n_threads
type:
<class 'int'>
default:
-1
description: How many threads to use for parallel computations. Set to -1 to use all logical cores.
env name:
SECCERTS_N_THREADS
cpe_matching_threshold
type:
<class 'int'>
default:
92
description: Level of required string similarity between CPE and certificate name on CC CPE matching, 0-100. Lower values yield more false negatives, higher values more false positives
env name:
SECCERTS_CPE_MATCHING_THRESHOLD
cpe_n_max_matches
type:
<class 'int'>
default:
99
description: Maximum number of candidate CPE items that may be related to given certificate, >0
env name:
SECCERTS_CPE_N_MAX_MATCHES
cc_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/cc/dataset.json
description: URL from where to fetch the latest snapshot of fully processed CC dataset.
env name:
SECCERTS_CC_LATEST_SNAPSHOT
cc_maintenances_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/cc/maintenance_updates.json
description: URL from where to fetch the latest snapshot of CC maintenance updates
env name:
SECCERTS_CC_MAINTENANCES_LATEST_SNAPSHOT
pp_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/static/pp.json
description: URL from where to fetch the latest snapshot of the PP dataset.
env name:
SECCERTS_PP_LATEST_SNAPSHOT
fips_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/fips/dataset.json
description: URL for the latest snapshot of FIPS dataset.
env name:
SECCERTS_FIPS_LATEST_SNAPSHOT
fips_iut_dataset
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/fips/iut/dataset.json
description: URL for the dataset of FIPS IUT data.
env name:
SECCERTS_FIPS_IUT_DATASET
fips_iut_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/fips/iut/latest.json
description: URL for the latest snapshot of FIPS IUT data.
env name:
SECCERTS_FIPS_IUT_LATEST_SNAPSHOT
fips_mip_dataset
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/fips/mip/dataset.json
description: URL for the dataset of FIPS MIP data
env name:
SECCERTS_FIPS_MIP_DATASET
fips_mip_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/fips/mip/latest.json
description: URL for the latest snapshot of FIPS MIP data
env name:
SECCERTS_FIPS_MIP_LATEST_SNAPSHOT
cpe_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/vuln/cpe/cpe.json.gz
description: URL for the latest snapshot of CPEDataset.
env name:
SECCERTS_CPE_LATEST_SNAPSHOT
cve_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/vuln/cve/cve.json.gz
description: URL for the latest snapshot of CVEDataset.
env name:
SECCERTS_CVE_LATEST_SNAPSHOT
cpe_match_latest_snapshot
type:
<class 'pydantic_core._pydantic_core.Url'>
default:
https://sec-certs.org/vuln/cpe/cpe_match.json.gz
description: URL for the latest snapshot of cpe match json.
env name:
SECCERTS_CPE_MATCH_LATEST_SNAPSHOT
fips_matching_threshold
type:
<class 'int'>
default:
90
description: Level of required similarity before FIPS IUT/MIP entry is considered to match a FIPS certificate.
env name:
SECCERTS_FIPS_MATCHING_THRESHOLD
minimal_token_length
type:
<class 'int'>
default:
3
description: Minimal length of a string that will be considered as a token during keyword extraction in CVE matching
env name:
SECCERTS_MINIMAL_TOKEN_LENGTH
ignore_first_page
type:
<class 'bool'>
default:
True
description: During keyword search, first page usually contains addresses - ignore it.
env name:
SECCERTS_IGNORE_FIRST_PAGE
cc_reference_annotator_dir
type:
typing.Optional[pathlib.Path]
default:
None
description: Path to directory with serialized reference annotator model. If set to
null
, tool will search default directory for the given dataset.env name:
SECCERTS_CC_REFERENCE_ANNOTATOR_DIR
cc_reference_annotator_should_train
type:
<class 'bool'>
default:
True
description: True if new reference annotator model shall be build, False otherwise.
env name:
SECCERTS_CC_REFERENCE_ANNOTATOR_SHOULD_TRAIN
cc_matching_threshold
type:
<class 'int'>
default:
90
description: Level of required similarity before CC scheme entry is considered to match a CC certificate.
env name:
SECCERTS_CC_MATCHING_THRESHOLD
cc_use_proxy
type:
<class 'bool'>
default:
False
description: Download CC artifacts through the sec-certs.org proxy.
env name:
SECCERTS_CC_USE_PROXY
fips_use_proxy
type:
<class 'bool'>
default:
False
description: Download FIPS artifacts through the sec-certs.org proxy.
env name:
SECCERTS_FIPS_USE_PROXY
enable_progress_bars
type:
<class 'bool'>
default:
True
description: If true, progress bars will be printed to stdout during computation.
env name:
SECCERTS_ENABLE_PROGRESS_BARS
nvd_api_key
type:
typing.Optional[str]
default:
None
description: NVD API key for access to CVEs and CPEs.
env name:
SECCERTS_NVD_API_KEY
preferred_source_nvd_datasets
type:
typing.Literal['sec-certs', 'api']
default:
sec-certs
description: If set to
sec-certs
, will fetch CPE and CVE datasets from sec-certs.org. If set toapi
, will fetch these resources from NVD API. It is advised to set annvd_api_key
when setting this tonvd
.env name:
SECCERTS_PREFERRED_SOURCE_NVD_DATASETS