Model package#

This package exposes model (mostly transformers and classifiers) that apply complex transformations. These are to be leveraged by members of Dataset package and are directly applied on members of Sample class (or on built-in objects).

Tip

The examples related to this package can be found in the model notebook.

CPEClassifier#

class sec_certs.model.CPEClassifier(match_threshold=80, n_max_matches=10, spacy_model_to_use='en_core_web_sm')#

Class that can predict CPE matches for certificate instances. Adheres to sklearn sklearn.base.BaseEstimator interface. Fit method is called on list of CPEs and build two look-up dictionaries, see description of attributes.

fit(X, y=None)#

Just creates look-up structures from provided list of CPEs

Parameters:

  • X (List[CPE]) – List of CPEs that can be matched with predict()

  • y (Optional[List[str]]) – will be ignored, specified to adhere to sklearn BaseEstimator interface, defaults to None

Return CPEClassifier:

return self to allow method chaining

predict(X)#

Will predict CPE uris for List of Tuples (vendor, product name, identified versions in product name)

Parameters:

X (List[Tuple[str, str, str]]) – tuples (vendor, product name, identified versions in product name)

Return List[Optional[Set[str]]]:

List of CPE uris that correspond to given input, None if nothing was found.

predict_single_cert(vendor, product_name, versions, relax_version=False, relax_title=False)#

Predict List of CPE uris for triplet (vendor, product_name, list_of_versions). The prediction is made as follows: 1. Sanitize vendor name, lemmatize product name. 2. Find vendors in CPE dataset that are related to the certificate 3. Based on (vendors, versions) find all CPE items that are considered as candidates for match 4. Compute string similarity of the candidate CPE matches and certificate name 5. Evaluate best string similarity, if above threshold, declare it a match. 6. If no CPE item is matched, try again but relax version and check CPEs that don’t have their version specified. 7. (Also, search for 100% CPE matches on item name instead of title.)

Parameters:

  • vendor (Optional[str]) – manufacturer of the certificate

  • product_name (str) – name of the certificate

  • versions (Set[str]) – List of versions that appear in the certificate name

  • relax_version (bool) – See step 6 above., defaults to False

  • relax_title (bool) – See step 7 above, defaults to False

Return Optional[Set[str]]:

Set of matching CPE uris, None if no matches found

SARTranformer#

class sec_certs.model.SARTransformer#

Class for transforming SARs defined in st_keywords and report_keywords dictionaries into SAR objects. This class implements sklearn.base.Transformer interface, so fit_transform() can be called on it.

fit(certificates)#

Just returns self, no fitting needed

Parameters:

certificates (Iterable[CCCertificate]) – Unused parameter

Return SARTransformer:

return self

transform(certificates)#

Just a wrapper around transform_single_cert() called on an iterable of CCCertificate.

Parameters:

certificates (Iterable[CCCertificate]) – Iterable of CCCertificate objects to perform the extraction on.

Return List[Optional[Set[SAR]]]:

Returns List of results from transform_single_cert().

transform_single_cert(cert)#

Given CCCertificate, will transform SAR keywords extracted from txt files into a set of SAR objects. Also handles extractin of correct SAR levels, duplicities and filtering. Uses three sources: CSV scan, security target, and certification report. The caller should assure that the certificates have the keywords extracted.

Parameters:

cert (CCCertificate) – Certificate to extract SARs from

Return Optional[Set[SAR]]:

Set of SARs, None if none were identified.

ReferenceFinder#

class sec_certs.model.ReferenceFinder#

The class assigns references of other certificate instances for each instance. Adheres to sklearn BaseEstimator interface. The fit is called on a dictionary of certificates, builds a hashmap of references, and assigns references for each certificate in the dictionary.

property duplicates#

Get the duplicates in the fitted dataset.

Return IDMapping:

Mapping of certificate ID to digests that share it.

fit(certificates, id_func, ref_lookup_func)#

Builds a list of references and assigns references for each certificate instance.

Parameters:

  • certificates (Certificates) – dictionary of certificates with hashes as key

  • id_func (IDLookupFunc) – lookup function for cert id

  • ref_lookup_func (ReferenceLookupFunc) – lookup for references

predict(dgst_list, keep_unknowns=True)#

Get the references for a list of certificate digests.

Parameters:

  • dgst_list – List of certificate digests.

  • keep_unknowns – Whether to keep references to and from unknown certificate IDs

Return Dict[str, References]:

Dict with certificate hash and References object.

predict_single_cert(dgst, keep_unknowns=True)#

Get the references object for specified certificate digest.

Parameters:

  • dgst – certificate digest

  • keep_unknowns – Whether to keep references to unknown certificate IDs

Return References:

References object

property unknown_references#

Get the unknown references in the fitted dataset (to unknown certificate IDs, not in the dataset during fit).

TransitiveVulnerabilityFinder#

class sec_certs.model.TransitiveVulnerabilityFinder(id_func)#

The class assigns vulnerabilities to each certificate instance caused by references among certificate instances. Adheres to sklearn BaseEstimator interface.

fit(certificates, ref_func)#

Method assigns each certificate vulnerabilities caused by references among certificates

Parameters:

certificates (Certificates) – Dictionary of certificates with digests

Return Vulnerabilities:

Dictionary of vulnerabilities of certificate instances

predict(dgst_list)#

Method returns vulnerabilities for a list of certificate digests

Parameters:

dgst_list (List[str]) – list of certificate digests

Return Dict[str, TransitiveCVE]:

Dictionary of TransitiveCVE objects for specified certificate digests

predict_single_cert(dgst)#

Method returns vulnerabilities for certificate digest

Parameters:

dgst (str) – Digest of certificate

Return TransitiveCVE:

TransitiveCVE object of certificate