Protection profiles example

This notebook illustrated basic functionality of the ProtectionProfileDataset class that holds protection profiles bound to Common Criteria certified products. The object that holds a single profile is called ProtectionProfile.

Note that there exists a front end to this functionality at sec-certs.org/pp. Before reinventing the wheel, it’s good idea to check our web. Maybe you don’t even need to run the code, but just use our web instead.

For full API documentation of the ProtectionProfileDataset class go to the dataset docs.

from sec_certs.dataset import ProtectionProfileDataset
from sec_certs.sample import ProtectionProfile

Get fresh dataset snapshot from mirror

There’s no need to do full processing of the dataset by yourself, unless you modified sec-certs code. You can simply fetch the processed version from the web.

dset = ProtectionProfileDataset.from_web()
print(len(dset)) # Print number of protection profiles in the dataset

Do some basic dataset serialization

The dataset can be saved/loaded into/from json

dset.to_json("./pp.json")
new_dset: ProtectionProfileDataset = ProtectionProfileDataset.from_json("./pp.json")
assert dset == new_dset

Simple dataset manipulation

The samples of the dataset are stored in a dictionary that maps sample’s primary key (we call it dgst) to the ProtectionProfile object. The primary key of the protection profile is simply a hash of the attributes that make the sample unique.

You can iterate over the dataset which is handy when selecting some subset of protection profiles.

for pp in dset:
    pass

# Get only collaborative protection profiles
collaborative_pps = [x for x in dset if x.web_data.is_collaborative]

# Get protection_profiles from 2015 and newer
from datetime import date
newer_than_2015 = [x for x in dset if x.web_data.not_valid_before > date(2014, 12, 31)]

Dissect a single protection profiles

The ProtectionProfile is basically a data structure that holds all the data we keep about a protection profile. Other classes (ProtectionProfile or model package members) are used to transform and process the samples. You can see all its attributes at API docs.

# Select a protection profile and print some attributes
pp: ProtectionProfile = dset["b02ed76d2545326a"]
print(f"{pp.name=}")
print(f"{pp.web_data.not_valid_before=}")
print(f"{pp.pdf_data=}")

Serialize a single protection profile

Again, a protection profile can be (de)serialized into/from json.

pp.to_json("./pp.json")
new_pp = pp.from_json("./pp.json")
assert pp == new_pp

Create new dataset and fully process it

The following piece of code roughly corresponds to $ sec-certs pp all CLI command – it fully processes the PP pipeline. This will create a folder in current working directory where the outputs will be stored.

Warning

It’s not good idea to run this from notebook. It may take several hours to finish. We recommend using from_web() or turning this into a Python script.

dset = ProtectionProfileDataset()
dset.get_certs_from_web()
dset.process_auxiliary_datasets()
dset.download_all_artifacts()
dset.convert_all_pdfs()
dset.analyze_certificates()