aboutsummaryrefslogtreecommitdiffhomepage
diff options
context:
space:
mode:
authormmstanone2021-07-28 22:59:07 +0200
committermmstanone2021-07-28 22:59:07 +0200
commit8b99d764425bdbc1cf880ee16e334f9a89026b4e (patch)
tree1eb283861984c519c1f704f549101b65a819189b
parent3830ef8f3062fd881706d1a78b0b1fb9ff958075 (diff)
downloadsec-certs-8b99d764425bdbc1cf880ee16e334f9a89026b4e.tar.gz
sec-certs-8b99d764425bdbc1cf880ee16e334f9a89026b4e.tar.zst
sec-certs-8b99d764425bdbc1cf880ee16e334f9a89026b4e.zip
WIP: FIPS CLI - readme
-rw-r--r--README.md116
-rw-r--r--setup.py2
2 files changed, 114 insertions, 4 deletions
diff --git a/README.md b/README.md
index 832698e7..03184cd4 100644
--- a/README.md
+++ b/README.md
@@ -11,9 +11,10 @@ This project is developed by the [Centre for Research On Cryptography and Securi
[![GitHub Workflow Status](https://img.shields.io/github/workflow/status/crocs-muni/sec-certs/Docker%20Image%20CI?label=Docker%20build&style=flat-square)](https://hub.docker.com/repository/docker/seccerts/sec-certs)
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crocs-muni/sec-certs/dev?filepath=notebooks%2Fcc_data_exploration.ipynb)
-## Installation (CC)
+## Installation
-The tool requires `Python >=3.8` and [pdftotext](https://www.xpdfreader.com/pdftotext-man.html) binary somewhere on the `PATH`.
+The tool requires `Python >=3.8`. Alongside `Python`, [pdftotext](https://www.xpdfreader.com/pdftotext-man.html), [graphviz](https://graphviz.org/),
+and [java](https://www.java.com/en) binaries must be accessible somewhere on `PATH`.
The stable release is published on [PyPi](https://pypi.org/project/sec-certs/) as well as on [DockerHub](https://hub.docker.com/repository/docker/seccerts/sec-certs), you can install it with:
@@ -35,7 +36,7 @@ source venv/bin/activate
pip install -e .
```
-## Usage
+## Usage (CC)
There are two main steps in exploring the world of Common Criteria certificates:
@@ -100,3 +101,112 @@ Options:
1. pull the image from the DockerHub repository : `docker pull seccerts/sec-certs`
2. run `docker run --volume ./processed_data:/opt/sec-certs/examples/debug_dataset -it seccerts/sec-certs`
3. All processed data will be in the `~/processed_data` directory
+
+
+## Usage (FIPS)
+
+
+
+Currently, the main goal of the FIPS module is to find dependencies between the certified products.
+
+#TODO MyBinder and online snapshot
+
+### Process FIPS data manually with Python
+
+You can also process FIPS data manually using `fips-certs` in terminal after installation.
+You can also use the `fips_cli.py` script.
+
+Calling `fips-certs --help` outputs following:
+```
+Usage: fips-certs [OPTIONS] [new-run|all|build|convert|update|web-scan|pdf-
+ scan|table-search|analysis|graphs]...
+
+ Specify actions, sequence of one or more strings from the following list:
+
+ ["new-run", "all", "build", "convert", "update", "web-scan", "pdf-scan",
+ "table-search", "analysis", "graphs"]
+
+ If 'new-run' is specified, a new dataset will be created and all the
+ actions will be run. If 'all' is specified, dataset will be updated and
+ all actions run against the dataset. Otherwise, only selected actions will
+ run in the correct order.
+
+ Dataset loading:
+
+ 'build' Create a skeleton of a new dataset from NIST pages.
+
+ 'update' Load a previously used dataset (created by 'build')
+ and update it with nonprocessed entries from NIST pages.
+
+ Both options download the files needed for analysis.
+
+ Analysis preparation:
+
+ 'convert' Convert all downloaded PDFs.
+
+ 'web-scan' Perform a scan of downloaded CMVP webpages.
+
+ 'pdf-scan' Perform a scan of downloaded CMVP security policy
+ documents - Keyword extraction.
+
+ 'table-search' Analyze algorithm implementation entries in tables in
+ security policy documents.
+
+ Analysis preparation actions are by default done only for
+ certificates, where each corresponding action failed. This
+ behaviour can be changed using '--redo-*' options. These actions
+ are also independent of each other.
+
+ Analysis:
+
+ 'analysis' Merge results from analysis preparation and find
+ dependencies between certificates.
+
+ 'graphs' Plot dependency graphs.
+
+Options:
+ -o, --output DIRECTORY Path where the output of the experiment will be
+ stored. May overwrite existing content.
+
+ -c, --config FILE Path to your own config yaml file that will
+ override the default one.
+
+ -i, --input FILE If set, the actions will be performed on a CC
+ dataset loaded from JSON from the input path.
+
+ -n, --name TEXT Name of the json object to be created in the
+ <<output>> directory. Defaults to
+ timestamp.json.
+
+ --no-download-algs Don't fetch new algorithm implementations
+ --redo-web-scan Redo HTML webpage scan from scratch
+ --redo-keyword-scan Redo PDF keyword scan from scratch
+ --higher-precision-results Redo table search for certificates with high
+ error rate. Behaviour undefined if used on a
+ newly instantiated dataset.
+
+ -s, --silent If set, will not print to stdout
+ --help Show this message and exit.
+```
+
+The *Analysis* part is designed to find dependecies between certificates.
+
+#### First run
+The first time you are using the FIPS module, use the following command:
+```
+fips-certs new-run --output <directory name> --name <dataset name>
+```
+where `<directory name>` is the name of the working directory of the FIPS module
+(e.g. where all the metadata will be stored), and `<dataset name>` is the name of the resulting dataset.
+
+This will download a large amount of data (4-5 GB) and can take up to 4 hours to finish.
+
+#### Next runs
+
+When a dataset is successfully created using `new-run`, you can use the command `all` to update the dataset
+(download latest files, redo scans for failed certificates, etc.). It is also **strongly advised** to use the `--higher-precision-results`
+switch on the **second run**. The following command should be used to update the dataset:
+```
+fips-certs all --input <path to the dataset>
+```
+where `<path to the dataset>` is the **path to the dataset file**, i.e. `<directory name>/<dataset name>.json` from the first run.
diff --git a/setup.py b/setup.py
index 758bf495..c1e97aa0 100644
--- a/setup.py
+++ b/setup.py
@@ -39,7 +39,7 @@ setup(
entry_points={
'console_scripts': [
'cc-certs=cc_cli:main',
- 'fips-certs=sec_certs.entrypoints.fips_certificates:main'
+ 'fips-certs=fips_cli:main'
]
}
)