# ECTester [![Build status](https://api.travis-ci.org/crocs-muni/ECTester.svg?branch=master)](https://travis-ci.org/crocs-muni/ECTester) [![GitHub release](https://img.shields.io/github/release/crocs-muni/ECTEster.svg)](https://github.com/crocs-muni/ECTester/releases) [![license](https://img.shields.io/github/license/crocs-muni/ECTester.svg)](https://github.com/crocs-muni/ECTester/blob/master/LICENSE) [![docs](https://img.shields.io/badge/docs-github.io-brightgreen.svg)](https://crocs-muni.github.io/ECTester/) Tests support and behavior of elliptic curve cryptography implementations on JavaCards (`TYPE_EC_FP` and `TYPE_EC_F2M`) and on selected software libraries. For more information on ECC support on JavaCards see the [github page](https://crocs-muni.github.io/ECTester/), with results, tables and docs. ## Build ECTester uses ant. There are three parts of ECTester, the JavaCard applet used for testing, the reader app which controls it and the standalone app which tests software libraries. ```bash git submodule update --init --recursive # To initialize submodules. ant -f build-reader.xml package # To build the reader tool (jar) -> "dist/ECTesterReader.jar" ant -f build-standalone.xml package # To build the standalone tool (jar) -> "dist/ECTesterStandalone.jar" ant -f build-applet.xml build # To build the applet (cap) -> "applet/ectester.cap". ``` Build produces both a lightweight version of the JARs and a full version of the JARs with dependencies included, the latter has the `*-dist.jar` suffix. The standalone build tries building test binaries for all the supported libraries, and silently fails if the library is not properly supported. ## JavaCard testing 1. Upload `!uploader/ectester.cap` using your favorite tool (e.g., [GlobalPlatformPro tool](https://github.com/martinpaljak/GlobalPlatform)) 2. Run `java -jar dist/ECTesterReader.jar -t` 3. Inspect output log with annotated results Following operations are tested: - Allocation of new KeyPair class for specified parameters - Generation of KeyPair with default curve - Setting of custom curve and KeyPair generation - Generation of shared secret via ECDH - Signature via ECDSA See `java -jar ECTesterReader.jar -h` for more. ### Options ``` -dsa,--ecdsa Sign data with ECDSA, [count] times. -t,--test Test ECC support. [test_suite]: - default: - invalid: - compression: - twist: - degenerate: - cofactor: - wrong: - composite: - test-vectors: -dh,--ecdh Do EC KeyAgreement (ECDH...), [count] times. -e,--export Export the defaut curve parameters of the card(if any). -V,--version Print version info. -ln,--list-named Print the list of supported named curves and keys. -h,--help Print help. -a,--all Test all curve sizes. -b,--bit-size Set curve size. -fp,--prime-field Use a prime field. -f2m,--binary-field Use a binary field. -c,--curve Use curve from file (field,a,b,gx,gy,r,k). -nc,--named-curve Use a named curve, from CurveDB: -u,--custom Use a custom curve (applet-side embedded, SECG curves). -npub,--named-public Use public key from KeyDB: -pub,--public Use public key from file (wx,wy). -priv,--private Use private key from file (s). -npriv,--named-private Use private key from KeyDB: -k,--key Use keyPair from file  (wx,wy,s). -nk,--named-key Use keyPair from KeyDB: -i,--input Input from file , for ECDSA signing. -o,--output Output into file . -l,--log Log output into file [log_file]. -v,--verbose Turn on verbose logging. --format Output format to use. One of: text,yml,xml. -f,--fresh Generate fresh keys (set domain parameters before every generation). --cleanup Send the cleanup command trigerring JCSystem.requestObjectDeletion() after some operations. -s,--simulate Simulate a card with jcardsim instead of using a terminal. -y,--yes Accept all warnings and prompts. -ka,--ka-type Set KeyAgreement object [type], corresponds to JC.KeyAgreement constants. -sig,--sig-type Set Signature object [type], corresponds to JC.Signature constants. -C,--color Print stuff with color, requires ANSI terminal. ``` ### Actions #### Export `-e / --export` Exports the default curves (if any) that are preset on the card. Use with `-o / --output [out_file]` to output the curve parameters to a file. For format of this file see [FORMAT](docs/FORMAT.md). #### Test `-t / --test [test_suite]` Perform support,performance and vulnerability tests of ECC. Use with `-o / --output [out_type:]` to output the test results to a file. For possible formats of this file see [FORMAT](docs/FORMAT.md). For more info about the test suites see [TESTS](docs/TESTS.md). #### Generate `-g / --generate [amount]` Generates batches of EC keypairs and exports them. Use with `-o / --output [out_file]` to output the generated keys to a file. For format of this file see [FORMAT](docs/FORMAT.md). #### ECDH `-dh / --ecdh [count]` Performs ECDH. Use with `-o / --output [out_file]` to output into a file. For format of this file see [FORMAT](docs/FORMAT.md). Respects the KeyAgreement type specified in `-ka / --ka-type [type]`. #### ECDSA `-dsa / --ecdsa [count]` Performs ECDSA. Useful with `-i / --input [in_file]` to sign the contents of a file. Use with `-o / --output [out_file]` to output into a file. For format of these files see [FORMAT](docs/FORMAT.md). Respects the Signature type specified in `-sig / --sig-type [type]`. #### List named curves `-ln / --list-named []` Lists categories of curves, keys and keypairs embedded in ECTester's jar, along with some information about them. These can be used as arguments to the `-n[c|k|pub|priv] / --named-[curve|key|public|private]` parameters. With the format: `category/name`. For example: `secg/secp192r1` identifies the SECG 192 bit prime field curve known as `secp192r1`. For more info about the curves and curve categories see [CURVES](docs/CURVES.md). ### Example Snippet below shows running the default test suite while simulating(`-s`), so using JCardSim. This shows that JCardsim simulates 112b Fp support with default curve present and supports ECDH, ECDHC and ECDSA. > java -jar ECTesterReader.jar -t -s ═══ Running test suite: default ═══ ═══ The default test suite tests basic support of ECDH and ECDSA. ═══ Date: 2018.05.02 20:29:38 ═══ ECTester version: v0.2.0 ═══ Card ATR: 3bfa1800008131fe454a434f5033315632333298 OK ┳ (0) Tests of 112b ALG_EC_FP support. ┃ SUCCESS ┃ All sub-tests matched the expected mask. ┣ OK ━ Allocate both keypairs 112b ALG_EC_FP ┃ SUCCESS ┃ 22 ms ┃ OK (0x9000) OK (0x9000) ┣ OK ━ Generate both keypairs ┃ SUCCESS ┃ 23 ms ┃ OK (0x9000) OK (0x9000) ┣ OK ━ Allocate both keypairs 112b ALG_EC_FP ┃ SUCCESS ┃ 0 ms ┃ OK (0x9000) OK (0x9000) ┣ OK ━ Set custom curve parameters on both keypairs ┃ SUCCESS ┃ 0 ms ┃ OK (0x9000) OK (0x9000) ┣ OK ━ Generate both keypairs ┃ SUCCESS ┃ 8 ms ┃ OK (0x9000) OK (0x9000) ┣ OK ┳ KeyAgreement tests. ┃ SUCCESS ┃ Some sub-tests did have the expected result. ┃ ┣ OK ┳ Test of the ALG_EC_SVDP_DH KeyAgreement. ┃ SUCCESS ┃ Some ECDH is supported. ┃ ┃ ┣ OK ━ Allocate KeyAgreement(ALG_EC_SVDP_DH) object ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┃ ┃ ┣ OK ━ ALG_EC_SVDP_DH of local pubkey and remote privkey(unchanged point) ┃ SUCCESS ┃ 2 ms ┃ OK (0x9000) ┃ ┃ ┣ OK ━ ALG_EC_SVDP_DH of local pubkey and remote privkey(COMPRESSED point) ┃ SUCCESS ┃ 2 ms ┃ OK (0x9000) ┃ ┃ ┗ OK ━ Mean = 1722885 ns, Median = 1718807 ns, Mode = 1614047 ns ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┃ ┣ OK ┳ Test of the ALG_EC_SVDP_DHC KeyAgreement. ┃ SUCCESS ┃ Some ECDH is supported. ┃ ┃ ┣ OK ━ Allocate KeyAgreement(ALG_EC_SVDP_DHC) object ┃ SUCCESS ┃ 0 ms ┃ OK (0x9000) ┃ ┃ ┣ OK ━ ALG_EC_SVDP_DHC of local pubkey and remote privkey(unchanged point) ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┃ ┃ ┣ OK ━ ALG_EC_SVDP_DHC of local pubkey and remote privkey(COMPRESSED point) ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┃ ┃ ┗ OK ━ Mean = 1563980 ns, Median = 1549170 ns, Mode = 1514747 ns ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┃ ┣ NOK ━ Allocate KeyAgreement(ALG_EC_SVDP_DH_PLAIN) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┃ ┣ NOK ━ Allocate KeyAgreement(ALG_EC_SVDP_DHC_PLAIN) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┃ ┣ NOK ━ Allocate KeyAgreement(ALG_EC_PACE_GM) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┃ ┗ NOK ━ Allocate KeyAgreement(ALG_EC_SVDP_DH_PLAIN_XY) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┗ OK ┳ Signature tests. ┃ SUCCESS ┃ Some sub-tests did have the expected result. ┣ OK ┳ Test of the ALG_ECDSA_SHA signature. ┃ SUCCESS ┃ All sub-tests had the expected result. ┃ ┣ OK ━ Allocate Signature(ALG_ECDSA_SHA) object ┃ SUCCESS ┃ 3 ms ┃ OK (0x9000) ┃ ┣ OK ━ ALG_ECDSA_SHA with local keypair(random data) ┃ SUCCESS ┃ 14 ms ┃ OK (0x9000) ┃ ┣ OK ━ Sign (Mean = 1890914 ns, Median = 1500125 ns, Mode = 1422588 ns) ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┃ ┗ OK ━ Verify (Mean = 1873952 ns, Median = 1870348 ns, Mode = 1843902 ns) ┃ SUCCESS ┃ 1 ms ┃ OK (0x9000) ┣ NOK ━ Allocate Signature(ALG_ECDSA_SHA_224) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┣ NOK ━ Allocate Signature(ALG_ECDSA_SHA_256) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┣ NOK ━ Allocate Signature(ALG_ECDSA_SHA_384) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) ┗ NOK ━ Allocate Signature(ALG_ECDSA_SHA_512) object ┃ FAILURE ┃ 0 ms ┃ fail (NO_SUCH_ALG, 0x0003) #### Legend - Some general information about the test suite and card is output first, test data follows after. - The **OK**/**NOK** values on the left represent the complete evaluated result of a test, as a test can be expected to succeed or fail, this is different than the values on the right: - **SUCCESS**: Is **OK**, the test was expected to pass and it did. - **FAILURE**: Is **NOK**, the test was expected to pass, but it did not. - **UXSUCCESS**: Is **NOK**, the test was expected to fail, but it did not. - **XFAILURE**: Is **OK**, the test was expected to fail, and it did. - **ERROR**: Is **NOK** an unexpected error during testing arose. - The tests can be compounded into compound tests, which are visible as a tree of tests and sub-tests. - The duration of non-compound tests is shown in the third column, this is a rough estimate, measured from before the APDU is sent, to just after the response is received. - The cause of the test result, is shown in the last column, for non-compound tests, these are JavaCard(or custom) status words, from operations done on the card. If you are interested in testing support for other JavaCard algorithms, please visit JCAlgTester project: https://github.com/crocs-muni/JCAlgTest ## Standalone library testing Currently supported libraries include: - BouncyCastle - SunEC - OpenSSL - Crypto++ - libtomcrypt - botan - Microsoft CNG For more information on ECC libraries see [LIBS](docs/LIBS.md). ### Setup Installing the Java Cryptography Extension Unlimited Strength policy files is necessary to do testing with quite a lot of practical key sizes, they are available for download: * [Java 6](http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.html) * [Java 7](http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html) * [Java 8](http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html) To install, place them in `${java.home}/jre/lib/security/`. ### Options ``` usage: ECTesterStandalone.jar [-V] [-h] [ (ecdh [-t ] [-n ] [-b ] [-nc ]) | (ecdsa [-t ] [-n ] [-b ] [-nc ] [-f ]) | (export [-t ] [-b ]) | (generate [-nc ] [-n ] [-t ] [-b ]) | (list-data [what]) | (list-libs) | (test [-gt ] [-kt ] [-st ] [-b ] [-nc ]) ] [lib] -V,--version Print version info. -h,--help Print help. [lib] What library to use. ecdh: -t,--type Set KeyAgreement object [type]. -n,--amount Do ECDH [amount] times. -b,--bits What size of curve to use. -nc,--named-curve Use a named curve, from CurveDB: ecdsa: -t,--type Set Signature object [type]. -n,--amount Do ECDSA [amount] times. -b,--bits What size of curve to use. -nc,--named-curve Use a named curve, from CurveDB: -f,--file Input [file] to sign. export: -t,--type Set KeyPair object [type]. -b,--bits What size of curve to use. generate: -nc,--named-curve Use a named curve, from CurveDB: -n,--amount Generate [amount] of EC keys. -t,--type Set KeyPairGenerator object [type]. -b,--bits What size of curve to use. list-data: [what] what to list. list-libs: test: -gt,--kpg-type Set the KeyPairGenerator object [type]. -kt,--ka-type Set the KeyAgreement object [type]. -st,--sig-type Set the Signature object [type]. -b,--bits What size of curve to use. -nc,--named-curve Use a named curve, from CurveDB: ```