aboutsummaryrefslogtreecommitdiffhomepage
path: root/paper_artifacts/chain_of_trust/data/model_evaluation/readme.md
blob: 7d93be173ac78c07411e612b4261c0633e76d9e8 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
## Reference annotation evaluation

- Balanced accuracy was used to optimimize the hyperparameters of the reference annotation task.
- Note that the balanced accuracy is defined as an average of recall over all classes
- Two classification problems:
  - 5 classes
  - 2 classes
- Three variants for each of the problems:
  - Random (prior knowledge) baseline
  - TF-IDF for feature extraction, then all as normal
  - Embeddings for feature extraction
- The respective folders contain results for the individual variants.