Merge pull request #299 from crocs-muni/issue/298-OCR-processing-fails

Issue/298 ocr processing fails, delete draft release action, download spacy model during test pipeline

4 files changed, 11 insertions, 27 deletions

diff --git a/.github/workflows/draft_release.yml b/.github/workflows/draft_release.yml
deleted file mode 100644
index 55c6a38a..00000000
--- a/.github/workflows/draft_release.yml
+++ /dev/null
@@ -1,19 +0,0 @@
-name: Create release draft
-
-on:
-  push:
-    tags:
-      - "*.*.*"
-
-jobs:
-  draft-release:
-    runs-on: ubuntu-latest
-    if: (github.repository == 'crocs-muni/sec-certs') && (github.ref == 'refs/heads/main')
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-      - name: Release
-        uses: softprops/action-gh-release@v1
-        with:
-          draft: true
-          generate_release_notes: true
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 8f2a037c..316e8679 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -21,7 +21,9 @@ jobs:
           pip install -r requirements/requirements.txt
           pip install -r requirements/test_requirements.txt
       - name: Install sec-certs
-        run: pip install -e .
+        run: |
+          pip install -e .
+          python -m spacy download en_core_web_sm
       - name: Run tests
         run: pytest --cov=sec_certs tests
       - name: Code coverage upload
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 958ba073..dd00667f 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -25,8 +25,6 @@ Requirements are maintained with [pip-tools](https://github.com/jazzband/pip-too
 
 ## Releases and version strings
 
-- On each revision pushed onto `main` that has `*.*.*` tag, a draft release is created with prepared changelog (this step can be skipped and the Release created right from the GitHub GUI).
-- This draft release is to be published manually by the maintainer.
 - Version string is not indexed in `git` but can be retreived maintained by `setuptools-scm` from git tags instead.
 - `setuptools-scm` will automatically, upon editable/real install of a package, infer its version and write it to `sec_certs/_version.py`. This file is not indexed as well. See more at [setuptools-scm GitHub](https://github.com/pypa/setuptools_scm)
 - On publishing a release, the tool is automatically published to [PyPi](https://pypi.org/project/sec-certs/) and [DockerHub](https://hub.docker.com/repository/docker/seccerts/sec-certs).
@@ -35,9 +33,8 @@ Note on single-sourcing the package version: More can be read [here](https://pac
 
 ### Currently, the release process is as follows
 
-1. (skip this optionally) Tag a revision with `*.*.*` tag -- this will create a draft release in GitHub.
-2. Modify changelog and publish the release (or create it from scratch with new tag).
-3. This will automatically update PyPi and DockerHub packages.
+1. Create a release from GitHub UI. Include release notes, add proper version tag and publish the release (or create it from scratch with new tag).
+2. This will automatically update PyPi and DockerHub packages.
 
 
 ## Quality assurance
diff --git a/src/sec_certs/utils/pdf.py b/src/sec_certs/utils/pdf.py
index 1d04a697..d97dff95 100644
--- a/src/sec_certs/utils/pdf.py
+++ b/src/sec_certs/utils/pdf.py
@@ -59,9 +59,13 @@ def ocr_pdf_file(pdf_path: Path) -> str:
             )
             if tes.returncode != 0:
                 raise ValueError(f"tesseract failed: {tes.returncode}")
+
         contents = ""
-        txt_paths = list(glob.glob(str(tmppath / "image*.txt")))
-        for txt_path in map(Path, sorted(txt_paths, key=lambda fname: int(fname[6:-4]))):
+
+        txt_paths = [x for x in tmppath.iterdir() if x.is_file() and "image-" in x.stem and x.suffix == ".txt"]
+        txt_paths = sorted(txt_paths, key=lambda txt_path: int(txt_path.stem.split("-")[1]))
+
+        for txt_path in txt_paths:
             with txt_path.open("r", encoding="utf-8") as f:
                 contents += f.read()
     return contents