Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added random sampling option for testing dataset in Kraken OCR #668

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jesusbft
Copy link

This pull request introduces a new feature that allows users to evaluate a random subset of the dataset during the testing phase. This is particularly useful for scenarios where testing the entire dataset is not necessary or feasible. Key details of the implementation include:

  • New Option: A --sample-percentage parameter was added to the test command, allowing users to specify the percentage (1-100) of the dataset to use for evaluation. The default is 100%.
  • Random Sampling: When a percentage lower than 100 is specified, a random subset of lines from the dataset is selected for testing.
  • Flexible Testing: Enables testing on the same dataset used for training while avoiding over-testing the entire dataset.

This feature helps in reducing testing time and resources while maintaining meaningful evaluation.

Let me know if you’d like further edits!

@jesusbft
Copy link
Author

jesusbft commented Dec 22, 2024

I propose adding an additional feature to calculate a case-insensitive Character Error Rate (CER) for the Kraken OCR test function. This metric would ignore errors caused by uppercase and lowercase differences in the predictions.

Use Case: OCR projects involving historical texts, where capitalization is often inconsistent. Scenarios where case differences are less important than the overall recognition accuracy.

The reports will appear like that:

=== report  ===

325047  Characters
17081   Errors
94.75%  Character Accuracy
95.09%  Character Accuracy (Case-insensitive)
80.31%  Word Accuracy

7322    Insertions
2110    Deletions
7649    Substitutions

Count   Missed  %Right
256356  10857   95.76%  Latin
61507   2712    95.59%  Common
7184    1402    80.48%  Inherited

Because it's a specific function that probably help only a small portion of users, may is better keep it private.

OBS: sorry sent it in the same branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant