Skip to content

Commit

Permalink
Merge pull request #32 from worldcoin/dev
Browse files Browse the repository at this point in the history
`v1.1.0`
  • Loading branch information
wiktorlazarski authored Jun 12, 2024
2 parents 1aa9ade + c4ef153 commit d19b8e3
Show file tree
Hide file tree
Showing 101 changed files with 1,839 additions and 426 deletions.
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Please, give a brief description of what was changed or fixed and how you did it
## Checklist
<!-- Please make sure you did all pre review requesting steps and check all with an 'x' ([x]). -->

- [ ] I've made sure that my code works as expected by writting unit tests.
- [ ] I've made sure that my code works as expected by writing unit tests.
- [ ] I've checked if my code doesn't generate warnings or errors.
- [ ] I've performed a self-review of my code.
- [ ] I've made sure that my code follows the style guidelines of the project.
Expand Down
20 changes: 12 additions & 8 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,15 @@ We pledge to act and interact in ways that contribute to an open, welcoming, div
## Our Standards

Examples of behavior that contributes to a positive environment for our community include:

- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
- Focusing on what is best not just for us as individuals, but for the overall community

Examples of unacceptable behavior include:

- The use of sexualized language or imagery, and sexual attention or advances of any kind
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
Expand All @@ -24,8 +26,9 @@ Examples of unacceptable behavior include:
- Other conduct which could reasonably be considered inappropriate in a professional setting

The Worldcoin open source community is united in its conviction that its open source materials not be used:
- for mass surveillance of individuals or for unlawful individual surveillance or other unlawful targeted actions, including on the basis on race, gender, sexual orientation, or religion; or
- in any activity or manner that violates, or supports, assists, facilitates, enables, constitutes or is otherwise deemed to be a violation of the Universal Declaration of Human Rights (https://www.un.org/en/universaldeclaration-human-rights/), the International Covenant on Civil and Political Rights (https://www.ohchr.org/en/professionalinterest/pages/ccpr.aspx), and the International Labor Organization Declaration on Fundamental Principles and Rights at Work (https://www.ilo.org/declaration/thedeclaration/textdeclaration/lang--en/index.htm), or any local laws implementing those instruments.

- For mass surveillance of individuals or for unlawful individual surveillance or other unlawful targeted actions, including on the basis on race, gender, sexual orientation, or religion; or
- In any activity or manner that violates, or supports, assists, facilitates, enables, constitutes or is otherwise deemed to be a violation of the Universal Declaration of Human Rights (https://www.un.org/en/universaldeclaration-human-rights/), the International Covenant on Civil and Political Rights (https://www.ohchr.org/en/professionalinterest/pages/ccpr.aspx), the International Labor Organization Declaration on Fundamental Principles and Rights at Work (https://www.ilo.org/declaration/thedeclaration/textdeclaration/lang--en/index.htm), or any local laws implementing those instruments.

## Enforcement Responsibilities

Expand All @@ -35,17 +38,18 @@ Community leaders have the right and responsibility to remove, edit, or reject c

## Scope

This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official email address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
This Code of Conduct also applies to actions taken outside of these spaces, and which have a negative impact on community health.
This Code of Conduct applies within all project spaces, and it also applies when an individual is representing the project or its community in public spaces. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

This Code of Conduct also applies outside the project spaces when there is a reasonable belief that an individual's behavior may have a negative impact on the project or its community.

## Enforcement

We encourage all communities to resolve issues on their own whenever possible. Instances of abusive, harassing, or otherwise unacceptable behavior should be reported to the community leaders responsible for enforcement in a given project or to [email protected]. All project and community leaders are obligated to respect the privacy and security of the reporter of any incident.

## Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.

Community Impact Guidelines were inspired by Mozilla’s code of conduct enforcement ladder.
This Code of Conduct is adapted from the Contributor Covenant, version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html

Expanding scope to include external impact on community health inspired by Microsoft’s Open Source Code of Conduct.
For answers to common questions about this code of conduct, see https://www.contributor-covenant.org/faq

For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations.
Expanding scope to include impact on community health inspired by Meta's Open Source Code of Conduct.
33 changes: 18 additions & 15 deletions IRIS_PERFORMANCE_CARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

## ICE 2005 dataset

ICE 2005 dataset is used by the National Institute of Standards and Technology (NIST) in Iris Challenge Evaluation (ICE) 2005 [1] and was made publicly available as part of the ND-Iris-0405 dataset from the University of Notre Dame du Lac [2]. This dataset was collected from 132 subjects using an LG EOU 2200 scanner with image resolution 640x480.
ICE 2005 dataset is used by the National Institute of Standards and Technology (NIST) in Iris Challenge Evaluation (ICE) 2005 [1] and was made publicly available as part of the ND-Iris-0405 dataset from the University of Notre Dame [2]. This dataset includes 2953 images collected from 132 subjects using an LG EOU 2200 scanner at 640x480 image resolution.

## Comparison with ICE 2005 participants (commercial and academia)
## Comparison with ICE 2005 participants

A list of public and private entities participated in the ICE 2005 is shown below. The full evaluation report can be found in [1].

<img src="./docs/performance_card/ice2005_results.png" alt="ICE 2005 results">
<img src="./docs/performance_card/ice2005_results.png" width="800" alt="ICE 2005 results">

Fig. 1(a) and 1(b) show the ROC on right and left eye, respectively, from all participants in Iris Challenge Evaluation (ICE) 2005 and Fig. 2(a) and 2(b) show our ROC in comparison. Although the exact error rates were not published in [1], these plots show IRIS performs as good as if not better than commercial algorithms available at the time.
Fig. 1(a) and 1(b) show the ROC on right and left eye, respectively, from all participants in Iris Challenge Evaluation (ICE) 2005 and Fig. 2(a) and 2(b) show our ROC in comparison*. Although the exact error rates were not published in [1], these plots show IRIS performs as good as if not better than commercial algorithms available at the time.

|<img src="./docs/performance_card/comparison_1a.png" alt="1a"> Fig. 1(a) participant ROC on ICE 2005 (right eye) [1]| <img src="./docs/performance_card/comparison_1b.png" alt="1b"> Fig. 1(b) participant ROC on ICE 2005 (left eye) [1]|
|---|---|
Expand All @@ -19,16 +19,18 @@ Fig. 1(a) and 1(b) show the ROC on right and left eye, respectively, from all pa

## Comparison with other open source algorithms

We also compare our performance with the latest available iris open source algorithm OSIRIS, which has three versions OSIRISV2, OSIRISV4, OSIRISV4.1 [3]. As we can see in the table below, IRIS achieves significantly lower False NonMatch Rate (FNMR) at False Match Rate (FMR) of 0.001 and 0.0001 than OSIRIS.
We also compare our performance with the latest available iris open source algorithm OSIRIS, which has three versions OSIRISV2, OSIRISV4, OSIRISV4.1 [3] as well as HDBIF [4]. As we can see in the table below, IRIS achieves significantly lower False NonMatch Rate (FNMR) at False Match Rate (FMR) of 0.001 and 0.0001 than these open source algorithms.

| | OSIRISV2 | OSIRISV3 | OSIRISV4 | IRIS |
|-------------------|----------|----------|----------|-------|
| FNMR @ FMR=0.001 | 0.174 | 0.031 | 0.019 | 0.003 |
| FNMR @ FMR=0.0001 | 0.268 | 0.058 | 0.034 | 0.006 |
| | OSIRISV2 <br>[2009] | OSIRISV4 <br>[2011] | OSIRISV4.1 <br>[2016] | HDBIF <br>[2019] | IRIS\* <br>[2023] |
|-------------------|----------|----------|------------|-------|--------|
| FNMR @ FMR=0.001 | 0.174 | 0.031 | 0.019 | 0.025 | 0.003 |
| FNMR @ FMR=0.0001 | 0.268 | 0.058 | 0.034 | 0.039 | 0.006 |

\*Note that four left eye images failed the default enrollment setting in IRIS with a Failure-To-Enroll (FTE) rate of 0.00135.

## ND-LG4000 dataset

ND-LG4000 dataset is part of the ND-CrossSensor-Iris-2013 public dataset from the University of Notre Dame [4]. This dataset was collected from 676 subjects using LG IrisAccess 4000 scanner with image resolution 640x480. We randomly chose 541 subjects for training, 67 subjects for validating the segmentation model in IRIS. The remaining 2187 images from 68 subjects were used for testing, resulting in a total of ~2.4 million matches.
ND-LG4000 dataset is part of the ND-CrossSensor-Iris-2013 public dataset from the University of Notre Dame [5]. This dataset was collected from 676 subjects using LG IrisAccess 4000 scanner with image resolution 640x480. We randomly chose 541 subjects for training, 67 subjects for validating the segmentation model in IRIS. The remaining 2187 images from 68 subjects were used for testing, resulting in a total of ~2.4 million matches.

|<img src="./docs/performance_card/lg4000_3a.png" alt="3a"> Fig. 3(a) IRIS ROC on ND-LG4000| <img src="./docs/performance_card/lg4000_3b.png" alt="3b"> Fig. 3(b) IRIS DET on ND-LG4000|
|---|---|
Expand All @@ -40,19 +42,20 @@ Fig. 3(a) shows the ROC for ND-LG4000 dataset, where Fig. 3(b) shows the DET in

**_Disclaimer: This data was not collected from Worldcoin users during field operations, but stems primarily from paid participants in a dedicated workstream, separately administered by a respected partner._**

The WLD-InHouse-v2 dataset was internally collected to simulate more challenging environments in both indoor and outdoor settings using the Orb [5]. This dataset comes from a research workstream which is isolated from Worldcoin’s operations and includes more than 2k images from 98 subjects with high resolution 1440x1080, resulting in a total of ~2.6 million matches.
The WLD-InHouse-v2 dataset was internally collected to simulate more challenging environments in both indoor and outdoor settings using the Orb [6]. This dataset comes from a research workstream which is isolated from Worldcoin’s operations and includes more than 2k images from 98 subjects with high resolution 1440x1080, resulting in a total of ~2.6 million matches.

|<img src="./docs/performance_card/wld_dataset_4a.png" alt="4a"> Fig. 4(a) IRIS* ROC on WLD-InHouse-v2| <img src="./docs/performance_card/wld_dataset_4b.png" alt="4b"> Fig. 4(b) IRIS* DET on WLD-InHouse-v2|
|---|---|

> **Note**: IRIS* is an optimised version of IRIS to accommodate high-resolution images captured by the Orb.
Because of the high quality image capture enabled by the Orb and the effectiveness of iris encoding enabled by IRIS*, we are able to achieve high accuracy with FNMR less than 0.0001 at FMR of 0.000001 in WLD-InHouse-v2 dataset.
Note that in the above public dataset tests, errors are mainly caused by poor image quality due to offgaze, motion blur and occlusion. These errors are greatly reduced in our WLD-InHouse-v2 dataset due to the auto-focus capability and stringent image quality control of the Orb. Together with the effectiveness of iris encoding enabled by IRIS*, we are able to achieve high accuracy with FNMR less than 0.0001 at FMR of 0.000001 in WLD-InHouse-v2 dataset.

## References

1. Phillips, P. , Bowyer, K. , Flynn, P. , Liu, X. and Scruggs, W. (2008), The Iris Challenge Evaluation 2005, IEEE Second International Conference on Biometrics: Theory, Applications and Systems (BTAS 08), Arlington, VA, (https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=890057)
2. Bowyer, K. , Flynn, P. (2016), The ND-IRIS-0405 Iris Image Dataset, (https://arxiv.org/abs/1606.04853)
3. Othman, N. , Dorizzi, B. , Garcia-Salicetti, S. (2016), OSIRIS: An open source iris recognition software, Pattern Recognition Letters, vol.82, part2, pp.124-131, 2016. (https://github.com/tohki/iris-osiris)
4. The ND-CrossSensor-Iris-2013 Dataset (Accessed 2018), (https://cvrl.nd.edu/projects/data/#nd-crosssensor-iris-2013-data-set)
5. The Worldcoin Foundation blog (2023), Opening The Orb: A look inside Worldcoin’s biometric imaging device, (https://worldcoin.org/blog/engineering/opening-orb-look-inside-worldcoin-biometric-imaging-device)
3. Othman, N. , Dorizzi, B. , Garcia-Salicetti, S. (2016), OSIRIS: An open source iris recognition software, Pattern Recognition Letters, vol.82, part2, pp.124-131, 2016. (https://github.com/tohki/iris-osiris)
4. Czajka, A. , Moreira, D. , Bowyer K. and Flynn, P. (2019), Domain-Specific Human-Inspired Binarized Statistical Image Features for Iris Recognition, IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, pp. 959-967, (https://ieeexplore.ieee.org/document/8658238, https://github.com/CVRL/OpenSourceIrisRecognition)
5. The ND-CrossSensor-Iris-2013 Dataset (Accessed 2018), (https://cvrl.nd.edu/projects/data/#nd-crosssensor-iris-2013-data-set)
6. The Worldcoin Foundation blog (2023), Opening The Orb: A look inside Worldcoin’s biometric imaging device, (https://worldcoin.org/blog/engineering/opening-orb-look-inside-worldcoin-biometric-imaging-device)
31 changes: 25 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ______________________________________________________________________
<div align="center">

<p align="center">
<a href="https://worldcoin.org/"><img src="https://github.com/worldcoin/open-iris/blob/main/docs/source/images/logos/wld.png" width=150px></img></a>
<a href="https://worldcoin.org/"><img src="https://github.com/worldcoin/open-iris/blob/main/docs/source/images/logos/wld.png?raw=true" width=150px></img></a>
</p>

# **_IRIS: Iris Recognition Inference System_**
Expand Down Expand Up @@ -31,10 +31,12 @@ ______________________________________________________________________
- [Installation](#installation)
- [Setup for development](#setup-for-development)
- [Running inference](#Running-inference)
- [Disclaimer](#disclaimer)
- [Project structure](#project-structure)
- [Example notebooks](#example-notebooks)
- [Documentation](#documentation)
- [Issues, pull requests and feature requests](#issues-pull-requests-and-feature-requests)
- [Citation](#citation)
- [License](#license)
- [Resources](#resources)

Expand Down Expand Up @@ -72,14 +74,14 @@ _The Iris Recognition Inference System (IRIS) software repository is owned and m
### Installation

Installation is as simple as running `pip install` with specifying `IRIS_ENV` installation global flag (`IRIS_ENV` flag may be skipped if `iris` is installed from PyPl server but this option is only available when `iris` is installed on local machine). The `IRIS_ENV` flag is used to indicate an "environment" in which package is meant to work. Possible options are:
1. `SERVER` - For installing `iris` package with dependecies required for running an inference on a local machines.
1. `SERVER` - For installing `iris` package with dependencies required for running an inference on a local machines.
```bash
# On a local machine
pip install open-iris
# or directly from GitHub
IRIS_ENV=SERVER pip install git+https://github.com/worldcoin/open-iris.git
```
2. `ORB` - For installing `iris` package with dependecies required for running an inference on the Orb.
2. `ORB` - For installing `iris` package with dependencies required for running an inference on the Orb.
```bash
# On the Orb
IRIS_ENV=ORB pip install git+https://github.com/worldcoin/open-iris.git
Expand Down Expand Up @@ -148,25 +150,42 @@ The `iris` package features a structured design with modular components, enhanci
| [`iris.nodes`](https://worldcoin.github.io/open-iris/) | a module that contains implementation of iris recognition pipeline nodes |
| [`iris.orchestration`](https://worldcoin.github.io/open-iris/) | a module that contains iris recognition pipeline's orchestration support mechanisms |
| [`iris.pipelines`](https://worldcoin.github.io/open-iris/) | a module that contains implementation of iris recognition pipelines |
| [`iris.utils`](https://worldcoin.github.io/open-iris/) | a module that contains utilities used throughout the code base and modules usefull for outputs analysis |
| [`iris.utils`](https://worldcoin.github.io/open-iris/) | a module that contains utilities used throughout the code base and modules useful for outputs analysis |

## Example notebooks

The Jupyter Notebooks provided present practical guides and real-world instances to demonstrate the complete capabilities of the `iris` package.

1. **Getting started** [![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/worldcoin/open-iris/blob/main/colab/GettingStarted.ipynb)
2. **Configuring custom pipeline** [![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/worldcoin/open-iris/blob/main/colab/ConfiguringCustomPipeline.ipynb)
3. **Matching entities** [![Open In Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/worldcoin/open-iris/blob/main/colab/MatchingEntities.ipynb)

## Documentation

For detailed documentation, including installation instructions, usage guidelines, and configuration options, please refer to the IRIS project [documentation](https://worldcoin.github.io/open-iris/).

## Issues, pull requests and feature requests

If you have any question or you found a bug or you feel like some feature is missing, please don't hesitate to file a new issue, discussion or PR with respective title and description.
If you have any question or you found a bug or you feel like some feature is missing, please don't hesitate to file a new [issue](https://github.com/worldcoin/open-iris/issues), discussion or [PR](https://github.com/worldcoin/open-iris/pulls) with respective title and description.
Any suggestion for potential project improvements are and will always be welcome!

Feel free to reach out to any project [contributor](https://github.com/worldcoin/open-iris/graphs/contributors)!
## Contact

If you have any questions, suggestions, or feedback, feel free to reach out to us:

- Email: [[email protected]](mailto:[email protected])
- GitHub Issues: [Open an issue](https://github.com/worldcoin/open-iris/issues)
- Contributors: Feel free to reach out to any project [contributor](https://github.com/worldcoin/open-iris/graphs/contributors) directly!

## Citation
```BibTeX
@misc{wldiris,
author = {Worldcoin AI},
title = {IRIS: Iris Recognition Inference System of the Worldcoin project},
year = {2023},
url = {https://github.com/worldcoin/open-iris}
}
```

## License

Expand Down
Loading

0 comments on commit d19b8e3

Please sign in to comment.