Skip to content

Commit

Permalink
Add sensitivity as secondary metric + specify optional TSV fields
Browse files Browse the repository at this point in the history
  • Loading branch information
danjjl committed Dec 6, 2024
1 parent 2a3e499 commit 4e415ed
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions website/content/challenge/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In partnership with [The International Conference on Artificial Intelligence in

Epilepsy is the most common chronic brain disease and affects people of all ages. More than 50 million people worldwide have epilepsy. Scalp EEG-based seizure detection algorithms can optimize and facilitate the diagnostic workup performed on people with epilepsy to improve patients' care and quality of life [\[1\]](https://doi.org/10.1111/epi.14052).

EEG-based seizure detection aims to detect the onset and duration of all seizures in an EEG recording. The task has benefited from advances in machine learning. However, a relative scarcity of public datasets and a lack of standardization hinder progress in the field. This likely explains the lack of adoption of state-of-the-art algorithms in clinical practices. Recently, [SzCORE]( https://doi.org/10.1111/epi.18113) has proposed a method to standardize dataset formats, evaluation methodology, and performance metrics.
EEG-based seizure detection aims to detect the onset and duration of all seizures in an EEG recording. The task has benefited from advances in machine learning. However, a relative scarcity of public datasets and a lack of standardization hinder progress in the field. This likely explains the lack of adoption of state-of-the-art algorithms in clinical practices. Recently, [SzCORE](https://doi.org/10.1111/epi.18113) has proposed a method to standardize dataset formats, evaluation methodology, and performance metrics.

In this machine learning challenge, we will leverage the standardization proposed by SzCORE. We ask the participants to build models on any combination of standardized publicly available datasets or private datasets. The model should perform a segmentation task by identifying the onset and duration of all epileptic seizures given a long-term, continuous, routine EEG as an input. The Models will then be evaluated on a large hold-out dataset using the event-based F1 score as the evaluation metric.

Expand Down Expand Up @@ -56,6 +56,8 @@ Here is an example of a HED-SCORE compliant annotation file with three seizures:
895.0 21.0 sz n/a n/a 2016-11-06 13:43:04 3600.00
```

In this challenge the `confidence` and `channels` fields are not used. They will not be evaluated.

### Training data

Challenge participants are encouraged to train their models on any combination of the three publicly available large datasets or any private datasets they might have access to. The main public datasets are:
Expand All @@ -81,7 +83,7 @@ The licenses of the other datasets require you download and convert them yoursel

### Evaluation

Submissions will be evaluated on event-based F1 score computed on a private dataset of more than 2500 hours of data recorded in an epilepsy monitoring unit.
Submissions will be evaluated on event-based F1 score computed on a private dataset of more than 2500 hours of data recorded in an epilepsy monitoring unit. In case of a tie on F1 score, sensitivity will be used as a secondary evaluation metric.

Event based scoring relies on overlap. If the reference event and the hypothesis event overlap, it is a correct detection (`True Positive`). If the hypothesis event does not overlap with a reference event it is a false detection (`False Positive`).

Expand Down

0 comments on commit 4e415ed

Please sign in to comment.