Skip to content

Commit

Permalink
New challenge updates
Browse files Browse the repository at this point in the history
  • Loading branch information
cioppaanthony authored Jan 30, 2023
1 parent 906cdd9 commit 3a3c3c6
Show file tree
Hide file tree
Showing 2 changed files with 40 additions and 18 deletions.
28 changes: 13 additions & 15 deletions ChallengeRules.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
# Guidelines for the Action Spotting challenge

The 2nd [Action Spotting Challenge]() will be held at the
official [ActivityNet Workshop](http://activity-net.org/challenges/2021/challenge.html) at CVPR 2022!
The 3rd [Action Spotting Challenge]() and the 1st [Ball Action Spotting Challenge]() will be held at the
official [CVSports Workshop](https://vap.aau.dk/cvsports/) at CVPR 2023!
Subscribe (watch) the repo to receive the latest info regarding timeline and prizes!


SoccerNet-v2 is a large-scale dataset build upon SoccerNet that benchmarks the tasks of action spotting, camera shot segmentation / boundary detection and replay grounding.
SoccerNet-v2 is composed of 300k manual annotations, span 500 complete soccer games from six main European leagues, covering three seasons from 2014 to 2017 and a total duration of 764 hours.

We propose the SoccerNet-v2 challenge to encourage the development of state-of-the-art algorithm for Generic Soccer Video Understanding.
It is composed of 2 tasks:
- **Action Spotting**: Spot the actions on a complete video of soccer game.
- **Replay Grounding**: Ground the timestamp of the actions represented in a specific replay.
We propose the SoccerNet challenge to encourage the development of state-of-the-art algorithm for Generic Soccer Video Understanding.
- **Action Spotting**: Spot the actions on a complete video of soccer game among 17 classes.
- **Ball Action Spotting**: Spot the actions related to the ball on a complete video of soccer game among 2 classes.

We provide an [evaluation server]() for anyone competing in the SoccerNet-v2 challenge.
We provide an [evaluation server]() for anyone competing in the SoccerNet challenge.
This evaluation server handles predictions for the open **test** set and the segregated **challenge** set.

Winners will be announced at ActivityNet Workshop at CVPR 2022.
Prizes 💲💲💲 include $1000 cash award ($500 for Action Spotting and $500 for Replay Grounding), sponsored by [SportRadar](https://www.sportradar.com/).
Winners will be announced at CVSports Workshop at CVPR 2023.
Prizes will be announced soon, stay tuned!


## Who can participate / How to participate?
Expand All @@ -34,21 +33,20 @@ Prizes 💲💲💲 include $1000 cash award ($500 for Action Spotting and $500
## How to win / What is the prize?

- For each task, the winner is the individual/team who reach the highest performance on the **challenge** set.
- The metrics taken into consideration are the **tight Average-mAP for Action Spotting** and the **tight Average-AP for Replay Grounding**.
- The deadline to submit your results is May 30th at 11.59 pm Pacific Time.
- The teams that perform best in each task will be granted $500 from our sponsor [SportRadar](https://www.sportradar.com/).
- In order to be eligible for the prize, we require the individual/team to provide a short report describing the details of the methodology (CVPR format, max 2 pages)
- The metrics taken into consideration are the **tight Average-mAP for Action Spotting** and the **mAP@1 for Ball Action Spotting**.
- The deadline to submit your results is May 30th 2023 at 11.59 pm Pacific Time.
- In order to be eligible for the prize, we require the individual/team to provide a short report describing the details of the methodology (CVPR format, max 2 pages).



## Important dates

Note that these dates are tentative and subject to changes if necessary.

- **February 1:** Open evaluation server on the (Open) Test set.
- **February 6:** Open evaluation server on the (Open) Test set.
- **February 15:** Open evaluation server on the (Seggregated) Challenge set.
- **May 30:** Close evaluation server.
- **June 6:** Deadline for submitting the report.
- **June TBD:** A full-day workshop at CVPR 2022.
- **June TBD:** A full-day workshop at CVPR 2023.

For any further doubt or concern, please raise an issue in that repository, or contact us directly on [Discord](https://discord.gg/SM8uHj9mkP).
30 changes: 27 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,36 @@ Welcome to the SoccerNet Development Kit for the Action Spotting Task and Challe
SoccerNet Action Spotting is part of the SoccerNet-v2 dataset, which is an extension of SoccerNet-v1 with new and challenging tasks including
action spotting, camera shot segmentation with boundary detection, and a novel replay grounding task.

__[News]__ For the 2023, we introduce a novel spotting challenge on ball events including drive and passes. Those events are much more dense and require a higher level of spotting precision. The density of those events, as well as the subtle underlying movement of the ball and players, make this new task even more challenging. For this new challenge, you only have access to 7 annotated games, so you may want to take a look at different training paradigms such as transfer learning, unsupervised learning or semi-supervised learning. Remember that you still have access to the 500 videos from soccernet to help you.


<p align="center"><img src="Images/GraphicalAbstract-SoccerNet-V2-1.png" width="640"></p>

The Action Spotting dataset consists of 500 complete soccer games including:
- Full untrimmed broadcast videos in both low and high resolution.
- Pre-computed features such as ResNET-152.
- Annotations of actions among 17 classes (Labels-v2.json).

Participate in our upcoming Challenge in the [CVPR 2022 International Challenge on Activity Recognition Workshop](http://activity-net.org/challenges/2021/index.html) and try to win 500$ sponsored by [SportRadar](https://www.sportradar.com/)! All details are available on the [challenge website](https://eval.ai/web/challenges/challenge-page/1536/overview), or on the [main page](https://soccer-net.org/).
The new Ball Action Spotting dataset consists of 7 complete soccer games including:
- Full untrimmed broadcast videos in both low and high resolution.
- Annotations of actions among 2 classes (Labels-ball.json).

Participate in our upcoming Challenges during the [CVPR 2023 International Challenge at the CVSports Workshop](https://vap.aau.dk/cvsports/)! All details will be available on the [challenge website](https://eval.ai/web/challenges/challenge-page/1536/overview), or on the [main soccernet page](https://www.soccer-net.org/).

The participation deadline is fixed at the 30th of May 2022.
The participation deadline is fixed at the 30th of May 2023.
The official rules and guidelines are available on [ChallengeRules.md](ChallengeRules.md).

<a href="https://youtu.be/tA9E1hkiyB0">
<p align="center"><img src="Images/Thumbnail.png" width="720"></p>
</a>

### 2022 Challenge leaderboard

The leaderboard will soon be provided here. In the meantime, please check out our paper on the [SoccerNet 2022 Challenge Results](https://arxiv.org/abs/2210.02365) published at the ACMM Workshop.

### 2021 Challenge leaderboard

This table summarizes the current performances on the 2021 challenge.
This table summarizes the current performances on the 2021 Action Spotting Challenge.
For the leaderboard on the 2022 challenge, please visit EvalAI [test](https://eval.ai/web/challenges/challenge-page/761/leaderboard/2072) and [challenge](https://eval.ai/web/challenges/challenge-page/761/leaderboard/2074) leaderboards.

| Model | tight Avg-mAP (challenge) | Avg-mAP (challenge) | tight Avg-mAP (test) | Avg-mAP (test) |
Expand Down Expand Up @@ -57,6 +68,8 @@ This table summarizes the current performances of published methods only. Last

## How to download the dataset

### SoccerNet-v2 Action Spotting

A [SoccerNet pip package](https://pypi.org/project/SoccerNet/) to easily download the data and the annotations is available.

To install the pip package simply run:
Expand Down Expand Up @@ -85,6 +98,12 @@ mySoccerNetDownloader.downloadGames(files=["1_baidu_soccer_embeddings.npy", "2_b
```


### SoccerNet Ball Action Spotting

You may use the same pip package as for SoccerNet-v2.
Information and data coming soon.


## How to extract video features

As it was one of the most requested features on SoccerNet-V1, this repository provides functions to automatically extract the ResNet-152 features and compute the PCA on your own broadcast videos. These functions allow you to test pre-trained action spotting, camera segmentation or replay grounding models on your own games.
Expand All @@ -95,8 +114,13 @@ The functions to extract the video features can be found in the [Features](Featu

## Benchmark Implementations

### SoccerNet-v2 Action Spotting

This repository contains several [benchmarks](Benchmarks) for action spotting, which are presented in the [SoccerNet-V2 paper](https://arxiv.org/pdf/2011.13367.pdf), or subsequent papers. You can use these codes to build upon our methods and improve the performances.

### SoccerNet Ball Action Spotting

The benchmark to beat this year is the [E2E-spot](https://arxiv.org/abs/2207.10213) method published at ECCV 2022 by J. Hong et. al trained with default parameters and 25fps videos. This method arrived second in the 2022 Action Spotting Challenge The code will be updated on their [github repository](https://github.com/jhong93/spot) for anyone to reproduce the baseline results and improve on them.

## Evaluation

Expand Down

0 comments on commit 3a3c3c6

Please sign in to comment.