Skip to content

Commit

Permalink
more info on the mediapipe utility
Browse files Browse the repository at this point in the history
  • Loading branch information
AmmarkoV committed Jun 1, 2022
1 parent 2c01fbb commit c3ec8ce
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 3 deletions.
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@
## News
------------------------------------------------------------------

1-6-2022

An added [python/mediapipe utility](https://github.com/FORTH-ModelBasedTracker/MocapNET/tree/master/src/python/mediapipe) can now help with generating 2D data for experiments!
This can help you create datasets that include hands that can be processed using [MocapNETv3](https://github.com/FORTH-ModelBasedTracker/MocapNET/tree/mnet3)


7-4-2022

The open call of [BONSAPPS (https://bonsapps.eu/)](https://bonsapps.eu/) for AI talents received 126 proposals from 31 EU countries.
Expand Down Expand Up @@ -359,7 +365,7 @@ To convert video files ready for use as input to MocapNET in a *relatively* easy

![MediaPipe Video 2 CSV utility](https://raw.githubusercontent.com/FORTH-ModelBasedTracker/MocapNET/master/doc/mediapipeConverter.jpg)

You can get mediapipe using the [setup.sh](https://github.com/FORTH-ModelBasedTracker/MocapNET/blob/master/src/python/mediapipe/setup.sh) script or by executing
You can get mediapipe using this [src/python/mediapipe/setup.sh](https://github.com/FORTH-ModelBasedTracker/MocapNET/blob/master/src/python/mediapipe/setup.sh) script or by executing

```
pip install --user mediapipe opencv-python
Expand All @@ -376,7 +382,7 @@ After the conversion finishes you can process the generated "dataset" using Moca
```
./MocapNET2CSV --from tester-mpdata/2dJoints_mediapipe.csv --show 3
```

Due to the higher accuracy of [mediapipe holistic](https://google.github.io/mediapipe/solutions/holistic.html) (as well as inclusion of heads and hands which makes data forward compatible with the next versions of MocapNET) this might be a very useful tool to use in conjunction with MocapNET. In particular if you use this dumper be sure to checkout [MocapNET version 3](https://github.com/FORTH-ModelBasedTracker/MocapNET/tree/mnet3) that also supports hand pose estimation!



Expand Down
27 changes: 26 additions & 1 deletion src/python/mediapipe/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,29 @@
Under construction..

To convert video files ready for use as input to MocapNET in a *relatively* easy way I have included a python converter that uses mediapipe/opencv to create the CSV files needed for MocapNET.

![MediaPipe Video 2 CSV utility](https://raw.githubusercontent.com/FORTH-ModelBasedTracker/MocapNET/master/doc/mediapipeConverter.jpg)

You can get mediapipe using this [src/python/mediapipe/setup.sh](https://github.com/FORTH-ModelBasedTracker/MocapNET/blob/master/src/python/mediapipe/setup.sh) script or by executing

```
pip install --user mediapipe opencv-python
```

The converter utility receives an input video stream and creates an output directory with all image frames and the CSV file with 2D joint estimations.

After going to the root directory of the project

```
python3 src/python/mediapipe/mediapipeHolistic2CSV.py --from shuffle.webm -o tester
```

After the conversion finishes you can process the generated "dataset" using MocapNET2CSV

```
./MocapNET2CSV --from tester-mpdata/2dJoints_mediapipe.csv --show 3
```
Due to the higher accuracy of [mediapipe holistic](https://google.github.io/mediapipe/solutions/holistic.html) (as well as inclusion of heads and hands which makes data forward compatible with the next versions of MocapNET) this might be a very useful tool to use in conjunction with MocapNET. In particular if you use this dumper be sure to checkout [MocapNET version 3](https://github.com/FORTH-ModelBasedTracker/MocapNET/tree/mnet3) that also supports hand pose estimation!


This is a python holistic mediapipe wrapper that can dump datasets in CSV files
https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html
Expand Down

0 comments on commit c3ec8ce

Please sign in to comment.