Skip to content

Commit

Permalink
Merge branch 'main' into 95-update-interface-to-remove-redundant-line…
Browse files Browse the repository at this point in the history 
…s-in-imports-and-api-calls
  • Loading branch information
@fabiocat93
fabiocat93 authored Sep 26, 2024
2 parents c4992bd + 7c9da40 commit 3a7a075
Show file tree
Hide file tree
Showing 5 changed files with 255 additions and 1 deletion.
84 changes: 84 additions & 0 deletions .github/workflows/greetings.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: Greetings

on: [pull_request_target, issues]

jobs:
greeting:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/first-interaction@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
issue-message: |
👋 Welcome to **Senselab**!
Thank you for your interest and contribution. Senselab is a comprehensive Python package designed to process behavioral data, including voice and speech patterns, with a focus on reproducibility and robust methodologies. Your issue will be reviewed soon. Stay tuned!
|
...:,::;;iiff11tfGGLf@Cftt1111iiii@80GG0f;;;
...;,,::;;itf111tGGLf@Cft111iiiiii@00GC0i;;;1
..,,,:::;;tf1111GGLt@Ctt11iiiiiit@0GCCG::;;1:
,,,....... ..,,,::;;1f1111GGLt@Ct11iiiiiiiG@0GCCf:::;i,,
;;;;;;;;;;;::::::,,,,,,.......... ..........,.,....... . ..... .:.,,::;;t1111GGL1@Ct11iiii;;;880CLGi:::i,,..
;;;;;;;;;:::::,,,..... .......,,,,,,,,...... .. .,.,,::;;tti11GCL1@C11iii;;;;;@8GCLG::::;,...,
,,,,,,.... ............. ..,,,::;;tt1i1GCL1@C11iii;;;;;@08LLL,,,::...i..
.,::: ...;,::;;;tf111CCL1@C11ii;;;;;i@GCLCt:,,;....,..
. ,::;;;;;::,. ...,;,::;;i1f111CCL1@L1ii;;;;;;888LLCf,,,;...,...
,:::, ,::;;ii111ii;;::,. ., ...,,,:::;;1t1iiLCL1@L1ii;;;;:188CLfC,,i::...:..
.,::;iii11111111ii;;::,,.. ... ..,,:,:;;itt1iifCL1@L1i;;;:::L80LLff,,,i......
...... ... . ,:::;;iiii111ttttt1111iii;;;:::: ....,,;,:::;;11iitLL1@L1ii;;:::080LfLi,.,:.i.;,.
,,,,,,,,,,,,........ .::;;iiiiii11111ttttttttt111iiii11i:,,,,,,,,,,:::;;:....,,:,:::;iiii1ff1@Lii;;::::t0GffL,,.:,......
::::::::::::,,,,,,,,,,.....,:1fLCG; .,;iii11i;ii111ttttttttttttttt11111tfffi1i1ifif1.C..;,...,,i,,:::;i;;iLfi8Li;;;::::00Cftf.,,;...;., .
;;;;;;;;;;;::::::::::::0Lftt1111111tfCf.... . ..,;11ttt1ii11tttttttfffffffffftttt11ifLLCt;it;1,.,.,i,.....:,,,,:;;:::ffffL1;:::::;00ftti..,:........
ii;;i;;;;;;;;;;;;;;i0Cfttt111iiiii,,,;1fC,,.....,,,:1tfff1ii1tttffffffffffLLLLLfffffft11LGCCL;..,:;.1.;,....,,.,,,::,,,,,::::::::::t0G11t,..:...:.... .
1iiiiiiiiiii;;;;;;GCfftt1111iii;;;;;;:: ;fG,,,,,,,,1tffLt111tffffffffffLLLLLLLLLLLLLLftttCGCff:::,:,,1,.....;....:;,,,,,,,,,,,::::CGCiit.L.;...,.....
ii1iiiiiiiiiiiii;0Lfftt111iii;: ;;;;;;;;tG,,,,,,:;fLLL1i1fLCCCCCCCLfffLLLLLLLCCCCCLft1:LGGG1;,:,..,,. ...:.........,,,,,,,,,;::GGL;;;..,,..:......
111t11111iiiiiiiGLfftt111ii;;: ,:;;;;;;;;tC,,,:,:;tLCLi1LCCLLLCCCCCLffLLLLG00000GG00f1,tG0Ct;;,,.. . ..........,,,,;i,GG1:;...:...:....
1111111111i1iiitCLftt11iii;: ,,::::;;;;1L,,,:,:;1LCiiffLLLLCCLLLLffLLLLLLLCCCCCLLLLf::GGft,,,:,.......,.........,.,,,.,,;;;GC::;...,..,.....
tttt11111111111fCLft11i , ,:;;;;1fi,,,,,;fLLitfLCLG000000GLfLLLLG088000GCGLt1, CGt,... , ......:;;;f:,:..,...:....
tttttt11t111111fCLft1i . .:;;;;1f:,,:,,:tf:1ffLGGCC08G00GCLLLLL0888888Ci8L11; ti1,.. . LL...::;i1,,...:.........
ttttttttt1t11111CLft1i . ,:;;iitC,,,,,,,iti1tftfLLCCGGGGLftLLftLG000000GLtft: ; ... :LCCf.::;1;,:......i.....
tttttttttttttt11fCLft1 , :;;;i1f;,,,,:,,,;iii1tfLLCCCLLft1tLLt1fCCGGGGCLLf1; . f.. LCCLL::;f:,:..:........
11111111111111111fCftt1i: : ittttfC;:::::,:,:iii11ttffLLLLLttitff;itCCCCCCLLf1;. , LLLLL11t:,..,:..1.....
iiiiii;;;;;;;;;;;;:Li;11ii;;;::::::ft111tL::i........iii1i1ttfffLLL111tff;i11GCLLfffti. . fLLLLf:::...,...,.....
;;;;;;;;;;;;;;;;;;;::t.,:11iii;;;;;LttfL.,,:C........1iiii1tttffLLCLGftff10GGGCCLfftti, : LLLLf1......,..i.....
;;;;;;;;;;;;;;:::::::::,fL1ii1ttttfLfi;:::;f.........iiii11tttfLLLC0000G0888800GCLfft1: , .ttftLf...............
;;;;;;;;;;:::::::::::,,,,,,,,,,.......tLL;... ;,.,.. ii11ttttLCG00888888888888888GLff1;, ,, itffffffLLff:i. ......,..... .
;;::::::::::::::::::,,,,,,,...................:,,:...:11LLLffC0000GCLLLLLCCCCLC088GffLLi,i 11tttttffLfttfftitt.i ....,......
:::::::::::::,,,,,,,,,,,.................... . .ifLCCLfG0G88Cti1;i:;i11L8808CLLLLt:. 1111ttfttttfffffff111i,;. ..........
,,,,,,,,,,,,,,,,,,,,.................... . .tLLCLLCLLfCCGGCCCfGGG0GLLCGLLCLL1. 1tt1tttttffffffffftt1i:: ....,......
,,,,,,,,,,,,,.......................... fLLCfCfftLfttffLLLLfttfLCLLLLf1. 1111111tffttttfffffftt1i,.. ........
,,.............................. .. fLLLfLfftffGGG0000CLffffLfLf1 iiii1t11ttttttttfftffftt11;....,......
............................. .LLLLLffffLCCGGCLLLfffffft i1iii11111tttffffffffffft1ii:.........
,,,,,,,,,,,,,,,:,,,,,,,,.,.................... .:LGGCCffLLCLLCCCCLft1Lf :;;iiiii1ftttttfftttftffftt11:,.,......
.... . ;ifC00GGGGCCGLCCGLLLL. ;;;1iiiii1111ttttttttttftttt1i:,......
....... .... i1f; .;i1fCC08800GGGG0GCf1, 1 ;;;;;iiiii11111tttttttfttttt1;,...... .
....... ..... .tLffff1,i.:;i11tffLLLLLLLLftt;:. :L1f1ii :::;;;;;iiiii11ttttttttttttttti;,.... ....
.. ,ii;tCCCLCCGCCt,:;i1111111ttfffftt11i;, ,LfCLCLf111 :;;iiiii;;iii11tttttttttttttti,,....,.......
.. ,:;1CCLLLCGGGCCGGG000880C1i111t111tttttt1111i;:,fCCCG0GGGLLCCf1ti, iiiii111ttttttttttttt;,,,,.........
.;1ffftfLLLCGG00000G0888@@@@@@@@@8GLtttttttttttfL0@@@880080880CC00LL1111i;., iiiii1tttttttttttttti,,,,,.........
;itfLLCCCCCGG000888888088@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8@@@@G08880GGLLftt;ii;,, ii1i111ttttttt1tfti,,,,..........
:itfffLCCGGG000088888@@@@@88@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8@@@@88800GCCLfftfi11;:: i1i11ttttttttttti:,,,...........
;1tfLLCfCCGG000088888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8@@@@@@@880880CGLLf111t1 ;i11111ttttfftt1i:,.,..........
ttfLLCCGCG0008888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8888000GGGCLLftt: .11111ttffffftt1i;,,...........
iffLCCCGC0008888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8888000CCCCCLf; .1111ttftfffttt1i;,,..........
.ttLCCCGG0G088888@@@8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@88888000GGCCCL11i :1111ttfffffttt1i:,..........
;ttLCGG0G0G88888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@888888000GGGGGCft; :ttttfffffftt111i;:.........
:fffCGG00000888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8888880000GGGCft1 itttffffffttt11iii:,.......
.fLCLGG00888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@88888880000GGCCCC. tttfffffffttt11iii;:......
:fCCGC00888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8888888800G0GCLfG tffffLfffftt11iii;;:,....
itCGG0G0888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8@@8888800GGCCL tffffffffftt111ii;;;,...
ttCG00008888@8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@888800000Ci ffLfLffffffttt11ii;;:,.
tffG0888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@, 8tL;8,@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8800000Gf; LLLLLLfLfffftt1iii;;;:
;fLCC088888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@f .L0@G:C:i.,,.@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@800CCGCCtCLLLffffffffftt1iii;;
ffCGG0888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 0@1t@@iCC@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8@@88@8888GCCLLLLfLLLLfftt11iii
,CCG00008@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ .f1t@,:.08@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@8GCLLLLLLLLLLfft11ii
iG0G08888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@L8@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@80GCCCLLLLLLfffftt11
iG0888888@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@GGCLLLLfffffffftt1
pr-message: |
🚀 **First Pull Request** 🎉
Welcome to **Senselab**, and thank you for submitting your first pull request! We’re thrilled to have your contribution. Our team will review it as soon as possible. Stay engaged, and let’s make behavioral data analysis even more powerful together!
27 changes: 27 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,30 @@
# 0.14.2 (Thu Sep 26 2024)

#### 🐛 Bug Fix

- ✨Add speaker verification tutorial and documentation [#100](https://github.com/sensein/senselab/pull/100) ([@ibevers](https://github.com/ibevers) [@fabiocat93](https://github.com/fabiocat93))
- First pr/issue wf [#166](https://github.com/sensein/senselab/pull/166) ([@fabiocat93](https://github.com/fabiocat93))

#### Authors: 2

- [@ibevers](https://github.com/ibevers)
- Fabio Catania ([@fabiocat93](https://github.com/fabiocat93))

---

# 0.14.1 (Thu Sep 26 2024)

#### 🐛 Bug Fix

- fixing transformers version to 4.44.2 until they solve some issues with the text-to-speech pipeline [#167](https://github.com/sensein/senselab/pull/167) ([@fabiocat93](https://github.com/fabiocat93))
- switch to poetry - last [#164](https://github.com/sensein/senselab/pull/164) ([@fabiocat93](https://github.com/fabiocat93))

#### Authors: 1

- Fabio Catania ([@fabiocat93](https://github.com/fabiocat93))

---

# 0.14.0 (Fri Sep 20 2024)

#### 🚀 Enhancement
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ datasets = "~=2.19"
torch = "~=2.2.2"
torchvision = "~=0.17"
torchaudio = "~=2.2"
transformers = "~=4.39"
transformers = "~=4.44.2"
ffmpeg-python = "~=0.2"
ipykernel = "*"
pydra = "~=0.23"
Expand Down
60 changes: 60 additions & 0 deletions src/senselab/audio/tasks/speaker_verification/doc.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Speaker Verification
Last updated: 08/13/2024

<button class="tutorial-button" onclick="window.location.href='https://github.com/sensein/senselab/blob/main/tutorials/speaker_verification.ipynb'">Tutorial</button>

## Task Overview
Speaker verification is identity authentication based on voice features.

This technology is widely used in various applications, including security systems, authentication processes, and personalized user experiences. The core concept revolves around comparing voice characteristics extracted from speech samples to verify the identity of the speaker.

SenseLab speaker verification extracts audio embeddings, finds their cosine similarity, and uses a similarity threshold to determine if two audio files came from the same speaker.

## Models
SenseLab speaker verification extracts audio embeddings using ECAPA-TDNN (`speechbrain/spkrec-ecapa-voxceleb`). This model is part of [SpeechBrain](https://huggingface.co/speechbrain), "an open-source and all-in-one conversational AI toolkit based on PyTorch." This model was trained on [Voxceleb1](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox1.html) and [Voxceleb2](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox2.html), celebrity voice datasets. It is important to ensure that the audio samples used for verification have a sampling rate of 16kHz, as this is the rate that `speechbrain/spkrec-ecapa-voxceleb` was trained on.

- **ECAPA-TDNN**
- [spkrec-ecapa-voxceleb](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb)


## Evaluation
### Metrics
The primary evaluation metric for speaker verification is the Equal Error Rate [EER](https://www.sciencedirect.com/topics/computer-science/equal-error-rate), which is the percent error when the false acceptance rate (FAR) and false rejection rate (FRR) are equal.
- **False Acceptance Rate (FAR)** The probability of a verification system accepting invalid inputs. Similar names may refer to this same metric.
- **False Rejection Rate (FRR)** The probability of a verification system rejecting valid inputs. Similar names may refer to this same metric.

Lower values on these metrics indicate better performance.

### Datasets
Common datasets used for evaluating speaker verification models include:
- [VoxCeleb1](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox1.html): A large-scale speaker verification dataset containing over 100,000 utterances from 1,251 celebrities.

Verification Split
| | Dev | Test |
|-----------------|------------|----------|
| # of speakers | 1,211 | 40 |
| # of videos | 21,819 | 677 |
| # of utterances | 148,642 | 4,874 |

Identification Split
| | Dev | Test |
|-----------------|------------|----------|
| # of speakers | 1,251 | 1,251 |
| # of videos | 21,245 | 1,251 |
| # of utterances | 145,265 | 8,251 |

- [VoxCeleb2](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox2.html): An extension of VoxCeleb1, with more speakers and more utterances, used for training and evaluation.

| | Dev | Test |
|-----------------|------------|----------|
| # of speakers | 5,994 | 118 |
| # of videos | 145,569 | 4,911 |
| # of utterances | 1,092,009 | 36,237 |

For more details on these datasets and the evaluation process, refer to the [VoxCeleb paper](https://arxiv.org/abs/1706.08612).

### Benchmark
See the [Papers With Code leaderboard](https://paperswithcode.com/sota/speaker-verification-on-voxceleb) for rankings of speaker verification by EER on VoxCeleb.

## Notes
- Fine-tuning the model on a specific dataset can further improve accuracy.
Loading

0 comments on commit 3a7a075

Please sign in to comment.