Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why is the speech signal not convolutional mixing #2

Open
bbbgan opened this issue Dec 25, 2023 · 4 comments
Open

Why is the speech signal not convolutional mixing #2

bbbgan opened this issue Dec 25, 2023 · 4 comments

Comments

@bbbgan
Copy link

bbbgan commented Dec 25, 2023

Hello, fakufaku, thanks for your amazing work!
I have just started exploring BSS and I have a question. Why is the signal not convolutional mixed?
It seems like linear mixed. Isn't this contradictory to the theory of IVA? IVA requires convolutional mixing in the time domain and linear mixing in the frequency domain.
this is the mixing code:

mix = np.sum(premix[:n_tgt, :], axis=0) + bg + un

I have been troubled by this issue for a long time. You are an expert in this field, can you give me some advices?

@fakufaku
Copy link
Owner

Hi @bbbgan , you are right that IVA is designed to handle convolutional mixtures.
In the code you point out, the array premix contains all the source signals already convolved with the appropriate impulse responses. So the resulting mixture is indeed convolutional.

@bbbgan
Copy link
Author

bbbgan commented Dec 26, 2023

Appreciate for your reply, the si_bss_eval in convergence_callback is seems to evaluate the performance of the IVA. However, it evaluates the ref_sig[:n_targets, :m].T and the result synth[:-1, :m].T, the ref_sig[:n_targets, :m].T is come from the premix as following codes:

refs[:n_targets, :] = premix[:n_targets, parameters["mix_params"]["ref_mic"], :]

So I'm confused. why compare the premix rather than the original signal with the results

sdr, sir, sar, perm = si_bss_eval(ref_sig[:, :m].T, synth[:, :m].T)

@fakufaku
Copy link
Owner

IVA will separate the reverberant signals without doing reverberation. It is thus correct to compare to the reverberant isolated signal rather than the anechoic original signal.

@bbbgan
Copy link
Author

bbbgan commented Dec 26, 2023

Reverberation will introduce dependencies between the signals, so it is correct. But one more question, if we evaluate the premix and the results, the mix is linear mixed by the premix. It's not convolutional mixing.
So it seems like IVA separating the mix which is linear mixed by original signal premix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants