Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Losses nan #241

Open
aabdumalikov opened this issue Nov 8, 2024 · 0 comments
Open

Losses nan #241

aabdumalikov opened this issue Nov 8, 2024 · 0 comments

Comments

@aabdumalikov
Copy link

Step 1000/900000, Total Loss: nan, Mel Loss: nan, Mel PostNet Loss: nan, Pitch Loss: nan, Energy Loss: nan, Duration Loss: nan
Training: 0%|▏ | 1000/900000 [09:22<148:18:56, 1.68it/s/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/matplotlib/image.py:452: UserWarning: Warning: converting a masked element to nan.██████████████████████████| 197/197 [09:22<00:00, 2.00s/it]
dv = np.float64(self.norm.vmax) - np.float64(self.norm.vmin)
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/matplotlib/image.py:459: UserWarning: Warning: converting a masked element to nan.
a_min = np.float64(newmin)
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/matplotlib/image.py:464: UserWarning: Warning: converting a masked element to nan.
a_max = np.float64(newmax)
<array_function internals>:5: UserWarning: Warning: converting a masked element to nan.
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/numpy/core/_asarray.py:83: UserWarning: Warning: converting a masked element to nan.
return array(a, dtype, copy=False, order=order)
/home/aziz/VScodeProjects/FastSpeech2/FastSpeech_eng/utils/tools.py:86: RuntimeWarning: invalid value encountered in true_divide
audio / max(abs(audio)),
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/matplotlib/image.py:452: UserWarning: Warning: converting a masked element to nan.
dv = np.float64(self.norm.vmax) - np.float64(self.norm.vmin)
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/matplotlib/image.py:459: UserWarning: Warning: converting a masked element to nan.
a_min = np.float64(newmin)
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/matplotlib/image.py:464: UserWarning: Warning: converting a masked element to nan.
a_max = np.float64(newmax)
<array_function internals>:5: UserWarning: Warning: converting a masked element to nan.
/home/aziz/VScodeProjects/FastSpeech2/env/lib/python3.8/site-packages/numpy/core/_asarray.py:83: UserWarning: Warning: converting a masked element to nan.
return array(a, dtype, copy=False, order=order)
/home/aziz/VScodeProjects/FastSpeech2/FastSpeech_eng/utils/tools.py:86: RuntimeWarning: invalid value encountered in true_divide
audio / max(abs(audio)),

It doesn't show total losses and it returns nan

@aabdumalikov aabdumalikov changed the title While training Losses nan Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant