Deep speech returns empty with new scorer

[JRMeyer]

[
[[Mahdi_Mohammadpour_Fard
[August 4, 2020, 6:59am]

Hi,

After moving to new released deep speech version 0.8.0, when I use old
scorer file generated by following commands, everything works fine:

./lmplz slash --text both.txt -S 20% slash --arpa words.arpa slash --o 3
slash --discount_fallback slash
./build_binary -T -s trie words.arpa lm.binary

and then using generate_package.py to generate scorer file.

But now when I use following command:

generate_lm.py slash --input_txt test.txt slash --output_dir . slash --top_k 500000
slash --kenlm_bins .../build/bin/ slash --arpa_order 3 slash --max_arpa_memory '20%'
slash --arpa_prune '0 slash |0 slash |1' slash

and then using native client generate_scorer_package to generate scorer
file, deep speech returns empty string for every input file. There is no
problem with loading scorer file, but it does not work.

Can someone help me pleas?

[This is an archived TTS discussion thread from discourse.mozilla.org/t/deep-speech-returns-empty-with-new-scorer]

[JRMeyer]

[
[[Mahdi_Mohammadpour_Fard
[August 4, 2020, 6:59am]

Hi,

After moving to new released deep speech version 0.8.0, when I use old
scorer file generated by following commands, everything works fine:

./lmplz slash --text both.txt -S 20% slash --arpa words.arpa slash --o 3
slash --discount_fallback slash
./build_binary -T -s trie words.arpa lm.binary

and then using generate_package.py to generate scorer file.

But now when I use following command:

generate_lm.py slash --input_txt test.txt slash --output_dir . slash --top_k 500000
slash --kenlm_bins .../build/bin/ slash --arpa_order 3 slash --max_arpa_memory '20%'
slash --arpa_prune '0 slash |0 slash |1' slash

and then using native client generate_scorer_package to generate scorer
file, deep speech returns empty string for every input file. There is no
problem with loading scorer file, but it does not work.

Can someone help me pleas?

[This is an archived TTS discussion thread from discourse.mozilla.org/t/deep-speech-returns-empty-with-new-scorer]

[SuperKogito]

any update on this?
I am facing the same issue training a model for dutch.
Without specifying a scorer aka using the default scorer I get a Test on /languages/nl/training/standard/test.csv - WER: 0.601479, CER: 0.223169, loss: 42.825607 and some sentences are predicted perfectly especially in the BEST WER cases. However, when I use my custom newly generated scorer I get a WER=1 and CER=1 and all predicted texts are "".

• Is this due to lack of data?
• do I need to normalize my text?

[jbflow]

Has any one manage to figure this out as I'm also seeing a similar problem. I have an open SO question about it here