Bug fix in the calculation of the validation loss (pytorch#427)

The last row of tokens in data_source in never used as input data in the last batch. Those tokens are always only used as targets and thus the total_loss variable shouldn't be divided by len(data_source) but len(data_source) - 1 instead.
KernLC · Oct 29, 2018 · 81f47e8 · 81f47e8
1 parent 05ed879
commit 81f47e8
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/word_language_model/main.py b/word_language_model/main.py
@@ -138,7 +138,7 @@ def evaluate(data_source):
             output_flat = output.view(-1, ntokens)
             total_loss += len(data) * criterion(output_flat, targets).item()
             hidden = repackage_hidden(hidden)
-    return total_loss / len(data_source)
+    return total_loss / (len(data_source) - 1)
 
 
 def train():