diff --git a/_posts/2019-08-12-illustrated-gpt2.md b/_posts/2019-08-12-illustrated-gpt2.md index 6e5107e89c658..c9c3d273a0587 100644 --- a/_posts/2019-08-12-illustrated-gpt2.md +++ b/_posts/2019-08-12-illustrated-gpt2.md @@ -486,7 +486,7 @@ GPT-2 holds on to the key and value vectors of the the ```a``` token. Every self
-Now in the next iteration, when the model processes the word ```robot```, it does not need to generate query, key, and value queries for the ```a``` token. It just reuses the ones it saved from the first iteration: +Now in the next iteration, when the model processes the word ```robot```, it does not need to regenerate the key, and value vectors for the ```a``` token. It just reuses the ones it saved from the first iteration: