From cba77860855a9654f342a389d84a9b74e502b6be Mon Sep 17 00:00:00 2001 From: Martin Blais Date: Sun, 17 Nov 2024 13:59:51 -0500 Subject: [PATCH] Fixed error in KV cache text in Illustrated GPT2 I think the saving and reuse of K, V across tokens does not include Q. (I'm definitely not an expert, please correct me if I'm wrong.) --- _posts/2019-08-12-illustrated-gpt2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_posts/2019-08-12-illustrated-gpt2.md b/_posts/2019-08-12-illustrated-gpt2.md index 6e5107e89c658..c9c3d273a0587 100644 --- a/_posts/2019-08-12-illustrated-gpt2.md +++ b/_posts/2019-08-12-illustrated-gpt2.md @@ -486,7 +486,7 @@ GPT-2 holds on to the key and value vectors of the the ```a``` token. Every self
-Now in the next iteration, when the model processes the word ```robot```, it does not need to generate query, key, and value queries for the ```a``` token. It just reuses the ones it saved from the first iteration: +Now in the next iteration, when the model processes the word ```robot```, it does not need to regenerate the key, and value vectors for the ```a``` token. It just reuses the ones it saved from the first iteration: