From 9fa227e0cbdca112b2c628831f1a9c1bcfb032b1 Mon Sep 17 00:00:00 2001 From: Mike DuPont Date: Sun, 28 Jul 2024 06:55:38 -0400 Subject: [PATCH] Update notes.org --- 2024/07/28/notes.org | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/2024/07/28/notes.org b/2024/07/28/notes.org index e84b6669..19a699c6 100644 --- a/2024/07/28/notes.org +++ b/2024/07/28/notes.org @@ -1,7 +1,9 @@ * Idea of the day -Now feeding a python program into this llm, we can embed the tokens into vectors. -The Abstract Syntax Tree (AST) of the compiler also embeds the tokens. +Consider a simple python program of "print(1+2+3)". + +Feeding this python program into a llm, we can embed the tokens into vectors. +The Abstract Syntax Tree (AST) of the compiler also embeds the tokens into a different but equivalent form. Consider positional encoding in a large language model, this takes each token and apply extra information to each token that gives it some