Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AdaKVPress #38

Merged
merged 27 commits into from
Jan 13, 2025
+222 −7
Merged

AdaKVPress #38

Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
82cc93e
Handle transformers breaking changes
SimJeg Jan 6, 2025
6d5f34e
Add AdaKVPress (first version)
SimJeg Jan 7, 2025
9a46d7a
Add alpha_safeguard
SimJeg Jan 7, 2025
2693558
Move from least squares to perceptron
SimJeg Jan 8, 2025
b16ab6a
Remove GQA
SimJeg Jan 8, 2025
d19edc7
Fix attention patch
SimJeg Jan 9, 2025
79156c7
Align with ScorerPress
SimJeg Jan 9, 2025
002ac9d
Update evaluate
SimJeg Jan 9, 2025
5817935
Fix attention patch
SimJeg Jan 9, 2025
31f6b12
Some cleaning
SimJeg Jan 9, 2025
f5cb200
Add check
SimJeg Jan 9, 2025
64f0e99
Add first test
SimJeg Jan 9, 2025
7f06fce
Merge branch 'main' into simon/adakv-press-448
SimJeg Jan 9, 2025
6227bcd
Adress PR feedback
SimJeg Jan 9, 2025
7e2ba3b
Update position embeddings
SimJeg Jan 9, 2025
21fb7a6
Fix ThinkPress
SimJeg Jan 9, 2025
64d0260
Fix ExpectedAttentionPress with wrapper
SimJeg Jan 13, 2025
bbbb755
Fix head_dim and patch
SimJeg Jan 13, 2025
a86534a
Fix flake8
SimJeg Jan 13, 2025
7ea10c7
Update notebook
SimJeg Jan 13, 2025
8edba20
merge
SimJeg Jan 13, 2025
54bb346
Merge branch 'main' into simon/adakv-press-448
SimJeg Jan 13, 2025
e78f8fc
Update python version
SimJeg Jan 13, 2025
67654a7
Update patch
SimJeg Jan 13, 2025
f804690
Back to 3.10.11
SimJeg Jan 13, 2025
9edb610
Add docs
SimJeg Jan 13, 2025
3ac3df2
Fix
SimJeg Jan 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into simon/adakv-press-448
  • Loading branch information
@SimJeg
SimJeg committed Jan 9, 2025
commit 7f06fce1cbee9cdaecd0058e307d725bd3fe2a76
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ scipy = "^1.13.1"
matplotlib = "^3.9.0"
bs4 = "^0.0.2"
torch = "^2.3.1"
transformers = "^4.48.0"
transformers = ">=4.48.0"
nvitop = "^1.3.2"
sentencepiece = "^0.2.0"
protobuf = "^5.27.2"
Expand Down
Loading
You are viewing a condensed version of this merge commit. You can view the full changes here.