Golf aes gcm fold #103

0xJepsen · 2024-10-22T23:32:16Z

Planing on
Closes #93
Closes #102
current status

non-linear constraints: 441903
linear constraints: 0
public inputs: 0
private inputs: 60
public outputs: 32
wires: 402452
labels: 2300725

thor314 · 2024-10-23T17:18:03Z

.github/workflows/circom.yml

+        cd circom-witnesscalc
+        cargo install --path .
+        echo $(which build-circuit)
+
+    - name: Build witness for aes-gcm
+      run: |
+        build-circuit circuits/aes-gcm/aes-gcm-fold.circom aes-gcm-fold.bin -l node_modules


nice this is super helpful for me to see

0xJepsen · 2024-10-24T01:35:50Z

Latest push is removing generics from the ghash modes and commenting.
It also add a protoc install to the CI so maybe that will fix the protoc C not found error in the lat CI run.

Tomorrow i will remove the generic from the ghashEnd template and then be able to remove it from the `SelectGhashBlocks1 template. I will also then have a good understanding of why we prepare 9 different (3 sets of 3 ghash blocks) when we should just be preparing one. this isn't adding up because we have 4 modes but three variations. I am hoping to be able to really start cutting constraints when that becomes clear.

Constraints are about the same so wont update that here.

0xJepsen · 2024-10-24T15:19:12Z

Okay so i decided to rip out authentication after some back and forth with team.

constraint count in now 133k

0xJepsen · 2024-10-24T16:07:10Z

Next things to do:

load full plaintext into step_in state (to commit to it)
handle this in all logic.

0xJepsen · 2024-10-24T23:29:16Z

Okay so i have a test passing for a single block, need to test more blocks now

0xJepsen · 2024-10-25T18:41:53Z

Okay this should be good for review. Here is a quick summary of what I did here:

I pulled out our folded ghash after a good amount of research it doesn't seem worth to fold and we can do the auth tag after we fold all our encryption. (Colin and I can explain more anyone would like). This reduced constraints by about 250k which was a big win for us as we wanted a goal of less than 250k. I then wrote tests to ensure this was done correctly.
I added witness calc to CI so we can always be confident that we can build with it.
After pulling out ghash i spoke with colin a lot on a design on how to fit this in with our other circuits more optimally. We settled on a design that we were both happy about and i got to work on this. I ran into an interesting problem here where i needed to build an efficient array builder for the step out so @Autoparallel and I jammed on the WriteToIndex template that is in this PR.
I then rewrote folding tests for the result.

Throughout all of this i was pulling out things that were unused or redundant. I know this is a lot and there was a bit of scope creep but i am very happy that is went smoothly and was coordinated well with @Autoparallel. The current constraint count with these changes for aesgcmfold are:

template instances: 63
non-linear constraints: 140770
linear constraints: 0
public inputs: 0
private inputs: 97 (77 belong to witness)
public outputs: 37
wires: 128844
labels: 803348

I do think it would be a good idea for us to have a multi block test for gctr so i did open an issue for it: #104. I am not sure where to find more test vectors. If really need be i can make some, byte by byte but that is likely not the best path here.

thor314 · 2024-10-26T01:02:33Z

circuits/aes-gcm/aes-gcm-fold.circom

-template AESGCMFOLD(bytesPerFold, totalBytes) {
-    // cannot fold outside chunk boundaries.
-    assert(bytesPerFold % 16 == 0);
-    assert(totalBytes % 16 == 0);
+template AESGCMFOLD(INPUT_LEN) {
+    assert(INPUT_LEN % 16 == 0);
+
+    var DATA_BYTES = (INPUT_LEN * 2) + 5;


nice complexity golf

thor314 · 2024-10-26T01:05:03Z

circuits/aes-gcm/aes-gcm-foldable.circom

 pragma circom 2.1.9;

-include "ghash-foldable.circom";
 include "aes/cipher.circom";


making note to self to annotate difference between files called foldable and fold suffix at module level comment next week

thor314 · 2024-10-26T01:09:08Z

circuits/aes-gcm/aes-gcm-foldable.circom

-template GhashStreamMode(l, ghashBlocks) {
-    signal input cipherText[l];
-    signal output blocks[ghashBlocks*4*4];
-
-    var blockIndex = 0;
-    // layout ciphertext (l*16 bytes)
-    for (var i=0; i<l; i++) {
-        blocks[blockIndex] <== cipherText[i];
-        blockIndex += 1;
-    }
-
-    // pad remainder 
-    for (var i=blockIndex; i<ghashBlocks*4*4; i++) {
-        blocks[i] <== 0x00; 
-    }
-}
-
-template GhashEndMode(l, totalBlocks, ghashBlocks) {
-    signal input cipherText[l];
-    signal output blocks[ghashBlocks*4*4];
-
-    var blockIndex = 0;
-    // layout ciphertext (l*16 bytes)
-    for (var i=0; i<l; i++) {
-        blocks[blockIndex] <== cipherText[i];
-        blockIndex += 1;
-    }
-
-    signal lengthData[8] <== [0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x80];
-    for (var i = 0; i<8; i++) {
-        blocks[blockIndex] <== lengthData[i];
-        blockIndex += 1;
-    }
-
-    // length of blocks as a u64 (8 bytes)
-    var len = totalBlocks * 128;
-    for (var i=0; i<8; i++) {
-        var byte_value = 0;
-        var val = 1;
-        for (var j=0; j<8; j++) {
-            var bit = (len >> i*8+j) & 1;
-            byte_value += bit*val;
-            val = val+val;
-        }
-        // Insert in reversed (big endian) order. 
-        blocks[blockIndex+7-i] <== byte_value;
-    }
-    blockIndex+=8; 
-    // NOTE: Added this so all of blocks is written
-    for (var i = 0; i<16; i++) {
-        blocks[blockIndex] <== 0;
-        blockIndex += 1;
+    // extract the counter column wise.
+    for (var i = 0; i < 4; i++) {
+        counter[i] <== J0[i][3];
    }


you love to see a big chunk killed like this. Was this all just stuff we wrote while figuring things out but didn't use?

No tracy wrote most of this, which i think was maybe not the best path forward.

thor314 · 2024-10-26T01:09:46Z

circuits/aes-gcm/aes-gcm.circom

    var blockCount = l\16;
    if(l%16 > 0){
        blockCount = blockCount + 1;
    }
-    var ghashblocks = 1 + blockCount + 1; // blocksize is 16 bytes


but the 1's are homoiconic

haha, It took me a while to dissect why there was three of them

thor314 · 2024-10-26T01:11:10Z

circuits/aes-gcm/ghash-foldable.circom

was this file unused?

yeah we decide to remove ghash entirely from this circuit. IT doesn't quite make sense to fold here since a since aes block does three ghash blocks that are not symmetric, which is where there was a bunch of strange ghash mode selector logic that gave us constraint bloat. I was talking a bit with @Autoparallel and i think we could fold ghash at a later step after we have all the cipher text because then we can do number of ciphertext blocks + 2 (as you see above) and fold that many ghash times, but then we will also need one gctr round on the result which is possible (see step 4-6 of nist spec). But it's pending that we will use at all for the test net at first.

thor314 · 2024-10-26T01:12:31Z

circuits/aes-gcm/utils.circom

@@ -436,3 +314,107 @@ template Selector(n) {

    out <== sums[n];
 }
+
+// TODO(WJ 2024-10-24): shared across parser circuits should consolidate.


thanks for this

thor314 · 2024-10-26T01:12:45Z

circuits/aes-gcm/utils.circom

+
+    // Note: this is underconstrained, we need to constrain that index + n <= m
+    // Need to constrain that index + n <= m -- can't be an assertion, because uses a signal
+    // ------------------------- //
+
+    // Here, we get an array of ALL zeros, except at the `index` AND `index + n`
+    //                                    beginning-------^^^^^ end---^^^^^^^^^  


great comments

thor314 · 2024-10-26T01:15:37Z

had a couple questions about some folding chunks you removed waylon. non blocking, good job

0xJepsen · 2024-10-28T17:29:11Z

Going to merge so we can start sheeping

0xJepsen added 5 commits October 22, 2024 11:42

add witness calc to CI

2fa03c1

idiomatic CI nameing

fa6dec4

constraints: 801698, removed redunant parameters

d7aeefd

constraints: 801676, ghash modes?

46d3b57

simplify a bit more

28749f6

thor314 reviewed Oct 23, 2024

View reviewed changes

0xJepsen added 4 commits October 23, 2024 12:56

CI Fix

b6d1250

comments: tracing our ghash mode selectors

16a09eb

mode logs

817ddf9

removing unneeded generics and adding documentation

9e3e7e0

0xJepsen added 2 commits October 23, 2024 19:55

remove unused test

41d3a52

removed all auth from ghash

72693d7

0xJepsen added 9 commits October 24, 2024 10:49

figure fix

60b9c0b

compiling array builder

bbe6cd7

accumulating all values

bb3ecad

witness calc works

a830f22

Update aes-gcm-fold.circom

496fc47

fix bug in WriteToIndex component

c20fce7

squashing bugs 🐞

4b0bd5f

squashing bugs 🐞

8bee83d

tests passing for a single block

d85a79f

0xJepsen added 3 commits October 24, 2024 17:34

remove misleading logs

f6dba09

remove unused components

0a45262

fix CI

2ecaca1

0xJepsen requested a review from Autoparallel October 24, 2024 23:52

cleanning up comments

93d53d6

0xJepsen added 4 commits October 24, 2024 18:05

cleanning up comments

1d901a7

bug in to blocks?

842ba21

passing non zero pr test

4ac3335

test passing for two block case

558f179

0xJepsen marked this pull request as ready for review October 25, 2024 18:29

0xJepsen added refactor 📦 repository structural changes test 🧪 testing optimize ⏰ make it fast labels Oct 25, 2024

0xJepsen added this to the AES-GCM P R O O F milestone Oct 25, 2024

0xJepsen self-assigned this Oct 25, 2024

0xJepsen mentioned this pull request Oct 25, 2024

readability: review the repo, document and comment for readability #100

Closed

0xJepsen added 2 commits October 25, 2024 14:00

remove unused foldable ghash

3021955

fix ci

d42a38f

thor314 approved these changes Oct 26, 2024

View reviewed changes

0xJepsen merged commit 9ef4926 into main Oct 28, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Golf aes gcm fold #103

Golf aes gcm fold #103

0xJepsen commented Oct 22, 2024 •

edited

Loading

thor314 Oct 23, 2024

0xJepsen commented Oct 24, 2024 •

edited

Loading

0xJepsen commented Oct 24, 2024

0xJepsen commented Oct 24, 2024

0xJepsen commented Oct 24, 2024

0xJepsen commented Oct 25, 2024 •

edited

Loading

thor314 Oct 26, 2024

thor314 Oct 26, 2024

thor314 Oct 26, 2024

0xJepsen Oct 27, 2024

thor314 Oct 26, 2024

0xJepsen Oct 27, 2024

thor314 Oct 26, 2024

0xJepsen Oct 27, 2024

thor314 Oct 26, 2024

thor314 Oct 26, 2024

thor314 commented Oct 26, 2024

0xJepsen commented Oct 28, 2024

Golf aes gcm fold #103

Golf aes gcm fold #103

Conversation

0xJepsen commented Oct 22, 2024 • edited Loading

Choose a reason for hiding this comment

0xJepsen commented Oct 24, 2024 • edited Loading

0xJepsen commented Oct 24, 2024

0xJepsen commented Oct 24, 2024

0xJepsen commented Oct 24, 2024

0xJepsen commented Oct 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thor314 commented Oct 26, 2024

0xJepsen commented Oct 28, 2024

0xJepsen commented Oct 22, 2024 •

edited

Loading

0xJepsen commented Oct 24, 2024 •

edited

Loading

0xJepsen commented Oct 25, 2024 •

edited

Loading