Skip to content

Commit

Permalink
chore(frontend): bench examples on hpc7a
Browse files Browse the repository at this point in the history 
close #854
  • Loading branch information
@aquint-zama @BourgerieQuentin
aquint-zama authored and BourgerieQuentin committed Oct 9, 2024
1 parent ea053ea commit fd9db12
Show file tree
Hide file tree
Showing 2 changed files with 167 additions and 34 deletions.
66 changes: 37 additions & 29 deletions frontends/concrete-python/examples/levenshtein_distance/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,29 +138,33 @@ Successful end
```
Typical performances for alphabet ACTG, with string of maximal length:
Computing Levenshtein between strings 'GCGA' and 'GTCA' - OK in 6.04 seconds
Computing Levenshtein between strings 'TCGA' and 'ACAA' - OK in 5.57 seconds
Computing Levenshtein between strings 'CAGT' and 'CGTT' - OK in 5.63 seconds
Computing Levenshtein between strings 'AGAC' and 'CACG' - OK in 6.22 seconds
Computing Levenshtein between strings 'CTGT' and 'TGGG' - OK in 5.43 seconds
Computing Levenshtein between strings 'GATC' and 'TGGA' - OK in 5.71 seconds
Typical performances for alphabet name, with string of maximal length:
Computing Levenshtein between strings 'ZtyQ' and 'Pvhf' - OK in 32.72 seconds
Computing Levenshtein between strings 'wHpP' and 'CbYj' - OK in 30.37 seconds
Computing Levenshtein between strings 'Npur' and 'oOAI' - OK in 30.44 seconds
Typical performances for alphabet string, with string of maximal length:
Computing Levenshtein between strings 'ctow' and 'qtor' - OK in 17.54 seconds
Computing Levenshtein between strings 'vwky' and 'enfh' - OK in 16.46 seconds
Computing Levenshtein between strings 'dqse' and 'spps' - OK in 16.49 seconds
Computing Levenshtein between strings 'plzn' and 'dctu' - OK in 30.90 seconds
Computing Levenshtein between strings 'pnux' and 'ssnz' - OK in 29.51 seconds
Computing Levenshtein between strings 'bfgf' and 'ngjd' - OK in 30.31 seconds
Typical performances for alphabet STRING, with string of maximal length:
Computing Levenshtein between strings 'TQBW' and 'LKIZ' - OK in 16.62 seconds
Computing Levenshtein between strings 'HANA' and 'CFVO' - OK in 16.32 seconds
Computing Levenshtein between strings 'BEXY' and 'YAWM' - OK in 16.58 seconds
Computing Levenshtein between strings 'YPZL' and 'RBUI' - OK in 30.87 seconds
Computing Levenshtein between strings 'QFZS' and 'YRCD' - OK in 29.54 seconds
Computing Levenshtein between strings 'YOUF' and 'BWCZ' - OK in 30.24 seconds
Typical performances for alphabet StRiNg, with string of maximal length:
Computing Levenshtein between strings 'iYmH' and 'ONnz' - OK in 30.56 seconds
Computing Levenshtein between strings 'hZyX' and 'vhHH' - OK in 30.11 seconds
Computing Levenshtein between strings 'sJdj' and 'strn' - OK in 30.48 seconds
FIXME: to be updated, by launching on hpc7a
Computing Levenshtein between strings 'YZon' and 'jFEH' - OK in 30.43 seconds
Computing Levenshtein between strings 'sJpz' and 'JApP' - OK in 30.41 seconds
Computing Levenshtein between strings 'ZmdY' and 'yfuW' - OK in 30.34 seconds
Successful end
```
Expand All @@ -183,34 +187,38 @@ it's only signed 3-bit PBS.

## Benchmarks on hpc7a

The benchmarks were done using Concrete 2.7 on `hpc7a` machine on AWS, and give:
The benchmarks were done using Concrete 2.8.1 on `hpc7a` machine on AWS, and give:

```
Typical performances for alphabet ACTG, with string of maximal length:
Computing Levenshtein between strings 'AGTC' and 'TGGA' - OK in 6.00 seconds
Computing Levenshtein between strings 'GTAA' and 'AGAC' - OK in 5.51 seconds
Computing Levenshtein between strings 'TCTT' and 'CACG' - OK in 5.49 seconds
Computing Levenshtein between strings 'AGAC' and 'CACG' - OK in 6.22 seconds
Computing Levenshtein between strings 'CTGT' and 'TGGG' - OK in 5.43 seconds
Computing Levenshtein between strings 'GATC' and 'TGGA' - OK in 5.71 seconds
Typical performances for alphabet name, with string of maximal length:
Computing Levenshtein between strings 'ZtyQ' and 'Pvhf' - OK in 32.72 seconds
Computing Levenshtein between strings 'wHpP' and 'CbYj' - OK in 30.37 seconds
Computing Levenshtein between strings 'Npur' and 'oOAI' - OK in 30.44 seconds
Typical performances for alphabet string, with string of maximal length:
Computing Levenshtein between strings 'jqdk' and 'zqlf' - OK in 17.43 seconds
Computing Levenshtein between strings 'uquc' and 'qvvp' - OK in 16.50 seconds
Computing Levenshtein between strings 'vebm' and 'ybqo' - OK in 16.46 seconds
Computing Levenshtein between strings 'plzn' and 'dctu' - OK in 30.90 seconds
Computing Levenshtein between strings 'pnux' and 'ssnz' - OK in 29.51 seconds
Computing Levenshtein between strings 'bfgf' and 'ngjd' - OK in 30.31 seconds
Typical performances for alphabet STRING, with string of maximal length:
Computing Levenshtein between strings 'UQES' and 'NWXQ' - OK in 16.53 seconds
Computing Levenshtein between strings 'LAJG' and 'NEGP' - OK in 16.26 seconds
Computing Levenshtein between strings 'OSQG' and 'OTEH' - OK in 16.52 seconds
Computing Levenshtein between strings 'YPZL' and 'RBUI' - OK in 30.87 seconds
Computing Levenshtein between strings 'QFZS' and 'YRCD' - OK in 29.54 seconds
Computing Levenshtein between strings 'YOUF' and 'BWCZ' - OK in 30.24 seconds
Typical performances for alphabet StRiNg, with string of maximal length:
Computing Levenshtein between strings 'ixgu' and 'cOSy' - OK in 30.94 seconds
Computing Levenshtein between strings 'QGCj' and 'Lknx' - OK in 29.82 seconds
Computing Levenshtein between strings 'fKVC' and 'xqaI' - OK in 30.27 seconds
FIXME: to be updated, by launching on hpc7a
Computing Levenshtein between strings 'YZon' and 'jFEH' - OK in 30.43 seconds
Computing Levenshtein between strings 'sJpz' and 'JApP' - OK in 30.41 seconds
Computing Levenshtein between strings 'ZmdY' and 'yfuW' - OK in 30.34 seconds
Successful end
```
135 changes: 130 additions & 5 deletions frontends/concrete-python/examples/prime-match/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,22 +37,147 @@ The corresponding resolution is

## Executing the classic protocol

We can run our `prime-match.py` to perform the computations: `FHE Simulation` is done in the clear to build expected results, while `FHE` is the real FHE computation. Our execution here was done on an `hpc7a` machine on AWS, with Concrete FIXME.
We can run our `prime-match.py` to perform the computations: `FHE Simulation` is done in the clear to build expected results, while `FHE` is the real FHE computation. Our execution here was done on an `hpc7a` machine on AWS, with Concrete v2.8.1, in about 3 seconds for 50 transactions on 10 symbols.

```
$ python prime-match.py
Key generation took: 10.746 seconds
FIXME: run that on hpc7a machine
Sample Input:
Bank Orders:
Buy 17 of E
Buy 30 of F
Sell 6 of D
Sell 34 of J
Buy 30 of C
Sell 54 of I
Buy 25 of G
Sell 8 of B
Sell 21 of H
Buy 24 of A
Client Orders:
Buy 56 of F
Buy 32 of A
Sell 44 of H
Buy 39 of J
Sell 50 of C
FHE Simulation:
Bank Orders:
Buy 0 of E
Buy 0 of F
Sell 0 of D
Sell 34 of J
Buy 30 of C
Sell 0 of I
Buy 0 of G
Sell 0 of B
Sell 0 of H
Buy 0 of A
Client Orders:
Buy 0 of F
Buy 0 of A
Sell 0 of H
Buy 34 of J
Sell 30 of C
FHE:
Bank Orders:
Buy 0 of E
Buy 0 of F
Sell 0 of D
Sell 34 of J
Buy 30 of C
Sell 0 of I
Buy 0 of G
Sell 0 of B
Sell 0 of H
Buy 0 of A
Client Orders:
Buy 0 of F
Buy 0 of A
Sell 0 of H
Buy 34 of J
Sell 30 of C
Complexity was: 158500926100.000
Nb of transactions: 50
Nb of Symbols: 10
Execution took: 3.237 seconds, ie 0.065 seconds per transaction
```

## Executing the semi honest protocol

We have executed the semi-honest protocol, still on an `hpc7a` machine on AWS, with Concrete FIXME.

We have executed the semi-honest protocol, still on an `hpc7a` machine on AWS, with Concrete v2.8.1, in 1 second for 20 transactions on 10 symbols.

```
$ python prime-match-semi-honest.py
FIXME: run that on hpc7a machine
Key generation took: 9.606 seconds
FHE Simulation:
Result Orders:
0 0 -> 0
23 20 -> 20
0 0 -> 0
7 27 -> 7
0 0 -> 0
0 0 -> 0
0 0 -> 0
41 13 -> 13
23 24 -> 23
0 0 -> 0
11 4 -> 4
0 0 -> 0
45 3 -> 3
0 0 -> 0
4 49 -> 4
33 31 -> 31
24 15 -> 15
0 0 -> 0
0 0 -> 0
2 23 -> 2
FHE:
Result Orders:
0 0 -> 0
23 20 -> 20
0 0 -> 0
7 27 -> 7
0 0 -> 0
0 0 -> 0
0 0 -> 0
41 13 -> 13
23 24 -> 23
0 0 -> 0
11 4 -> 4
0 0 -> 0
45 3 -> 3
0 0 -> 0
4 49 -> 4
33 31 -> 31
24 15 -> 15
0 0 -> 0
0 0 -> 0
2 23 -> 2
Complexity was: 23066687400.000
Quantities in [1, 50]
Nb of transactions: 20
Nb of Symbols: 10
Execution took: 1.011 seconds, ie 0.051 seconds per transaction
```

0 comments on commit fd9db12

Please sign in to comment.