docs(frontend): adding a use-case for Levenshtein distance

[bcm-at-zama]

Adding a use-case for Levenshtein distance

closes #https://github.com/zama-ai/concrete-internal/issues/750

[bcm-at-zama]

Works quite well but fail from time to time:

Computations in FHE

    Computing Levenshtein between strings '' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'a' - OK in 0.01 seconds
    Computing Levenshtein between strings 'b' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings 'a' and 'a' - OK in 10.76 seconds
    Computing Levenshtein between strings 'a' and 'b' - OK in 11.03 seconds
    Computing Levenshtein between strings '' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'a' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'tv' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'gag' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'wywd' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'oezbr' - OK in 0.01 seconds
    Computing Levenshtein between strings '' and 'ctuugd' - OK in 0.01 seconds
    Computing Levenshtein between strings 'n' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings 'o' and 'g' - OK in 11.78 seconds
    Computing Levenshtein between strings 'p' and 'sl' - OK in 21.41 seconds
    Computing Levenshtein between strings 'r' and 'qbd' - OK in 30.90 seconds
    Computing Levenshtein between strings 't' and 'vbej' - OK in 43.54 seconds
    Computing Levenshtein between strings 'b' and 'srvxs' - OK in 52.11 seconds

    Computing Levenshtein between strings 'n' and 'fkuftz' - OK in 63.67 seconds
    Computing Levenshtein between strings 'ey' and '' - OK in 0.01 seconds
    Computing Levenshtein between strings 'kd' and 'f' - OK in 20.81 seconds
    Computing Levenshtein between strings 'fv' and 'xh' - OK in 60.51 seconds
    Computing Levenshtein between strings 'mv' and 'dnr' - OK in 120.93 seconds
    Computing Levenshtein between strings 'db' and 'msvl' - OK in 199.30 seconds

    Computing Levenshtein between strings 'hn' and 'whoql'Traceback (most recent call last):
  File "frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.py", line 189, in <module>
    assert l1_fhe == l1_clear, f"    {l1_fhe=} and {l1_clear=} are different"
AssertionError:     l1_fhe=5 and l1_clear=4 are different

I have p_error=10**-8 which is already quite low

[bcm-at-zama]

Blocked by https://github.com/zama-ai/concrete-internal/issues/754, and a bit by https://github.com/zama-ai/concrete-internal/issues/752 and https://github.com/zama-ai/concrete-internal/issues/753

[bcm-at-zama]

Now:

• https://github.com/zama-ai/concrete-internal/issues/754: no more semantic issue (with wires + @rudy-6-4 has a fix for the issue)
• https://github.com/zama-ai/concrete-internal/issues/752: good now
• https://github.com/zama-ai/concrete-internal/issues/753: closed

[bcm-at-zama]

Hey @umut-sahin , would you have a look, to see if we can make it faster, please?

[bcm-at-zama]

Now I have added a system for alphabet: eg, one can do

python frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.py --autotest --alphabet ACTG --show_mlir

and you'll see it generates random string of A, C, T and G.

[bcm-at-zama]

It's very easy to add a new alphabet

[bcm-at-zama]

And there is a new option, to make perks on all alphabets:

python frontends/concrete-python/examples/levenshtein_distance/levenshtein_distance.py --autoperf

Typical performances for alphabet ACTG, with string of maximal length:

    Computing Levenshtein between strings 'AACG' and 'GATT' - OK in 5.24 seconds
    Computing Levenshtein between strings 'ACCG' and 'GTGG' - OK in 4.71 seconds
    Computing Levenshtein between strings 'TTTC' and 'TTTA' - OK in 4.87 seconds

Typical performances for alphabet string, with string of maximal length:

    Computing Levenshtein between strings 'skon' and 'iisi' - OK in 14.67 seconds
    Computing Levenshtein between strings 'qukm' and 'vufu' - OK in 15.01 seconds
    Computing Levenshtein between strings 'afoe' and 'kbwh' - OK in 14.62 seconds

Typical performances for alphabet STRING, with string of maximal length:

    Computing Levenshtein between strings 'RPJX' and 'MLZU' - OK in 14.33 seconds
    Computing Levenshtein between strings 'GYAQ' and 'IEWC' - OK in 13.73 seconds
    Computing Levenshtein between strings 'GEUC' and 'CLUI' - OK in 15.11 seconds

Typical performances for alphabet StRiNg, with string of maximal length:

    Computing Levenshtein between strings 'oXcM' and 'Igjh' - OK in 30.11 seconds
    Computing Levenshtein between strings 'pgBk' and 'GuOp' - OK in 28.20 seconds
    Computing Levenshtein between strings 'jScn' and 'yRRN' - OK in 30.81 seconds

Successful end

[bcm-at-zama]

What do you think guys? When you like the code, I add a small .md and measure that on AWS

[bcm-at-zama]

Blocked by https://github.com/zama-ai/concrete-internal/issues/794

[bcm-at-zama]

No more blocked by 794

[BourgerieQuentin]

Minor comment feel free to include or not, thanks.

[BourgerieQuentin]

@bcm-at-zama I updated the code with a constant function fell free to keep it or not, still bad as we need the "encrypt function" on server side, but is more a lack of the API that we need the client to export clear value

[bcm-at-zama]

Thx for 380a523 @BourgerieQuentin . I'll reapply it by hand sorry, because too much big conflicts with my coming Alphabet things.

[bcm-at-zama]

I've done an Alphabet class as requested, even if I am not a big fan. We may let some functions outside of the class, tell me what you want @umut-sahin

[bcm-at-zama]

Related to https://github.com/zama-ai/concrete-internal/issues/808

[bcm-at-zama]

@umut-sahin : hopefully it's the final review!

[umut-sahin]

A few small things, but it's good to merge otherwise!

[bcm-at-zama]

Rebasing

[bcm-at-zama]

@umut-sahin , perf have been done on hpc, could you tell me it's fine for you? Then I squash and merge!