dsa: migrate to crypto-bigint

[baloo]

This is a rebase of #784

There are two commits on top that I think should fix tests. There is a bit of cleanup to do still.

[baloo]

cc @aumetra

[aumetra]

Apologies, I was a bit absent lately. I'll have a look when I get the time (which should be in like two hours at max)

[baloo]

No need to apologize! Please take care.
That was only to keep you updated on the progress and let you chime in as needed.

If there are specific things you wanted to be taken care of, please let me know I'd be happy to.

[baloo]

out of the last 3 tests that still fail:

• sign_message: that one needs the vectors to be adjusted, we're switching the backend and by doing so the primes we get from the RNG are different because their generation is now different.
• signer_verifier_signature: that one is a precision that needs to be adjusted.
• verify_signature: I'm not quite sure yet.

Overall having to adjust precisions seems to be ... inefficent? error-prone? I guess there is a way better method for doing that. I need to have a look at RSA how they did it there.

[aumetra]

Overall having to adjust precisions seems to be ... inefficent? error-prone? I guess there is a way better method for doing that. I need to have a look at RSA how they did it there.

Seems like even rsa did the same thing. Just via the .widen method which is, I guess, a newer method that didn't exist when I started this PR a year ago

[aumetra]

Scratch that, it existed, I just didn't see it at that time

[tarcieri]

@baloo the precision aspect is definitely a bit tricky. num-bigint does it automatically, however this leads to variable-time operation depending on the value’s precision which is unsuitable for private key operations

[baloo]

Oh I understand why it's done! I'm just not sure I know what I'm doing [insert here the picture of a dog].