allow additional entropy sources

[AndySchroder]

Sparrow currently sources its entropy for new software wallets from /dev/random. It would be nice to also allow for additional entropy from dice rolls and a camera like seed signer does. Possibly, it would be good to have the option to mix all 3 together too.

[ghscuuo]

A few of the reasons why I think this is a really important new feature. (Just flip pennies.)

From ChatGPT4o:

Here are some notable RNG bugs in history:

RANDU: A notorious RNG from the 1960s and 1970s used a linear congruential generator with the formula X_(n+1) = 65539 * X_n (mod 2^31). It produced very poor-quality random numbers, leading to predictable patterns​ (Wikipedia)​​ (Apache Commons)​.

Debian OpenSSL Bug (2008): A bug in Debian's version of OpenSSL reduced the entropy of generated keys, making them predictable and compromising security​ (Wikipedia)​​ (Wikipedia)​.

Kerberos RNG Bug (1996): The Kerberos authentication system had a flaw in its RNG, leading to predictable cryptographic keys and security vulnerabilities​ (Wikipedia)​.

Sony PlayStation 3 (PS3) RNG Bug (2010): A flaw in the implementation of the Elliptic Curve Digital Signature Algorithm (ECDSA) in the PS3 led to predictable keys and the system being hacked​ (Wikipedia)​.

Netscape SSL RNG Bug (1995): Netscape's early SSL implementation used a flawed RNG seeded with predictable values like the system clock, making it vulnerable to attack​ (Wikipedia)​.

X9.31 RNG and Dual_EC_DRBG Backdoors: RNGs suspected of having backdoors, with Dual_EC_DRBG being particularly controversial due to possible NSA influence​ (Wikipedia)​.

Windows 2000 PRNG Bug: A bug in Windows 2000's RNG caused it to produce non-random results after 49.7 days of system uptime due to an integer overflow​ (Apache Commons)​.

Libbitcoin's "Milk-SAD" Vulnerability (2023): The "Milk-SAD" vulnerability in the Libbitcoin Explorer (BX) tool used the Mersenne Twister (MT19937) PRNG, providing only 32 bits of entropy. This made it feasible for attackers to brute-force private keys, leading to the theft of approximately $900,000 from over 2,600 Bitcoin wallets​ (Milksad)​​ (Milksad)​​ (Decrypt)​​ (Unchained)​.

Ethereum "Istanbul" Hard Fork RNG Issue (2019): A bug in the implementation of the Ethereum "Istanbul" hard fork affected the RNG used in smart contracts, potentially leading to predictable outputs in certain conditions​ (Wikipedia)​.

Trust Wallet RNG Bug (2021): A similar issue to the "Milk-SAD" vulnerability, where the Trust Wallet used a non-cryptographically secure RNG, leading to weak entropy and potential security risks​ (Milksad)​​ (Unchained)​.

These examples underscore the importance of using robust, cryptographically secure RNGs, particularly in applications where security is critical.

[craigraw]

These examples underscore the importance of using robust, cryptographically secure RNGs, particularly in applications where security is critical.

I completely agree. That said, it's possible to find examples of implementation errors in every aspect of security. There is no specific reason at this time to believe that the PRNGs in modern operating systems are faulty - certainly, if there were known issues we would expect to see loss of funds reported, which is not the case.

What is true however is that humans are very bad at creating entropy. We have many recent examples of Coldcard users losing funds by simply rolling a dice too few times. There is no particular reason to believe that the image from a camera will generate good entropy either. If you are a sufficiently advanced user (and most people who think they are, are not) then certainly creating your own entropy may provide some marginal additional level of security. But I don't think adding such features to Sparrow would be a net positive - indeed the reverse, as inexperienced users attempt to create their own entropy, fail to do so robustly, and lose funds as a result.

[jdlcdl]

There is no particular reason to believe that the image from a camera will generate good entropy either.

If you have time, could you expand on this and/or point me to where I can learn of supporting opinions?

I'm of the opinion that the size of images and depth of colors common in today's sensors (even if every 3rd pixel were representing only one rgb channel) result in overkill when looking to achieve enough entropy for 12 or 24 words. Thanks in advance for challenging my belief and helping me to learn more about this.

[AndySchroder]

I am curious to know if a camera can provide good entropy. I have doubts that it can, but that is one reason why I was suggesting the option to mix entropy from a camera, dice, /dev/random/, and possibly coin tosses too. I'd be more comfortable not relying on a single source.

I think it is important to also provide the user the option to not mix as well (but with Sparrow being a desktop application, there is a lot more screen space to provide people good warnings how easily you can mess up raw manual (unmixed) entropy, that you don't get on a tiny black and white Coldcard screen), that way people can verify that different software implementations are working consistently. For example, https://www.youtube.com/watch?v=mVgPoQrbi7A shows a comparison between several devices. This video replaces an earlier video he did where the Keystone did not produce the same result (this video has since been deleted). It turns out they use a different algorithm and hopefully the industry can standardize on common methods so that lay users can do basic checks like this and easily get consistent results and build confidence.

[ghscuuo]

For those who want to use Sparrow with their own entropy, an option, for discussion purposes only, suggestions welcome:

1. Everything in Bitcoin flows from entropy/RNG. Everything. Screw that up, game over.
2. I do not have the credentials to argue with Craig.
3. The length of that list of RNG fails freaks me out. Those RNG fails were catastrophic; I was personally hurt by one of them. That list is only partial. Other RNG fails will be discovered. I don't trust /dev/(u)random to not have a catastrophic flaw on all hardware variants, for all versions, of all operating systems, for all time.
4. Flipping coins has been "since ancient times" pretty much universally regarded as an extremely convenient, transparent, simple, elegant, and irrefutable source of entropy.
5. Flipping coins for 5 or 10 minutes to better protect generational wealth is time well spent.
6. (Flipping U.S. junk silver "mercury" dimes would give this tedious chore a certain joie de vivre, je ne sais quoi. 😄)
7. Ian Coleman's bip39 standalone version 0.5.4 seems well-exercised:
   https://github.com/iancoleman/bip39/releases/tag/0.5.4
   8b8e3c1be03501f57e395781de8a59fd553808e1eb1278710bd7b96dacb6d0f6
8. We're only trusting Ian's standalone html to generate our seed words correctly--not too heavy a trust burden, fairly easily verified. We could do it by hand.
9. Download Ian's standalone html to an SD card, verify it, and copy the html file from the SD card to your forever-offline Sparrow computer. Verify the copied target version, just in case...
10. On your forever-offline Sparrow computer, open Ian's standalone html, enable "Show entropy details", "Valid entropy values include: Binary", and "Auto compute".
11. Read and heed the red Entropy Warning and links, and sub-links, and sub-links.
12. In the Entropy box, type in 0s and 1s for your coin flips. (I shake 11 coins at a time for convenience.)
13. See your BIP39 Mnemonic seed words.
14. In your forever-offline Sparrow, import a software wallet, Mnemonic Words (BIP39), Use # Words, type in your words.
15. Backup, secure, bequeath, shred, burn.

Option B:

On your forever-offline Sparrow machine, send /dev/urandom through "ent" for a week or a month or something before generating your seed phrase in Sparrow.

[AndySchroder]

On your forever-offline Sparrow machine, send /dev/urandom through "ent" for a week or a month or something before generating your seed phrase in Sparrow.

What do you mean by this?

[AndySchroder]

The Ian Coleman approach works, but that to me seems convoluted. My view is Sparrow wallet should be a unified toolbox that provides all options in single .deb package that can be verified and transferred to the offline computer. It would be better if Sparrow got more and more public scrutiny, had deterministic builds, and many signers of the binary builds and git tags like bitcoin core, giving it more trust in a large user base. If we have to manually collect, verify, and transfer a variety of tools, that seems like it could introduce a lot more error and confusion to users and slow the onboarding process.

Would be curious to know Craig's goals because you can make clear tradeoffs between having a unified toolbox that provides all options and a streamlined tool with training wheels. It depends where he wants the project to go.

[ghscuuo]

@AndySchroder

Half-joking a way to confirm, with a generational-wealth level of paranoia, that my current /dev/urandom* isn't too obviously broken:

$ for paranoia in {0..1000000} ; do time dd if=/dev/urandom bs=1M count=1024 | ent ; done
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 25.0135 s, 42.9 MB/s
Entropy = 8.000000 bits per byte.

Optimum compression would reduce the size
of this 1073741824 byte file by 0 percent.

Chi square distribution for 1073741824 samples is 236.79, and randomly
would exceed this value 78.72 percent of the times.

Arithmetic mean value of data bytes is 127.5000 (127.5 = random).
Monte Carlo value for Pi is 3.141876821 (error 0.01 percent).
Serial correlation coefficient is 0.000003 (totally uncorrelated = 0.0).

real 0m25.028s
user 0m22.394s
sys 0m5.572s
...

Edit: *presuming my version of Sparrow on my version of java SecureRandom on my forever-offline Sparrow machine uses /dev/urandom...

[AndySchroder]

Chi square distribution for 1073741824 samples is 236.79, and randomly would exceed this value 78.72 percent of the times.

Arithmetic mean value of data bytes is 127.5000 (127.5 = random). Monte Carlo value for Pi is 3.141876821 (error 0.01 percent). Serial correlation coefficient is 0.000003 (totally uncorrelated = 0.0).

Curious if you could provide some more information on what these metrics mean.

[ghscuuo]

@AndySchroder Sorry, all I know is the man page:
https://www.fourmilab.ch/random/
and that Walker provided multiple outputs because he was trying to address the fact that different folks have different needs for different (kinds of?) entropy. I like this statement:
"care about unpredictability. As long as nobody can get any information about the next random number, we’re fine." from https://www.2uo.de/myths-about-urandom/

I guess most of those RNG fails were of the ilk like, we thought the pool was 2^128 bits deep, but, oops, it was only 2^32, sorry, money gone. Not sure, but I hope that sending a sufficiently large chunk of /dev/urandom through ent (or other utility), more smartly than my joke above, would at least give me a fighting change to detect a problem with /dev/urandom or whatever java SecureRandom does, on my specific version of everything (TPM?, HRNG?, kernel, kernel modules, /dev/(u)random code, etc) on my specific forever-offline Sparrow computer, before I ask Sparrow to generate my wallet. (/dev/(u)random was beyond reproach before kernel 4.8, but they somehow made it even more perfect in kernel 4.8; will it be made even more perfect again soon?)

If, instead, we can just document exactly how a normie (and by normie, I mean me) can spend a few minutes flipping coins to protect their generational wealth, then all that statistical nuance and java SecureRandom logic and the /dev/(u)random debates all become moot.

[AndySchroder]

Very interesting to know about ent. Wondering, would it be good to run a coin toss through this program too as a safety check to see if the coin or flipper has some bias?

[ghscuuo]

The only biased coin flipping that I saw in the chain of links above was intentionally biased, as part of research or a magic trick or a con game. Make each flip unique in some way. Inconsistency is a virtue.

The security of the entire Bitcoin ecosystem is foundationally based on the combination of the promises of both the incomprehensible vastness, and the verified unpredictability, of the 2^128+ bit space. (256 bits strengthens the vastness promise, but not the unpredictability promise, I don't think.) If any subtle flaw is hiding, at any level, anywhere in one's unique personal technology stack that selects one's seed on one's behalf, and that flaw breaks either one or both of those promises, then a brute-force attack is more profitable. Savor, celebrate, the few minutes flipping your lucky silver (or gold!) dime, your beneficiaries will thank you.

[remcoros]

I've seen cases where people flip a hundred coins, get a streak of 15 heads and then think: ,,well, that is not random, let's start over''. Or even worse: they manually flip some coins of the streak to make it 'look more random'.

Humans are really, really, REALLY bad at creating randomness. Leave it to the scientists and proven methods done in the OS or OS level libraries.

If you really want to go this route, use a specialized or external tool for it. It has no place in an end-user facing Wallet software in my opinion.

[ghscuuo]

I don't know anything, but I don't think it should be this easy to create identical wallets.

Connect to a free gcp cloud console.

Create a "bad_random" Dockerfile:
FROM ubuntu:latest

RUN apt-get update &&
    apt-get install -y libfaketime xxd &&
    rm -rf /var/lib/apt/lists/*

COPY sparrow-server.deb /tmp/

RUN cd /tmp &&
    dpkg --install sparrow-server.deb &&
    rm -rf /var/lib/apt/lists/*

Build and run and exec into that "bad_random" Dockerfile:
$ docker build -t bad_random .
$ docker run -it bad_random

Break the rng:
root@c0d7966986de:/# rm -f /dev/random && ln -s /dev/zero /dev/random && rm -f /dev/urandom && ln -s /dev/zero /dev/urandom

(Not sure, but iirc, my testing showed that Sparrow/SecureRandom uses both. That is, I had to break both to make this work.)

Verify the broken rng:
root@c0d7966986de:/# dd if=/dev/random bs=1 count=32 | xxd
00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................

(Same for /dev/urandom)

Prepare to break the clock:
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1

Break the clock and start a wallet app in which you plan to store your generation wealth...:
FAKETIME="1970-01-01 00:00:00" /opt/sparrow/bin/Sparrow --terminal --network testnet --level trace

Create a default wallet. The first wallet is always, for me:
cave purse kitten transfer they barrel ...

(I was half-heartedly expecting "abandon abandon abandon ...". Craig can probably explain.)

(Don't "Show seed", that crashes Sparrow and throws: "Exception in Application init method".)

Create another default wallet. The second wallet is always, for me:
modify toddler movie shrug same verb ...

No warning.  The only warning is the app crashing when trying to "Show seed". 

Not sure what all that means, maybe it's too extreme, too contrived, but it kinda freaks me out.

Flip coins, no matter how many heads in a row you get. Ian and Craig are your friends. 😄

[ghscuuo]

The "Show seed " crash:

2024-06-05 00:39:07,435 DEBUG [JavaFX Application Thread] o.f.c.i.s.DefaultSqlScriptExecutor [null:-1] Executing SQL: create table walletConfig (id identity not null, ic
onData varbinary(4096), userIcon boolean, usePayNym boolean, wallet bigint not null)

2024-06-05 00:39:07,437 DEBUG [JavaFX Application Thread] o.f.c.i.s.DefaultSqlScriptExecutor [null:-1] 0 rows affected

2024-06-05 00:39:07,442 DEBUG [JavaFX Application Thread] o.f.c.Flyway [null:-1] Memory usage: 26 of 40M

2024-06-05 00:39:07,575 TRACE [JavaFX Application Thread] o.j.v.c.Jdbi [null:-1] Jdbi [org.jdbi.v3.core.Jdbi@4cb4808] obtain handle [org.jdbi.v3.core.Handle@7a478fab] in
 0ms

2024-06-05 00:39:07,836 TRACE [JavaFX Application Thread] o.j.v.c.Handle [null:-1] Handle [org.jdbi.v3.core.Handle@7a478fab] released

2024-06-05 00:40:09,758 ERROR [main] c.s.s.t.SparrowTerminal [null:-1] Exception in thread "main"

java.lang.RuntimeException: Exception in Application init method
        at javafx.graphics@18/com.sun.javafx.application.LauncherImpl.launchApplication1(Unknown Source)
        at javafx.graphics@18/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)

Caused by: java.lang.NullPointerException: Cannot invoke "com.sun.javafx.font.FontFactory.isPlatformFont(String)" because "<local2>" is null
        at javafx.graphics@18/com.sun.javafx.font.PrismFontLoader.loadFont(Unknown Source)

[craigraw]

Appreciate the discussion here. I tend to agree with @remcoros though - there are (significantly) more disadvantages to advantages in adding an interface to incrementally create entropy, starting from zero. I'm less opposed to incrementally adding entropy starting from good entropy, but consider this low priority at best. Again, we have no reason to expect the entropy of the modern operating systems that Sparrow runs on to be poor, and this is reflected in practice by a lack of reports of missing funds (unlike with the Coldcard cases mentioned previously).

@ghscuuo the error you got is unrelated to entropy, and simply to the lack of a default platform font on the docker image. As to the proposed 'bad_random' technique - for me, it's good to see this, as it means things are working as expected.

[AndySchroder]

Here's something else relevant: https://www.crowdsupply.com/leetronics/infinite-noise-trng . Not sure if this is something that might be useful to support in addition to /dev/random?