Use IPFS for initial miner download of params and artifacts

[Zorlin]

I would like to propose that we enhance Lotus (may also apply to Venus and other implementations) by adding the option for downloading Filecoin parameters and artifacts over IPFS instead of fetching them from proofs.filecoin.io/ipfs/ over HTTPS.

The motivation behind this suggestion: I am trying to set up a Filecoin miner in Western Australia for testing purposes, and the initial download of 300GiB of artifacts means pulling data from across the world to my location in Australia, at high latency, which results in much slower speeds than expected. On a symmetric gigabit connection, my real world speed was between 1-3MiB/s. This would result in a 1-4 day process just for starting a miner for the first time, a huge delay and bad user experience for a first time SP.

As an experiment, I dug into the code for go-paramfetch (the library used in this process) and found that it allows you to override "IPFS_GATEWAY" with your own gateway (before it was pointed out that we have some documentation of this here), and so I overrode it with a local IPFS gateway. No major difference in speed. I then spun up an IPFS node in Singapore on Whatbox, and pinned the Filecoin collab cluster dataset, and tried again - I am now consistently seeing 10-20MiB/s speeds or better, vs the original 1-3MiB/s.

The proposal is - we have a "legacy" download option which uses the current implementation, and a native IPFS one which uses IPFS to pull all the artifacts as necessary.

Other thoughts:

• For people running more than one Lotus miner, they might run an IPFS (cluster) node that pins all the Filecoin proofs and params for very fast setup time, as new nodes that come online from that point should see fast downloads.

[anjor]

Why is this a fip? Isn't this an implementation detail, and as such a feature request to lotus/Venus? Not sure why this needs to be at the protocol layer.

[Zorlin]

I was requested to put it in as a FIP - https://filecoinproject.slack.com/archives/CPFTWMY7N/p1692708087811069?thread_ts=1692675580.389589&cid=CPFTWMY7N

[anorth]

I'm afraid you were misled. I will transfer this to the Lotus repo.