Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement RetrieveBlockData as described by ADR 002 #232

Merged
merged 34 commits into from
Apr 13, 2021
Merged

Implement RetrieveBlockData as described by ADR 002 #232

Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
3c24d62
implement GetLeafData
evan-forbes Mar 12, 2021
1f12d6b
use a real ipfs node instead of a mocked one during testing
evan-forbes Mar 16, 2021
d8f9190
Merge branch 'master' into evan/read-ipfs
evan-forbes Mar 16, 2021
1a5f9e3
use the ResolveNode method of the ipfs api instead of dag get
evan-forbes Mar 16, 2021
b69c0af
update CalcCIDPath tests
evan-forbes Mar 16, 2021
6dff524
first draft of RetrieveBlockData implementation
evan-forbes Mar 17, 2021
25a521d
Merge branch 'master' into evan/RetrieveBlockData
evan-forbes Mar 29, 2021
e692b0e
remove the parser interface
evan-forbes Mar 30, 2021
0d6b0f5
Merge branch 'master' into evan/RetrieveBlockData
evan-forbes Apr 6, 2021
e4be5d4
refactor leaf counter, fix bug
evan-forbes Apr 7, 2021
1b2335a
remove prints
evan-forbes Apr 7, 2021
28c6dee
update to the latest commit of rsmt2d
evan-forbes Apr 8, 2021
ca8c5b2
refactor RetrieveBlockData and fix bugs
evan-forbes Apr 8, 2021
cc7b7c4
increase test timeouts for CI
evan-forbes Apr 8, 2021
0b8c2ca
increase timeouts again :disappointed:
evan-forbes Apr 8, 2021
baf2fcc
add polish and change up timeouts for CI
evan-forbes Apr 8, 2021
af6a815
fix typo causing tests to fail
evan-forbes Apr 8, 2021
d10e141
revert remnant of debugging
evan-forbes Apr 8, 2021
60afd13
polish
evan-forbes Apr 8, 2021
4199979
re adjust timeouts
evan-forbes Apr 8, 2021
bb7fd2f
even longer timeout
evan-forbes Apr 8, 2021
f5677be
add buffer to hopefully help CI
evan-forbes Apr 8, 2021
152bb82
polish
evan-forbes Apr 8, 2021
98e1d72
Merge branch 'master' into evan/RetrieveBlockData
evan-forbes Apr 8, 2021
ffe1719
don't pin data to ipfs during testing
evan-forbes Apr 9, 2021
92e4132
use max block size
evan-forbes Apr 9, 2021
eba479b
retrieve block with minimum number of ipfs requests
evan-forbes Apr 12, 2021
0879f7a
go mod tidy + use cyrpto/rand instead of math/rand
evan-forbes Apr 12, 2021
43d420b
add no lint directive to weak random number generator
evan-forbes Apr 12, 2021
0510c67
set absurd timeout
evan-forbes Apr 12, 2021
69a7037
only run large tests with the race detector of
evan-forbes Apr 12, 2021
48360ca
review feedback: better docs
evan-forbes Apr 13, 2021
20d4c19
review feedback: better docs
evan-forbes Apr 13, 2021
50b691e
review feedback: smoother test sorting for a large tests
evan-forbes Apr 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion p2p/ipld/read.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ package ipld

import (
"context"
"crypto/rand"
"errors"
"fmt"
"math"
"math/rand"

"github.com/ipfs/go-cid"
coreiface "github.com/ipfs/interface-go-ipfs-core"
Expand Down Expand Up @@ -85,6 +85,7 @@ func uniqueRandNumbers(count, max int) []uint32 {
}
samples := make(map[uint32]struct{}, count)
for i := 0; i < count; {
// nolint:gosec // G404: Use of weak random number generator
sample := uint32(rand.Intn(max))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this OK?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I think in this case it doesn't really matter as we are downloading enough data to reconstruct the whole EDS anyways. For the sampling, we should use something "more random" IMO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only concern around using a weak PRNG is that it could allow some weird timing attacks to correlate requests to nodes. Only really problematic for DAS rather than full downloading.

if _, has := samples[sample]; has {
continue
Expand Down