Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reshape and document the dht-cache #36

Open
wants to merge 52 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
53331ba
Document dht-cache
lu-zero Jul 10, 2023
f175b75
Hide unneeded API surface
lu-zero Jul 10, 2023
66c1af6
Re-export to the root of dht-cache
lu-zero Jul 10, 2023
f9c876b
Initial error type
lu-zero Jul 20, 2023
c4f778e
Simplify the DomoEvent handling
lu-zero Jul 20, 2023
308d5b6
wip: Add get_peers_stats
lu-zero Jul 20, 2023
390743a
Hide even more of the DomoCache
lu-zero Jul 25, 2023
505fd2e
Move the data structures in a separate module
lu-zero Jul 25, 2023
a9a8aa5
Derive Default for DomoCacheElement
lu-zero Jul 27, 2023
8bbfcd4
Move the rsa key generator in domolibp2p
lu-zero Jul 27, 2023
ce34b61
Make the pg test use the DOMO_DHT_TEST_DB env var if available
lu-zero Jul 28, 2023
47d6527
Split the behavior creation from the swarm setup
lu-zero Jul 28, 2023
81d40d9
wip: Abstract over the network code
lu-zero Jul 28, 2023
8b3ab42
Fix documentation
lu-zero Jul 29, 2023
63310d9
wip: Rework the cache layer
lu-zero Aug 6, 2023
6afefdc
wip: Update the broker to handle the NewPeers event
lu-zero Aug 9, 2023
c8c1a73
fixup: use any() instead of find().is_some()
lu-zero Aug 10, 2023
5351b75
fixup: Move the comments to the right place
lu-zero Aug 10, 2023
7cff78f
Wire in the persistent storage
lu-zero Aug 10, 2023
dfb9449
Allow to not use a local db at all
lu-zero Aug 10, 2023
b673ce7
fixup: Clippy lints
lu-zero Aug 11, 2023
de440c0
Test everything including pg integration
lu-zero Aug 11, 2023
75cb13b
wip: Add Builder
lu-zero Aug 14, 2023
5f02fdb
Refactor the psk logic
lu-zero Aug 14, 2023
030caee
fixme: move to dev-dependencies
lu-zero Aug 14, 2023
19817ac
Use pnet in the ephemeral swarms to ensure tests do not interact due …
lu-zero Aug 18, 2023
1587597
Speed up the test execution
lu-zero Aug 18, 2023
625a1bb
Speedup the test execution
lu-zero Aug 18, 2023
c56cf1b
fixup: Spurious comment
lu-zero Aug 18, 2023
4c9a15e
fixup: is_syncronised is not async
lu-zero Aug 18, 2023
691a18d
Do not keep the write lock longer than needed
lu-zero Aug 18, 2023
c9fc865
Explicitly yield in the swarm task
lu-zero Aug 18, 2023
7e3c3ef
Compact the CI
lu-zero Aug 18, 2023
69056d7
Add a mean to access the list of peers
lu-zero Aug 18, 2023
5b4cadf
Use more workers in the tests
lu-zero Aug 18, 2023
cc9ca28
fixme: Check for a subset of peers
lu-zero Aug 18, 2023
aca4ae7
Add Cache::get_hash
lu-zero Aug 21, 2023
6cf9414
Apply the impl Into<String>/owned Value pattern
lu-zero Aug 21, 2023
5f79a70
Use futures-concurrency
lu-zero Aug 21, 2023
c4000eb
Signal when the nodes are available, not when they are discovered
lu-zero Aug 21, 2023
b8a9410
Do not panic on non-utf8 data
lu-zero Aug 22, 2023
d2e3540
use into_iter
lu-zero Aug 22, 2023
eef07a8
Cleanup the peers check to be more readable
lu-zero Aug 22, 2023
5f336c4
drop redundant drops
lu-zero Aug 22, 2023
33881b7
Avoid redundant path
lu-zero Aug 22, 2023
339272f
Use a worker task to persist the data
lu-zero Aug 21, 2023
7020d08
fixup: multiple_peer tests should wait for all of them
lu-zero Aug 24, 2023
c7f0568
Take ownership of the elem in the local cache API as well
lu-zero Aug 24, 2023
37210e4
Refine the Ready peers event
lu-zero Sep 11, 2023
4fd2dbe
fix: export structs from cache::local
dodomorandi Sep 28, 2023
abff470
feat!: overhaul dht_cache::cache::Query API
dodomorandi Sep 29, 2023
ff3cc90
Mark Cache as Clone
lu-zero Oct 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 18 additions & 82 deletions .github/workflows/libp2p-rust-dht.yml
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,19 @@ jobs:

runs-on: ubuntu-latest

services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: mysecretpassword
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432

steps:
- uses: actions/checkout@v3

Expand Down Expand Up @@ -179,7 +192,7 @@ jobs:
RUSTFLAGS: "-Cinstrument-coverage"
LLVM_PROFILE_FILE: "libp2p-rust-dht-%p-%m.profraw"
run: |
cargo test --verbose
cargo test --verbose --all

- name: Get coverage data for codecov
run: |
Expand Down Expand Up @@ -220,34 +233,6 @@ jobs:
exit 1
fi

weighted-code-coverage:

needs: [build, docs]

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Install protobuf compiler
run: |
sudo apt-get update
sudo apt-get install protobuf-compiler

- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable

- name: Install grcov
env:
GRCOV_LINK: https://github.com/mozilla/grcov/releases/download
GRCOV_VERSION: v0.8.13
GRCOV_BINARY: grcov-x86_64-unknown-linux-musl.tar.bz2
run: |
curl -L "$GRCOV_LINK/$GRCOV_VERSION/$GRCOV_BINARY" |
tar xj -C $HOME/.cargo/bin

- name: Install weighted-code-coverage
env:
WCC_LINK: https://github.com/SoftengPoliTo/weighted-code-coverage/releases/download
Expand All @@ -257,23 +242,7 @@ jobs:
curl -L "$WCC_LINK/$WCC_VERSION/$WCC_BINARY" |
tar xz -C $HOME/.cargo/bin

- name: Install llvm-tools-preview
run: |
rustup component add llvm-tools-preview

# Not necessary on a newly created image, but strictly advised
- name: Run cargo clean
run: |
cargo clean

- name: Run tests
env:
RUSTFLAGS: "-Cinstrument-coverage"
LLVM_PROFILE_FILE: "libp2p-rust-dht-%p-%m.profraw"
run: |
cargo test --verbose

- name: Run grcov
- name: Run grcov to produce a coveralls json
run: |
grcov . --binary-path ./target/debug/ -t coveralls -s . --token YOUR_COVERALLS_TOKEN > coveralls.json

Expand All @@ -292,7 +261,7 @@ jobs:

audit:

needs: [code-coverage, weighted-code-coverage]
needs: [code-coverage]

runs-on: ubuntu-latest

Expand Down Expand Up @@ -322,7 +291,7 @@ jobs:

deny:

needs: [code-coverage, weighted-code-coverage]
needs: [code-coverage]

runs-on: ubuntu-latest

Expand Down Expand Up @@ -368,7 +337,7 @@ jobs:

udeps:

needs: [code-coverage, weighted-code-coverage]
needs: [code-coverage]

runs-on: ubuntu-latest

Expand Down Expand Up @@ -446,39 +415,6 @@ jobs:

################################## UNSAFE CHECKS LEVEL #########################

valgrind:

needs: cache-level

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Install protobuf compiler
run: |
sudo apt-get update
sudo apt-get install protobuf-compiler

- name: Install valgrind
run: |
sudo apt-get install valgrind

- name: Install cargo-valgrind
env:
VALGRIND_LINK: https://github.com/jfrimmel/cargo-valgrind/releases/download
VALGRIND_VERSION: 2.1.0
run: |
curl -L "$VALGRIND_LINK/v$VALGRIND_VERSION/cargo-valgrind-$VALGRIND_VERSION-x86_64-unknown-linux-musl.tar.gz" |
tar xz -C $HOME/.cargo/bin

# Usage of the `help` command as base command, please replace it
# with the effective command that valgrind has to analyze
- name: Run cargo-valgrind
run: |
cargo valgrind run -- --help
# cargo valgrind test

careful:

needs: cache-level
Expand Down
10 changes: 9 additions & 1 deletion dht-cache/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,19 @@ time = "0.3.17"
tokio = { version = "1.19.0", features = ["full"] }
url = "2.2.2"
rsa = "0.9"
pem-rfc7468 = { version = "0.7", features = ["alloc"] }
pem-rfc7468 = { version = "0.7", features = ["std"] }
sifis-config = { path = "../dht-config" }
openssl-sys = "*"
libsqlite3-sys = "*"
thiserror = "1.0.43"
anyhow = "1.0.72"
tokio-stream = "0.1.14"
futures-concurrency = "7.4.1"

[dev-dependencies]
env_logger = "0.10.0"
libp2p-swarm-test = "0.2.0"
libp2p = { version = "0.52.0", features = ["plaintext"] }

[package.metadata.cargo-udeps.ignore]
normal = ["openssl-sys", "libsqlite3-sys", "libc"]
Loading