Skip to content
This repository has been archived by the owner on May 23, 2022. It is now read-only.

Implement DAS and query for app specific data #17

Merged
merged 36 commits into from
Jan 14, 2022
Merged

Conversation

evan-forbes
Copy link
Member

@evan-forbes evan-forbes commented Nov 12, 2021

Description

Adds DAS and fetching optimint blocks from the data availability layer.

closes: #5 #6

@evan-forbes evan-forbes changed the base branch from main to evan/config November 12, 2021 06:04
@evan-forbes evan-forbes changed the base branch from evan/config to main November 12, 2021 06:04
@Wondertan
Copy link
Member

Wondertan commented Nov 13, 2021

@evan-forbes, a small note that you can already implement querying of a specific namespace and checking if DAH is available as #170 and #171 are merged.

No need to wait for DASer, as it is just a component that continuously executes DASing. Also, #182 is not blocking you.

Copy link
Member Author

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this ready for review even though it still needs a refactor to properly utilize celestia-node, as it already contains too many changes. Instead of properly utilizing celestia-node, this version of the DALC is still downloading the entire celestia block to create the Data availability header by hand.

I'll have an issue up before this is merged describing the changes that need to be made to remedy this.

.github/workflows/test.yml Show resolved Hide resolved
config/config.go Show resolved Hide resolved
Comment on lines +11 to +16
// HeightMapper keeps track of which optimint blocks where posted to which celestia
// blocks
type HeightMapper struct {
Heights map[string]int64
mut *sync.RWMutex
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tzdybal perhaps keeping track of optimint block height: celestia block height, belongs in optimint. I didn't want to add yet another KV database, but we will have to do something in the long term, as using a simple map and saving it as a json will obviously not be ideal for a blockchain with a few million blocks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think keeping track is one of DALC functions, and Optimint should be agnostic. Also, I don't see any reason not to use KV store - it's operation won't be resource-consuming anyways.

This implementation is going to work only for aggregator node (to be super specific: DALC instance that will be used by aggregator, will also be able to serve optimint block data to other nodes). I described in #24 results of our previous conversations about locating blocks in celestia chain.

IMHO we should merge as is and create follow-up issue.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 follow-up #39

server/server.go Show resolved Hide resolved
server/server.go Show resolved Hide resolved
server/test/server_integration_test.go Show resolved Hide resolved
server/test/single-node.sh Show resolved Hide resolved
@evan-forbes evan-forbes marked this pull request as ready for review January 12, 2022 20:28
@evan-forbes evan-forbes requested a review from jbowen93 January 12, 2022 20:28
cmd/dalc/main.go Outdated Show resolved Hide resolved
cmd/dalc/main.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
cmd/dalc/main.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
cmd/dalc/main.go Outdated Show resolved Hide resolved
config/config.go Outdated Show resolved Hide resolved
server/height_map.go Outdated Show resolved Hide resolved
Comment on lines +11 to +16
// HeightMapper keeps track of which optimint blocks where posted to which celestia
// blocks
type HeightMapper struct {
Heights map[string]int64
mut *sync.RWMutex
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think keeping track is one of DALC functions, and Optimint should be agnostic. Also, I don't see any reason not to use KV store - it's operation won't be resource-consuming anyways.

This implementation is going to work only for aggregator node (to be super specific: DALC instance that will be used by aggregator, will also be able to serve optimint block data to other nodes). I described in #24 results of our previous conversations about locating blocks in celestia chain.

IMHO we should merge as is and create follow-up issue.

@evan-forbes evan-forbes self-assigned this Jan 13, 2022
config/config.go Outdated Show resolved Hide resolved
evan-forbes and others added 2 commits January 13, 2022 18:44
Copy link
Member

@tzdybal tzdybal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As follow-up issues are created, I think we should move forward with this PR.

@evan-forbes evan-forbes merged commit 1d8ce20 into main Jan 14, 2022
@liamsi liamsi deleted the evan/sample branch March 2, 2022 09:35
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Add data availability sampling
5 participants