Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update module structure #56

Merged
merged 37 commits into from
Oct 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
78c3a33
update structure
NagaTulasi Sep 25, 2024
4ffc45f
feat: make proto gen
Teja2045 Sep 25, 2024
71c0f4c
fix imports
NagaTulasi Sep 25, 2024
af93a01
Merge branch 'tulasi/update_structure' of https://github.com/vitwit/a…
NagaTulasi Sep 25, 2024
6aba59d
update proto imports
NagaTulasi Sep 25, 2024
7b7fb5d
fix: proto
Teja2045 Sep 25, 2024
83b35fd
update cli tests
NagaTulasi Sep 25, 2024
c3a3091
Merge branch 'tulasi/update_structure' of https://github.com/vitwit/a…
NagaTulasi Sep 25, 2024
ef22957
add sims tests
NagaTulasi Sep 25, 2024
78b7d19
Merge branch 'main' of https://github.com/vitwit/avail-da-module into…
NagaTulasi Sep 27, 2024
a984083
fix: lint
NagaTulasi Sep 27, 2024
89034c3
fix: simulation tests
NagaTulasi Sep 27, 2024
8cbd753
update sims workflow
NagaTulasi Sep 27, 2024
c507428
fix: sims workflow
NagaTulasi Sep 27, 2024
c2df3e6
fix: sim tests
NagaTulasi Sep 27, 2024
3b1d7d4
fix: sim tests
NagaTulasi Sep 27, 2024
cf4ecc1
fix: sim tests
NagaTulasi Sep 27, 2024
807dc69
fix: build
NagaTulasi Sep 27, 2024
0384df3
WIP
NagaTulasi Sep 30, 2024
86cd894
fix specs path
NagaTulasi Sep 30, 2024
36d23ae
add new line in makefile
NagaTulasi Sep 30, 2024
4016b10
fix: add working sims tests
NagaTulasi Sep 30, 2024
7ef2b18
fix lint
NagaTulasi Sep 30, 2024
c24d1eb
fix lint
NagaTulasi Sep 30, 2024
6165944
update runism in workflow
NagaTulasi Sep 30, 2024
de34992
add go bin path
NagaTulasi Sep 30, 2024
7084142
remove notify failure & success
NagaTulasi Sep 30, 2024
77f52ee
fix runsim issue
NagaTulasi Sep 30, 2024
4c7aad9
WIP
NagaTulasi Sep 30, 2024
6f8c16f
WIP
NagaTulasi Sep 30, 2024
b5ce372
WIP
NagaTulasi Sep 30, 2024
ca64b37
WIP
NagaTulasi Oct 1, 2024
f95ac9b
WIP
NagaTulasi Oct 1, 2024
88ca7c7
WIP
NagaTulasi Oct 1, 2024
60f8433
WIP
NagaTulasi Oct 1, 2024
2e8d43b
add non determinism workflow test
NagaTulasi Oct 1, 2024
f5f4559
remove comments
NagaTulasi Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions .github/workflows/sims.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Sims
# Sims workflow runs multiple types of simulations (nondeterminism, import-export, after-import, multi-seed-short)
# This workflow will run on all Pull Requests, if a .go, .mod or .sum file have been changed
on:
pull_request:
push:
branches:
- main

concurrency:
group: ci-${{ github.ref }}-sims
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, 'skip-sims')"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
check-latest: true
- run: make -C simapp build

install-runsim:
permissions:
contents: none
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/setup-go@v5
with:
go-version: "1.22"
check-latest: true
- name: Install runsim
run: go install github.com/cosmos/tools/cmd/[email protected]
- uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-runsim-binary

test-sim-nondeterminism:
runs-on: ubuntu-latest
needs: [build]
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
check-latest: true
- uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-runsim-binary
- name: test-sim-import-export
run: |
make test-sim-nondeterminism

test-sim-import-export:
runs-on: ubuntu-latest
needs: [build, install-runsim]
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
check-latest: true
- uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-runsim-binary
- name: test-sim-import-export
run: |
make test-sim-import-export

test-sim-after-import:
runs-on: ubuntu-latest
needs: [build, install-runsim]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
check-latest: true
- uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-runsim-binary
- name: test-sim-after-import
run: |
make test-sim-after-import

test-sim-multi-seed-short:
runs-on: ubuntu-latest
needs: [build, install-runsim]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.22"
check-latest: true
- uses: actions/cache@v4
with:
path: ~/go/bin
key: ${{ runner.os }}-go-runsim-binary
- name: test-sim-multi-seed-short
run: |
make test-sim-multi-seed-short
90 changes: 89 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
DOCKER := $(shell which docker)

export GO111MODULE = on
SIMAPP = ./simapp/app/
BINDIR ?= $(GOPATH)/bin

###############################################################################
### e2e ###
Expand Down Expand Up @@ -75,6 +77,7 @@ lint-fix:
GO := go
TARGET := cada
BINDIR ?= $(GOPATH)/bin
CURRENT_DIR = $(shell pwd)

.PHONY: all build install clean

Expand All @@ -92,7 +95,6 @@ clean:
rm -f $(TARGET)



###############################################################################
### testnet ###
###############################################################################
Expand All @@ -109,12 +111,21 @@ set-testnet-configs:
### Tests & Simulation ###
###############################################################################

# make init-simapp initializes a single local node network
# it is useful for testing and development
# Usage: make install && make init-simapp && simd start
# Warning: make init-simapp will remove all data in simapp home directory
init-simapp:
./simapp/init-chain.sh

test: test-unit
test-all: test-unit test-ledger-mock test-race test-cover

TEST_PACKAGES=./...
TEST_TARGETS := test-unit test-unit-proto test-ledger-mock test-race test-ledger test-race

include tools/Makefile

# Test runs-specific rules. To add a new test target, just add
# a new rule, customise ARGS or TEST_PACKAGES ad libitum, and
# append the new rule to the TEST_TARGETS list.
Expand All @@ -140,3 +151,80 @@ else
endif

.PHONY: run-tests test test-all $(TEST_TARGETS)

test-sim-nondeterminism:
@echo "Running non-determinism test..."
@cd ${CURRENT_DIR}/simapp/app && go test -mod=readonly -run TestAppStateDeterminism -Enabled=true \
-NumBlocks=20 -BlockSize=200 -Commit=true -Period=0 -v -timeout 24h

test-sim-custom-genesis-fast:
@echo "Running custom genesis simulation..."
@echo "By default, ${HOME}/.cada/config/genesis.json will be used."
@cd ${CURRENT_DIR}/simapp/app && go test -mod=readonly -run TestFullAppSimulation -Genesis=${HOME}/.cada/config/genesis.json \
-Enabled=true -NumBlocks=100 -BlockSize=200 -Commit=true -Seed=99 -Period=5 -v -timeout 24h

test-sim-import-export: runsim
@echo "Running application import/export simulation. This may take several minutes..."
@cd ${CURRENT_DIR}/simapp/app && $(BINDIR)/runsim -Jobs=4 -SimAppPkg=. -ExitOnFail 50 5 TestAppImportExport

test-sim-after-import: runsim
@echo "Running application simulation-after-import. This may take several minutes..."
@cd ${CURRENT_DIR}/simapp/app && $(BINDIR)/runsim -Jobs=4 -SimAppPkg=. -ExitOnFail 50 5 TestAppSimulationAfterImport

test-sim-custom-genesis-multi-seed: runsim
@echo "Running multi-seed custom genesis simulation..."
@echo "By default, ${HOME}/.cada/config/genesis.json will be used."
@cd ${CURRENT_DIR}/simapp/app && $(BINDIR)/runsim -Genesis=${HOME}/.cada/config/genesis.json -SimAppPkg=. -ExitOnFail 400 5 TestFullAppSimulation

test-sim-multi-seed-long: runsim
@echo "Running long multi-seed application simulation. This may take awhile!"
@cd ${CURRENT_DIR}/simapp/app && $(BINDIR)/runsim -Jobs=4 -SimAppPkg=. -ExitOnFail 500 50 TestFullAppSimulation

test-sim-multi-seed-short: runsim
@echo "Running short multi-seed application simulation. This may take awhile!"
@cd ${CURRENT_DIR}/simapp/app && $(BINDIR)/runsim -Jobs=4 -SimAppPkg=. -ExitOnFail 50 10 TestFullAppSimulation

test-sim-benchmark-invariants:
@echo "Running simulation invariant benchmarks..."
cd ${CURRENT_DIR}/simapp/app && @go test -mod=readonly -benchmem -bench=BenchmarkInvariants -run=^$ \
-Enabled=true -NumBlocks=1000 -BlockSize=200 \
-Period=1 -Commit=true -Seed=57 -v -timeout 24h

.PHONY: \
test-sim-nondeterminism \
test-sim-custom-genesis-fast \
test-sim-import-export \
test-sim-after-import \
test-sim-custom-genesis-multi-seed \
test-sim-multi-seed-short \
test-sim-multi-seed-long \
test-sim-benchmark-invariants

SIM_NUM_BLOCKS ?= 500
SIM_BLOCK_SIZE ?= 200
SIM_COMMIT ?= true

test-sim-benchmark:
@echo "Running application benchmark for numBlocks=$(SIM_NUM_BLOCKS), blockSize=$(SIM_BLOCK_SIZE). This may take awhile!"
@go test -mod=readonly -benchmem -run=^$$ $(APP) -bench ^BenchmarkFullAppSimulation$$ \
-Enabled=true -NumBlocks=$(SIM_NUM_BLOCKS) -BlockSize=$(SIM_BLOCK_SIZE) -Commit=$(SIM_COMMIT) -timeout 24h

test-sim-profile:
@echo "Running application benchmark for numBlocks=$(SIM_NUM_BLOCKS), blockSize=$(SIM_BLOCK_SIZE). This may take awhile!"
@go test -mod=readonly -benchmem -run=^$$ $(APP) -bench ^BenchmarkFullAppSimulation$$ \
-Enabled=true -NumBlocks=$(SIM_NUM_BLOCKS) -BlockSize=$(SIM_BLOCK_SIZE) -Commit=$(SIM_COMMIT) -timeout 24h -cpuprofile cpu.out -memprofile mem.out

.PHONY: test-sim-profile test-sim-benchmark

test-cover:
@export VERSION=$(VERSION); bash -x scripts/test_cover.sh
.PHONY: test-cover

test-rosetta:
docker build -t rosetta-ci:latest -f contrib/rosetta/node/Dockerfile .
docker-compose -f contrib/rosetta/docker-compose.yaml up --abort-on-container-exit --exit-code-from test_rosetta --build
.PHONY: test-rosetta

benchmark:
@go test -mod=readonly -bench=. $(PACKAGES_NOSIMULATION)
.PHONY: benchmark
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ CADA is a module designed to connect Cosmos sovereign chains with the Avail netw

# Integration Guide

To integrate the CADA module into your application, follow the steps outlined in the [integration guide](./integration_docs/README.md)
To integrate the CADA module into your application, follow the steps outlined in the [integration guide](./x/cada/integration_docs/README.md)

Note: Ensure that the Avail light client URL is correctly configured for the module to function as expected. For instructions on setup Avail locally, please refer to [this documentation](https://github.com/rollkit/avail-da?tab=readme-ov-file#avail-da).

Expand All @@ -31,6 +31,7 @@ These are main components in the workflow:
The core functionality of the **CADA** module is integrated with and operates on the Cosmos blockchain.

In the CADA module:

- At each block interval, the `PreBlocker` ABCI method sends a request to the `Relayer`, specifying the range of block heights that are ready to be posted to the **Avail** Data Availability (DA) network.
![Data Submission](https://github.com/user-attachments/assets/fc4d23cc-f6bd-4210-8407-47a57adcc290)

Expand All @@ -40,6 +41,7 @@ In the CADA module:
![Vote Extension](https://github.com/user-attachments/assets/ea5b10ab-fb64-4ed0-8761-44675a852a01)

## 2. Relayer

The **Relayer** facilitates communication between the Cosmos Chain, the Avail light client, and the Cosmos Provider.

- **Data Submission**: The relayer is responsible for fetching block data from the Cosmos provider and posting it to the Avail light client via an HTTP request.
Expand All @@ -48,21 +50,21 @@ The **Relayer** facilitates communication between the Cosmos Chain, the Avail li
- **Data Verification**: During verification, the relayer communicates with the Avail light client to confirm whether the data is truly available at the specified height.



## 3. Avail Light Node

The **Avail Light Client** allows interaction with the Avail DA network without requiring a full node, and without having to trust remote peers. It leverages **Data Availability Sampling (DAS)**, which the light client performs on every newly created block.

- The chain communicates with the Avail light client via the relayer during the data submission and data availability verification processes.

Find more details about the Avail Light Client [here](https://docs.availproject.org/docs/operate-a-node/run-a-light-client/Overview).

## 4. Cosmos Provider

The **Cosmos Provider** is responsible for fetching block data via RPC so that the data can be posted to Avail for availability checks.


# Workflow


- At each block interval, a request is sent from the `PreBlocker` ABCI method to the Keeper, specifying the range of block heights that are ready to be posted to the `Avail` DA network.
- The range of block heights should be from `provenHeight + 1` to `min(provenHeight + MaxBlocksLimitForBlob, CurrentBlockHeight)`.

Expand Down
2 changes: 1 addition & 1 deletion chainclient/broadcast_tx.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
"github.com/cosmos/cosmos-sdk/crypto/keyring"
sdk "github.com/cosmos/cosmos-sdk/types"
authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"
"github.com/vitwit/avail-da-module/types"
"github.com/vitwit/avail-da-module/x/cada/types"
)

// GetBinPath returns the path to the cada home directory within the user's home directory.
Expand Down
2 changes: 0 additions & 2 deletions chainclient/create_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@ const (
defaultGasLimit = 300000
)

// var availdHomePath = xfilepath.JoinFromHome(xfilepath.Path("availsdk"))

func NewClientCtx(kr keyring.Keyring, c *cometrpc.HTTP, chainID string,
cdc codec.BinaryCodec, homepath string, fromAddress sdk.AccAddress,
) client.Context {
Expand Down
Loading
Loading