-
Notifications
You must be signed in to change notification settings - Fork 202
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
test(simulation): re-enable simulation tests (#1735)
* test(simulation): fix non-determinism sim test * fix: CLI flags and app params * refactor: clean up TestAppStateDeterminism * test(sim): fix TestFullAppSimulation * refactor: make ModuleManager public * test(sim): add TestAppImportExport * test(sim): add TestAppSimulationAfterImport * chore: update changelog * ci: add GHA workflow to run sim tests * ci: update workflow name * docs: add simulation tests readme * Update CHANGELOG.md * fix: remove println --------- Co-authored-by: Unique Divine <[email protected]>
- Loading branch information
1 parent
022600b
commit a8ff7ef
Showing
15 changed files
with
564 additions
and
202 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| name: Simulation tests | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| # every push to default branch | ||
| - main | ||
| schedule: | ||
| # once per day | ||
| - cron: "0 0 * * *" | ||
| pull_request: | ||
| branches: | ||
| # every pull request to default branch | ||
| - main | ||
|
|
||
| jobs: | ||
| test-sim-nondeterminism: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: 1.21 | ||
| cache: true | ||
| - name: TestAppStateDeterminism | ||
| run: | | ||
| make test-sim-nondeterminism | ||
| test-sim-default-genesis-fast: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: 1.21 | ||
| cache: true | ||
| - name: TestFullAppSimulation | ||
| run: | | ||
| make test-sim-default-genesis-fast | ||
| test-sim-import-export: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: 1.21 | ||
| cache: true | ||
| - name: TestAppImportExport | ||
| run: | | ||
| make test-sim-import-export | ||
| test-sim-after-import: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: 1.21 | ||
| cache: true | ||
| - name: TestAppSimulationAfterImport | ||
| run: | | ||
| make test-sim-after-import |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,43 +1,51 @@ | ||
| BINDIR = $(GOPATH)/bin | ||
| RUNSIM = $(BINDIR)/runsim | ||
| SIMAPP = ./simapp | ||
|
|
||
| .PHONY: runsim | ||
| runsim: $(RUNSIM) | ||
| $(RUNSIM): | ||
| @echo "Installing runsim..." | ||
| @(cd /tmp && go install github.com/cosmos/tools/cmd/[email protected]) | ||
|
|
||
| .PHONY: test-sim-nondeterminism | ||
| test-sim-nondeterminism: | ||
| @echo "Running non-determinism test..." | ||
| @go test -mod=readonly $(SIMAPP) -run TestAppStateDeterminism -Enabled=true \ | ||
| -NumBlocks=100 -BlockSize=200 -Commit=true -Period=0 -v -timeout 24h | ||
| @go test -mod=readonly -v $(SIMAPP) \ | ||
| -run TestAppStateDeterminism \ | ||
| -Enabled=true \ | ||
| -Params=params.json \ | ||
| -NumBlocks=100 \ | ||
| -BlockSize=200 \ | ||
| -Commit=true \ | ||
| -Period=0 \ | ||
| -Verbose=true | ||
|
|
||
| .PHONY: test-sim-default-genesis-fast | ||
| test-sim-default-genesis-fast: | ||
| @echo "Running default genesis simulation..." | ||
| @go test -mod=readonly $(SIMAPP) -run TestFullAppSimulation \ | ||
| -Enabled=true -NumBlocks=100 -BlockSize=200 -Commit=true -Seed=99 -Period=5 -v | ||
|
|
||
| .PHONY: test-sim-custom-genesis-multi-seed | ||
| test-sim-custom-genesis-multi-seed: runsim | ||
| @echo "Running multi-seed custom genesis simulation..." | ||
| @$(RUNSIM) -SimAppPkg=$(SIMAPP) -ExitOnFail 400 5 TestFullAppSimulation | ||
|
|
||
| .PHONY: test-sim-multi-seed-long | ||
| test-sim-multi-seed-long: runsim | ||
| @echo "Running long multi-seed application simulation. This may take awhile!" | ||
| @$(RUNSIM) -Jobs=4 -SimAppPkg=$(SIMAPP) -ExitOnFail 500 50 TestFullAppSimulation | ||
| @go test -mod=readonly -v $(SIMAPP) \ | ||
| -run TestFullAppSimulation \ | ||
| -Params=params.json \ | ||
| -Enabled=true \ | ||
| -NumBlocks=100 \ | ||
| -BlockSize=200 \ | ||
| -Commit=true \ | ||
| -Seed=99 \ | ||
| -Period=0 | ||
|
|
||
| .PHONY: test-sim-multi-seed-short | ||
| test-sim-multi-seed-short: runsim | ||
| @echo "Running short multi-seed application simulation. This may take awhile!" | ||
| @$(RUNSIM) -Jobs=4 -SimAppPkg=$(SIMAPP) -ExitOnFail 50 10 TestFullAppSimulation | ||
| .PHONY: test-sim-import-export | ||
| test-sim-import-export: | ||
| @echo "Running application import/export simulation. This may take several minutes..." | ||
| @go test -mod=readonly -v $(SIMAPP) \ | ||
| -run TestAppImportExport \ | ||
| -Params=params.json \ | ||
| -Enabled=true \ | ||
| -NumBlocks=100 \ | ||
| -Commit=true \ | ||
| -Seed=99 \ | ||
| -Period=5 | ||
|
|
||
| .PHONY: test-sim-benchmark-invariants | ||
| test-sim-benchmark-invariants: | ||
| @echo "Running simulation invariant benchmarks..." | ||
| @go test -mod=readonly $(SIMAPP) -benchmem -bench=BenchmarkInvariants -run=^$ \ | ||
| -Enabled=true -NumBlocks=1000 -BlockSize=200 \ | ||
| -Period=1 -Commit=true -Seed=57 -v -timeout 24h | ||
| .PHONY: test-sim-after-import | ||
| test-sim-after-import: | ||
| @echo "Running application simulation-after-import. This may take several minutes..." | ||
| @go test -mod=readonly -v $(SIMAPP) \ | ||
| -run TestAppSimulationAfterImport \ | ||
| -Params=params.json \ | ||
| -Enabled=true \ | ||
| -NumBlocks=50 \ | ||
| -Commit=true \ | ||
| -Seed=99 \ | ||
| -Period=5 |
Oops, something went wrong.