Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref: oracle rename #294

Draft
wants to merge 56 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
e976a84
use base64 of hash to emit nst-balance-change-event, don't reset inde…
leonz789 Dec 26, 2024
2e9a962
don't slash on oracle downtime
leonz789 Dec 27, 2024
62c5198
use lowercase to compare nst prefix
leonz789 Dec 27, 2024
2e30ee9
fix: update roundID in staker's balance list
leonz789 Dec 27, 2024
18804ce
ref:feedermanager. todo:events, uts, aggregator-tally
leonz789 Jan 6, 2025
9b736cc
remove unused file
leonz789 Jan 6, 2025
6cb052f
fix:pepareround,logs. complete:round tally related functions,state co…
leonz789 Jan 8, 2025
0b0e1b2
feat: add recovery check in beginblock for devmode-build
leonz789 Jan 9, 2025
bce690f
feat: update snapshot of feedermanager as state for checkTx mode
leonz789 Jan 9, 2025
928ea61
emit events in endblock for updated feeders
leonz789 Jan 9, 2025
4f3bd49
revert changes to GrowRoundID
leonz789 Jan 9, 2025
a9a34fa
add unit tests for aggregation
leonz789 Jan 9, 2025
1e95036
add equal methods for recovery logic check
leonz789 Jan 9, 2025
9e8b9e4
remove outdated aggregator
leonz789 Jan 10, 2025
6dbfd73
add build tag devmode and corresponding beginblock for recovery check
leonz789 Jan 12, 2025
6dd0806
fix lint
leonz789 Jan 13, 2025
33f8330
fix lint
leonz789 Jan 13, 2025
38643a0
revert localnodesh
leonz789 Jan 13, 2025
7342e0f
fix checking for overflow
leonz789 Jan 13, 2025
03dba54
move e2e out of ut-cover test from ci
leonz789 Jan 13, 2025
8ee52c8
separate test jobs
leonz789 Jan 13, 2025
b1c1f15
fix map iteration, ref
leonz789 Jan 13, 2025
e41e269
print log instead of panic in endBlock
leonz789 Jan 13, 2025
3fd2510
set expected height in query to get deterministic when do e2e oracle …
leonz789 Jan 13, 2025
e9c1032
fix lint
leonz789 Jan 13, 2025
a9dd690
refactor
leonz789 Jan 14, 2025
a6e9245
documentation
leonz789 Jan 14, 2025
fd31114
refactor
leonz789 Jan 14, 2025
90727b2
add version to sync nst validatorlist, fix aggregation for multiple p…
leonz789 Jan 15, 2025
6f307ec
test: wait one more block to make sure state is committed
leonz789 Jan 15, 2025
7c91056
remove slashing params for oracle downtime
leonz789 Jan 16, 2025
99cffd2
lint proto
leonz789 Jan 16, 2025
f946f89
remove package cache, refactor
leonz789 Jan 17, 2025
14ce217
remove unnecessary condition check
leonz789 Jan 17, 2025
348404d
use sorted keys for logs consistance
leonz789 Jan 17, 2025
4c5a5f2
comments for types
leonz789 Jan 17, 2025
14784a8
refactor:remove unused code
leonz789 Jan 17, 2025
47b9ebf
revert sorted keys and use map range directly for nonce remove, sort …
leonz789 Jan 17, 2025
2e1e84c
remove redundant init for validators in endblock, remove function dup…
leonz789 Jan 17, 2025
24c8be6
hanld quote behavior in deterministic order with sorted keys, add com…
leonz789 Jan 17, 2025
989ef09
test: add an e2e case for quoting after delegation
leonz789 Jan 19, 2025
19b64e0
refactor
leonz789 Jan 19, 2025
e116b03
refactor
leonz789 Jan 20, 2025
66648b5
refactor nil compare for Equals
leonz789 Jan 20, 2025
f9d31d5
typo
leonz789 Jan 20, 2025
e431f70
add e2e and ut cases, add copy algo for fchecktx
leonz789 Jan 22, 2025
689917f
refactor, typo
leonz789 Jan 22, 2025
82eabfa
fix nil compare
leonz789 Jan 22, 2025
23bdeba
refactor
leonz789 Jan 22, 2025
9ee6746
remove commentted codes, refactor
leonz789 Jan 23, 2025
1b8667e
use math.pow to calculate count
leonz789 Jan 23, 2025
3f0d2a4
regenerate swagger doc
leonz789 Jan 23, 2025
6c95eac
regenerate proto
leonz789 Jan 24, 2025
3f98a3a
typo: fix annatation
leonz789 Jan 26, 2025
30313dc
regenerate proto, swagger
leonz789 Jan 26, 2025
665a141
rename createprice to pricefeed
leonz789 Jan 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,21 @@ jobs:
run: |
make test-unit-cover
if: env.GIT_DIFF
test-unit-e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v4
with:
go-version: '1.21'
check-latest: true
- uses: actions/checkout@v4
- uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Test e2e cases
run: |
make test-unit-e2e
if: env.GIT_DIFF
28 changes: 20 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ all: build

build-all: tools build lint test vulncheck

.PHONY: distclean clean build-all
.PHONY: distclean clean build-all build

###############################################################################
### makTools & Dependencies ###
Expand Down Expand Up @@ -305,23 +305,35 @@ test-all: test-unit test-race
# we want to include all unit tests in the subfolders (tests/e2e/*)
# We also want to exclude the testutil folder because it contains only
# helper functions for the tests.
PACKAGES_UNIT=$(shell go list ./... | grep -v '/tests/e2e$$' | grep -v 'testutil')
PACKAGES_UNIT=$(shell go list ./... | grep -v '/tests/e2e' | grep -v 'testutil')
PACKAGES_UNIT_E2E=$(shell go list ./... | grep '/tests/e2e')
TEST_PACKAGES=./...
TEST_TARGETS := test-unit test-unit-cover test-race
TEST_TARGETS := test-unit test-unit-cover test-race test-unit-e2e test-unit-cover-local test-unit-e2e-local

# Test runs-specific rules. To add a new test target, just add
# a new rule, customise ARGS or TEST_PACKAGES ad libitum, and
# append the new rule to the TEST_TARGETS list.
test-unit: ARGS=-timeout=15m -gcflags=all=-l
test-unit: ARGS=-timeout=15m -gcflags=all=-l --tags devmode
test-unit: TEST_PACKAGES=$(PACKAGES_UNIT)

test-race: ARGS=-race
test-race: TEST_PACKAGES=$(PACKAGES_NOSIMULATION)
$(TEST_TARGETS): run-tests

test-unit-cover: ARGS=-timeout=15m -coverprofile=cover.out -covermode=atomic -gcflags=all=-l
test-unit-cover: ARGS=-timeout=15m -coverprofile=cover.out -covermode=atomic -gcflags=all=-l --tags devmode
test-unit-cover: TEST_PACKAGES=$(PACKAGES_UNIT)

test-unit-e2e: ARGS=-timeout=15m --tags devmode
test-unit-e2e: TEST_PACKAGES=$(PACKAGES_UNIT_E2E)

test-unit-cover-local: ARGS=-timeout=30m -coverprofile=cover.out -covermode=atomic -gcflags=all=-l --tags 'devmode local'
test-unit-cover-local: TEST_PACKAGES=$(PACKAGES_UNIT)

test-unit-e2e-local: TEST_OPTION=local
test-unit-e2e-local: ARGS=-timeout=30m --tags devmode
test-unit-e2e-local: TEST_PACKAGES=$(PACKAGES_UNIT_E2E)


test-e2e:
@if [ -z "$(TARGET_VERSION)" ]; then \
echo "Building docker image from local codebase"; \
Expand All @@ -335,9 +347,9 @@ test-e2e:

run-tests:
ifneq (,$(shell which tparse 2>/dev/null))
go test -mod=readonly -json $(ARGS) $(EXTRA_ARGS) $(TEST_PACKAGES) | tparse
TEST_OPTION=$(TEST_OPTION) go test -mod=readonly -json $(ARGS) $(EXTRA_ARGS) $(TEST_PACKAGES) | tparse
else
go test -mod=readonly $(ARGS) $(EXTRA_ARGS) $(TEST_PACKAGES)
TEST_OPTION=$(TEST_OPTION) go test -mod=readonly $(ARGS) $(EXTRA_ARGS) $(TEST_PACKAGES)
endif

test-import:
Expand Down Expand Up @@ -634,4 +646,4 @@ check-licenses:
@python3 scripts/check_licenses.py .

swagger-ui:
docker run -p 8080:8080 -e SWAGGER_JSON=/app/swagger.json -v $(pwd)/client/docs/swagger-ui:/app swaggerapi/swagger-ui
docker run -p 8080:8080 -e SWAGGER_JSON=/app/swagger.json -v $(pwd)/client/docs/swagger-ui:/app swaggerapi/swagger-ui
6 changes: 3 additions & 3 deletions app/ante/cosmos/context.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,9 @@ func (sud SetUpContextDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate
return newCtx, sdkerrors.ErrTxDecode.Wrap("Tx must be GasTx")
}

// If the transaction is an OracleCreatePriceTx, we set the gas meter to infinite
// NOTE: this works fine with cosmossdk-noopmempool+tendermint-mempool, since tenderminit used gasMeter.Limit() to check the gaswanted of tx insteand of tx.GetGas(), but if works with application mempool, we need to be caution of the tx.GetGas() since oracle create-price tx doesn't really pay.
if anteutils.IsOracleCreatePriceTx(tx) {
// If the transaction is an OraclePriceFeedTx, we set the gas meter to infinite
// NOTE: this works fine with cosmossdk-noopmempool+tendermint-mempool, since tenderminit used gasMeter.Limit() to check the gaswanted of tx insteand of tx.GetGas(), but if works with application mempool, we need to be caution of the tx.GetGas() since oracle price-feed tx doesn't really pay.
if anteutils.IsOraclePriceFeedTx(tx) {
newCtx = ctx.WithGasMeter(evmostypes.NewInfiniteGasMeterWithLimit(0))
return next(newCtx, tx, simulate)
}
Expand Down
2 changes: 1 addition & 1 deletion app/ante/cosmos/fees.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ func (dfd DeductFeeDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bo
return ctx, errorsmod.Wrap(errortypes.ErrTxDecode, "Tx must be a FeeTx")
}

if anteutils.IsOracleCreatePriceTx(tx) {
if anteutils.IsOraclePriceFeedTx(tx) {
newCtx := ctx.WithPriority(math.MaxInt64)
return next(newCtx, tx, simulate)
}
Expand Down
2 changes: 1 addition & 1 deletion app/ante/cosmos/min_price.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func (mpd MinGasPriceDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate
return ctx, errorsmod.Wrapf(errortypes.ErrInvalidType, "invalid transaction type %T, expected sdk.FeeTx", tx)
}

if anteutils.IsOracleCreatePriceTx(tx) {
if anteutils.IsOraclePriceFeedTx(tx) {
return next(ctx, tx, simulate)
}

Expand Down
17 changes: 10 additions & 7 deletions app/ante/cosmos/sigverify.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ func NewSetPubKeyDecorator(ak authante.AccountKeeper) SetPubKeyDecorator {
}

func (spkd SetPubKeyDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) {
// skip publickkey set for oracle create-price message
if utils.IsOracleCreatePriceTx(tx) {
// skip publickkey set for oracle price-feed message
if utils.IsOraclePriceFeedTx(tx) {
sigTx, ok := tx.(authsigning.SigVerifiableTx)
if !ok {
return ctx, sdkerrors.ErrTxDecode.Wrap("invalid transaction type, expected SigVerifiableTx")
Expand Down Expand Up @@ -173,7 +173,7 @@ func (sgcd SigGasConsumeDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simula
return ctx, sdkerrors.ErrTxDecode.Wrap("invalid transaction type, expected SigVerifiableTx")
}

if utils.IsOracleCreatePriceTx(tx) {
if utils.IsOraclePriceFeedTx(tx) {
return next(ctx, tx, simulate)
}

Expand Down Expand Up @@ -258,7 +258,7 @@ func OnlyLegacyAminoSigners(sigData signing.SignatureData) bool {
}

func (svd SigVerificationDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (newCtx sdk.Context, err error) {
if utils.IsOracleCreatePriceTx(tx) {
if utils.IsOraclePriceFeedTx(tx) {
sigTx, ok := tx.(authsigning.SigVerifiableTx)
if !ok {
return ctx, sdkerrors.ErrTxDecode.Wrap("invalid transaction type, expected SigVerifiableTx")
Expand Down Expand Up @@ -385,12 +385,14 @@ func NewIncrementSequenceDecorator(ak authante.AccountKeeper, oracleKeeper utils
}

func (isd IncrementSequenceDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (sdk.Context, error) {
// oracle create-price message dont need to increment sequence, check its nonce instead
if utils.IsOracleCreatePriceTx(tx) {
// oracle price-feed message dont need to increment sequence, check its nonce instead
if utils.IsOraclePriceFeedTx(tx) {
for _, msg := range tx.GetMsgs() {
msg := msg.(*oracletypes.MsgCreatePrice)
msg := msg.(*oracletypes.MsgPriceFeed)
if accAddress, err := sdk.AccAddressFromBech32(msg.Creator); err != nil {
return ctx, errors.New("invalid address")
// #nosec G115 // safe conversion
// TODO: define msg.Nonce as uint32 to avoid conversion
} else if _, err := isd.oracleKeeper.CheckAndIncreaseNonce(ctx, sdk.ConsAddress(accAddress).String(), msg.FeederID, uint32(msg.Nonce)); err != nil {
return ctx, err
}
Expand Down Expand Up @@ -445,6 +447,7 @@ func (vscd ValidateSigCountDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, sim
sigCount := 0
for _, pk := range pubKeys {
sigCount += CountSubKeys(pk)
// #nosec G115
if uint64(sigCount) > params.TxSigLimit {
return ctx, sdkerrors.ErrTooManySignatures.Wrapf("signatures: %d, limit: %d", sigCount, params.TxSigLimit)
}
Expand Down
8 changes: 5 additions & 3 deletions app/ante/cosmos/txsize_gas.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ func (cgts ConsumeTxSizeGasDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, sim
return ctx, sdkerrors.ErrTxDecode.Wrap("invalid tx type")
}

// Skip gas consumption if tx is an OracleCreatePriceTx
if anteutils.IsOracleCreatePriceTx(tx) {
// Skip gas consumption if tx is an OraclePriceFeedTx
if anteutils.IsOraclePriceFeedTx(tx) {
if len(ctx.TxBytes()) > anteutils.TxSizeLimit {
return ctx, sdkerrors.ErrTxTooLarge.Wrapf("oracle create-price tx has exceeds size limit, limit:%d, got:%d", anteutils.TxSizeLimit, len(ctx.TxBytes()))
return ctx, sdkerrors.ErrTxTooLarge.Wrapf("oracle price-feed tx has exceeds size limit, limit:%d, got:%d", anteutils.TxSizeLimit, len(ctx.TxBytes()))
}
return next(ctx, tx, simulate)
}
Expand Down Expand Up @@ -76,12 +76,14 @@ func (cgts ConsumeTxSizeGasDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, sim
}

// use stdsignature to mock the size of a full signature
// #nosec G115
simSig := legacytx.StdSignature{ // nolint:staticcheck // this will be removed when proto is ready
Signature: simSecp256k1Sig[:],
PubKey: pubkey,
}

sigBz := legacy.Cdc.MustMarshal(simSig)
// #nosec G115
cost := sdk.Gas(len(sigBz) + 6)

// If the pubkey is a multi-signature pubkey, then we estimate for the maximum
Expand Down
2 changes: 1 addition & 1 deletion app/ante/evm/fee_market.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ func NewGasWantedDecorator(
}

func (gwd GasWantedDecorator) AnteHandle(ctx sdk.Context, tx sdk.Tx, simulate bool, next sdk.AnteHandler) (newCtx sdk.Context, err error) {
if anteutils.IsOracleCreatePriceTx(tx) {
if anteutils.IsOraclePriceFeedTx(tx) {
return next(ctx, tx, simulate)
}
evmParams := gwd.evmKeeper.GetParams(ctx)
Expand Down
6 changes: 3 additions & 3 deletions app/ante/utils/oracle.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ import (
sdk "github.com/cosmos/cosmos-sdk/types"
)

// TxSizeLimit limits max size of a create-price tx, this is calculated based on one nativeTokenbalance message of worst case(max size), which will need 576 bytes for balance update
// TxSizeLimit limits max size of a price-feed tx, this is calculated based on one nativeTokenbalance message of worst case(max size), which will need 576 bytes for balance update
const TxSizeLimit = 1000

func IsOracleCreatePriceTx(tx sdk.Tx) bool {
func IsOraclePriceFeedTx(tx sdk.Tx) bool {
msgs := tx.GetMsgs()
if len(msgs) == 0 {
return false
}
for _, msg := range msgs {
if _, ok := msg.(*oracletypes.MsgCreatePrice); !ok {
if _, ok := msg.(*oracletypes.MsgPriceFeed); !ok {
return false
}
}
Expand Down
22 changes: 12 additions & 10 deletions app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,7 @@ func NewExocoreApp(

// Setup Mempool and Proposal Handlers
baseAppOptions = append(baseAppOptions, func(app *baseapp.BaseApp) {
// NOTE: we use a NoOpMempool here, for oracle create-price, it works fine since we have set a infinitgasmeterwithlimit in the ante handler to avoid the out-of-gas error no matter what the amount/gas is set by tx builder, and we set the highest priority for oracle create-price txs to work properly with tendermint mempool to make sure oracle creat-prie tx will be included in the mempool if received. And if we want to use some other application mempool, we need to take care of the gas limit and gas price in the oracle create-price txs.(we don't need to bother this since tendermint mempool use gasMeter.limit() instead of tx.Gas())
// NOTE: we use a NoOpMempool here, for oracle price-feed, it works fine since we have set a infinitgasmeterwithlimit in the ante handler to avoid the out-of-gas error no matter what the amount/gas is set by tx builder, and we set the highest priority for oracle price-feed txs to work properly with tendermint mempool to make sure oracle creat-prie tx will be included in the mempool if received. And if we want to use some other application mempool, we need to take care of the gas limit and gas price in the oracle price-feed txs.(we don't need to bother this since tendermint mempool use gasMeter.limit() instead of tx.Gas())
mempool := mempool.NoOpMempool{}
app.SetMempool(mempool)
handler := baseapp.NewDefaultProposalHandler(mempool, app)
Expand Down Expand Up @@ -915,14 +915,17 @@ func NewExocoreApp(
app.mm.SetOrderBeginBlockers(
upgradetypes.ModuleName, // to upgrade the chain
capabilitytypes.ModuleName, // before any module with capabilities like IBC
epochstypes.ModuleName, // to update the epoch
feemarkettypes.ModuleName, // set EIP-1559 gas prices
evmtypes.ModuleName, // stores chain id in memory
slashingtypes.ModuleName, // TODO after reward
evidencetypes.ModuleName, // TODO after reward
stakingtypes.ModuleName, // track historical info
ibcexported.ModuleName, // handles upgrades of chain and hence client
authz.ModuleName, // clear expired approvals
// being block of oracle will fill params cache, need to be put before epochs will use the params
// it need to put before other modules to fill params cahce before access
oracleTypes.ModuleName,
epochstypes.ModuleName, // to update the epoch
feemarkettypes.ModuleName, // set EIP-1559 gas prices
evmtypes.ModuleName, // stores chain id in memory
slashingtypes.ModuleName, // TODO after reward
evidencetypes.ModuleName, // TODO after reward
stakingtypes.ModuleName, // track historical info
ibcexported.ModuleName, // handles upgrades of chain and hence client
authz.ModuleName, // clear expired approvals
// no-op modules
ibctransfertypes.ModuleName,
icatypes.ModuleName,
Expand All @@ -943,7 +946,6 @@ func NewExocoreApp(
rewardTypes.ModuleName,
exoslashTypes.ModuleName,
avsManagerTypes.ModuleName,
oracleTypes.ModuleName,
distrtypes.ModuleName,
)

Expand Down
Loading
Loading