Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

do not merge: testing CI with all DB / logs changes #876

Closed
wants to merge 70 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a406f7f
feat: add new txIndex column to event meta response
paologalligit Sep 24, 2024
e22616b
test: add convert event test
paologalligit Sep 25, 2024
856701b
feat: make txLog and txIndex as optional return params
paologalligit Sep 26, 2024
37920c1
chore: update swagger with new event optional data
paologalligit Sep 26, 2024
c30b6a0
feat: save logIndex in sequence
paologalligit Oct 7, 2024
99d5fff
feat: tweaked bits in sequence
paologalligit Oct 11, 2024
5663cb4
refactor: rename optional log meta field
paologalligit Oct 11, 2024
10a4bc0
refactor: comments, yaml and txIndex counts
paologalligit Nov 5, 2024
ca8b38b
Darren/logdb remove leading zeros (#865)
darrenvechain Oct 29, 2024
f9173e4
feat: add new txIndex column to event meta response
paologalligit Sep 24, 2024
f3bd272
test: add convert event test
paologalligit Sep 25, 2024
c5b3c63
feat: make txLog and txIndex as optional return params
paologalligit Sep 26, 2024
76a38d9
chore: update swagger with new event optional data
paologalligit Sep 26, 2024
ad6204e
feat: save logIndex in sequence
paologalligit Oct 7, 2024
1415b40
feat: tweaked bits in sequence
paologalligit Oct 11, 2024
94f4070
refactor: rename optional log meta field
paologalligit Oct 11, 2024
464f530
refactor: comments, yaml and txIndex counts
paologalligit Nov 5, 2024
9114c0b
rebase to master
paologalligit Nov 7, 2024
a80a62a
fix: remove stale struct
paologalligit Nov 6, 2024
9217741
add txIndex to returned logdb query
paologalligit Nov 7, 2024
297ffd5
reset to 0 eventCount and transferCount each receipt and write blockI…
paologalligit Nov 7, 2024
9ff47cb
fix lint
paologalligit Nov 7, 2024
6f4c9ef
rephrase logIndex description in yaml file
paologalligit Nov 7, 2024
f81bc85
refactor: use filter.Option instead of eventFilter.Option
paologalligit Nov 7, 2024
a2a74f8
move includeIndexes to api
libotony Nov 8, 2024
d3e7903
Darren/logdb remove leading zeros (#865)
darrenvechain Oct 29, 2024
a7cf6fd
cmd/thor: update instance dir to v4
qianbin Dec 14, 2023
b81910e
trie: implement varint-prefix coder
qianbin Jan 9, 2024
19c7810
deps: add github.com/qianbin/drlp
qianbin Jan 9, 2024
7edff9c
trie: implement appendHexToCompact & compactLen
qianbin Jan 9, 2024
997429c
trie: temporarily remove merkle proof stuff
qianbin Jan 9, 2024
e0ceb39
trie: many changes
qianbin Jan 11, 2024
08d2283
trie: optimize full-node encoding/decoding
qianbin Jan 18, 2024
3260ede
trie: tweak shortnode encoding
qianbin Jan 19, 2024
4c83982
muxdb: move engine pkg
qianbin Jan 19, 2024
97debf7
trie: add Version() method for node interface
qianbin Jan 26, 2024
da44763
muxdb: refactor due to trie updates and:
qianbin Jan 26, 2024
5a7942f
chain: a lot of changes
qianbin Jan 28, 2024
acf9b3c
state: changes due to update of trie
qianbin Feb 1, 2024
f4d7ec1
lowrlp: remove this pkg
qianbin Feb 1, 2024
5f85854
txpool: changes due to underlying pkg update
qianbin Feb 2, 2024
2801e81
genesis: changes due to underlying pkg update
qianbin Feb 2, 2024
4f437c7
consensus: changes due to underlying pkg update
qianbin Feb 2, 2024
547c8f9
builtin: changes due to underlying pkg update
qianbin Feb 2, 2024
356a0ba
runtime: changes due to underlying pkg update
qianbin Feb 2, 2024
d3040c1
api: changes due to underlying pkg update
qianbin Feb 2, 2024
aaa4b8f
cmd/thor/pruner: rename pkg optimizer to pruner
qianbin Feb 2, 2024
aa2f9a4
cmd/thor: changes due to underlying pkg update
qianbin Feb 2, 2024
73213b2
muxdb: abandon leaf filter
qianbin Feb 12, 2024
63d8853
cmd/thor/pruner: use smaller period when nearly synced
qianbin Feb 12, 2024
b66b4ed
muxdb: improve trie node path encoding
qianbin Feb 12, 2024
982e02e
trie: treat short nodes as standalone nodes when skipping hash
qianbin Feb 12, 2024
235c964
cmd/thor: fix disablePrunerFlag not work
qianbin Feb 13, 2024
79ece8f
trie: improve refNode encoding/decoding
qianbin Feb 19, 2024
b6cb3f6
muxdb: improve history node key encoding
qianbin Feb 19, 2024
bc552d6
cmd/thor: adjust pruner parameters
qianbin Feb 20, 2024
f550058
build: fix test cases
qianbin Feb 21, 2024
e1d7f4a
lint: fix lint error
qianbin Feb 21, 2024
6f40214
muxdb: fix ver encoding in node blob cache
qianbin Feb 28, 2024
55e7f76
muxdb: add test cases for cache
qianbin Feb 28, 2024
16eaa32
runtime: fix test compile error
qianbin Mar 25, 2024
230ecf7
make build and test pass after rebase
libotony Aug 6, 2024
c649d54
add back pruner tests
libotony Oct 25, 2024
29f904a
add tests for node encoding
libotony Nov 1, 2024
b7a42b3
minor typo
libotony Nov 1, 2024
ae23b6d
update named store space prefix
libotony Nov 5, 2024
7b883f1
add more tests
libotony Nov 6, 2024
67da142
fix block summary in repo
libotony Nov 6, 2024
ca14f79
merge branch 'feat/tx-log-seq' into feat/maindb-v4-candidate-pre-logdb
darrenvechain Nov 8, 2024
08def69
Merge branch 'feat/tx-log-seq' into feat/maindb-v4-candidate-pre-logdb
darrenvechain Nov 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions api/debug/debug_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import (
"github.com/vechain/thor/v2/thor"
"github.com/vechain/thor/v2/thorclient"
"github.com/vechain/thor/v2/tracers/logger"
"github.com/vechain/thor/v2/trie"
"github.com/vechain/thor/v2/tx"

// Force-load the tracer native engines to trigger registration
Expand Down Expand Up @@ -95,7 +96,7 @@ func TestDebug(t *testing.T) {

func TestStorageRangeFunc(t *testing.T) {
db := muxdb.NewMem()
state := state.New(db, thor.Bytes32{}, 0, 0, 0)
state := state.New(db, trie.Root{})

// Create an account and set storage values
addr := thor.BytesToAddress([]byte("account1"))
Expand Down Expand Up @@ -125,7 +126,7 @@ func TestStorageRangeFunc(t *testing.T) {

func TestStorageRangeMaxResult(t *testing.T) {
db := muxdb.NewMem()
state := state.New(db, thor.Bytes32{}, 0, 0, 0)
state := state.New(db, trie.Root{})

addr := thor.BytesToAddress([]byte("account1"))
for i := 0; i < 1001; i++ {
Expand Down
38 changes: 37 additions & 1 deletion api/doc/thor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1325,6 +1325,16 @@ components:
description: The index of the clause in the transaction, from which the log was generated.
example: 0
nullable: false
txIndex:
description: The index of the transaction in the block, from which the log was generated.
type: integer
nullable: true
example: 1
logIndex:
description: The index of the log in the receipt's outputs. This is an overall index among all clauses.
type: integer
nullable: true
example: 1

Block:
title: Block
Expand Down Expand Up @@ -1855,6 +1865,11 @@ components:
The limit of records to be included in the output. Use this parameter for pagination.

Default's to all results.
includeIndexes:
type: boolean
example: true
nullable: true
description: Include both transaction and log index in the response.
description: |
Include these parameters to receive filtered results in a paged format.

Expand All @@ -1865,7 +1880,8 @@ components:
{
"options": {
"offset": 0,
"limit": 10
"limit": 10,
"includeIndexes": true
}
}
```
Expand Down Expand Up @@ -1916,6 +1932,26 @@ components:
}
```
This refers to the range from block 10 to block 1000.

EventOptionalData:
nullable: true
type: object
title: EventOptionalData
properties:
txIndex:
type: boolean
example: true
nullable: true
description: |
Specifies whether to include in the response the event transaction index.
loglIndex:
type: boolean
example: true
nullable: true
description: |
Specifies whether to include in the response the event log index.
description: |
Specifies all the optional data that can be included in the response.

EventCriteria:
type: object
Expand Down
9 changes: 5 additions & 4 deletions api/events/events.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ func (e *Events) filter(ctx context.Context, ef *EventFilter) ([]*FilteredEvent,
}
fes := make([]*FilteredEvent, len(events))
for i, e := range events {
fes[i] = convertEvent(e)
fes[i] = convertEvent(e, ef.Options.IncludeIndexes)
}
return fes, nil
}
Expand All @@ -60,9 +60,10 @@ func (e *Events) handleFilter(w http.ResponseWriter, req *http.Request) error {
if filter.Options == nil {
// if filter.Options is nil, set to the default limit +1
// to detect whether there are more logs than the default limit
filter.Options = &logdb.Options{
Offset: 0,
Limit: e.limit + 1,
filter.Options = &Options{
Offset: 0,
Limit: e.limit + 1,
IncludeIndexes: false,
}
}

Expand Down
52 changes: 51 additions & 1 deletion api/events/events_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,56 @@ func TestEvents(t *testing.T) {
testEventWithBlocks(t, blocksToInsert)
}

func TestOptionalIndexes(t *testing.T) {
thorChain := initEventServer(t, defaultLogLimit)
defer ts.Close()
insertBlocks(t, thorChain.LogDB(), 5)
tclient = thorclient.New(ts.URL)

testCases := []struct {
name string
includeIndexes bool
expected *uint32
}{
{
name: "do not include indexes",
includeIndexes: false,
expected: nil,
},
{
name: "include indexes",
includeIndexes: true,
expected: new(uint32),
},
}

for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
filter := events.EventFilter{
CriteriaSet: make([]*events.EventCriteria, 0),
Range: nil,
Options: &events.Options{Limit: 6, IncludeIndexes: tc.includeIndexes},
Order: logdb.DESC,
}

res, statusCode, err := tclient.RawHTTPClient().RawHTTPPost("/logs/event", filter)
assert.NoError(t, err)
assert.Equal(t, http.StatusOK, statusCode)
var tLogs []*events.FilteredEvent
if err := json.Unmarshal(res, &tLogs); err != nil {
t.Fatal(err)
}
assert.Equal(t, http.StatusOK, statusCode)
assert.Equal(t, 5, len(tLogs))

for _, tLog := range tLogs {
assert.Equal(t, tc.expected, tLog.Meta.TxIndex)
assert.Equal(t, tc.expected, tLog.Meta.LogIndex)
}
})
}
}

func TestOption(t *testing.T) {
thorChain := initEventServer(t, 5)
defer ts.Close()
Expand All @@ -65,7 +115,7 @@ func TestOption(t *testing.T) {
filter := events.EventFilter{
CriteriaSet: make([]*events.EventCriteria, 0),
Range: nil,
Options: &logdb.Options{Limit: 6},
Options: &events.Options{Limit: 6},
Order: logdb.DESC,
}

Expand Down
63 changes: 27 additions & 36 deletions api/events/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
package events

import (
"fmt"
"math"

"github.com/ethereum/go-ethereum/common/hexutil"
Expand All @@ -23,6 +22,8 @@ type LogMeta struct {
TxID thor.Bytes32 `json:"txID"`
TxOrigin thor.Address `json:"txOrigin"`
ClauseIndex uint32 `json:"clauseIndex"`
TxIndex *uint32 `json:"txIndex,omitempty"`
LogIndex *uint32 `json:"logIndex,omitempty"`
}

type TopicSet struct {
Expand All @@ -42,8 +43,8 @@ type FilteredEvent struct {
}

// convert a logdb.Event into a json format Event
func convertEvent(event *logdb.Event) *FilteredEvent {
fe := FilteredEvent{
func convertEvent(event *logdb.Event, addIndexes bool) *FilteredEvent {
fe := &FilteredEvent{
Address: event.Address,
Data: hexutil.Encode(event.Data),
Meta: LogMeta{
Expand All @@ -55,50 +56,37 @@ func convertEvent(event *logdb.Event) *FilteredEvent {
ClauseIndex: event.ClauseIndex,
},
}

if addIndexes {
fe.Meta.TxIndex = &event.TxIndex
fe.Meta.LogIndex = &event.LogIndex
}

fe.Topics = make([]*thor.Bytes32, 0)
for i := 0; i < 5; i++ {
if event.Topics[i] != nil {
fe.Topics = append(fe.Topics, event.Topics[i])
}
}
return &fe
}

func (e *FilteredEvent) String() string {
return fmt.Sprintf(`
Event(
address: %v,
topics: %v,
data: %v,
meta: (blockID %v,
blockNumber %v,
blockTimestamp %v),
txID %v,
txOrigin %v,
clauseIndex %v)
)`,
e.Address,
e.Topics,
e.Data,
e.Meta.BlockID,
e.Meta.BlockNumber,
e.Meta.BlockTimestamp,
e.Meta.TxID,
e.Meta.TxOrigin,
e.Meta.ClauseIndex,
)
return fe
}

type EventCriteria struct {
Address *thor.Address `json:"address"`
TopicSet
}

type Options struct {
Offset uint64
Limit uint64
IncludeIndexes bool
}

type EventFilter struct {
CriteriaSet []*EventCriteria `json:"criteriaSet"`
Range *Range `json:"range"`
Options *logdb.Options `json:"options"`
Order logdb.Order `json:"order"`
CriteriaSet []*EventCriteria
Range *Range
Options *Options
Order logdb.Order // default asc
}

func convertEventFilter(chain *chain.Chain, filter *EventFilter) (*logdb.EventFilter, error) {
Expand All @@ -107,9 +95,12 @@ func convertEventFilter(chain *chain.Chain, filter *EventFilter) (*logdb.EventFi
return nil, err
}
f := &logdb.EventFilter{
Range: rng,
Options: filter.Options,
Order: filter.Order,
Range: rng,
Options: &logdb.Options{
Offset: filter.Options.Offset,
Limit: filter.Options.Limit,
},
Order: filter.Order,
}
if len(filter.CriteriaSet) > 0 {
f.CriteriaSet = make([]*logdb.EventCriteria, len(filter.CriteriaSet))
Expand Down
Loading
Loading