Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include operation information in the ingest.Change struct #5536

Merged
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d158205
Include operation information in the Change struct
karthikiyer56 Nov 20, 2024
66abef5
Dont error out when operation index out of range
karthikiyer56 Nov 20, 2024
fb94fdf
Fix failing unit test
karthikiyer56 Nov 21, 2024
06139d3
Reafactor ledger_transaction.GetChanges() to use internal functions
karthikiyer56 Nov 21, 2024
070c552
reformat comments
karthikiyer56 Nov 21, 2024
26780fa
Add all types of change reasons to the Change struct. Simplify Ledger…
karthikiyer56 Nov 21, 2024
b168415
Add LCM and transaction info altogether in change entry
karthikiyer56 Nov 21, 2024
29bc9bc
Wrte help doc for Change Entry
karthikiyer56 Nov 21, 2024
73e4494
Add TxHash to LedgerTransaction struct
karthikiyer56 Nov 21, 2024
c06e241
reorg comments
karthikiyer56 Nov 21, 2024
fe7cef8
Comments cleanup
karthikiyer56 Nov 21, 2024
06c515c
Address PR review changes
karthikiyer56 Nov 25, 2024
01b86b0
rename fields
karthikiyer56 Nov 25, 2024
e363afc
uncommit half baked changes
karthikiyer56 Nov 25, 2024
e2f95d6
Add helpers in intergration tests for creating captive core config
karthikiyer56 Nov 25, 2024
564c3bb
fix updates to change struct
karthikiyer56 Nov 27, 2024
88af498
Updates to integration.go
karthikiyer56 Nov 27, 2024
343373b
Integration tests for change - part 1
karthikiyer56 Nov 27, 2024
c7f37fb
Undo all changes to parameters_test.go
karthikiyer56 Nov 28, 2024
5313aa6
Make updates to comments and rename variables
karthikiyer56 Nov 30, 2024
dccf1e2
- Several changes to integration.go to pull out common functions from…
karthikiyer56 Nov 30, 2024
6d97032
cosmetic doc style changes
karthikiyer56 Nov 30, 2024
adf96c1
check err before file close
karthikiyer56 Nov 30, 2024
fee6685
Add tx test and in change_test.go
karthikiyer56 Dec 1, 2024
157c12e
code cleanup
karthikiyer56 Dec 1, 2024
b82a9dd
Rework test fixtures
karthikiyer56 Dec 3, 2024
3bd80ec
reformat file
karthikiyer56 Dec 3, 2024
e4b3a67
Fix breaking test
karthikiyer56 Dec 3, 2024
c3fbe24
Address code review comments
karthikiyer56 Dec 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 79 additions & 4 deletions ingest/change.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,90 @@ import (
// It also provides some helper functions to quickly check if a given
// change has occurred in an entry.
//
// If an entry is created: Pre is nil and Post is not nil.
// If an entry is updated: Pre is not nil and Post is not nil.
// If an entry is removed: Pre is not nil and Post is nil.
// Change represents a modification to a ledger entry, capturing both the before and after states
// of the entry along with the context that explains what caused the change. It is primarily used to
// track changes during transactions and/or operations within a transaction
// and can be helpful in identifying the specific cause of changes to the LedgerEntry state. (https://github.com/stellar/go/issues/5535
//
// Behavior:
//
// - **Created entries**: Pre is nil, and Post is not nil.
//
// - **Updated entries**: Both Pre and Post are non-nil.
//
// - **Removed entries**: Pre is not nil, and Post is nil.
//
// A `Change` can be caused primarily by either a transaction or by an operation within a transaction:
//
// - **Operations**:
// Each successful operation can cause multiple ledger entry changes.
// For example, a path payment operation may affect the source and destination account entries,
// as well as potentially modify offers and/or liquidity pools.
//
// - **Transactions**:
// Some ledger changes, such as those involving fees or account balances, may be caused by
// the transaction itself and may not be tied to a specific operation within a transaction.
// For instance, fees for all operations in a transaction are debited from the source account,
// triggering ledger changes without operation-specific details.
type Change struct {
// The type of the ledger entry being changed.
Type xdr.LedgerEntryType
Pre *xdr.LedgerEntry

// The state of the LedgerEntry before the change. This will be nil if the entry was created.
Pre *xdr.LedgerEntry

// The state of the LedgerEntry after the change. This will be nil if the entry was removed.
Post *xdr.LedgerEntry

// Specifies why the change occurred, represented as a LedgerEntryChangeReason
Reason LedgerEntryChangeReason

// The index of the operation within the transaction that caused the change.
// This field is relevant only when the Reason is LedgerEntryChangeReasonOperation
// This field cannot be relied upon when the compactingChangeReader is used.
OperationIndex uint32

// The LedgerTransaction responsible for the change.
// It contains details such as transaction hash, envelope, result pair, and fees.
// This field is populated only when the Reason is one of:
// LedgerEntryChangeReasonTransaction, LedgerEntryChangeReasonOperation or LedgerEntryChangeReasonFee
Transaction *LedgerTransaction

// The LedgerCloseMeta that precipitated the change.
// This is useful only when the Change is caused by an upgrade or by an eviction, i.e. outside a transaction
// This field is populated only when the Reason is one of:
// LedgerEntryChangeReasonUpgrade or LedgerEntryChangeReasonEviction
// For changes caused by transaction or operations, look at the Transaction field
Ledger *xdr.LedgerCloseMeta

// Information about the upgrade, if the change occurred as part of an upgrade
// This field is relevant only when the Reason is LedgerEntryChangeReasonUpgrade
LedgerUpgrade *xdr.LedgerUpgrade
}

// LedgerEntryChangeReason represents the reason for a ledger entry change.
type LedgerEntryChangeReason uint16
karthikiyer56 marked this conversation as resolved.
Show resolved Hide resolved

const (
// LedgerEntryChangeReasonUnknown indicates an unknown or unsupported change reason
LedgerEntryChangeReasonUnknown LedgerEntryChangeReason = iota

// LedgerEntryChangeReasonOperation indicates a change caused by an operation in a transaction
LedgerEntryChangeReasonOperation

// LedgerEntryChangeReasonTransaction indicates a change caused by the transaction itself
LedgerEntryChangeReasonTransaction

// LedgerEntryChangeReasonFee indicates a change related to transaction fees.
LedgerEntryChangeReasonFee

// LedgerEntryChangeReasonUpgrade indicates a change caused by a ledger upgrade.
LedgerEntryChangeReasonUpgrade

// LedgerEntryChangeReasonEviction indicates a change caused by entry eviction.
LedgerEntryChangeReasonEviction
)

// String returns a best effort string representation of the change.
// If the Pre or Post xdr is invalid, the field will be omitted from the string.
func (c Change) String() string {
Expand Down
16 changes: 13 additions & 3 deletions ingest/ledger_change_reader.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ func (r *LedgerChangeReader) Read() (Change, error) {
r.pending = append(r.pending, metaChanges...)
}
return r.Read()

case evictionChangesState:
entries, err := r.lcm.EvictedPersistentLedgerEntries()
if err != nil {
Expand All @@ -185,21 +186,30 @@ func (r *LedgerChangeReader) Read() (Change, error) {
entry := entries[i]
// when a ledger entry is evicted it is removed from the ledger
changes[i] = Change{
Type: entry.Data.Type,
Pre: &entry,
Post: nil,
Type: entry.Data.Type,
Pre: &entry,
Post: nil,
Reason: LedgerEntryChangeReasonEviction,
Ledger: &r.lcm,
}
}
sortChanges(changes)
r.pending = append(r.pending, changes...)
r.state++
return r.Read()

case upgradeChangesState:
// Get upgrade changes
if r.upgradeIndex < len(r.LedgerTransactionReader.lcm.UpgradesProcessing()) {
changes := GetChangesFromLedgerEntryChanges(
r.LedgerTransactionReader.lcm.UpgradesProcessing()[r.upgradeIndex].Changes,
)
ledgerUpgrades := r.LedgerTransactionReader.lcm.UpgradesProcessing()
for i := range changes {
changes[i].Reason = LedgerEntryChangeReasonUpgrade
changes[i].Ledger = &r.lcm
changes[i].LedgerUpgrade = &ledgerUpgrades[r.upgradeIndex].Upgrade
}
r.pending = append(r.pending, changes...)
r.upgradeIndex++
return r.Read()
Expand Down
82 changes: 54 additions & 28 deletions ingest/ledger_transaction.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ type LedgerTransaction struct {
FeeChanges xdr.LedgerEntryChanges
UnsafeMeta xdr.TransactionMeta
LedgerVersion uint32
Ledger xdr.LedgerCloseMeta // This is read-only and not to be modified by downstream functions
Shaptic marked this conversation as resolved.
Show resolved Hide resolved
Hash xdr.Hash
}

func (t *LedgerTransaction) txInternalError() bool {
Expand All @@ -26,7 +28,21 @@ func (t *LedgerTransaction) txInternalError() bool {
// GetFeeChanges returns a developer friendly representation of LedgerEntryChanges
// connected to fees.
func (t *LedgerTransaction) GetFeeChanges() []Change {
return GetChangesFromLedgerEntryChanges(t.FeeChanges)
changes := GetChangesFromLedgerEntryChanges(t.FeeChanges)
for i := range changes {
changes[i].Reason = LedgerEntryChangeReasonFee
changes[i].Transaction = t
}
return changes
}

func (t *LedgerTransaction) getTransactionChanges(ledgerEntryChanges xdr.LedgerEntryChanges) []Change {
changes := GetChangesFromLedgerEntryChanges(ledgerEntryChanges)
for i := range changes {
changes[i].Reason = LedgerEntryChangeReasonTransaction
changes[i].Transaction = t
}
return changes
}

// GetChanges returns a developer friendly representation of LedgerEntryChanges.
Expand All @@ -42,42 +58,48 @@ func (t *LedgerTransaction) GetChanges() ([]Change, error) {
return changes, errors.New("TransactionMeta.V=0 not supported")
case 1:
v1Meta := t.UnsafeMeta.MustV1()
txChanges := GetChangesFromLedgerEntryChanges(v1Meta.TxChanges)
// The var `txChanges` reflect the ledgerEntryChanges that are changed because of the transaction as a whole
txChanges := t.getTransactionChanges(v1Meta.TxChanges)
changes = append(changes, txChanges...)

// Ignore operations meta if txInternalError https://github.com/stellar/go/issues/2111
if t.txInternalError() && t.LedgerVersion <= 12 {
return changes, nil
}

for _, operationMeta := range v1Meta.Operations {
opChanges := GetChangesFromLedgerEntryChanges(
operationMeta.Changes,
)
// These changes reflect the ledgerEntry changes that were caused by the operations in the transaction
// Populate the operationInfo for these changes in the `Change` struct

operationMeta := v1Meta.Operations
// operationMeta is a list of lists.
// Each element in operationMeta is a list of ledgerEntryChanges
// caused by the operation at that index of the element
for opIdx := range operationMeta {
opChanges := t.operationChanges(v1Meta.Operations, uint32(opIdx))
changes = append(changes, opChanges...)
}
case 2, 3:
var (
beforeChanges, afterChanges xdr.LedgerEntryChanges
operationMeta []xdr.OperationMeta
txBeforeChanges, txAfterChanges xdr.LedgerEntryChanges
operationMeta []xdr.OperationMeta
)

switch t.UnsafeMeta.V {
case 2:
v2Meta := t.UnsafeMeta.MustV2()
beforeChanges = v2Meta.TxChangesBefore
afterChanges = v2Meta.TxChangesAfter
txBeforeChanges = v2Meta.TxChangesBefore
txAfterChanges = v2Meta.TxChangesAfter
operationMeta = v2Meta.Operations
case 3:
v3Meta := t.UnsafeMeta.MustV3()
beforeChanges = v3Meta.TxChangesBefore
afterChanges = v3Meta.TxChangesAfter
txBeforeChanges = v3Meta.TxChangesBefore
txAfterChanges = v3Meta.TxChangesAfter
operationMeta = v3Meta.Operations
default:
panic("Invalid meta version, expected 2 or 3")
}

txChangesBefore := GetChangesFromLedgerEntryChanges(beforeChanges)
txChangesBefore := t.getTransactionChanges(txBeforeChanges)
changes = append(changes, txChangesBefore...)

// Ignore operations meta and txChangesAfter if txInternalError
Expand All @@ -86,14 +108,15 @@ func (t *LedgerTransaction) GetChanges() ([]Change, error) {
return changes, nil
}

for _, operationMeta := range operationMeta {
opChanges := GetChangesFromLedgerEntryChanges(
operationMeta.Changes,
)
// operationMeta is a list of lists.
// Each element in operationMeta is a list of ledgerEntryChanges
// caused by the operation at that index of the element
for opIdx := range operationMeta {
opChanges := t.operationChanges(operationMeta, uint32(opIdx))
changes = append(changes, opChanges...)
}

txChangesAfter := GetChangesFromLedgerEntryChanges(afterChanges)
txChangesAfter := t.getTransactionChanges(txAfterChanges)
changes = append(changes, txChangesAfter...)
default:
return changes, errors.New("Unsupported TransactionMeta version")
Expand All @@ -114,15 +137,13 @@ func (t *LedgerTransaction) GetOperation(index uint32) (xdr.Operation, bool) {
// GetOperationChanges returns a developer friendly representation of LedgerEntryChanges.
// It contains only operation changes.
func (t *LedgerTransaction) GetOperationChanges(operationIndex uint32) ([]Change, error) {
changes := []Change{}

if t.UnsafeMeta.V == 0 {
return changes, errors.New("TransactionMeta.V=0 not supported")
return []Change{}, errors.New("TransactionMeta.V=0 not supported")
}

// Ignore operations meta if txInternalError https://github.com/stellar/go/issues/2111
if t.txInternalError() && t.LedgerVersion <= 12 {
return changes, nil
return []Change{}, nil
}

var operationMeta []xdr.OperationMeta
Expand All @@ -134,21 +155,26 @@ func (t *LedgerTransaction) GetOperationChanges(operationIndex uint32) ([]Change
case 3:
operationMeta = t.UnsafeMeta.MustV3().Operations
default:
return changes, errors.New("Unsupported TransactionMeta version")
return []Change{}, errors.New("Unsupported TransactionMeta version")
}

return operationChanges(operationMeta, operationIndex), nil
return t.operationChanges(operationMeta, operationIndex), nil
}

func operationChanges(ops []xdr.OperationMeta, index uint32) []Change {
func (t *LedgerTransaction) operationChanges(ops []xdr.OperationMeta, index uint32) []Change {
if int(index) >= len(ops) {
return []Change{}
}

operationMeta := ops[index]
return GetChangesFromLedgerEntryChanges(
operationMeta.Changes,
)
changes := GetChangesFromLedgerEntryChanges(operationMeta.Changes)

for i := range changes {
changes[i].Reason = LedgerEntryChangeReasonOperation
changes[i].Transaction = t
changes[i].OperationIndex = index
}
return changes
}

// GetDiagnosticEvents returns all contract events emitted by a given operation.
Expand Down
2 changes: 2 additions & 0 deletions ingest/ledger_transaction_reader.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ func (reader *LedgerTransactionReader) Read() (LedgerTransaction, error) {
UnsafeMeta: reader.lcm.TxApplyProcessing(i),
FeeChanges: reader.lcm.FeeProcessing(i),
LedgerVersion: uint32(reader.lcm.LedgerHeaderHistoryEntry().Header.LedgerVersion),
Ledger: reader.lcm,
Hash: hash,
}, nil
}

Expand Down
Loading
Loading