From 2de4ea0deff12a276f74df57ef3a14dab2c5dfb8 Mon Sep 17 00:00:00 2001 From: Alexander Slesarenko Date: Thu, 27 Jun 2024 18:49:29 +0100 Subject: [PATCH 1/4] eip46: first draft --- eip-0046.md | 160 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) create mode 100644 eip-0046.md diff --git a/eip-0046.md b/eip-0046.md new file mode 100644 index 00000000..a53f273f --- /dev/null +++ b/eip-0046.md @@ -0,0 +1,160 @@ + + +# EIP-0046: Atomic Transaction DAGs + +* Author: aslesarenko +* Status: Proposed +* Created: 27-06-2024 +* License: CC0 +* Forking: Not Needed + +## Contents +- [Description](#description) +- [Background And Motivation](#background-and-motivation) +- [Example Use Cases](#example-use-cases) +- [Atomic Transaction DAG Serialization Format](#atomic-transaction-dag-serialization-format) +- [JSON Format](#json-format) +- [Integration with Miners](#integration-with-miners) +- [Security Considerations](#security-considerations) + +## Description +This EIP proposes a new extension to the transaction format to support Atomic Transaction +DAGs (ATDs) in the Ergo blockchain, where DAG refers to Directed Acyclic Graph. ATDs +ensure that a series of chained transactions are either all included in a block or none +are included, thus preserving the integrity and preventing undesirable (for DApps and L2 +protocols) states on the blockchain. + +## Background And Motivation + +When a transaction is sent to the Ergo node, it is first placed into a mempool +queue. From the mempool, the node can pick up transactions to include in the next block. +Chained transactions (or transaction chains) are sequences where the outputs (UTXOs) of +one transaction are immediately consumed by the next transaction in the sequence, forming +a chain. + +Although many transactions can be chained together as long as the input UTXOs (called +Boxes in Ergo) are not yet spent, there is a problem: the default mempool procedure can +break the chain by including only part of it in the next block. This can create +undesirable states for some protocols. + +The solution is to introduce Atomic Transaction DAGs (ATDs), which extend the unconfirmed +transaction data structure to include special markers, ensuring that miners either include +all chained transactions in a block or none at all. This will preserve the integrity of +the transaction chains and ensure consistent states on the blockchain. + +## Example Use Cases + +- **Complex DApps**: Decentralized applications (DApps) that rely on multi-step + transactions can ensure atomic execution, preventing intermediate states that could lead + to vulnerabilities or inconsistent data. +- **Batch Payments**: Businesses or services that need to process batch payments can use + ATDs to ensure all payments in the batch are processed together. This also includes the + case when the batched transaction are independent but should be processed together, in + which case ADT markings connect them together. +- **Cross-Chain Operations**: Ensuring atomicity in transactions that interact with + multiple blockchains or off-chain components. +- **DeFi Protocols**: Transactions across decentralized finance (DeFi) protocols can be + DAGed to ensure atomic execution, enhancing security and reliability. + Example: 1) Withdraw from SigUSD; b) Swap on DEX; c) Send to Rosen. +- **Simplified DApp design**: DApps can simplify their design by chaining multiple + transactions together without worrying about partial execution. +- **Easier Implementation of L2 Protocols**: Layer 2 protocols can use ATDs to ensure + atomic execution of their transactions, simplifying the implementation and improving + security. +- **Improved User Experience**: Integration with wallets like Nautilus will allow DApps to use + the mempool to improve user experience with transaction chaining and instant balance + updates. + +## Implementation Design + +Ergo node can be configured to support ATDs (after updating the software) to +recognize and process the new ATDs transaction markers. The process involves: + +1. **Mempool Handling**: Collecting all transactions belonging to the same DAG from the + mempool. +2. **DAG Verification**: Ensuring that the DAG is complete and no transactions are + missing. +3. **Atomic Inclusion**: Scheduling the inclusion of the entire DAG into the next block, + ensuring atomicity. + +DApps can selectively send ATDs to Ergo nodes that announce their support for this feature, +ensuring that their transactions are processed atomically. + +## Changes in Ergo Node + +To support Atomic Transaction DAGs (ATDs), modifications are required in the Ergo node, +specifically in the handling of the mempool and the UnconfirmedTransaction class. The +changes are as follows: + +### UnconfirmedTransaction Class + +The UnconfirmedTransaction class needs to be extended with optional fields to support +chaining of transactions. These fields include: + +- `chainId`: This is an identifier of the first transaction in the chain. It is obtained + by topologically ordering the transactions in the Directed Acyclic Graph (DAG) and using + id of the first transaction. +- `indexInChain`: This is the index of the UnconfirmedTransaction in the chain. +- `numTransactions`: This is the total number of transactions in the DAG. + +### Mempool Handling + +The mempool handling process needs to be updated to support ATDs. + +### Support in DApps + +When a DApp wants to submit a new ATD to the blockchain, it should: + +- Topologically order the DAG into a list of transactions. +- Use the `id` of the first transaction as the `chainId`. +- Pair each transaction with the chaining fields. +- Post each transaction individually to the node API endpoint along with the chaining + fields. + +All of this implementation details can be encapsulated in Ergo SDKs and hidden from dApp +developers. + +### New API Endpoints + +To facilitate interaction with DApps, new API endpoints need to be created. The +operations supported should include posting new ATDs, retrieving existing ones, looking up +ATDs by their identifiers, searching for ATDs based on certain criteria, and replacing +existing ATDs. + +## Security Considerations + +- **DAG Integrity**: Ensuring that all transactions in a DAG are valid and that the + DAG is not tampered with. +- **Mempool Management**: Preventing mempool spam or abuse by enforcing limits on the + number of transactions in a DAG. +- **Protocol Updates**: While this proposal doesn't require protocol update, in the future + it may be possible to update the L1 protocol to support ATDs which will give better + security guarantees to the DApps. + +### Raw + +The UnconfirmedTransaction class needs to be extended with optional fields to support chaining of transactions. These fields include: +chainId: This is an identifier of the first transaction in the chain. It is obtained by topologically ordering the transactions in the Directed Acyclic Graph (DAG). +indexInChain: This is the index of the UnconfirmedTransaction in the chain. +numTransactions: This is the total number of transactions in the DAG. + +To facilitate interaction with DApps, new API endpoints need to be created. These endpoints should be capable of interacting with ATDs in the mempool. The operations supported should include posting new ATDs, retrieving existing ones, looking up ATDs by their identifiers, searching for ATDs based on certain criteria, and replacing existing ATDs. + +By implementing these changes, Ergo nodes will be able to support ATDs, thereby enhancing the atomicity of transaction execution and improving the overall integrity and consistency of states on the blockchain. + +Implementation ATCs on the level of mempools is rather straighforward. All we need to do +is to add optional fields to the UnconfirmedTransaction class (chaining fields): +- chainId: ModifierId - id of the first transaction in the chain obtained by + topologically ordering DAG's transactions +- indexInChain: Int - index of the UnconfirmedTransaction in the chain +- numTransactions: Int - number of transaction in the DAG. + +When DApp wants to submit a new ATD to the blockchain, it performs the following steps: +- topologically order DAG into list of transactions +- use `id` of the first transaction as `chainId` +- pair each transaction with chaining fields +- post each transaction individually to node API endpoint together with chaining fields + +To support interaction with dApps, a set of new endpoints should be created which can +interact with ATDs in the mempool (post, get, lookup, search, replace etc.) + From ad1adf0062bb9b1380093d639ecf7ba2fbf04e55 Mon Sep 17 00:00:00 2001 From: Alexander Slesarenko Date: Thu, 27 Jun 2024 22:24:44 +0100 Subject: [PATCH 2/4] eip46: second draft --- eip-0046.md | 75 +++++++++++++++++++++++------------------------------ 1 file changed, 33 insertions(+), 42 deletions(-) diff --git a/eip-0046.md b/eip-0046.md index a53f273f..752bf68a 100644 --- a/eip-0046.md +++ b/eip-0046.md @@ -1,9 +1,9 @@ -# EIP-0046: Atomic Transaction DAGs +# EIP-0046: Atomic Transaction DAGs in Mempool * Author: aslesarenko -* Status: Proposed +* Status: Draft * Created: 27-06-2024 * License: CC0 * Forking: Not Needed @@ -12,9 +12,12 @@ - [Description](#description) - [Background And Motivation](#background-and-motivation) - [Example Use Cases](#example-use-cases) -- [Atomic Transaction DAG Serialization Format](#atomic-transaction-dag-serialization-format) -- [JSON Format](#json-format) -- [Integration with Miners](#integration-with-miners) +- [Summary of The Proposal](#summary-of-the-proposal) +- [Changes in the Reference Implementation of Ergo Client](#changes-in-the-reference-implementation-of-ergo-client) + - [UnconfirmedTransaction Class](#unconfirmedtransaction-class) + - [Transaction handling changes](#transaction-handling-changes) + - [Support in DApps](#support-in-dapps) + - [New API Endpoints](#new-api-endpoints) - [Security Considerations](#security-considerations) ## Description @@ -65,9 +68,9 @@ the transaction chains and ensure consistent states on the blockchain. the mempool to improve user experience with transaction chaining and instant balance updates. -## Implementation Design +## Summary of the Proposal -Ergo node can be configured to support ATDs (after updating the software) to +Ergo node can be configured to support ATDs (after this EIP is implemented in Ergo node) to recognize and process the new ATDs transaction markers. The process involves: 1. **Mempool Handling**: Collecting all transactions belonging to the same DAG from the @@ -80,26 +83,40 @@ recognize and process the new ATDs transaction markers. The process involves: DApps can selectively send ATDs to Ergo nodes that announce their support for this feature, ensuring that their transactions are processed atomically. -## Changes in Ergo Node +## Changes in the Reference Implementation of Ergo Client -To support Atomic Transaction DAGs (ATDs), modifications are required in the Ergo node, -specifically in the handling of the mempool and the UnconfirmedTransaction class. The -changes are as follows: +To support Atomic Transaction DAGs (ATDs) in the mempool, modifications are required in +the Ergo node, specifically: +- in the API enpoints +- in handling of UnconfirmedTransaction in the mempool +- wire format (network message format) of transaction data +- +The changes are described in the following sections. ### UnconfirmedTransaction Class The UnconfirmedTransaction class needs to be extended with optional fields to support chaining of transactions. These fields include: -- `chainId`: This is an identifier of the first transaction in the chain. It is obtained +- `chainId` - identifier of the first transaction in the chain. It is obtained by topologically ordering the transactions in the Directed Acyclic Graph (DAG) and using id of the first transaction. -- `indexInChain`: This is the index of the UnconfirmedTransaction in the chain. -- `numTransactions`: This is the total number of transactions in the DAG. +- `indexInChain` - index of this UnconfirmedTransaction in the whole chain +- `numTransactions` - the total number of transactions in the DAG. -### Mempool Handling +By examining these fields, the Ergo node can determine the transaction is part of a DAG +and ensure it is processed as part of it in mempool operations (see the next section). -The mempool handling process needs to be updated to support ATDs. +### Transaction handling changes + +The mempool handling process needs to be updated to support ATDs. The main classes that +need to be changed are ErgoMemPool, OrderedTxPool and ErgoNodeViewSynchronizer: +- The OrderedTxPool should be able to store and collect transactions belonging to the same + DAG by grouping them by the new `chainId` field. +- The ErgoMemPool should be able to verify the DAG integrity and ensure that all + transactions in the DAG are either all included in the next block or none are included. +- ErgoNodeViewSynchronizer needs to broadcast transactions with new ATDs fields (if they + are present) and also be able to receive and process such ATD transactions from other nodes. ### Support in DApps @@ -131,30 +148,4 @@ existing ATDs. it may be possible to update the L1 protocol to support ATDs which will give better security guarantees to the DApps. -### Raw - -The UnconfirmedTransaction class needs to be extended with optional fields to support chaining of transactions. These fields include: -chainId: This is an identifier of the first transaction in the chain. It is obtained by topologically ordering the transactions in the Directed Acyclic Graph (DAG). -indexInChain: This is the index of the UnconfirmedTransaction in the chain. -numTransactions: This is the total number of transactions in the DAG. - -To facilitate interaction with DApps, new API endpoints need to be created. These endpoints should be capable of interacting with ATDs in the mempool. The operations supported should include posting new ATDs, retrieving existing ones, looking up ATDs by their identifiers, searching for ATDs based on certain criteria, and replacing existing ATDs. - -By implementing these changes, Ergo nodes will be able to support ATDs, thereby enhancing the atomicity of transaction execution and improving the overall integrity and consistency of states on the blockchain. - -Implementation ATCs on the level of mempools is rather straighforward. All we need to do -is to add optional fields to the UnconfirmedTransaction class (chaining fields): -- chainId: ModifierId - id of the first transaction in the chain obtained by - topologically ordering DAG's transactions -- indexInChain: Int - index of the UnconfirmedTransaction in the chain -- numTransactions: Int - number of transaction in the DAG. - -When DApp wants to submit a new ATD to the blockchain, it performs the following steps: -- topologically order DAG into list of transactions -- use `id` of the first transaction as `chainId` -- pair each transaction with chaining fields -- post each transaction individually to node API endpoint together with chaining fields - -To support interaction with dApps, a set of new endpoints should be created which can -interact with ATDs in the mempool (post, get, lookup, search, replace etc.) From 9b78218fe963f336f956ee744d648f4894de686f Mon Sep 17 00:00:00 2001 From: Alexander Slesarenko Date: Thu, 27 Jun 2024 22:40:00 +0100 Subject: [PATCH 3/4] eip46: cleanup --- eip-0046.md | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/eip-0046.md b/eip-0046.md index 752bf68a..c2b45b4e 100644 --- a/eip-0046.md +++ b/eip-0046.md @@ -21,11 +21,11 @@ - [Security Considerations](#security-considerations) ## Description -This EIP proposes a new extension to the transaction format to support Atomic Transaction -DAGs (ATDs) in the Ergo blockchain, where DAG refers to Directed Acyclic Graph. ATDs -ensure that a series of chained transactions are either all included in a block or none -are included, thus preserving the integrity and preventing undesirable (for DApps and L2 -protocols) states on the blockchain. +This EIP proposes a new extension to the transaction processing in the mempool to support +Atomic Transaction DAGs (ATDs) in the Ergo blockchain, where DAG refers to Directed +Acyclic Graph. ATDs ensure that a series of chained transactions are either all included +in a block or none are included, thus preserving the integrity and preventing undesirable +(for DApps and L2 protocols) states on the blockchain. ## Background And Motivation @@ -38,22 +38,23 @@ a chain. Although many transactions can be chained together as long as the input UTXOs (called Boxes in Ergo) are not yet spent, there is a problem: the default mempool procedure can break the chain by including only part of it in the next block. This can create -undesirable states for some protocols. +undesirable states for dApps and L2 protocols. -The solution is to introduce Atomic Transaction DAGs (ATDs), which extend the unconfirmed -transaction data structure to include special markers, ensuring that miners either include -all chained transactions in a block or none at all. This will preserve the integrity of -the transaction chains and ensure consistent states on the blockchain. +The solution proposed in this EIP is to introduce Atomic Transaction DAGs (ATDs), which +extend the unconfirmed transaction data structure to include special markers, ensuring +that miners either include all chained transactions in a block or none at all. This will +preserve the integrity of the transaction chains and ensure consistent states on the +blockchain. ## Example Use Cases - **Complex DApps**: Decentralized applications (DApps) that rely on multi-step transactions can ensure atomic execution, preventing intermediate states that could lead to vulnerabilities or inconsistent data. -- **Batch Payments**: Businesses or services that need to process batch payments can use - ATDs to ensure all payments in the batch are processed together. This also includes the - case when the batched transaction are independent but should be processed together, in - which case ADT markings connect them together. +- **Batch Payments**: DApps or services that need to process batch payments can use ATDs + to ensure all payments in the batch are processed together. This also includes the case + when the batched transactions are independent, but should be processed atomically + together, in which case ADT markings connect them together. - **Cross-Chain Operations**: Ensuring atomicity in transactions that interact with multiple blockchains or off-chain components. - **DeFi Protocols**: Transactions across decentralized finance (DeFi) protocols can be @@ -89,8 +90,8 @@ To support Atomic Transaction DAGs (ATDs) in the mempool, modifications are requ the Ergo node, specifically: - in the API enpoints - in handling of UnconfirmedTransaction in the mempool -- wire format (network message format) of transaction data -- +- wire format (network message format) of the broadcasted transaction data + The changes are described in the following sections. ### UnconfirmedTransaction Class From 585683d37b07faa2b8fb693840dd002c5ce6e5ff Mon Sep 17 00:00:00 2001 From: Alexander Slesarenko Date: Fri, 28 Jun 2024 00:17:58 +0100 Subject: [PATCH 4/4] eip46: clarification about L1 --- eip-0046.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/eip-0046.md b/eip-0046.md index c2b45b4e..f5a106ee 100644 --- a/eip-0046.md +++ b/eip-0046.md @@ -46,6 +46,12 @@ that miners either include all chained transactions in a block or none at all. T preserve the integrity of the transaction chains and ensure consistent states on the blockchain. +HOWEVER, it is important to note that ATDs don't guarantee atomicity when implemented as +mempool feature only. Full implementation will require L1 protocol changes, such that ATDs +are enforced by all miners, i.e. it is consensus level requirement, not mempool level +optimization. From this perspective, this EIP is a first step towards full ATD support on +L1. + ## Example Use Cases - **Complex DApps**: Decentralized applications (DApps) that rely on multi-step