From a453c4d35d0898764a68cde2a788ffab0a3c164c Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 19 May 2020 17:00:42 -0400 Subject: [PATCH 01/49] Add basic state representation. --- specs/data_structures.md | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 5618a8e..09d4dc0 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -43,6 +43,8 @@ Data Structures - [MessageData](#messagedata) - [Message](#message) - [State](#state) + - [Account](#account) + - [Validator](#validator) - [Consensus Parameters](#consensus-parameters) # Data Structures Overview @@ -475,7 +477,28 @@ enum VoteType : uint8_t { # State -TODO validator set repr +The state of the LazyLedger chain contains only account balances and the validator set (which is really just extra metadata on top of the account balances). + +Two [Sparse Merkle Trees](#sparse-merkle-tree) are maintained: one of [accounts](#account) and one of [validators](#validator). The state root is computed as the [hash](#hashdigest) of concatenation of the account tree root and the validator tree root. + +## Account + +| name | type | description | +| -------------------- | ------------------- | ---------------------------------------------------------------------------------------- | +| `balance` | `uint64` | Coin balance. | +| `isDelegating` | `bool` | Whether this account is delegating its stake or not. | +| `delegatedValidator` | [Address](#address) | _Optional._ The validator this is account is delegating to. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. `0` is this account is not a validator. | + +In the account tree, accounts are keyed by the [hash](#hashdigest) of their [address](#address). + +## Validator + +| name | type | description | +| ------------- | --------- | --------------- | +| `votingPower` | `uint256` | Voting balance. | + +In the validator tree, validators are keyed by their [address](#address). # Consensus Parameters From 62420a96c2b665722e438578c6555ab86b0b4576 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 19 May 2020 17:02:42 -0400 Subject: [PATCH 02/49] Fix typo is size of validator voting power. --- specs/data_structures.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 09d4dc0..d9a036d 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -494,9 +494,9 @@ In the account tree, accounts are keyed by the [hash](#hashdigest) of their [add ## Validator -| name | type | description | -| ------------- | --------- | --------------- | -| `votingPower` | `uint256` | Voting balance. | +| name | type | description | +| ------------- | -------- | --------------- | +| `votingPower` | `uint64` | Voting balance. | In the validator tree, validators are keyed by their [address](#address). From cf6e484fe83681f174294241e8c2dc065d8f5a53 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 19 May 2020 20:52:54 -0400 Subject: [PATCH 03/49] Update specs/data_structures.md Co-authored-by: Ismail Khoffi --- specs/data_structures.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index d9a036d..21c753b 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -487,7 +487,7 @@ Two [Sparse Merkle Trees](#sparse-merkle-tree) are maintained: one of [accounts] | -------------------- | ------------------- | ---------------------------------------------------------------------------------------- | | `balance` | `uint64` | Coin balance. | | `isDelegating` | `bool` | Whether this account is delegating its stake or not. | -| `delegatedValidator` | [Address](#address) | _Optional._ The validator this is account is delegating to. | +| `delegatedValidator` | [Address](#address) | _Optional._ The validator this account is delegating to. | | `delegatedCount` | `uint32` | Number of accounts delegating to this validator. `0` is this account is not a validator. | In the account tree, accounts are keyed by the [hash](#hashdigest) of their [address](#address). From a1340ff75515404f084eaf794d9c7565e18270ce Mon Sep 17 00:00:00 2001 From: John Adler Date: Wed, 20 May 2020 09:56:43 -0400 Subject: [PATCH 04/49] Add nonce field to accounts. --- specs/data_structures.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 21c753b..d78a5cf 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -486,8 +486,9 @@ Two [Sparse Merkle Trees](#sparse-merkle-tree) are maintained: one of [accounts] | name | type | description | | -------------------- | ------------------- | ---------------------------------------------------------------------------------------- | | `balance` | `uint64` | Coin balance. | +| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | | `isDelegating` | `bool` | Whether this account is delegating its stake or not. | -| `delegatedValidator` | [Address](#address) | _Optional._ The validator this account is delegating to. | +| `delegatedValidator` | [Address](#address) | _Optional._ The validator this account is delegating to. | | `delegatedCount` | `uint32` | Number of accounts delegating to this validator. `0` is this account is not a validator. | In the account tree, accounts are keyed by the [hash](#hashdigest) of their [address](#address). From a2adee4d542525d239a8add20eb43e65a08287e9 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 25 May 2020 14:14:41 -0400 Subject: [PATCH 05/49] First draft refactor: validators and accounts in a single tree. --- specs/data_structures.md | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index d78a5cf..19c02e1 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -44,7 +44,6 @@ Data Structures - [Message](#message) - [State](#state) - [Account](#account) - - [Validator](#validator) - [Consensus Parameters](#consensus-parameters) # Data Structures Overview @@ -477,9 +476,14 @@ enum VoteType : uint8_t { # State -The state of the LazyLedger chain contains only account balances and the validator set (which is really just extra metadata on top of the account balances). +| name | type | description | +| ----------------- | ------------------------- | ---------------------------- | +| `accountTrieRoot` | [HashDigest](#hashdigest) | Merkle root of account trie. | +| `numValidators` | `uint32` | Number of active validators | -Two [Sparse Merkle Trees](#sparse-merkle-tree) are maintained: one of [accounts](#account) and one of [validators](#validator). The state root is computed as the [hash](#hashdigest) of concatenation of the account tree root and the validator tree root. +The state of the LazyLedger chain contains only account balances and the validator set (which is extra metadata on top of the plain account balances). + +One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both account account balances and validator metadata, the _account trie_. The state root is computed as the [hash](#hashdigest) of the serialized account trie root and number of active validators. The latter is necessary to ensure light nodes can determine the entire validator set from a single state root commitment. ## Account @@ -490,16 +494,9 @@ Two [Sparse Merkle Trees](#sparse-merkle-tree) are maintained: one of [accounts] | `isDelegating` | `bool` | Whether this account is delegating its stake or not. | | `delegatedValidator` | [Address](#address) | _Optional._ The validator this account is delegating to. | | `delegatedCount` | `uint32` | Number of accounts delegating to this validator. `0` is this account is not a validator. | +| `votingPower` | `uint64` | Voting balance. | -In the account tree, accounts are keyed by the [hash](#hashdigest) of their [address](#address). - -## Validator - -| name | type | description | -| ------------- | -------- | --------------- | -| `votingPower` | `uint64` | Voting balance. | - -In the validator tree, validators are keyed by their [address](#address). +In the account trie, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). # Consensus Parameters From 6201f4cdd0116d387e311eb476f800562d22c569 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 26 May 2020 11:49:37 -0400 Subject: [PATCH 06/49] Add consensus constants for unbonding duration and maximum active validators. --- specs/consensus.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specs/consensus.md b/specs/consensus.md index 0fced16..efa90fc 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -37,6 +37,8 @@ Consensus Rules | `SHARE_RESERVED_BYTES` | `uint64` | `1` | `byte` | Bytes reserved at the beginning of each [share](data_structures.md#share). Must be sufficient to represent `SHARE_SIZE`. | | `AVAILABLE_DATA_ORIGINAL_SQUARE_SIZE` | `uint64` | | `share` | Number of rows/columns of the original data [shares](data_structures.md#share) in [square layout](data_structures.md#arranging-available-data-into-shares). | | `GENESIS_COIN_COUNT` | `uint64` | `10**8` | `4u` | `(= 100000000)` Number of coins at genesis. | +| `UNBONDING_DURATION` | `uint32` | | `block` | Duration, in blocks, for unbonding a validator or delegation. | +| `MAX_VALIDATORS` | `uint16` | `64` | | Maximum number of active validators. | ## Types From 2b14d8d4eaf4e5267a8c0910d3cf84bbf1d2e68f Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 26 May 2020 12:09:00 -0400 Subject: [PATCH 07/49] Add validator and delegation structs to accounts. --- specs/data_structures.md | 43 +++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 9 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 19c02e1..818640a 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -44,6 +44,8 @@ Data Structures - [Message](#message) - [State](#state) - [Account](#account) + - [Validator](#validator) + - [Delegation](#delegation) - [Consensus Parameters](#consensus-parameters) # Data Structures Overview @@ -479,7 +481,7 @@ enum VoteType : uint8_t { | name | type | description | | ----------------- | ------------------------- | ---------------------------- | | `accountTrieRoot` | [HashDigest](#hashdigest) | Merkle root of account trie. | -| `numValidators` | `uint32` | Number of active validators | +| `numValidators` | `uint32` | Number of active validators. | The state of the LazyLedger chain contains only account balances and the validator set (which is extra metadata on top of the plain account balances). @@ -487,17 +489,40 @@ One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both ac ## Account -| name | type | description | -| -------------------- | ------------------- | ---------------------------------------------------------------------------------------- | -| `balance` | `uint64` | Coin balance. | -| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | -| `isDelegating` | `bool` | Whether this account is delegating its stake or not. | -| `delegatedValidator` | [Address](#address) | _Optional._ The validator this account is delegating to. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. `0` is this account is not a validator. | -| `votingPower` | `uint64` | Voting balance. | +| name | type | description | +| ---------------- | ------------------------- | --------------------------------------------------------------------------------- | +| `balance` | `uint64` | Coin balance. | +| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | +| `isValidator` | `bool` | Whether this account is a validator or not. | +| `validatorInfo` | [Validator](#validator) | _Optional_, only if `isValidator` is set. Validator info. | +| `isDelegating` | `bool` | Whether this account is delegating its stake or not. | +| `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | In the account trie, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). +## Validator + +```C++ +enum ValidatorStatus : uint8_t { + Queued = 1, + Bonded = 2, + Unbonding = 3, + Unbonded = 4, +}; +``` + +| name | type | description | +| ---------------- | ----------------- | ------------------------------------------------ | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `votingPower` | `uint64` | Voting balance. | + +## Delegation + +| name | type | description | +| -------------------- | ------------------- | ------------------------------------- | +| `delegatedValidator` | [Address](#address) | The validator being is delegating to. | + # Consensus Parameters Various [consensus parameters](consensus.md#system-parameters) are committed to in the block header, such a limits and constants. From 9ce5d6c90b0ce5555683014f48e7da14157cddeb Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 26 May 2020 22:32:49 -0400 Subject: [PATCH 08/49] Add additional validator and delegation fields. --- specs/data_structures.md | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 818640a..b8731ea 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -46,6 +46,7 @@ Data Structures - [Account](#account) - [Validator](#validator) - [Delegation](#delegation) + - [Decimal](#decimal) - [Consensus Parameters](#consensus-parameters) # Data Structures Overview @@ -489,14 +490,14 @@ One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both ac ## Account -| name | type | description | -| ---------------- | ------------------------- | --------------------------------------------------------------------------------- | -| `balance` | `uint64` | Coin balance. | -| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | -| `isValidator` | `bool` | Whether this account is a validator or not. | -| `validatorInfo` | [Validator](#validator) | _Optional_, only if `isValidator` is set. Validator info. | -| `isDelegating` | `bool` | Whether this account is delegating its stake or not. | -| `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | +| name | type | description | +| ---------------- | ------------------------- | ------------------------------------------------------------------------------------------- | +| `balance` | `uint64` | Coin balance. | +| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | +| `isValidator` | `bool` | Whether this account is a validator or not. Mutually exclusive with `isDelegating`. | +| `validatorInfo` | [Validator](#validator) | _Optional_, only if `isValidator` is set. Validator info. | +| `isDelegating` | `bool` | Whether this account is delegating its stake or not. Mutually exclusive with `isValidator`. | +| `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | In the account trie, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). @@ -511,17 +512,24 @@ enum ValidatorStatus : uint8_t { }; ``` -| name | type | description | -| ---------------- | ----------------- | ------------------------------------------------ | -| `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `votingPower` | `uint64` | Voting balance. | +| name | type | description | +| ----------------- | ------------------- | ------------------------------------------------ | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `votingPower` | `uint64` | Voting balance. | +| `unbondingHeight` | `uint64` | Block height validator began unbonding. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | ## Delegation -| name | type | description | -| -------------------- | ------------------- | ------------------------------------- | -| `delegatedValidator` | [Address](#address) | The validator being is delegating to. | +| name | type | description | +| ----------- | ------------------- | ------------------------------------- | +| `validator` | [Address](#address) | The validator being is delegating to. | +| `start` | `uint64` | Start block. | + +## Decimal + +TODO define a format for numbers in the range `[0,1]` # Consensus Parameters From 06f6436d576515635620c6dbfd942c165ce65835 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 1 Jun 2020 17:17:52 -0400 Subject: [PATCH 09/49] Add explanation for validator status. --- specs/data_structures.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index b8731ea..ff5c62d 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -520,12 +520,26 @@ enum ValidatorStatus : uint8_t { | `unbondingHeight` | `uint64` | Block height validator began unbonding. | | `commissionRate` | [Decimal](#decimal) | Commission rate. | +Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: +1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. +1. `Bonded`: This validator is active and bonded. It can propose new blocks and vote on proposed blocks. Once bonded, an active validator must go through an unbonding process until its stake can be freed. +1. `Unbonding`: This validator is in the process of unbonding, which can be voluntary (the validator decided to stop being an active validator) to forced (the validator committed a slashable offence and was kicked from the active validator set). Validators will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. +1. `Unbonded`: This validator has completed its unbonding and has withdrawn its stake. The validator object will remain in this status until `delegatedCount` reaches zero, at which point it is destroyed. + ## Delegation -| name | type | description | -| ----------- | ------------------- | ------------------------------------- | -| `validator` | [Address](#address) | The validator being is delegating to. | -| `start` | `uint64` | Start block. | +```C++ +enum DelegationStatus : uint8_t { + Bonded = 1, + Unbonding = 2, +}; +``` + +| name | type | description | +| ------------- | ------------------- | ----------------------------------- | +| `status` | `DelegationStatus` | Status of this delegation. | +| `validator` | [Address](#address) | The validator being delegating to. | +| `startHeight` | `uint64` | Block height when delegation began. | ## Decimal From 05b2d47bea3e149658c2425e0b812a70530c377f Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 1 Jun 2020 17:38:01 -0400 Subject: [PATCH 10/49] Add explication for delegation status. --- specs/data_structures.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/specs/data_structures.md b/specs/data_structures.md index ff5c62d..0505d17 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -541,6 +541,10 @@ enum DelegationStatus : uint8_t { | `validator` | [Address](#address) | The validator being delegating to. | | `startHeight` | `uint64` | Block height when delegation began. | +Delegation objects represent a delegation. They have two statuses: +1. `Bonded`: This delegation is enabled for a `Queued` _or_ `Bonded` validator. Delegations to a `Queued` validator can be withdrawn immediately, while delegations for a `Bonded` validator must be unbonded first. +1. `Unbonding`: This delegation is unbonding. It will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. Once the unbonding duration has expired, the delegation can be withdrawn. + ## Decimal TODO define a format for numbers in the range `[0,1]` From 00d46754090c13a334f7efcdded6e98739019e47 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 1 Jun 2020 17:40:12 -0400 Subject: [PATCH 11/49] Add slashing fields to validator. --- specs/data_structures.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 0505d17..0225d0a 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -512,18 +512,20 @@ enum ValidatorStatus : uint8_t { }; ``` -| name | type | description | -| ----------------- | ------------------- | ------------------------------------------------ | -| `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `votingPower` | `uint64` | Voting balance. | -| `unbondingHeight` | `uint64` | Block height validator began unbonding. | -| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| name | type | description | +| ----------------- | ------------------- | -------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `votingPower` | `uint64` | Voting balance. | +| `unbondingHeight` | `uint64` | Block height validator began unbonding. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| `isSlashed` | `bool` | If this validator has been slashed or not. | +| `slashRate` | [Decimal](#decimal) | _Optional._ Only if `isSlashed` is set. Rate at which this validator has been slashed. | Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: 1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. 1. `Bonded`: This validator is active and bonded. It can propose new blocks and vote on proposed blocks. Once bonded, an active validator must go through an unbonding process until its stake can be freed. -1. `Unbonding`: This validator is in the process of unbonding, which can be voluntary (the validator decided to stop being an active validator) to forced (the validator committed a slashable offence and was kicked from the active validator set). Validators will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. +1. `Unbonding`: This validator is in the process of unbonding, which can be voluntary (the validator decided to stop being an active validator) or forced (the validator committed a slashable offence and was kicked from the active validator set). Validators will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. 1. `Unbonded`: This validator has completed its unbonding and has withdrawn its stake. The validator object will remain in this status until `delegatedCount` reaches zero, at which point it is destroyed. ## Delegation From 5a60665cd55b6a147b60a477b7635d77e53505c0 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 1 Jun 2020 17:40:52 -0400 Subject: [PATCH 12/49] Clean up. --- specs/data_structures.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 0225d0a..6a215f2 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -520,7 +520,7 @@ enum ValidatorStatus : uint8_t { | `unbondingHeight` | `uint64` | Block height validator began unbonding. | | `commissionRate` | [Decimal](#decimal) | Commission rate. | | `isSlashed` | `bool` | If this validator has been slashed or not. | -| `slashRate` | [Decimal](#decimal) | _Optional._ Only if `isSlashed` is set. Rate at which this validator has been slashed. | +| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: 1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. From 2df269d3c116aadd5ac11268889eb5c2dc7fbed3 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 10:47:16 -0400 Subject: [PATCH 13/49] Add accumulation of voting power and rewards to validators and delegations. --- specs/data_structures.md | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 6a215f2..135f89d 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -512,15 +512,18 @@ enum ValidatorStatus : uint8_t { }; ``` -| name | type | description | -| ----------------- | ------------------- | -------------------------------------------------------------------------------------- | -| `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `votingPower` | `uint64` | Voting balance. | -| `unbondingHeight` | `uint64` | Block height validator began unbonding. | -| `commissionRate` | [Decimal](#decimal) | Commission rate. | -| `isSlashed` | `bool` | If this validator has been slashed or not. | -| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | +| name | type | description | +| ------------------------------- | ------------------- | -------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake. | +| `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | +| `pendingRewards` | `uint64` | Rewards collected but not withdrawn. | +| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. TODO too few bits? | +| `unbondingHeight` | `uint64` | Block height validator began unbonding. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| `isSlashed` | `bool` | If this validator has been slashed or not. | +| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: 1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. @@ -537,11 +540,13 @@ enum DelegationStatus : uint8_t { }; ``` -| name | type | description | -| ------------- | ------------------- | ----------------------------------- | -| `status` | `DelegationStatus` | Status of this delegation. | -| `validator` | [Address](#address) | The validator being delegating to. | -| `startHeight` | `uint64` | Block height when delegation began. | +| name | type | description | +| --------------------- | ------------------- | -------------------------------------------------- | +| `status` | `DelegationStatus` | Status of this delegation. | +| `validator` | [Address](#address) | The validator being delegating to. | +| `votingPower` | `uint64` | Delegated stake. | +| `startHeight` | `uint64` | Block height when delegation began. | +| `startPendingRewards` | `uint64` | Validator's pending rewards when delegation began. | Delegation objects represent a delegation. They have two statuses: 1. `Bonded`: This delegation is enabled for a `Queued` _or_ `Bonded` validator. Delegations to a `Queued` validator can be withdrawn immediately, while delegations for a `Bonded` validator must be unbonded first. From 4cf479a20f9341a199aaee2d6b1d9903ca097123 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 10:58:24 -0400 Subject: [PATCH 14/49] Reduce the precision of voting power to whole coins (i.e. drop 9 zeroes in decimal notation). --- specs/data_structures.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 135f89d..181da40 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -516,7 +516,7 @@ enum ValidatorStatus : uint8_t { | ------------------------------- | ------------------- | -------------------------------------------------------------------------------------- | | `status` | `ValidatorStatus` | Status of this validator. | | `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake. | +| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | | `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | | `pendingRewards` | `uint64` | Rewards collected but not withdrawn. | | `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. TODO too few bits? | @@ -544,7 +544,7 @@ enum DelegationStatus : uint8_t { | --------------------- | ------------------- | -------------------------------------------------- | | `status` | `DelegationStatus` | Status of this delegation. | | `validator` | [Address](#address) | The validator being delegating to. | -| `votingPower` | `uint64` | Delegated stake. | +| `votingPower` | `uint64` | Delegated stake, in `4u`. | | `startHeight` | `uint64` | Block height when delegation began. | | `startPendingRewards` | `uint64` | Validator's pending rewards when delegation began. | From 411925c4f3ddcf1a2a38bc7b3cb50d2bfaf9d826 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 11:42:09 -0400 Subject: [PATCH 15/49] Remove todo. --- specs/data_structures.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 181da40..ffb572e 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -519,7 +519,7 @@ enum ValidatorStatus : uint8_t { | `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | | `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | | `pendingRewards` | `uint64` | Rewards collected but not withdrawn. | -| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. TODO too few bits? | +| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. | | `unbondingHeight` | `uint64` | Block height validator began unbonding. | | `commissionRate` | [Decimal](#decimal) | Commission rate. | | `isSlashed` | `bool` | If this validator has been slashed or not. | From 7f311128ff19459d04dd724735bbde15879d53dd Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 21:45:19 -0400 Subject: [PATCH 16/49] Add rules for calculating rewards and penalties for delegations and validators. --- specs/consensus.md | 103 +++++++++++++++++++++++++++++++++++++++ specs/data_structures.md | 41 ++++++++-------- 2 files changed, 125 insertions(+), 19 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index efa90fc..fd254ec 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -11,6 +11,9 @@ Consensus Rules - [Leader Selection](#leader-selection) - [Fork Choice](#fork-choice) - [Block Validity](#block-validity) + - [State Transitions](#state-transitions) + - [Validators and Delegations](#validators-and-delegations) + - [Calculating Rewards and Penalties](#calculating-rewards-and-penalties) - [Formatting](#formatting) - [Availability](#availability) @@ -68,6 +71,106 @@ Consensus Rules # Block Validity +## State Transitions + +### Validators and Delegations + +A transaction `tx` that requests a new validator initializes the [Validator](data_structures.md#validator) field of that account as follows: +| name | value | +| ------------------------------- | ------------------------ | +| `status` | `ValidatorStatus.Queued` | +| `delegatedCount` | `0` | +| `stakedBalance` | `tx.amount` | +| `votingPower` | `tx.amount` | +| `startHeight` | `0` | +| `heightOfLastVotingPowerChange` | `0` | +| `pendingRewards` | `0` | +| `accumulatedVotingPower` | `0` | +| `unbondingHeight` | `0` | +| `commissionRate` | `tx.commissionRate` | +| `isSlashed` | `false` | + +At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validators by voting power are or become active. For newly-bonded validators, their status is changed to bonded and height values initialized. +| name | value | +| ------------------------------- | ------------------------ | +| `status` | `ValidatorStatus.Bonded` | +| `startHeight` | `block.height + 1` | +| `heightOfLastVotingPowerChange` | `block.height + 1` | + +For validators that were bonded but are no longer (either by being outside the top validators or through a transaction that requests unbonding), they begin unbonding. +| name | value | +| ----------------- | --------------------------- | +| `status` | `ValidatorStatus.Unbonding` | +| `unbondingHeight` | `block.height + 1` | + +Once an unbonding validator has waited at least `UNBONDING_DURATION` blocks, they can be unbonded, collecting their reward: +| name | value | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus.Unbonded` | +| `stakedBalance` | `0` | +| `votingPower` | `old.votingPower - old.stakedBalance` | +| `pendingRewards` | `old.pendingRewards - calculatedReward` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | +| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | + +Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. +| name | value | +| ------------------------------- | --------------------------------------------------------------------------------------------------- | +| `heightOfLastVotingPowerChange` | `block.height` | +| `accumulatedVotingPower` | `old.accumulatedVotingPower + old.votingPower * (block.height - old.heightOfLastVotingPowerChange)` | + +A transactions `tx` that requests a new delegation first updates the target validator's voting power: +| name | value | +| ---------------- | ----------------------------- | +| `delegatedCount` | `old.delegatedCount + 1` | +| `votingPower` | `old.votingPower + tx.amount` | +then initializes the [Delegation](data_structures.md#delegation) field of that account as follows: +| name | value | +| ----------------- | ------------------------- | +| `status` | `DelegationStatus.Bonded` | +| `validator` | `tx.validator` | +| `votingPower` | `tx.amount` | +| `startHeight` | `block.height + 1` | +| `unbondingHeight` | `0` | +| `pendingRewards` | `0` | + +A transaction `tx` that requests withdrawing a delegation first updates the delegation field: +| name | value | +| ----------------- | ----------------------------------------------------------------------------------------------- | +| `status` | `DelegationStatus.Unbonding` | +| `unbondingHeight` | `block.height + 1` | +| `pendingRewards` | `calculatedReward` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | +then updates the target validator's voting power: +| name | value | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `delegatedCount` | `old.delegatedCount - 1` | +| `votingPower` | `old.votingPower - delegation.votingPower` | +| `pendingRewards` | `old.pendingRewards - calculatedReward` (Same as above.) | +| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | + + +### Calculating Rewards and Penalties + +Due to the requirement that all incorrect state transitions be provable with a compact fraud proof that is cheap enough to verify within a smart contract on a remote chain, computing rewards and penalties must involve minimal or no iterations. The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." + +F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. + +Rewards with penalties for validators: + +``` +calculatedAccumulatedVotingPower = (block.height - validator.startHeight) * validator.stakedBalance +calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower +if (validator.isSlashed) + calculatedReward *= validator.slashRate +``` + +Rewards with penalties for delegations: +``` +calculatedAccumulatedVotingPower = (block.height - delegation.startHeight) * delegation.votingPower +calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower +if (validator.isSlashed) + calculatedReward *= validator.slashRate +``` + ## Formatting Leaves in the message [Namespace Merkle Tree](data_structures.md#namespace-merkle-tree) must be ordered lexicographically by namespace ID. diff --git a/specs/data_structures.md b/specs/data_structures.md index ffb572e..57c3e15 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -512,18 +512,20 @@ enum ValidatorStatus : uint8_t { }; ``` -| name | type | description | -| ------------------------------- | ------------------- | -------------------------------------------------------------------------------------- | -| `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | -| `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | -| `pendingRewards` | `uint64` | Rewards collected but not withdrawn. | -| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. | -| `unbondingHeight` | `uint64` | Block height validator began unbonding. | -| `commissionRate` | [Decimal](#decimal) | Commission rate. | -| `isSlashed` | `bool` | If this validator has been slashed or not. | -| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | +| name | type | description | +| ------------------------------- | ------------------- | -------------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `stakedBalance` | `uint64` | Validator's personal staked balance, in `4u`. | +| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | +| `startHeight` | `uint64` | Block height when validator became bonded. `0` if not bonded or in the process of unbonding. | +| `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | +| `pendingRewards` | `uint64` | Rewards collected but not withdrawn, in `1u`. | +| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. | +| `unbondingHeight` | `uint64` | Block height validator began unbonding. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| `isSlashed` | `bool` | If this validator has been slashed or not. | +| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: 1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. @@ -540,13 +542,14 @@ enum DelegationStatus : uint8_t { }; ``` -| name | type | description | -| --------------------- | ------------------- | -------------------------------------------------- | -| `status` | `DelegationStatus` | Status of this delegation. | -| `validator` | [Address](#address) | The validator being delegating to. | -| `votingPower` | `uint64` | Delegated stake, in `4u`. | -| `startHeight` | `uint64` | Block height when delegation began. | -| `startPendingRewards` | `uint64` | Validator's pending rewards when delegation began. | +| name | type | description | +| ----------------- | ------------------- | ---------------------------------------------- | +| `status` | `DelegationStatus` | Status of this delegation. | +| `validator` | [Address](#address) | The validator being delegating to. | +| `votingPower` | `uint64` | Delegated stake, in `4u`. | +| `startHeight` | `uint64` | Block height when delegation began. | +| `unbondingHeight` | `uint64` | Block height delegation began unbonding. | +| `pendingRewards` | `uint64` | Pending rewards when delegation ends, in `1u`. | Delegation objects represent a delegation. They have two statuses: 1. `Bonded`: This delegation is enabled for a `Queued` _or_ `Bonded` validator. Delegations to a `Queued` validator can be withdrawn immediately, while delegations for a `Bonded` validator must be unbonded first. From 34d9ddcda93b3bcaa626f4a079b2615741d6e092 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 21:47:07 -0400 Subject: [PATCH 17/49] Clean up. --- specs/data_structures.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 57c3e15..e1db2ec 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -512,20 +512,20 @@ enum ValidatorStatus : uint8_t { }; ``` -| name | type | description | -| ------------------------------- | ------------------- | -------------------------------------------------------------------------------------------- | -| `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `stakedBalance` | `uint64` | Validator's personal staked balance, in `4u`. | -| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | -| `startHeight` | `uint64` | Block height when validator became bonded. `0` if not bonded or in the process of unbonding. | -| `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | -| `pendingRewards` | `uint64` | Rewards collected but not withdrawn, in `1u`. | -| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. | -| `unbondingHeight` | `uint64` | Block height validator began unbonding. | -| `commissionRate` | [Decimal](#decimal) | Commission rate. | -| `isSlashed` | `bool` | If this validator has been slashed or not. | -| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | +| name | type | description | +| ------------------------------- | ------------------- | -------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `stakedBalance` | `uint64` | Validator's personal staked balance, in `4u`. | +| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | +| `startHeight` | `uint64` | Block height when validator became bonded. `0` if not bonded. | +| `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | +| `pendingRewards` | `uint64` | Rewards collected but not withdrawn, in `1u`. | +| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. | +| `unbondingHeight` | `uint64` | Block height validator began unbonding. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| `isSlashed` | `bool` | If this validator has been slashed or not. | +| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: 1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. From 8339f8668eab63a5a366b41d12d5f4fee8648d81 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 21:49:18 -0400 Subject: [PATCH 18/49] Add rule to update accumulated voting power also when validator begins unbonding. --- specs/consensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index fd254ec..0f82af3 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -112,7 +112,7 @@ Once an unbonding validator has waited at least `UNBONDING_DURATION` blocks, the | `pendingRewards` | `old.pendingRewards - calculatedReward` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | | `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | -Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. +Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. | name | value | | ------------------------------- | --------------------------------------------------------------------------------------------------- | | `heightOfLastVotingPowerChange` | `block.height` | From 61cbe5554a06643bcf8226ae9be80c716d8593b7 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 21:51:00 -0400 Subject: [PATCH 19/49] Clarify that accumulated voting power is in whole coins. --- specs/data_structures.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index e1db2ec..f98d35d 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -521,7 +521,7 @@ enum ValidatorStatus : uint8_t { | `startHeight` | `uint64` | Block height when validator became bonded. `0` if not bonded. | | `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | | `pendingRewards` | `uint64` | Rewards collected but not withdrawn, in `1u`. | -| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks. | +| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks, in `4u`. | | `unbondingHeight` | `uint64` | Block height validator began unbonding. | | `commissionRate` | [Decimal](#decimal) | Commission rate. | | `isSlashed` | `bool` | If this validator has been slashed or not. | From f7bc92b87ef063f0e4decda298e5593d4893bcdd Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 22:04:48 -0400 Subject: [PATCH 20/49] Add commission calculations. --- specs/consensus.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index 0f82af3..66f6daa 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -155,10 +155,11 @@ Due to the requirement that all incorrect state transitions be provable with a c F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. Rewards with penalties for validators: - ``` calculatedAccumulatedVotingPower = (block.height - validator.startHeight) * validator.stakedBalance calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower +if (validator.status != ValidatorStatus.Unbonded) + calculatedReward -= calculatedReward * validator.commissionRate if (validator.isSlashed) calculatedReward *= validator.slashRate ``` @@ -167,6 +168,7 @@ Rewards with penalties for delegations: ``` calculatedAccumulatedVotingPower = (block.height - delegation.startHeight) * delegation.votingPower calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower +calculatedReward += validator.pendingRewards * validator.commissionRate if (validator.isSlashed) calculatedReward *= validator.slashRate ``` From 0828bf35aa93b52d84ea449ead757ea763ed5513 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 22:08:04 -0400 Subject: [PATCH 21/49] Fix tables. --- specs/consensus.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/specs/consensus.md b/specs/consensus.md index 66f6daa..1728353 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -123,6 +123,7 @@ A transactions `tx` that requests a new delegation first updates the target vali | ---------------- | ----------------------------- | | `delegatedCount` | `old.delegatedCount + 1` | | `votingPower` | `old.votingPower + tx.amount` | + then initializes the [Delegation](data_structures.md#delegation) field of that account as follows: | name | value | | ----------------- | ------------------------- | @@ -139,6 +140,7 @@ A transaction `tx` that requests withdrawing a delegation first updates the dele | `status` | `DelegationStatus.Unbonding` | | `unbondingHeight` | `block.height + 1` | | `pendingRewards` | `calculatedReward` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | + then updates the target validator's voting power: | name | value | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | From 3f320e0e63067fac8fc4c39a9fc25e9254df1e04 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 2 Jun 2020 22:09:22 -0400 Subject: [PATCH 22/49] Fix commissions. --- specs/consensus.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index 1728353..17c2386 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -160,8 +160,7 @@ Rewards with penalties for validators: ``` calculatedAccumulatedVotingPower = (block.height - validator.startHeight) * validator.stakedBalance calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower -if (validator.status != ValidatorStatus.Unbonded) - calculatedReward -= calculatedReward * validator.commissionRate +calculatedReward += validator.pendingRewards * validator.commissionRate if (validator.isSlashed) calculatedReward *= validator.slashRate ``` @@ -170,7 +169,8 @@ Rewards with penalties for delegations: ``` calculatedAccumulatedVotingPower = (block.height - delegation.startHeight) * delegation.votingPower calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower -calculatedReward += validator.pendingRewards * validator.commissionRate +if (validator.status != ValidatorStatus.Unbonded) + calculatedReward -= calculatedReward * validator.commissionRate if (validator.isSlashed) calculatedReward *= validator.slashRate ``` From d87ef538023f546765e708f7a46455b266776225 Mon Sep 17 00:00:00 2001 From: John Adler Date: Sat, 6 Jun 2020 14:02:06 -0400 Subject: [PATCH 23/49] Rename calculating rewards and penalties to distributing. --- specs/consensus.md | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index 17c2386..cda17ed 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -13,7 +13,7 @@ Consensus Rules - [Block Validity](#block-validity) - [State Transitions](#state-transitions) - [Validators and Delegations](#validators-and-delegations) - - [Calculating Rewards and Penalties](#calculating-rewards-and-penalties) + - [Distributing Rewards and Penalties](#distributing-rewards-and-penalties) - [Formatting](#formatting) - [Availability](#availability) @@ -104,13 +104,13 @@ For validators that were bonded but are no longer (either by being outside the t | `unbondingHeight` | `block.height + 1` | Once an unbonding validator has waited at least `UNBONDING_DURATION` blocks, they can be unbonded, collecting their reward: -| name | value | -| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | -| `status` | `ValidatorStatus.Unbonded` | -| `stakedBalance` | `0` | -| `votingPower` | `old.votingPower - old.stakedBalance` | -| `pendingRewards` | `old.pendingRewards - calculatedReward` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | -| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | +| name | value | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus.Unbonded` | +| `stakedBalance` | `0` | +| `votingPower` | `old.votingPower - old.stakedBalance` | +| `pendingRewards` | `old.pendingRewards - calculatedReward` (Calculated [here](#distributing-rewards-and-penalties).) | +| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated [here](#distributing-rewards-and-penalties).) | Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. | name | value | @@ -135,22 +135,22 @@ then initializes the [Delegation](data_structures.md#delegation) field of that a | `pendingRewards` | `0` | A transaction `tx` that requests withdrawing a delegation first updates the delegation field: -| name | value | -| ----------------- | ----------------------------------------------------------------------------------------------- | -| `status` | `DelegationStatus.Unbonding` | -| `unbondingHeight` | `block.height + 1` | -| `pendingRewards` | `calculatedReward` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | +| name | value | +| ----------------- | ---------------------------------------------------------------------------- | +| `status` | `DelegationStatus.Unbonding` | +| `unbondingHeight` | `block.height + 1` | +| `pendingRewards` | `calculatedReward` (Calculated [here](#distributing-rewards-and-penalties).) | then updates the target validator's voting power: -| name | value | -| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | -| `delegatedCount` | `old.delegatedCount - 1` | -| `votingPower` | `old.votingPower - delegation.votingPower` | -| `pendingRewards` | `old.pendingRewards - calculatedReward` (Same as above.) | -| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated in [rewards and penalties](#calculating-rewards-and-penalties).) | +| name | value | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------- | +| `delegatedCount` | `old.delegatedCount - 1` | +| `votingPower` | `old.votingPower - delegation.votingPower` | +| `pendingRewards` | `old.pendingRewards - calculatedReward` (Same as above.) | +| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated [here](#distributing-rewards-and-penalties).) | -### Calculating Rewards and Penalties +### Distributing Rewards and Penalties Due to the requirement that all incorrect state transitions be provable with a compact fraud proof that is cheap enough to verify within a smart contract on a remote chain, computing rewards and penalties must involve minimal or no iterations. The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." From 08dfbc536f10aaffa94e672b582b8231bc7251b7 Mon Sep 17 00:00:00 2001 From: John Adler Date: Sat, 6 Jun 2020 14:08:42 -0400 Subject: [PATCH 24/49] Clean up. --- specs/consensus.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index cda17ed..686d16c 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -149,7 +149,6 @@ then updates the target validator's voting power: | `pendingRewards` | `old.pendingRewards - calculatedReward` (Same as above.) | | `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated [here](#distributing-rewards-and-penalties).) | - ### Distributing Rewards and Penalties Due to the requirement that all incorrect state transitions be provable with a compact fraud proof that is cheap enough to verify within a smart contract on a remote chain, computing rewards and penalties must involve minimal or no iterations. The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." From 313388e8acb204d301363947b5d993037d714080 Mon Sep 17 00:00:00 2001 From: John Adler Date: Sat, 6 Jun 2020 17:02:13 -0400 Subject: [PATCH 25/49] Migrate rationale for reward distribution to dedicated document. --- rationale/distributing_rewards.md | 12 ++++++++++++ specs/consensus.md | 4 +--- 2 files changed, 13 insertions(+), 3 deletions(-) create mode 100644 rationale/distributing_rewards.md diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md new file mode 100644 index 0000000..0217081 --- /dev/null +++ b/rationale/distributing_rewards.md @@ -0,0 +1,12 @@ +Rationale: Distributing Rewards and Penalties +=== + +- [Rationale: Distributing Rewards and Penalties](#rationale-distributing-rewards-and-penalties) +- [Distributing Rewards and Penalties](#distributing-rewards-and-penalties) + +# Distributing Rewards and Penalties + +Due to the requirement that all incorrect state transitions be provable with a compact fraud proof that is cheap enough to verify within a smart contract on a remote chain, computing rewards and penalties must involve minimal or no iterations. The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." + +F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. + diff --git a/specs/consensus.md b/specs/consensus.md index 686d16c..1b68d36 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -151,9 +151,7 @@ then updates the target validator's voting power: ### Distributing Rewards and Penalties -Due to the requirement that all incorrect state transitions be provable with a compact fraud proof that is cheap enough to verify within a smart contract on a remote chain, computing rewards and penalties must involve minimal or no iterations. The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." - -F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. +For rationale behind the distribution scheme for rewards and penalties, see [rationale document](../rationale/distributing_rewards.md). Rewards with penalties for validators: ``` From ecda995a12028ebaa0bd9daae7501163e23fc469 Mon Sep 17 00:00:00 2001 From: John Adler Date: Sat, 6 Jun 2020 17:03:23 -0400 Subject: [PATCH 26/49] Fix commission calculation for validator. --- specs/consensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index 1b68d36..ec50430 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -157,7 +157,7 @@ Rewards with penalties for validators: ``` calculatedAccumulatedVotingPower = (block.height - validator.startHeight) * validator.stakedBalance calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower -calculatedReward += validator.pendingRewards * validator.commissionRate +calculatedReward += (validator.pendingRewards - calculatedReward) * validator.commissionRate if (validator.isSlashed) calculatedReward *= validator.slashRate ``` From 4fd51b7028e8188aa8f7fcf357899bb506d08ed8 Mon Sep 17 00:00:00 2001 From: John Adler Date: Sun, 7 Jun 2020 23:08:22 -0400 Subject: [PATCH 27/49] Remove some rationale from consensus document for reward distribution. --- specs/consensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index ec50430..7b41c8b 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -112,7 +112,7 @@ Once an unbonding validator has waited at least `UNBONDING_DURATION` blocks, the | `pendingRewards` | `old.pendingRewards - calculatedReward` (Calculated [here](#distributing-rewards-and-penalties).) | | `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated [here](#distributing-rewards-and-penalties).) | -Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. +Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power to date is calculated. | name | value | | ------------------------------- | --------------------------------------------------------------------------------------------------- | | `heightOfLastVotingPowerChange` | `block.height` | From fa9307b24739004592b9ed2b29363e5dce654ca4 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 8 Jun 2020 14:06:09 -0400 Subject: [PATCH 28/49] Add preamble to reward distribution doc, clearn up. --- rationale/distributing_rewards.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md index 0217081..5b749f5 100644 --- a/rationale/distributing_rewards.md +++ b/rationale/distributing_rewards.md @@ -2,11 +2,28 @@ Rationale: Distributing Rewards and Penalties === - [Rationale: Distributing Rewards and Penalties](#rationale-distributing-rewards-and-penalties) -- [Distributing Rewards and Penalties](#distributing-rewards-and-penalties) +- [Preamble](#preamble) +- [Background](#background) +- [Distribution Scheme](#distribution-scheme) -# Distributing Rewards and Penalties +# Preamble -Due to the requirement that all incorrect state transitions be provable with a compact fraud proof that is cheap enough to verify within a smart contract on a remote chain, computing rewards and penalties must involve minimal or no iterations. The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." +Due to the requirement that all incorrect state transitions on LazyLedger be provable with a [compact fraud proof](https://arxiv.org/abs/1809.09044) that is cheap enough to verify within a smart contract on a remote chain (e.g. Ethereum), computing how rewards and penalties are distributed must involve minimal or (ideally) no iterations. To understand why, let us consider the following desiderata in a staking system: +1. In-protocol stake delegation: this makes it easier for users to participate in the consensus process, and reduces reliance on custodial staking services. +1. In-protocol enforcement of proper distribution of rewards and penalities to delegators: rewards and penalties collected by validators should be distributed to delegators trustlessly. -F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. +Naively, rewards and penalties (henceforth referred to collectively as "rewards", since penalties are simply negative rewards) can be distributed immediately. For example, when a validator produces a new block and is entitled to collecting transaction fees, these fees can be distributed to every single account delegating stake to this validator. This requires iterating over potentially a huge number of state elements for a single state transition (i.e. transaction), which is computationally expensive. The specific problem is that it would be infeasible to prove that such a state transition was _incorrect_ (i.e. with a fraud proof) within the execution system of a remote blockchain (i.e. with a smart contract). +This forms the primary motivation of the reward distribution scheme presented in this document: a mechanism for distributing rewards that is "good enough" while requiring no iteration over state elements for any state transition. + +# Background + +F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. + +# Distribution Scheme + + The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." + + The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. + +Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. From e1d624c7c7ea52c6ba3e0274f9078d6a8a39b73d Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 8 Jun 2020 14:11:51 -0400 Subject: [PATCH 29/49] Clean up. --- rationale/distributing_rewards.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md index 5b749f5..a4fe8bd 100644 --- a/rationale/distributing_rewards.md +++ b/rationale/distributing_rewards.md @@ -18,12 +18,12 @@ This forms the primary motivation of the reward distribution scheme presented in # Background +The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." + F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. # Distribution Scheme - The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." - - The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. +The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. From 39bc623036d01e423323a0d7f6133923078dc5ea Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 01:28:21 -0400 Subject: [PATCH 30/49] Revamp reward distribution rationale. --- rationale/distributing_rewards.md | 45 ++++++++++++++++++++++++------- 1 file changed, 36 insertions(+), 9 deletions(-) diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md index a4fe8bd..0cb0396 100644 --- a/rationale/distributing_rewards.md +++ b/rationale/distributing_rewards.md @@ -3,27 +3,54 @@ Rationale: Distributing Rewards and Penalties - [Rationale: Distributing Rewards and Penalties](#rationale-distributing-rewards-and-penalties) - [Preamble](#preamble) -- [Background](#background) - [Distribution Scheme](#distribution-scheme) + - [State-Efficient Implementation](#state-efficient-implementation) # Preamble -Due to the requirement that all incorrect state transitions on LazyLedger be provable with a [compact fraud proof](https://arxiv.org/abs/1809.09044) that is cheap enough to verify within a smart contract on a remote chain (e.g. Ethereum), computing how rewards and penalties are distributed must involve minimal or (ideally) no iterations. To understand why, let us consider the following desiderata in a staking system: +Due to the requirement that all incorrect state transitions on LazyLedger be provable with a [compact fraud proof](https://arxiv.org/abs/1809.09044) that is cheap enough to verify within a smart contract on a remote chain (e.g. Ethereum), computing how rewards and penalties are distributed must involve no iterations. To understand why, let us consider the following desiderata in a staking system: 1. In-protocol stake delegation: this makes it easier for users to participate in the consensus process, and reduces reliance on custodial staking services. 1. In-protocol enforcement of proper distribution of rewards and penalities to delegators: rewards and penalties collected by validators should be distributed to delegators trustlessly. Naively, rewards and penalties (henceforth referred to collectively as "rewards", since penalties are simply negative rewards) can be distributed immediately. For example, when a validator produces a new block and is entitled to collecting transaction fees, these fees can be distributed to every single account delegating stake to this validator. This requires iterating over potentially a huge number of state elements for a single state transition (i.e. transaction), which is computationally expensive. The specific problem is that it would be infeasible to prove that such a state transition was _incorrect_ (i.e. with a fraud proof) within the execution system of a remote blockchain (i.e. with a smart contract). -This forms the primary motivation of the reward distribution scheme presented in this document: a mechanism for distributing rewards that is "good enough" while requiring no iteration over state elements for any state transition. +This forms the primary motivation of the scheme discussed here: a mechanism for distributing rewards that is state-efficient and requires no iteration over state elements for any state transition. -# Background +# Distribution Scheme -The scheme presented here is inspired by Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf) and the concept of "[coin days](https://bitcointalk.org/index.php?topic=6172.msg90789#msg90789)." +store first entry in delegation on creation? -F1 requires iterating over (potentially) a large number of blocks, but avoids needing to iterate over every delegation, all while being approximation-free. As such, it cannot be used for LazyLedger directly. +The scheme presented here is an incarnation of Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf). F1 has the nice property of being approximation-free and, with proper implementation details, can be highly efficient with state usage and completely iteration-free in all cases. -# Distribution Scheme +Naively, when considering a single block, the reward that should be given to a delegator with stake $x$, who's delegating to a validator with total voting power $n$, whose reward in that block is $T$, is: + +$$ +\text{naive reward} = x \frac{T}{n} +$$ + +In other words, the voting power contributed by the delegator multiplied by the _reward rate_, i.e. the rewards per unit of voting power. We note that if the total voting power of a validator remains constant forever, then the above equations hold and is approximation-free. However, changes to the total voting power need to be accounted for. + +Blocks between two changes to a validator's voting power (i.e. whenever a user delegates or undelegates stake) are a _period_. Every time a validator's voting power changes (i.e. a new period $f$ begins), an entry $Entry_f$ for this period is saved in state, which records _the reward rate up to the beginning of_ $f$. This is simply the sum of the reward rate up to the beginning of previous period $f-1$ and the reward rate of the period $f$ itself: + +$$ +Entry_f = \begin{cases} + 0 & f = 0 \\ + Entry_{f-1} + \frac{T_f}{n_f} & f > 0 \\ +\end{cases} +$$ + +Finally, the raw reward for a delegation is simply proportional to the difference in entries between the period where undelegation ended ($f$) and where it began ($k$). + +$$ +\text{reward} = x (Entry_f - Entry_k) +$$ + +This raw reward can be scaled by additional factors, such as commissions or slashing penalties. + +## State-Efficient Implementation -The scheme presented here requires no iterations at all, but is not approximation-free. The intuition is that rewards of a delegation (or validator) are simply proportional to the accumulated voting power contributed by that delegation (or validator's stake) and the remaining accumulated voting power. This has the effect of "smoothing out" rewards, hence the lack of approximation-freeness. +The F1 paper doesn't specify where entries are stored in state, but the understanding is that they are placed in independent state elements. This has the downside of requiring multiple Merkle branches to prove the inclusion of entries for e.g. fraud proofs. We can improve on this by leveraging a specific property of entries, namely that each entry is only used in exactly two cases: +1. To compute the next entry. +1. To compute the reward of a delegator. -Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. Intuitively, this "accumulated voting power" is similar to "coin days," but measures voting power over a number of blocks instead of coins over a number of days. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power is increased. +Intuitively, after having being used twice, an entry can be pruned from the state. We can make use of this by storing only the latest entry with its respective validator object, and a copy of the two entries each delegation needs with the delegation object. By storing entries directly with the objects that require them, state transitions can be statelessly validated without extra inclusion proofs. From 6bba8c6f8bffa4110a98e21a715f5db582993431 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 01:58:01 -0400 Subject: [PATCH 31/49] Fix state data structures for reward distribution. --- specs/data_structures.md | 53 +++++++++++++++++++++++----------------- 1 file changed, 31 insertions(+), 22 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index f98d35d..debca3a 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -44,6 +44,7 @@ Data Structures - [Message](#message) - [State](#state) - [Account](#account) + - [PeriodEntry](#periodentry) - [Validator](#validator) - [Delegation](#delegation) - [Decimal](#decimal) @@ -501,6 +502,14 @@ One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both ac In the account trie, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). +## PeriodEntry + +| name | type | description | +| ------------ | -------- | ------------------------------------------------------------- | +| `rewardRate` | `uint64` | Rewards per unit of voting power accumulated so far, in `1u`. | + +For explanation on entries, see the [reward distribution rationale document](../rationale/distributing_rewards.md). + ## Validator ```C++ @@ -512,20 +521,20 @@ enum ValidatorStatus : uint8_t { }; ``` -| name | type | description | -| ------------------------------- | ------------------- | -------------------------------------------------------------------------------------- | -| `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | -| `stakedBalance` | `uint64` | Validator's personal staked balance, in `4u`. | -| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | -| `startHeight` | `uint64` | Block height when validator became bonded. `0` if not bonded. | -| `heightOfLastVotingPowerChange` | `uint64` | Block height of the last time this validator's voting power changed. | -| `pendingRewards` | `uint64` | Rewards collected but not withdrawn, in `1u`. | -| `accumulatedVotingPower` | `uint64` | Accumulated voting power over blocks, in `4u`. | -| `unbondingHeight` | `uint64` | Block height validator began unbonding. | -| `commissionRate` | [Decimal](#decimal) | Commission rate. | -| `isSlashed` | `bool` | If this validator has been slashed or not. | -| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | +| name | type | description | +| ----------------- | --------------------------- | -------------------------------------------------------------------------------------- | +| `status` | `ValidatorStatus` | Status of this validator. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | +| `stakedBalance` | `uint64` | Validator's personal staked balance, in `4u`. | +| `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | +| `latestEntry` | [PeriodEntry](#periodentry) | Latest entry, used for calculating reward distribution. | +| `pendingRewards` | `uint64` | Rewards collected so far this period, in `1u`. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| `beginEntry` | [PeriodEntry](#periodentry) | Entry when validator bonding began. | +| `endEntry` | [PeriodEntry](#periodentry) | Entry when validator bonding ended (i.e. began unbonding). | +| `unbondingHeight` | `uint64` | Block height validator began unbonding. | +| `isSlashed` | `bool` | If this validator has been slashed or not. | +| `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | Validator objects represent all the information needed to be keep track of a validator. Validators have four statuses: 1. `Queued`: This validator has entered the queue to become an active validator. Once the next validator set transition occurs, if this validator has sufficient voting power (including its own stake and stake delegated to it) to be in the top `MAX_VALIDATORS` validators by voting power, it will become an active, i.e. `Bonded` validator. Until bonded, this validator can immediately exit the queue. @@ -542,14 +551,14 @@ enum DelegationStatus : uint8_t { }; ``` -| name | type | description | -| ----------------- | ------------------- | ---------------------------------------------- | -| `status` | `DelegationStatus` | Status of this delegation. | -| `validator` | [Address](#address) | The validator being delegating to. | -| `votingPower` | `uint64` | Delegated stake, in `4u`. | -| `startHeight` | `uint64` | Block height when delegation began. | -| `unbondingHeight` | `uint64` | Block height delegation began unbonding. | -| `pendingRewards` | `uint64` | Pending rewards when delegation ends, in `1u`. | +| name | type | description | +| ----------------- | --------------------------- | --------------------------------------------------- | +| `status` | `DelegationStatus` | Status of this delegation. | +| `validator` | [Address](#address) | The validator being delegating to. | +| `stakedBalance` | `uint64` | Delegated stake, in `4u`. | +| `beginEntry` | [PeriodEntry](#periodentry) | Entry when delegation began. | +| `endEntry` | [PeriodEntry](#periodentry) | Entry when delegation ended (i.e. began unbonding). | +| `unbondingHeight` | `uint64` | Block height delegation began unbonding. | Delegation objects represent a delegation. They have two statuses: 1. `Bonded`: This delegation is enabled for a `Queued` _or_ `Bonded` validator. Delegations to a `Queued` validator can be withdrawn immediately, while delegations for a `Bonded` validator must be unbonded first. From 6e036ad2a9971e44513c509724a807b9ac8b4fba Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 01:59:29 -0400 Subject: [PATCH 32/49] Remove redundant entry from validator. --- specs/data_structures.md | 1 - 1 file changed, 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index debca3a..c053ae7 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -530,7 +530,6 @@ enum ValidatorStatus : uint8_t { | `latestEntry` | [PeriodEntry](#periodentry) | Latest entry, used for calculating reward distribution. | | `pendingRewards` | `uint64` | Rewards collected so far this period, in `1u`. | | `commissionRate` | [Decimal](#decimal) | Commission rate. | -| `beginEntry` | [PeriodEntry](#periodentry) | Entry when validator bonding began. | | `endEntry` | [PeriodEntry](#periodentry) | Entry when validator bonding ended (i.e. began unbonding). | | `unbondingHeight` | `uint64` | Block height validator began unbonding. | | `isSlashed` | `bool` | If this validator has been slashed or not. | From 747b594f939ad893d0bcde08bba9acd4186eddc5 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 02:02:24 -0400 Subject: [PATCH 33/49] Remove other redundant entry from validator. --- specs/data_structures.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index c053ae7..d7da20e 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -524,13 +524,12 @@ enum ValidatorStatus : uint8_t { | name | type | description | | ----------------- | --------------------------- | -------------------------------------------------------------------------------------- | | `status` | `ValidatorStatus` | Status of this validator. | -| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | | `stakedBalance` | `uint64` | Validator's personal staked balance, in `4u`. | +| `commissionRate` | [Decimal](#decimal) | Commission rate. | +| `delegatedCount` | `uint32` | Number of accounts delegating to this validator. | | `votingPower` | `uint64` | Total voting power as staked balance + delegated stake, in `4u`. | -| `latestEntry` | [PeriodEntry](#periodentry) | Latest entry, used for calculating reward distribution. | | `pendingRewards` | `uint64` | Rewards collected so far this period, in `1u`. | -| `commissionRate` | [Decimal](#decimal) | Commission rate. | -| `endEntry` | [PeriodEntry](#periodentry) | Entry when validator bonding ended (i.e. began unbonding). | +| `latestEntry` | [PeriodEntry](#periodentry) | Latest entry, used for calculating reward distribution. | | `unbondingHeight` | `uint64` | Block height validator began unbonding. | | `isSlashed` | `bool` | If this validator has been slashed or not. | | `slashRate` | [Decimal](#decimal) | _Optional_, only if `isSlashed` is set. Rate at which this validator has been slashed. | From 62f759c7af45b69e75ddee15693e34339d62fbd8 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 16:22:59 -0400 Subject: [PATCH 34/49] Fix consensus rules for distributing rewards. --- specs/consensus.md | 110 ++++++++++++++++----------------------------- 1 file changed, 39 insertions(+), 71 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index 7b41c8b..4049c54 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -13,7 +13,6 @@ Consensus Rules - [Block Validity](#block-validity) - [State Transitions](#state-transitions) - [Validators and Delegations](#validators-and-delegations) - - [Distributing Rewards and Penalties](#distributing-rewards-and-penalties) - [Formatting](#formatting) - [Availability](#availability) @@ -76,26 +75,22 @@ Consensus Rules ### Validators and Delegations A transaction `tx` that requests a new validator initializes the [Validator](data_structures.md#validator) field of that account as follows: -| name | value | -| ------------------------------- | ------------------------ | -| `status` | `ValidatorStatus.Queued` | -| `delegatedCount` | `0` | -| `stakedBalance` | `tx.amount` | -| `votingPower` | `tx.amount` | -| `startHeight` | `0` | -| `heightOfLastVotingPowerChange` | `0` | -| `pendingRewards` | `0` | -| `accumulatedVotingPower` | `0` | -| `unbondingHeight` | `0` | -| `commissionRate` | `tx.commissionRate` | -| `isSlashed` | `false` | +| name | value | +| ----------------- | ------------------------ | +| `status` | `ValidatorStatus.Queued` | +| `stakedBalance` | `tx.amount` | +| `commissionRate` | `tx.commissionRate` | +| `delegatedCount` | `0` | +| `votingPower` | `tx.amount` | +| `pendingRewards` | `0` | +| `latestEntry` | `PeriodEntry(0)` | +| `unbondingHeight` | `0` | +| `isSlashed` | `false` | At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validators by voting power are or become active. For newly-bonded validators, their status is changed to bonded and height values initialized. -| name | value | -| ------------------------------- | ------------------------ | -| `status` | `ValidatorStatus.Bonded` | -| `startHeight` | `block.height + 1` | -| `heightOfLastVotingPowerChange` | `block.height + 1` | +| name | value | +| -------- | ------------------------ | +| `status` | `ValidatorStatus.Bonded` | For validators that were bonded but are no longer (either by being outside the top validators or through a transaction that requests unbonding), they begin unbonding. | name | value | @@ -104,21 +99,19 @@ For validators that were bonded but are no longer (either by being outside the t | `unbondingHeight` | `block.height + 1` | Once an unbonding validator has waited at least `UNBONDING_DURATION` blocks, they can be unbonded, collecting their reward: -| name | value | -| ------------------------ | ------------------------------------------------------------------------------------------------------------------------- | -| `status` | `ValidatorStatus.Unbonded` | -| `stakedBalance` | `0` | -| `votingPower` | `old.votingPower - old.stakedBalance` | -| `pendingRewards` | `old.pendingRewards - calculatedReward` (Calculated [here](#distributing-rewards-and-penalties).) | -| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated [here](#distributing-rewards-and-penalties).) | - -Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rate at which accumulated voting power grows also changes. The height of the last time the voting power of this validator was changed is updated to the current block height and the accumulated voting power to date is calculated. -| name | value | -| ------------------------------- | --------------------------------------------------------------------------------------------------- | -| `heightOfLastVotingPowerChange` | `block.height` | -| `accumulatedVotingPower` | `old.accumulatedVotingPower + old.votingPower * (block.height - old.heightOfLastVotingPowerChange)` | - -A transactions `tx` that requests a new delegation first updates the target validator's voting power: +| name | value | +| --------------- | ------------------------------------- | +| `status` | `ValidatorStatus.Unbonded` | +| `stakedBalance` | `0` | +| `votingPower` | `old.votingPower - old.stakedBalance` | + +Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rewards per unit of voting power accumulated so far are calculated: +| name | value | +| ---------------- | -------------------------------------------------------- | +| `pendingRewards` | `0` | +| `latestEntry` | `old.latestEntry + old.pendingRewards / old.votingPower` | + +A transaction `tx` that requests a new delegation first updates the target validator's voting power: | name | value | | ---------------- | ----------------------------- | | `delegatedCount` | `old.delegatedCount + 1` | @@ -129,48 +122,23 @@ then initializes the [Delegation](data_structures.md#delegation) field of that a | ----------------- | ------------------------- | | `status` | `DelegationStatus.Bonded` | | `validator` | `tx.validator` | -| `votingPower` | `tx.amount` | -| `startHeight` | `block.height + 1` | +| `stakedBalance` | `tx.amount` | +| `beginEntry` | `validator.latestEntry` | +| `endEntry` | `PeriodEntry(0)` | | `unbondingHeight` | `0` | -| `pendingRewards` | `0` | A transaction `tx` that requests withdrawing a delegation first updates the delegation field: -| name | value | -| ----------------- | ---------------------------------------------------------------------------- | -| `status` | `DelegationStatus.Unbonding` | -| `unbondingHeight` | `block.height + 1` | -| `pendingRewards` | `calculatedReward` (Calculated [here](#distributing-rewards-and-penalties).) | +| name | value | +| ----------------- | ---------------------------- | +| `status` | `DelegationStatus.Unbonding` | +| `endEntry` | `validator.latestEntry` | +| `unbondingHeight` | `block.height + 1` | then updates the target validator's voting power: -| name | value | -| ------------------------ | ------------------------------------------------------------------------------------------------------------------------- | -| `delegatedCount` | `old.delegatedCount - 1` | -| `votingPower` | `old.votingPower - delegation.votingPower` | -| `pendingRewards` | `old.pendingRewards - calculatedReward` (Same as above.) | -| `accumulatedVotingPower` | `old.accumulatedVotingPower - calculatedAccumulatedVotingPower` (Calculated [here](#distributing-rewards-and-penalties).) | - -### Distributing Rewards and Penalties - -For rationale behind the distribution scheme for rewards and penalties, see [rationale document](../rationale/distributing_rewards.md). - -Rewards with penalties for validators: -``` -calculatedAccumulatedVotingPower = (block.height - validator.startHeight) * validator.stakedBalance -calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower -calculatedReward += (validator.pendingRewards - calculatedReward) * validator.commissionRate -if (validator.isSlashed) - calculatedReward *= validator.slashRate -``` - -Rewards with penalties for delegations: -``` -calculatedAccumulatedVotingPower = (block.height - delegation.startHeight) * delegation.votingPower -calculatedReward = validator.pendingRewards * calculatedAccumulatedVotingPower / validator.accumulatedVotingPower -if (validator.status != ValidatorStatus.Unbonded) - calculatedReward -= calculatedReward * validator.commissionRate -if (validator.isSlashed) - calculatedReward *= validator.slashRate -``` +| name | value | +| ---------------- | ------------------------------------------ | +| `delegatedCount` | `old.delegatedCount - 1` | +| `votingPower` | `old.votingPower - delegation.votingPower` | ## Formatting From 09f95d249a83c6f722615f3688d4e77cec103eb8 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 16:33:11 -0400 Subject: [PATCH 35/49] Cleanup. --- rationale/distributing_rewards.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md index 0cb0396..1f98bb4 100644 --- a/rationale/distributing_rewards.md +++ b/rationale/distributing_rewards.md @@ -18,8 +18,6 @@ This forms the primary motivation of the scheme discussed here: a mechanism for # Distribution Scheme -store first entry in delegation on creation? - The scheme presented here is an incarnation of Cosmos' [F1 fee distribution scheme](https://github.com/cosmos/cosmos-sdk/blob/master/docs/spec/_proposals/f1-fee-distribution/f1_fee_distr.pdf). F1 has the nice property of being approximation-free and, with proper implementation details, can be highly efficient with state usage and completely iteration-free in all cases. Naively, when considering a single block, the reward that should be given to a delegator with stake $x$, who's delegating to a validator with total voting power $n$, whose reward in that block is $T$, is: From 5449c59fb8e7bc9b094565e428dd2e375c5ea692 Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 19:53:39 -0400 Subject: [PATCH 36/49] Update specs/consensus.md Co-authored-by: Ismail Khoffi --- specs/consensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index 4049c54..b76ed77 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -74,7 +74,7 @@ Consensus Rules ### Validators and Delegations -A transaction `tx` that requests a new validator initializes the [Validator](data_structures.md#validator) field of that account as follows: +A transaction `tx` that requests a new validator initializes the [Validator](data_structures.md#validator) fields of that account as follows: | name | value | | ----------------- | ------------------------ | | `status` | `ValidatorStatus.Queued` | From e8cc2d34f7e62237124699468545e8b5b79512ea Mon Sep 17 00:00:00 2001 From: John Adler Date: Tue, 9 Jun 2020 20:04:59 -0400 Subject: [PATCH 37/49] Simplify naming of accounts tree. --- specs/data_structures.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index d7da20e..455584b 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -480,14 +480,14 @@ enum VoteType : uint8_t { # State -| name | type | description | -| ----------------- | ------------------------- | ---------------------------- | -| `accountTrieRoot` | [HashDigest](#hashdigest) | Merkle root of account trie. | -| `numValidators` | `uint32` | Number of active validators. | +| name | type | description | +| --------------- | ------------------------- | ---------------------------- | +| `accountsRoot` | [HashDigest](#hashdigest) | Merkle root of account tree. | +| `numValidators` | `uint32` | Number of active validators. | The state of the LazyLedger chain contains only account balances and the validator set (which is extra metadata on top of the plain account balances). -One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both account account balances and validator metadata, the _account trie_. The state root is computed as the [hash](#hashdigest) of the serialized account trie root and number of active validators. The latter is necessary to ensure light nodes can determine the entire validator set from a single state root commitment. +One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both account account balances and validator metadata, the _accounts tree_. The final state root is computed as the [hash](#hashdigest) of the accounts tree root and number of active validators. The latter is necessary to ensure light nodes can determine the entire validator set from a single state root commitment. ## Account @@ -500,7 +500,7 @@ One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both ac | `isDelegating` | `bool` | Whether this account is delegating its stake or not. Mutually exclusive with `isValidator`. | | `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | -In the account trie, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). +In the accounts tree, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). ## PeriodEntry From e4005c8c1bf1c17341aa8ea2e592ba08f3c38dcc Mon Sep 17 00:00:00 2001 From: John Adler Date: Wed, 10 Jun 2020 10:06:41 -0400 Subject: [PATCH 38/49] Clean up. --- rationale/distributing_rewards.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md index 1f98bb4..b221173 100644 --- a/rationale/distributing_rewards.md +++ b/rationale/distributing_rewards.md @@ -26,7 +26,7 @@ $$ \text{naive reward} = x \frac{T}{n} $$ -In other words, the voting power contributed by the delegator multiplied by the _reward rate_, i.e. the rewards per unit of voting power. We note that if the total voting power of a validator remains constant forever, then the above equations hold and is approximation-free. However, changes to the total voting power need to be accounted for. +In other words, the voting power contributed by the delegator multiplied by the _reward rate_, i.e. the rewards per unit of voting power. We note that if the total voting power of a validator remains constant forever, then the above equation holds and is approximation-free. However, changes to the total voting power need to be accounted for. Blocks between two changes to a validator's voting power (i.e. whenever a user delegates or undelegates stake) are a _period_. Every time a validator's voting power changes (i.e. a new period $f$ begins), an entry $Entry_f$ for this period is saved in state, which records _the reward rate up to the beginning of_ $f$. This is simply the sum of the reward rate up to the beginning of previous period $f-1$ and the reward rate of the period $f$ itself: From 8785e68fb6a23c226973cbe7b5b6a1002d684f33 Mon Sep 17 00:00:00 2001 From: John Adler Date: Wed, 10 Jun 2020 11:35:43 -0400 Subject: [PATCH 39/49] Clean up informal language. --- rationale/distributing_rewards.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rationale/distributing_rewards.md b/rationale/distributing_rewards.md index b221173..e035f1d 100644 --- a/rationale/distributing_rewards.md +++ b/rationale/distributing_rewards.md @@ -47,7 +47,7 @@ This raw reward can be scaled by additional factors, such as commissions or slas ## State-Efficient Implementation -The F1 paper doesn't specify where entries are stored in state, but the understanding is that they are placed in independent state elements. This has the downside of requiring multiple Merkle branches to prove the inclusion of entries for e.g. fraud proofs. We can improve on this by leveraging a specific property of entries, namely that each entry is only used in exactly two cases: +The F1 paper does not specify where entries are stored in state, but the understanding is that they are placed in independent state elements. This has the downside of requiring multiple Merkle branches to prove the inclusion of entries for e.g. fraud proofs. We can improve on this by leveraging a specific property of entries, namely that each entry is only used in exactly two cases: 1. To compute the next entry. 1. To compute the reward of a delegator. From 70ac923a669f84bc7d8e8e0280845d335b262312 Mon Sep 17 00:00:00 2001 From: John Adler Date: Wed, 10 Jun 2020 23:46:15 -0400 Subject: [PATCH 40/49] Refactor consensus rules for validators and delegations to use code snippets instead of tables. --- specs/consensus.md | 101 ++++++++++++++++++++++++--------------------- 1 file changed, 53 insertions(+), 48 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index b76ed77..36006a8 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -75,70 +75,75 @@ Consensus Rules ### Validators and Delegations A transaction `tx` that requests a new validator initializes the [Validator](data_structures.md#validator) fields of that account as follows: -| name | value | -| ----------------- | ------------------------ | -| `status` | `ValidatorStatus.Queued` | -| `stakedBalance` | `tx.amount` | -| `commissionRate` | `tx.commissionRate` | -| `delegatedCount` | `0` | -| `votingPower` | `tx.amount` | -| `pendingRewards` | `0` | -| `latestEntry` | `PeriodEntry(0)` | -| `unbondingHeight` | `0` | -| `isSlashed` | `false` | +``` +validator.status = ValidatorStatus.Queued +validator.stakedBalance = tx.amount +validator.commissionRate = tx.commissionRate +validator.delegatedCount = 0 +validator.votingPower = tx.amount +validator.pendingRewards = 0 +validator.latestEntry = PeriodEntry(0) +validator.unbondingHeight = 0 +validator.isSlashed = false +``` At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validators by voting power are or become active. For newly-bonded validators, their status is changed to bonded and height values initialized. -| name | value | -| -------- | ------------------------ | -| `status` | `ValidatorStatus.Bonded` | +``` +validator.status = ValidatorStatus.Bonded +``` For validators that were bonded but are no longer (either by being outside the top validators or through a transaction that requests unbonding), they begin unbonding. -| name | value | -| ----------------- | --------------------------- | -| `status` | `ValidatorStatus.Unbonding` | -| `unbondingHeight` | `block.height + 1` | +``` +validator.status = ValidatorStatus.Unbonding +validator.unbondingHeight = block.height + 1 +``` Once an unbonding validator has waited at least `UNBONDING_DURATION` blocks, they can be unbonded, collecting their reward: -| name | value | -| --------------- | ------------------------------------- | -| `status` | `ValidatorStatus.Unbonded` | -| `stakedBalance` | `0` | -| `votingPower` | `old.votingPower - old.stakedBalance` | +``` +old_stakedBalance = validator.stakedBalance + +validator.status = ValidatorStatus.Unbonded +validator.stakedBalance = 0 +validator.votingPower -= old_stakedBalance +``` Every time a bonded validator's voting power changes (i.e. when a delegation is added or removed), or when a validator begins unbonding, the rewards per unit of voting power accumulated so far are calculated: -| name | value | -| ---------------- | -------------------------------------------------------- | -| `pendingRewards` | `0` | -| `latestEntry` | `old.latestEntry + old.pendingRewards / old.votingPower` | +``` +old_pendingRewards = validator.pendingRewards +old_votingPower = validator.votingPower + +validator.pendingRewards = 0 +validator.latestEntry += old_pendingRewards / old_votingPower +``` A transaction `tx` that requests a new delegation first updates the target validator's voting power: -| name | value | -| ---------------- | ----------------------------- | -| `delegatedCount` | `old.delegatedCount + 1` | -| `votingPower` | `old.votingPower + tx.amount` | +``` +validator.delegatedCount += 1 +validator.votingPower += tx.amount +``` then initializes the [Delegation](data_structures.md#delegation) field of that account as follows: -| name | value | -| ----------------- | ------------------------- | -| `status` | `DelegationStatus.Bonded` | -| `validator` | `tx.validator` | -| `stakedBalance` | `tx.amount` | -| `beginEntry` | `validator.latestEntry` | -| `endEntry` | `PeriodEntry(0)` | -| `unbondingHeight` | `0` | +``` +delegation.status = DelegationStatus.Bonded +delegation.validator = tx.validator +delegation.stakedBalance = tx.amount +delegation.beginEntry = validator.latestEntry +delegation.endEntry = PeriodEntry(0) +delegation.unbondingHeight = 0 +``` A transaction `tx` that requests withdrawing a delegation first updates the delegation field: -| name | value | -| ----------------- | ---------------------------- | -| `status` | `DelegationStatus.Unbonding` | -| `endEntry` | `validator.latestEntry` | -| `unbondingHeight` | `block.height + 1` | +``` +delegation.status = DelegationStatus.Unbonding +delegation.endEntry = validator.latestEntry +delegation.unbondingHeight = block.height + 1 +``` then updates the target validator's voting power: -| name | value | -| ---------------- | ------------------------------------------ | -| `delegatedCount` | `old.delegatedCount - 1` | -| `votingPower` | `old.votingPower - delegation.votingPower` | +``` +validator.delegatedCount -= 1 +validator.votingPower -= delegation.votingPower +``` ## Formatting From f1cba758138d074b7a18b59d62b3236587f920d0 Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 13:08:21 -0400 Subject: [PATCH 41/49] Refactor state tree to use a single unified tree with distinct subtrees. --- specs/consensus.md | 17 +++++++-- specs/data_structures.md | 80 +++++++++++++++++++++++----------------- 2 files changed, 61 insertions(+), 36 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index 36006a8..2a8ce22 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -7,6 +7,7 @@ Consensus Rules - [Constants](#constants) - [Types](#types) - [Reserved Namespace IDs](#reserved-namespace-ids) + - [Reserved State Subtree IDs](#reserved-state-subtree-ids) - [Rewards and Penalties](#rewards-and-penalties) - [Leader Selection](#leader-selection) - [Fork Choice](#fork-choice) @@ -41,12 +42,14 @@ Consensus Rules | `GENESIS_COIN_COUNT` | `uint64` | `10**8` | `4u` | `(= 100000000)` Number of coins at genesis. | | `UNBONDING_DURATION` | `uint32` | | `block` | Duration, in blocks, for unbonding a validator or delegation. | | `MAX_VALIDATORS` | `uint16` | `64` | | Maximum number of active validators. | +| `STATE_SUBTREE_RESERVED_BYTES` | `uint64` | `1` | `byte` | Number of bytes reserved to identify state subtrees. | ## Types -| name | type | -| ------------- | -------- | -| `NamespaceID` | `uint64` | +| name | type | +| ---------------- | -------- | +| `NamespaceID` | `uint64` | +| `StateSubtreeID` | `byte` | ## Reserved Namespace IDs @@ -57,6 +60,14 @@ Consensus Rules | `EVIDENCE_NAMESPACE_ID` | `NamespaceID` | `0x0000000000000000000000000000000000000000000000000000000000000003` | | `PARITY_SHARE_NAMESPACE_ID` | `NamespaceID` | `0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF` | +## Reserved State Subtree IDs + +| name | type | value | +| ---------------------------- | ---------------- | ------ | +| `ACCOUNTS_SUBTREE_ID` | `StateSubtreeID` | `0x01` | +| `VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x02` | +| `VALIDATOR_COUNT_SUBTREE_ID` | `StateSubtreeID` | `0x03` | + ## Rewards and Penalties | name | type | value | unit | description | diff --git a/specs/data_structures.md b/specs/data_structures.md index 455584b..1f1ee29 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -44,9 +44,10 @@ Data Structures - [Message](#message) - [State](#state) - [Account](#account) - - [PeriodEntry](#periodentry) - - [Validator](#validator) - [Delegation](#delegation) + - [Validator](#validator) + - [ActiveValidatorCount](#activevalidatorcount) + - [PeriodEntry](#periodentry) - [Decimal](#decimal) - [Consensus Parameters](#consensus-parameters) @@ -480,14 +481,18 @@ enum VoteType : uint8_t { # State -| name | type | description | -| --------------- | ------------------------- | ---------------------------- | -| `accountsRoot` | [HashDigest](#hashdigest) | Merkle root of account tree. | -| `numValidators` | `uint32` | Number of active validators. | +| name | type | description | +| ----------- | ------------------------- | -------------------------- | +| `stateRoot` | [HashDigest](#hashdigest) | Merkle root of state tree. | -The state of the LazyLedger chain contains only account balances and the validator set (which is extra metadata on top of the plain account balances). +The state of the LazyLedger chain is intentionally restricted to containing only account balances and the validator set metadata. One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for the entire chain state, the _state tree_. -One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both account account balances and validator metadata, the _accounts tree_. The final state root is computed as the [hash](#hashdigest) of the accounts tree root and number of active validators. The latter is necessary to ensure light nodes can determine the entire validator set from a single state root commitment. +The state tree is separated into `2**(8*STATE_SUBTREE_RESERVED_BYTES)` subtrees, each of which can be used to store a different component of the state. This is done by slicing off the highest `STATE_SUBTREE_RESERVED_BYTES` bytes from the key and replacing them with the appropriate [reserved state subtree ID](consensus.md#reserved-state-subtree-ids). Reducing the key size within subtrees also reduces the collision resistance of keys by `8*STATE_SUBTREE_RESERVED_BYTES` bits, but this is not an issue due the number of bits removed being small. + +Three subtrees are maintained: +1. [Accounts](#account) +1. [Active validator set](#validator) +1. [Active validator count](#activevalidatorcount) ## Account @@ -496,19 +501,32 @@ One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for both ac | `balance` | `uint64` | Coin balance. | | `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | | `isValidator` | `bool` | Whether this account is a validator or not. Mutually exclusive with `isDelegating`. | -| `validatorInfo` | [Validator](#validator) | _Optional_, only if `isValidator` is set. Validator info. | | `isDelegating` | `bool` | Whether this account is delegating its stake or not. Mutually exclusive with `isValidator`. | | `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | -In the accounts tree, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). +In the accounts subtree, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). The first byte is then replaced with `ACCOUNTS_SUBTREE_ID`. -## PeriodEntry +## Delegation -| name | type | description | -| ------------ | -------- | ------------------------------------------------------------- | -| `rewardRate` | `uint64` | Rewards per unit of voting power accumulated so far, in `1u`. | +```C++ +enum DelegationStatus : uint8_t { + Bonded = 1, + Unbonding = 2, +}; +``` -For explanation on entries, see the [reward distribution rationale document](../rationale/distributing_rewards.md). +| name | type | description | +| ----------------- | --------------------------- | --------------------------------------------------- | +| `status` | `DelegationStatus` | Status of this delegation. | +| `validator` | [Address](#address) | The validator being delegating to. | +| `stakedBalance` | `uint64` | Delegated stake, in `4u`. | +| `beginEntry` | [PeriodEntry](#periodentry) | Entry when delegation began. | +| `endEntry` | [PeriodEntry](#periodentry) | Entry when delegation ended (i.e. began unbonding). | +| `unbondingHeight` | `uint64` | Block height delegation began unbonding. | + +Delegation objects represent a delegation. They have two statuses: +1. `Bonded`: This delegation is enabled for a `Queued` _or_ `Bonded` validator. Delegations to a `Queued` validator can be withdrawn immediately, while delegations for a `Bonded` validator must be unbonded first. +1. `Unbonding`: This delegation is unbonding. It will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. Once the unbonding duration has expired, the delegation can be withdrawn. ## Validator @@ -540,27 +558,23 @@ Validator objects represent all the information needed to be keep track of a val 1. `Unbonding`: This validator is in the process of unbonding, which can be voluntary (the validator decided to stop being an active validator) or forced (the validator committed a slashable offence and was kicked from the active validator set). Validators will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. 1. `Unbonded`: This validator has completed its unbonding and has withdrawn its stake. The validator object will remain in this status until `delegatedCount` reaches zero, at which point it is destroyed. -## Delegation +In the validators subtree, validators are keyed by the [hash](#hashdigest) of their [address](#address). The first byte is then replaced with `VALIDATORS_SUBTREE_ID`. By construction, the validators subtree will be a subset of a mirror of the [accounts subtree](#account). -```C++ -enum DelegationStatus : uint8_t { - Bonded = 1, - Unbonding = 2, -}; -``` +## ActiveValidatorCount -| name | type | description | -| ----------------- | --------------------------- | --------------------------------------------------- | -| `status` | `DelegationStatus` | Status of this delegation. | -| `validator` | [Address](#address) | The validator being delegating to. | -| `stakedBalance` | `uint64` | Delegated stake, in `4u`. | -| `beginEntry` | [PeriodEntry](#periodentry) | Entry when delegation began. | -| `endEntry` | [PeriodEntry](#periodentry) | Entry when delegation ended (i.e. began unbonding). | -| `unbondingHeight` | `uint64` | Block height delegation began unbonding. | +| name | type | description | +| --------------- | -------- | ---------------------------- | +| `numValidators` | `uint32` | Number of active validators. | -Delegation objects represent a delegation. They have two statuses: -1. `Bonded`: This delegation is enabled for a `Queued` _or_ `Bonded` validator. Delegations to a `Queued` validator can be withdrawn immediately, while delegations for a `Bonded` validator must be unbonded first. -1. `Unbonding`: This delegation is unbonding. It will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. Once the unbonding duration has expired, the delegation can be withdrawn. +Since the [validator set](#validator) is stored in a Sparse Merkle Tree, there is no compact way of proving that the number of active validators exceeds `MAX_VALIDATORS` without keeping track of the number of active validators. There is only a single leaf in the active validator count subtree, which is keyed with zero (i.e. `0x0000000000000000000000000000000000000000000000000000000000000000`), and the first byte replaced with `VALIDATOR_COUNT_SUBTREE_ID`. + +## PeriodEntry + +| name | type | description | +| ------------ | -------- | ------------------------------------------------------------- | +| `rewardRate` | `uint64` | Rewards per unit of voting power accumulated so far, in `1u`. | + +For explanation on entries, see the [reward distribution rationale document](../rationale/distributing_rewards.md). ## Decimal From a9d5149254c756f27258629850894f635723282d Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 13:11:17 -0400 Subject: [PATCH 42/49] Fix typo. --- specs/data_structures.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 1f1ee29..bd3c1c1 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -485,7 +485,7 @@ enum VoteType : uint8_t { | ----------- | ------------------------- | -------------------------- | | `stateRoot` | [HashDigest](#hashdigest) | Merkle root of state tree. | -The state of the LazyLedger chain is intentionally restricted to containing only account balances and the validator set metadata. One unified [Sparse Merkle Trees](#sparse-merkle-tree) is maintained for the entire chain state, the _state tree_. +The state of the LazyLedger chain is intentionally restricted to containing only account balances and the validator set metadata. One unified [Sparse Merkle Tree](#sparse-merkle-tree) is maintained for the entire chain state, the _state tree_. The state tree is separated into `2**(8*STATE_SUBTREE_RESERVED_BYTES)` subtrees, each of which can be used to store a different component of the state. This is done by slicing off the highest `STATE_SUBTREE_RESERVED_BYTES` bytes from the key and replacing them with the appropriate [reserved state subtree ID](consensus.md#reserved-state-subtree-ids). Reducing the key size within subtrees also reduces the collision resistance of keys by `8*STATE_SUBTREE_RESERVED_BYTES` bits, but this is not an issue due the number of bits removed being small. From 4fbff8a4133876c8133c03268245804be7a8f4a9 Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 13:16:57 -0400 Subject: [PATCH 43/49] Remove redundant state root definition. --- specs/data_structures.md | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index bd3c1c1..0751f39 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -72,16 +72,16 @@ Blocks are the top-level data structure of the LazyLedger blockchain. Block header, which is fully downloaded by both full clients and light clients. -| name | type | description | -| ------------------- | ------------------------- | -------------------------------------------------------------------------------------------- | -| `height` | `uint64` | Block height. The genesis block is at height `1`. | -| `time` | [Time](#time) | Timestamp of this block. | -| `lastBlockID` | [BlockID](#blockid) | Previous block's ID. | -| `lastCommitRoot` | [HashDigest](#hashdigest) | Previous block's Tendermint commit root. | -| `consensusRoot` | [HashDigest](#hashdigest) | Merkle root of [consensus parameters](#consensus-parameters) for this block. | -| `stateCommitment` | [HashDigest](#hashdigest) | Commitment to state root and validator set root after this block's transactions are applied. | -| `availableDataRoot` | [HashDigest](#hashdigest) | Root of [commitments to erasure-coded data](#availabledataheader). | -| `proposerAddress` | [Address](#address) | Address of this block's proposer. | +| name | type | description | +| ------------------- | ------------------------- | ---------------------------------------------------------------------------- | +| `height` | `uint64` | Block height. The genesis block is at height `1`. | +| `time` | [Time](#time) | Timestamp of this block. | +| `lastBlockID` | [BlockID](#blockid) | Previous block's ID. | +| `lastCommitRoot` | [HashDigest](#hashdigest) | Previous block's Tendermint commit root. | +| `consensusRoot` | [HashDigest](#hashdigest) | Merkle root of [consensus parameters](#consensus-parameters) for this block. | +| `stateCommitment` | [HashDigest](#hashdigest) | The [state root](#state) after this block's transactions are applied. | +| `availableDataRoot` | [HashDigest](#hashdigest) | Root of [commitments to erasure-coded data](#availabledataheader). | +| `proposerAddress` | [Address](#address) | Address of this block's proposer. | ## AvailableDataHeader @@ -481,11 +481,7 @@ enum VoteType : uint8_t { # State -| name | type | description | -| ----------- | ------------------------- | -------------------------- | -| `stateRoot` | [HashDigest](#hashdigest) | Merkle root of state tree. | - -The state of the LazyLedger chain is intentionally restricted to containing only account balances and the validator set metadata. One unified [Sparse Merkle Tree](#sparse-merkle-tree) is maintained for the entire chain state, the _state tree_. +The state of the LazyLedger chain is intentionally restricted to containing only account balances and the validator set metadata. One unified [Sparse Merkle Tree](#sparse-merkle-tree) is maintained for the entire chain state, the _state tree_. The root of this tree is committed to in the [block header](#header). The state tree is separated into `2**(8*STATE_SUBTREE_RESERVED_BYTES)` subtrees, each of which can be used to store a different component of the state. This is done by slicing off the highest `STATE_SUBTREE_RESERVED_BYTES` bytes from the key and replacing them with the appropriate [reserved state subtree ID](consensus.md#reserved-state-subtree-ids). Reducing the key size within subtrees also reduces the collision resistance of keys by `8*STATE_SUBTREE_RESERVED_BYTES` bits, but this is not an issue due the number of bits removed being small. From 4a508078ab0b41c4c2bcd45938bd1068aa354435 Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 13:30:08 -0400 Subject: [PATCH 44/49] Remove redundant validator flag in accounts. --- specs/data_structures.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/specs/data_structures.md b/specs/data_structures.md index 0751f39..d0a2a10 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -492,13 +492,12 @@ Three subtrees are maintained: ## Account -| name | type | description | -| ---------------- | ------------------------- | ------------------------------------------------------------------------------------------- | -| `balance` | `uint64` | Coin balance. | -| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | -| `isValidator` | `bool` | Whether this account is a validator or not. Mutually exclusive with `isDelegating`. | -| `isDelegating` | `bool` | Whether this account is delegating its stake or not. Mutually exclusive with `isValidator`. | -| `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | +| name | type | description | +| ---------------- | ------------------------- | --------------------------------------------------------------------------------- | +| `balance` | `uint64` | Coin balance. | +| `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | +| `isDelegating` | `bool` | Whether this account is delegating its stake or not. | +| `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | In the accounts subtree, accounts (i.e. leaves) are keyed by the [hash](#hashdigest) of their [address](#address). The first byte is then replaced with `ACCOUNTS_SUBTREE_ID`. From a8c17a78403eaaa6bd5fb56de782c1414fb3d67e Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 14:10:12 -0400 Subject: [PATCH 45/49] Add protobuf definitions for state elements. --- proto/types.proto | 55 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 54 insertions(+), 1 deletion(-) diff --git a/proto/types.proto b/proto/types.proto index f9abb13..a60ba01 100644 --- a/proto/types.proto +++ b/proto/types.proto @@ -137,4 +137,57 @@ message MessageData { message Message { bytes namespace_id = 1; bytes raw_data = 2; -} \ No newline at end of file +} + +message Account { + uint64 balance = 1; + uint64 nonce = 2; + bool isDelegating = 3; + Delegation delegationInfo = 4; +} + +message Delegation { + enum DelegationStatus { + DELEGATION_STATUS_UNSPECIFIED = 0; + DELEGATION_STATUS_BONDED = 1; + DELEGATION_STATUS_UNBONDING = 2; + } + + DelegationStatus status = 1; + Address validator = 2; + uint64 stakedBalance = 3; + PeriodEntry beginEntry = 4; + PeriodEntry endEntry = 5; + uint64 unbondingHeight = 6; +} + +message Validator { + enum ValidatorStatus { + VALIDATOR_STATUS_UNSPECIFIED = 0; + VALIDATOR_STATUS_QUEUED = 1; + VALIDATOR_STATUS_BONDED = 2; + VALIDATOR_STATUS_UNBONDING = 3; + VALIDATOR_STATUS_UNBONDED = 4; + } + + ValidatorStatus status = 1; + uint64 stakedBalance = 2; + Decimal commissionRate = 3; + uint32 delegatedCount = 4; + uint64 votingPower = 5; + uint64 pendingRewards = 6; + PeriodEntry latestEntry = 7; + uint64 unbondingHeight = 8; + bool isSlashed = 9; + Decimal slashRate = 10; +} + +message ActiveValidatorCount { + uint32 numValidators = 1; +} + +message PeriodEntry { + uint64 rewardRate = 1; +} + +message Decimal {} \ No newline at end of file From 73de678d395c68a868e00675bc4cbcba9c60bc5f Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 14:46:52 -0400 Subject: [PATCH 46/49] Add another subtree for inactive validators. --- proto/types.proto | 5 +++-- specs/consensus.md | 18 ++++++++++-------- specs/data_structures.md | 8 +++++--- 3 files changed, 18 insertions(+), 13 deletions(-) diff --git a/proto/types.proto b/proto/types.proto index a60ba01..b977e64 100644 --- a/proto/types.proto +++ b/proto/types.proto @@ -142,8 +142,9 @@ message Message { message Account { uint64 balance = 1; uint64 nonce = 2; - bool isDelegating = 3; - Delegation delegationInfo = 4; + bool isValidator = 3; + bool isDelegating = 4; + Delegation delegationInfo = 5; } message Delegation { diff --git a/specs/consensus.md b/specs/consensus.md index 2a8ce22..4775cf2 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -62,11 +62,13 @@ Consensus Rules ## Reserved State Subtree IDs -| name | type | value | -| ---------------------------- | ---------------- | ------ | -| `ACCOUNTS_SUBTREE_ID` | `StateSubtreeID` | `0x01` | -| `VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x02` | -| `VALIDATOR_COUNT_SUBTREE_ID` | `StateSubtreeID` | `0x03` | +| name | type | value | +| ----------------------------------- | ---------------- | ------ | +| `ACCOUNTS_SUBTREE_ID` | `StateSubtreeID` | `0x01` | +| `ACTIVE_VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x02` | +| `ACTIVE_VALIDATOR_COUNT_SUBTREE_ID` | `StateSubtreeID` | `0x03` | +| `INACTIVE_VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x04` | + ## Rewards and Penalties @@ -85,7 +87,7 @@ Consensus Rules ### Validators and Delegations -A transaction `tx` that requests a new validator initializes the [Validator](data_structures.md#validator) fields of that account as follows: +A transaction `tx` that requests a new validator initializes a new [Validator](data_structures.md#validator) leaf in the inactive validators subtree for that account as follows: ``` validator.status = ValidatorStatus.Queued validator.stakedBalance = tx.amount @@ -98,12 +100,12 @@ validator.unbondingHeight = 0 validator.isSlashed = false ``` -At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validators by voting power are or become active. For newly-bonded validators, their status is changed to bonded and height values initialized. +At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validators by voting power are or become active (bonded). For newly-bonded validators, the entire validator object is moved to the active validators subtree and their status is changed to bonded. ``` validator.status = ValidatorStatus.Bonded ``` -For validators that were bonded but are no longer (either by being outside the top validators or through a transaction that requests unbonding), they begin unbonding. +For validators that were bonded but are no longer (either by being outside the top `MAX_VALIDATORS` validators or through a transaction that requests unbonding), the validator object is moved to the inactive validators subtree they begin unbonding. ``` validator.status = ValidatorStatus.Unbonding validator.unbondingHeight = block.height + 1 diff --git a/specs/data_structures.md b/specs/data_structures.md index d0a2a10..81ef42e 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -485,10 +485,11 @@ The state of the LazyLedger chain is intentionally restricted to containing only The state tree is separated into `2**(8*STATE_SUBTREE_RESERVED_BYTES)` subtrees, each of which can be used to store a different component of the state. This is done by slicing off the highest `STATE_SUBTREE_RESERVED_BYTES` bytes from the key and replacing them with the appropriate [reserved state subtree ID](consensus.md#reserved-state-subtree-ids). Reducing the key size within subtrees also reduces the collision resistance of keys by `8*STATE_SUBTREE_RESERVED_BYTES` bits, but this is not an issue due the number of bits removed being small. -Three subtrees are maintained: +Four subtrees are maintained: 1. [Accounts](#account) 1. [Active validator set](#validator) 1. [Active validator count](#activevalidatorcount) +1. [Inactive validator set](#validator) ## Account @@ -496,6 +497,7 @@ Three subtrees are maintained: | ---------------- | ------------------------- | --------------------------------------------------------------------------------- | | `balance` | `uint64` | Coin balance. | | `nonce` | `uint64` | Account nonce. Every outgoing transaction from this account increments the nonce. | +| `isValidator` | `bool` | Whether this account is a validator or not. | | `isDelegating` | `bool` | Whether this account is delegating its stake or not. | | `delegationInfo` | [Delegation](#delegation) | _Optional_, only if `isDelegating` is set. Delegation info. | @@ -553,7 +555,7 @@ Validator objects represent all the information needed to be keep track of a val 1. `Unbonding`: This validator is in the process of unbonding, which can be voluntary (the validator decided to stop being an active validator) or forced (the validator committed a slashable offence and was kicked from the active validator set). Validators will remain in this status for at least `UNBONDING_DURATION` blocks, and while unbonding may still be slashed. 1. `Unbonded`: This validator has completed its unbonding and has withdrawn its stake. The validator object will remain in this status until `delegatedCount` reaches zero, at which point it is destroyed. -In the validators subtree, validators are keyed by the [hash](#hashdigest) of their [address](#address). The first byte is then replaced with `VALIDATORS_SUBTREE_ID`. By construction, the validators subtree will be a subset of a mirror of the [accounts subtree](#account). +In the validators subtrees, validators are keyed by the [hash](#hashdigest) of their [address](#address). The first byte is then replaced with `ACTIVE_VALIDATORS_SUBTREE_ID` for the active validator set or `INACTIVE_VALIDATORS_SUBTREE_ID` for the inactive validator set. Active validators are `Bonded`, while inactive validators are not `Bonded`. By construction, the validators subtrees will be a subset of a mirror of the [accounts subtree](#account). ## ActiveValidatorCount @@ -561,7 +563,7 @@ In the validators subtree, validators are keyed by the [hash](#hashdigest) of th | --------------- | -------- | ---------------------------- | | `numValidators` | `uint32` | Number of active validators. | -Since the [validator set](#validator) is stored in a Sparse Merkle Tree, there is no compact way of proving that the number of active validators exceeds `MAX_VALIDATORS` without keeping track of the number of active validators. There is only a single leaf in the active validator count subtree, which is keyed with zero (i.e. `0x0000000000000000000000000000000000000000000000000000000000000000`), and the first byte replaced with `VALIDATOR_COUNT_SUBTREE_ID`. +Since the [active validator set](#validator) is stored in a Sparse Merkle Tree, there is no compact way of proving that the number of active validators exceeds `MAX_VALIDATORS` without keeping track of the number of active validators. There is only a single leaf in the active validator count subtree, which is keyed with zero (i.e. `0x0000000000000000000000000000000000000000000000000000000000000000`), and the first byte replaced with `VALIDATOR_COUNT_SUBTREE_ID`. ## PeriodEntry From b4db7e329f4e3f1ff16536b7d000acb2410be326 Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 15:00:20 -0400 Subject: [PATCH 47/49] Clean up. --- specs/consensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index 4775cf2..ab1b20f 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -105,7 +105,7 @@ At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validator validator.status = ValidatorStatus.Bonded ``` -For validators that were bonded but are no longer (either by being outside the top `MAX_VALIDATORS` validators or through a transaction that requests unbonding), the validator object is moved to the inactive validators subtree they begin unbonding. +For validators that were bonded but are no longer (either by being outside the top `MAX_VALIDATORS` validators, through a transaction that requests unbonding, or being by slashing), the validator object is moved to the inactive validators subtree and they begin unbonding. ``` validator.status = ValidatorStatus.Unbonding validator.unbondingHeight = block.height + 1 From 2c4e44c408613b472e61556f5e477118ef8fdd58 Mon Sep 17 00:00:00 2001 From: John Adler Date: Thu, 11 Jun 2020 15:00:48 -0400 Subject: [PATCH 48/49] Fix typo. --- specs/consensus.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/consensus.md b/specs/consensus.md index ab1b20f..edcf96f 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -105,7 +105,7 @@ At the end of a block at the end of an epoch, the top `MAX_VALIDATORS` validator validator.status = ValidatorStatus.Bonded ``` -For validators that were bonded but are no longer (either by being outside the top `MAX_VALIDATORS` validators, through a transaction that requests unbonding, or being by slashing), the validator object is moved to the inactive validators subtree and they begin unbonding. +For validators that were bonded but are no longer (either by being outside the top `MAX_VALIDATORS` validators, through a transaction that requests unbonding, or by being slashing), the validator object is moved to the inactive validators subtree and they begin unbonding. ``` validator.status = ValidatorStatus.Unbonding validator.unbondingHeight = block.height + 1 From 4d879965bb8f76825e9bf4f814dc3062b6388eb1 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 15 Jun 2020 17:40:20 -0400 Subject: [PATCH 49/49] Move the active validator count to the active validators subtree. --- specs/consensus.md | 11 +++++------ specs/data_structures.md | 5 ++--- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/specs/consensus.md b/specs/consensus.md index edcf96f..5130f6c 100644 --- a/specs/consensus.md +++ b/specs/consensus.md @@ -62,12 +62,11 @@ Consensus Rules ## Reserved State Subtree IDs -| name | type | value | -| ----------------------------------- | ---------------- | ------ | -| `ACCOUNTS_SUBTREE_ID` | `StateSubtreeID` | `0x01` | -| `ACTIVE_VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x02` | -| `ACTIVE_VALIDATOR_COUNT_SUBTREE_ID` | `StateSubtreeID` | `0x03` | -| `INACTIVE_VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x04` | +| name | type | value | +| -------------------------------- | ---------------- | ------ | +| `ACCOUNTS_SUBTREE_ID` | `StateSubtreeID` | `0x01` | +| `ACTIVE_VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x02` | +| `INACTIVE_VALIDATORS_SUBTREE_ID` | `StateSubtreeID` | `0x03` | ## Rewards and Penalties diff --git a/specs/data_structures.md b/specs/data_structures.md index 81ef42e..8e49040 100644 --- a/specs/data_structures.md +++ b/specs/data_structures.md @@ -485,10 +485,9 @@ The state of the LazyLedger chain is intentionally restricted to containing only The state tree is separated into `2**(8*STATE_SUBTREE_RESERVED_BYTES)` subtrees, each of which can be used to store a different component of the state. This is done by slicing off the highest `STATE_SUBTREE_RESERVED_BYTES` bytes from the key and replacing them with the appropriate [reserved state subtree ID](consensus.md#reserved-state-subtree-ids). Reducing the key size within subtrees also reduces the collision resistance of keys by `8*STATE_SUBTREE_RESERVED_BYTES` bits, but this is not an issue due the number of bits removed being small. -Four subtrees are maintained: +Three subtrees are maintained: 1. [Accounts](#account) 1. [Active validator set](#validator) -1. [Active validator count](#activevalidatorcount) 1. [Inactive validator set](#validator) ## Account @@ -563,7 +562,7 @@ In the validators subtrees, validators are keyed by the [hash](#hashdigest) of t | --------------- | -------- | ---------------------------- | | `numValidators` | `uint32` | Number of active validators. | -Since the [active validator set](#validator) is stored in a Sparse Merkle Tree, there is no compact way of proving that the number of active validators exceeds `MAX_VALIDATORS` without keeping track of the number of active validators. There is only a single leaf in the active validator count subtree, which is keyed with zero (i.e. `0x0000000000000000000000000000000000000000000000000000000000000000`), and the first byte replaced with `VALIDATOR_COUNT_SUBTREE_ID`. +Since the [active validator set](#validator) is stored in a [Sparse Merkle Tree](#sparse-merkle-tree), there is no compact way of proving that the number of active validators exceeds `MAX_VALIDATORS` without keeping track of the number of active validators. The active validator count is stored in the active validators subtree, and is keyed with zero (i.e. `0x0000000000000000000000000000000000000000000000000000000000000000`), with the first byte replaced with `ACTIVE_VALIDATORS_SUBTREE_ID`. ## PeriodEntry