|
| 1 | +# ICS 23 |
| 2 | + |
| 3 | +As already stated in the README.md, ICS 23 attempts to be a generic library to represent and verify merkle proofs for (ideally) many different merkle tree storage implementations. The library is heavily used in [ibc-go](https://github.com/cosmos/ibc-go) for proof verification. The following documentation uses the Golang implementation as a reference. |
| 4 | + |
| 5 | +The two most important top level types in the library are `ProofSpec` and `CommitmmentProof`. We will explain them in the coming sections. |
| 6 | + |
| 7 | +## Proof specs |
| 8 | + |
| 9 | +The [`ProofSpec`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L145-L170) message defines what the expected parameters are for a given proof type. Different types of merkle trees will have different proof specs. For example, for [IAVL](https://github.com/cosmos/iavl) trees, [this](https://github.com/cosmos/ics23/blob/go/v0.10.0/go/proof.go#L9-L26) is the corresponding proof spec. The proof spec defines any contraints (e.g. minimum and maximum allowed depth of trees), what data should be added (e.g. prefix or suffix data), what operations are executed on [leaf](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L96-L120) and [inner](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L172-L194) nodes of the tree when calculating the root hash, etc. |
| 10 | + |
| 11 | +```proto |
| 12 | +message ProofSpec { |
| 13 | + LeafOp leaf_spec = 1; |
| 14 | + InnerSpec inner_spec = 2; |
| 15 | + int32 max_depth = 3; |
| 16 | + int32 min_depth = 4; |
| 17 | + bool prehash_key_before_comparison = 5; |
| 18 | +} |
| 19 | +``` |
| 20 | + |
| 21 | +where: |
| 22 | + |
| 23 | +- `max_depth` is the maximum number of inner nodes in the tree. |
| 24 | +- `min-depth` is the minimum number of inner nodes in the tree. |
| 25 | +- `prehash_key_before_comparison` is a flag that indicates whether to prehash the key using the `prehash_key` hash operation specified in the `leaf_spec`. This is used to compare lexical ordering of keys for non-existence proofs. |
| 26 | + |
| 27 | +The `leaf_spec` is a [`LeafOp`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L96-L120). The `LeafOp` type specifies the internal transformation from the original key/value pair of the leaf node into a hash. It specifies any prefix that should prepended and the hash operations to transform the key and value, among other things: |
| 28 | + |
| 29 | +```proto |
| 30 | +message LeafOp { |
| 31 | + HashOp hash = 1; |
| 32 | + HashOp prehash_key = 2; |
| 33 | + HashOp prehash_value = 3; |
| 34 | + LengthOp length = 4; |
| 35 | + bytes prefix = 5; |
| 36 | +} |
| 37 | +``` |
| 38 | + |
| 39 | +where: |
| 40 | + |
| 41 | +- `hash` is [hash operation](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L7-L16) that is applied to the result of `(prefix || length(prehash_key(key)) || length(prehash_value(value))`, where the `||` operator is concatenation of binary data and `key` and `value` are the key and value of the leaf node. |
| 42 | +- `prehash_key` is the hash operation that is applied to the key of the leaf. |
| 43 | +- `prehash_value` is the hash operation that is applied to the value of the leaf. |
| 44 | +- `length` is the [length encoding operation](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L18-L43) that determines the length of the input and returns it prepended to the data. |
| 45 | +- `prefix` is a fixed set of bytes that may optionally be prepended to differentiate a leaf node from an inner node. |
| 46 | + |
| 47 | +And `inner_spec` is an [`InnerSpec`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L172-L194). The `InnerSpec` specifies all store-specific structure information to determine if two proofs from a |
| 48 | +given store are neighbors. |
| 49 | + |
| 50 | +```proto |
| 51 | +message InnerSpec { |
| 52 | + repeated int32 child_order = 1; |
| 53 | + int32 child_size = 2; |
| 54 | + int32 min_prefix_length = 3; |
| 55 | + int32 max_prefix_length = 4; |
| 56 | + bytes empty_child = 5; |
| 57 | + HashOp hash = 6; |
| 58 | +} |
| 59 | +``` |
| 60 | + |
| 61 | +where: |
| 62 | + |
| 63 | +- `child_order` is the ordering of the children node and counts from 0. For example, for an IAVL tree is [0, 1] (left, then right); for a merk tree is [0, 2, 1] (left, right, here). |
| 64 | +- `child_size` is the size of the child node when calculating the root hash. For example, the child size for an [IAVL tree](https://github.com/cosmos/ics23/blob/4fd3a5af9a290e80bca166fd40f0ef316e167245/go/proof.go#L22) is 33, as the child is: `length op (1 byte) | child_hash (32 bytes)` |
| 65 | +- `min_prefix_length` is the minimum allowed prefix length. For example, for an IAVL tree the `min_prefix_length` is [4](https://github.com/cosmos/ics23/blob/4fd3a5af9a290e80bca166fd40f0ef316e167245/go/proof.go#L20). This derives from the logic that when the child comes from the left, the prefix is: `IAVL height | IAVL size | IAVL version | length byte` |
| 66 | +- `max_prefix_length` is the maximum allowed prefix length. It must be strictly less than the `min_prefix_length` + `child_size`. |
| 67 | +- `empty_child` is the prehash image that is used when one child is `nil`. |
| 68 | +- `hash` is the hashing algorithm that must be used in each element of an `InnerOp` list in an `ExistenceProof`. |
| 69 | + |
| 70 | +## Proof types |
| 71 | + |
| 72 | +The [`CommitmentProof`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L84-L94) is either an `ExistenceProof` or a `NonExistenceProof`, or a batch of such messages: |
| 73 | + |
| 74 | +```proto |
| 75 | +message CommitmentProof { |
| 76 | + oneof proof { |
| 77 | + ExistenceProof exist = 1; |
| 78 | + NonExistenceProof nonexist = 2; |
| 79 | + BatchProof batch = 3; |
| 80 | + CompressedBatchProof compressed = 4; |
| 81 | + } |
| 82 | +} |
| 83 | +``` |
| 84 | + |
| 85 | +At the moment the library has support for [`ExistenceProof`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L45-L71) and [`NonExistenceProof`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L73-L82). |
| 86 | + |
| 87 | +The `ExistenceProof` |
| 88 | + |
| 89 | +```proto |
| 90 | +message ExistenceProof { |
| 91 | + bytes key = 1; |
| 92 | + bytes value = 2; |
| 93 | + LeafOp leaf = 3; |
| 94 | + repeated InnerOp path = 4; |
| 95 | +} |
| 96 | +``` |
| 97 | + |
| 98 | +where: |
| 99 | + |
| 100 | +- `key` is the key of the leaf node. |
| 101 | +- `value` is the value of the leaf node. |
| 102 | +- `leaf` specifies the operations to perform on the leaf node to internally transform its data into a hash. |
| 103 | +- `path` is a list of items of type `InnerOp` that specify the operations to iteratively perform on inner nodes of a tree to calculate a root hash. `InnerOp` represents a merkle-proof step that is not a leaf. It represents concatenating two children and hashing them to provide the next result: |
| 104 | + |
| 105 | +```proto |
| 106 | +message InnerOp { |
| 107 | + HashOp hash = 1; |
| 108 | + bytes prefix = 2; |
| 109 | + bytes suffix = 3; |
| 110 | +} |
| 111 | +``` |
| 112 | + |
| 113 | +where: |
| 114 | + |
| 115 | +- `hash` is the hash operation that is applied to the result of `(prefix || child || suffix)`, where the `||` operator is concatenation of binary data and where `child` the result of hashing all the tree below this step. |
| 116 | +- `prefix` is is a fixed set of bytes that may optionally be prepended to differentiate leaf nodes from inner nodes. |
| 117 | +- `suffix` is a fixed set of bytes that may optionally be appended to differentiate leaf nodes from inner nodes. |
| 118 | + |
| 119 | +We will explain more details of the different supported proof types in the following sections, where we see an example of how each proof type is used. |
| 120 | + |
| 121 | +## Membership verification |
| 122 | + |
| 123 | +For membership verification of a key/value pair in a merkle tree we need an [`ExistenceProof`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L45-L71). An `ExistenceProof` contains the key and value to prove existence of, and a set of steps to calculate a hash that can be compared with a merkle tree root hash. The `ExistenceProof` struct contains the method `Verify`: |
| 124 | + |
| 125 | +```go |
| 126 | +func (p *ExistenceProof) Verify( |
| 127 | + spec *ProofSpec, |
| 128 | + root CommitmentRoot, |
| 129 | + key []byte, |
| 130 | + value []byte |
| 131 | +) error |
| 132 | +``` |
| 133 | + |
| 134 | +where `spec` is the corresponding `ProofSpec` for the concrete merkle tree, `root` is a byte slice that represents the merkle tree root hash (with which the calculated hash from the proof will be compared), and `key` and `value` are byte slices for the leaf node pair to be verified. We will see this method in action in the following sample program: |
| 135 | + |
| 136 | +```go |
| 137 | +package main |
| 138 | + |
| 139 | +import ( |
| 140 | + "fmt" |
| 141 | + |
| 142 | + log "cosmossdk.io/log" |
| 143 | + iavl "github.com/cosmos/iavl" |
| 144 | + dbm "github.com/cosmos/iavl/db" |
| 145 | + ics23 "github.com/cosmos/ics23/go" |
| 146 | +) |
| 147 | + |
| 148 | +func main() { |
| 149 | + key := []byte("c") |
| 150 | + value := []byte{3} |
| 151 | + |
| 152 | + tree := iavl.NewMutableTree(dbm.NewMemDB(), 0, false, log.NewNopLogger()) |
| 153 | + |
| 154 | + _, err := tree.Set(key, value) |
| 155 | + if err != nil { |
| 156 | + fmt.Println(err) |
| 157 | + } |
| 158 | + |
| 159 | + tree.Set([]byte("f"), []byte{6}) |
| 160 | + tree.Set([]byte("d"), []byte{4}) |
| 161 | + tree.Set([]byte("c"), []byte{3}) |
| 162 | + tree.Set([]byte("b"), []byte{2}) |
| 163 | + tree.Set([]byte("a"), []byte{1}) |
| 164 | + |
| 165 | + // retrieve root hash of merkle tree |
| 166 | + rootHash, version, err := tree.SaveVersion() |
| 167 | + if err != nil { |
| 168 | + fmt.Println(err) |
| 169 | + os.Exit(1) |
| 170 | + } |
| 171 | + |
| 172 | + fmt.Printf("saved version %v with root hash %x\n", version, rootHash) |
| 173 | + |
| 174 | + // retrieve membership proof for key "c" |
| 175 | + proof, err := tree.GetMembershipProof(key) |
| 176 | + if err != nil { |
| 177 | + fmt.Println(err) |
| 178 | + os.Exit(1) |
| 179 | + } |
| 180 | + |
| 181 | + // check membership of key/value "c"/3 |
| 182 | + ok := ics23.VerifyMembership(ics23.IavlSpec, rootHash, proof, key, value) |
| 183 | + fmt.Println(ok) |
| 184 | +} |
| 185 | +``` |
| 186 | + |
| 187 | +This simple program creates an IAVL tree, adds a few key/value pairs, and verifies an existence proof. The following diagram represents the IAVL tree created with the sample code above and the steps that the ics23 library performs [during membership verification](https://github.com/cosmos/ics23/blob/go/v0.10.0/go/proof.go#L110) (the hexadecimal values in the picture are the actual values generated during the verification for this particular tree and proof): |
| 188 | + |
| 189 | + |
| 190 | + |
| 191 | +If we run the program with the debugger and step through it, the entry point for ics23 is the [`VerifyMembership`](https://github.com/cosmos/ics23/blob/go/v0.10.0/go/ics23.go#L36) function: |
| 192 | + |
| 193 | + |
| 194 | + |
| 195 | +Inside `VerifyMembership` the `proof`, which is of type [`CommitmentProof`](https://github.com/cosmos/ics23/blob/go/v0.10.0/proto/cosmos/ics23/v1/proofs.proto#L84-L94) is converted to an `ExistenceProof` and then the function `Verify` is called: |
| 196 | + |
| 197 | + |
| 198 | + |
| 199 | +The existence proof contains a `LeafOp` and a `Path`, which is a list of items of type `InnerOp` (there are as many items as inner nodes on the tree - i.e. excluding the leaf and the root nodes): |
| 200 | + |
| 201 | + |
| 202 | + |
| 203 | +Inside `Verify` the existence proof is checked against the corresponging proof spec (i.e. an existence proof for an IAVL tree will be checked against the IAVL proof spec). [This means that the parameters of the leaf operation and the parameters of each inner operation in the path of the existence proof will be checked](https://github.com/cosmos/ics23/blob/go/v0.10.0/go/proof.go#L181-L206) against the specs for leaf and inner nodes: |
| 204 | + |
| 205 | + |
| 206 | + |
| 207 | + |
| 208 | + |
| 209 | +If the check against the proof spec passes, then the leaf operation is applied: |
| 210 | + |
| 211 | + |
| 212 | + |
| 213 | + |
| 214 | +And the result of that operation is fed into the firs inner operation. The result of each inner operation will be fed into the next one until we iterate over the complete list of inner operations: |
| 215 | + |
| 216 | + |
| 217 | + |
| 218 | + |
| 219 | +The result from the last inner operation can be compared against the input root hash and if they match then the verification succeeds: |
| 220 | + |
| 221 | + |
| 222 | + |
| 223 | +## Non-membership verification |
| 224 | + |
| 225 | +TODO: https://github.com/cosmos/ics23/issues/385 |
0 commit comments