ADR 008 Updating to tendermint v0.35.4 (#739)

* add ADR 008 and point to it in 007

* fix date

Co-authored-by: John Adler <[email protected]>

* fix date

Co-authored-by: John Adler <[email protected]>

Co-authored-by: John Adler <[email protected]>

docs/celestia-architecture/adr-007-minimal-changes-to-tendermint.md

@@ -2,7 +2,8 @@
 
 ## Changelog
 
-- 20-08-2021: Initial Description
+- 2021-08-20: Initial Description
+- 2022-05-03: Update pointing to ADR 008
 
 ## Context
 
@@ -14,6 +15,8 @@ some of the remaining necessary changes to be more isolated from the rest of the
 strategies should significantly streamline pulling updates from upstream, and allow us to iterate faster since most
 changes will be isolated to celestia-node.
 
+Update: many of the changes described below have since been minimized or removed. Please see ADR 008 for a summarized list of changes. Notably, we removed intermediate state roots, adopted two new methods from ABCI++ instead of PreprocessTxs, and are still signing over the PartSetHeader. 
+
 ## Decision
 
 Treat tendermint more as a "black box".
@@ -27,7 +30,7 @@ tendermint node in a separate process, via celestia-node, which communicates wit
 availability sampling logic, including all Celestia-specific networking logic not already provided by tendermint, is
 moved into celestia node:
 
-![](./img/core-node-relation.png)
+![core node relation](./img/core-node-relation.png)
 
 The detailed design of celestia-node will be defined in the repository itself.
 

docs/celestia-architecture/adr-008-updating-to-tendermint-v0.35.x.md

@@ -0,0 +1,53 @@
+# ADR 008: Updating to tendermint v0.35.x 
+
+## Changelog
+
+- 2022-05-03: Initial document describing changes to tendermint v0.35.x
+
+## Context
+
+Building off of ADR 007, we have further distilled the necessary changes to tendermint and continued to move added logic to other repos. Specifically, we have moved generation of the data hash, efficient construction of the data square, and a message inclusion check to celestia-app via adopting two new methods from ABCI++. This document is to serve as a guide for the remaining changes made on top of tendermint v0.35.4.
+
+### Changes to tendermint
+
+#### Misc
+
+- [update github templates](https://github.com/celestiaorg/celestia-core/pull/405)
+- [update README.md](https://github.com/celestiaorg/celestia-core/pull/737/commits/be9039d4e0f5d876ec3d8d4521be3374172d7992)
+- [updating to go 1.17](https://github.com/celestiaorg/celestia-core/pull/737/commits/6094b7338082d106f81da987dffa56eb540a675e)
+- [adding the consts package](https://github.com/celestiaorg/celestia-core/pull/737/commits/fea8528b0177230b7e75396ae05f7a9b5da23741)
+
+#### Changing the way the data hash is calculated
+
+To enable data availability sampling, Celestia uses a proprietary data square format to encode its block data. The data hash is generated from this data square by calculating namespace merkle tree root over each row and column. In the following changes, we implement encoding and decoding of block data to the data square format and tooling to generate the data hash. More details over this design can be found in our (archived but still very useful) [specs repo](https://github.com/celestiaorg/celestia-specs)
+
+- [Adding the Data Availability Header](https://github.com/celestiaorg/celestia-core/pull/737/commits/116b7af4000920030a373363487ef9a9f084e066)
+- [Adding a wrapper for namespaced merkle trees](https://github.com/celestiaorg/celestia-core/pull/737/commits/eee8f352cb6a1687a9f6b470abe28bbd4eb66413)
+- [Adding Messages and Evidence to the block data](https://github.com/celestiaorg/celestia-core/pull/737/commits/86df6529a7c0cc1112c34b6bf1b5364aa0518dec)
+- [Adding share splitting and merging for block encoding](https://github.com/celestiaorg/celestia-core/pull/737/commits/bf2d8b46c1caf1fed52e7db9bf8aa6a9847d84ab)
+- [Modifying MakeBlock to also accept Messages](https://github.com/celestiaorg/celestia-core/pull/737/commits/bb970a417356ab030c934ccd2bd39c9641af45f8)
+
+#### Adding PrepareProposal and ProcessProposal ABCI methods from ABCI++
+
+- [PrepareProposal](https://github.com/celestiaorg/celestia-core/pull/737/commits/07f9a05444db763c44ff81f564e7350ddf57e5a4)
+- [ProcessProposal](https://github.com/celestiaorg/celestia-core/pull/737/commits/2c9552db09841f2bbebc1ec34653b2441def9f13)
+
+more details on how we use these new methods in the app can be found in the [ABCI++ Adoption ADR](https://github.com/celestiaorg/celestia-app/blob/master/docs/architecture/ADR-001-ABCI%2B%2B.md).
+
+#### Wrapping Malleated Transactions
+
+Tendermint and the cosmos-sdk were not built to handle malleated transactions (txs that are submitted by the user, but modified by the block producer before being included in a block). While not a final solution, we have resorted to adding the hash of the original transaction (the one that is not modified by the block producer) to the modified one. This allows us to track the transaction in the event system and mempool.
+
+- [Index malleated Txs](https://github.com/celestiaorg/celestia-core/pull/737/commits/a54e3599a5ef6b2ba8b63f586aed8185a3f59e4d)
+
+#### Create NMT Inclusion Proofs for Transactions
+
+Since the block data that is committed over is encoded as a data square and we use namespaced merkle trees to generate the row and column roots of that square, we have to create transaction inclusion proofs also using nmts and a data square. The problem is that the block data isn't stored as a square, so in order to generate the inclusion proofs, we have to regenerate a portion of the square. We do that here.
+
+- [Create namespace merkle tree inclusion proofs for transactions included in the block](https://github.com/celestiaorg/celestia-core/pull/737/commits/01051aa5fef0693bf3bda801e39c80e5746b9c33)
+
+#### Adding the DataCommitment RPC endpoint
+
+This RPC endpoint is used by quantum gravity bridge orchestrators to create a commitment over the block data of a range of blocks.
+
+- [Adding the DataCommitment RPC endpoint](https://github.com/celestiaorg/celestia-core/pull/737/commits/134eeefb7af41afe760d4adc5b22a9d55e36bc3e)