chore: ABCI methods benchmarks (#3904)

Works on this: #3898 To run the benchmarks, run the following in the root directory: ```shell go test -bench=<benchmark_name> app/benchmark_test.go ``` with `<benchmark_name>` is: - `BenchmarkCheckTx_MsgSend_1`: for a benchmark of a `MsgSend` transaction in `CheckTx` - `BenchmarkCheckTx_MsgSend_8MB`: for a benchmark of 8mb block worth of `MsgSend` transactions in `CheckTx` - `BenchmarkDeliverTx_MsgSend_1`: for a benchmark of a `MsgSend` transaction in `DeliverTx` - `BenchmarkDeliverTx_MsgSend_8MB`: for a benchmark of 8mb block worth of `MsgSend` transactions in `DeliverTx` - `BenchmarkPrepareProposal_MsgSend_1`: for a benchmark of a block containing a single`MsgSend` transaction in `PrepareProposal` - `BenchmarkPrepareProposal_MsgSend_8MB`: for a benchmark of an 8mb block containing `MsgSend` transactions in `PrepareProposal` - `BenchmarkProcessProposal_MsgSend_1`: for a benchmark of a block containing a single`MsgSend` transaction in `ProcessProposal` - `BenchmarkProcessProposal_MsgSend_8MB`: for a benchmark of an 8mb block containing`MsgSend` transactions in `ProcessProposal` - ... Note: keeping this as a draft because it doesn't necessarily need to be merged Benchmark run on: Macbook pro M3 max 48GB RAM --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Rootul P <[email protected]>
celestiaorg · Oct 21, 2024 · d4eb75e · d4eb75e
1 parent f217064
commit d4eb75e
Show file tree

Hide file tree

Showing 9 changed files with 2,133 additions and 13 deletions.
diff --git a/app/benchmarks/README.md b/app/benchmarks/README.md
@@ -0,0 +1,27 @@
+# Benchmarks
+
+This package contains benchmarks for the ABCI methods with the following transaction types:
+
+- Message send
+- IBC update client
+- PayForBlobs
+
+## How to run
+
+To run the benchmarks, run the following in the root directory:
+
+```shell
+go test -tags=bench_abci_methods -bench=<benchmark_name> app/benchmarks/benchmark_*
+```
+
+## Results
+
+The results are outlined in the [results](results.md) document.
+
+## Key takeaways
+
+We decided to softly limit the number of messages contained in a block, via introducing the `MaxPFBMessages` and `MaxNonPFBMessages`, and checking against them in prepare proposal.
+
+This way, the default block construction mechanism will only propose blocks that respect these limitations. And if a block that doesn't respect them reached consensus, it will still be accepted since this rule is not consensus breaking.
+
+As specified in [results](results.md) document, those results were generated on 16 core 48GB RAM machine, and gave us certain thresholds. However, when we run the same experiments on the recommended validator setup, 4 cores 16GB RAM, the numbers were lower. These low numbers are what we used in the limits.