Add document about how to benchmark the performance using Hyperledger…

… Caliper (#1238) Add document about how to benchmark the performance using Hyperledger Caliper Signed-off-by: takayuki-nagai <[email protected]> Co-authored-by: Tatsuya Sato <[email protected]>
hyperledger · Jul 17, 2024 · 50b69f6 · 50b69f6
1 parent 604a0a5
commit 50b69f6
Show file tree

Hide file tree

Showing 2 changed files with 206 additions and 0 deletions.
diff --git a/test-network-k8s/docs/CALIPER.md b/test-network-k8s/docs/CALIPER.md
@@ -0,0 +1,205 @@
+# Benchmarking the performance using Hyperledger Caliper
+
+This document introduces how to use [Hyperledger Caliper](https://hyperledger.github.io/caliper/) to benchmark the performance of the Hyperledger Fabric environment created with test-network-k8s.
+
+[Fabric adapter manual of Hyperledger Caliper v0.6.0](https://hyperledger.github.io/caliper/v0.6.0/fabric-config/new/) only describes how to connect to test-network. Furthermore, these chaincodes need to be executed as services to run in a K8s environment, but this is not supported by default, requiring customization. So we will explain how to benchmark the performance of the Kubernetes test network using Hyperledger Caliper and Asset Transfer Basic chaincode, which is most basic in current sample chaincodes.
+
+The following documentation assumes that test-network-k8s and Hyperledger Caliper v0.6.0 are located on the same host.
+
+## Setting of test-network-k8s side
+
+As described in the README of test-network-k8s, launch the network, create a channel, and deploy and invoke the basic-asset-transfer smart contract:
+
+```shell
+./network kind
+
+./network cluster init
+
+./network up
+
+./network channel create
+
+./network chaincode deploy asset-transfer-basic ../asset-transfer-basic/chaincode-java
+
+./network chaincode invoke asset-transfer-basic '{"Args":["InitLedger"]}'
+```
+
+REST API will not be used in the procedure described below, but the connection profile will be generated by launching it: 
+```shell
+./network rest-easy
+```
+
+## Setting of Hyperledger Caliper side
+
+Following [Install manual of Hyperledger Caliper v0.6.0](https://hyperledger.github.io/caliper/v0.6.0/installing-caliper/), install Hyperledger Caliper from npm:
+
+```shell
+git clone https://github.com/hyperledger/caliper-benchmarks.git
+cd caliper-benchmarks
+npm install --only=prod @hyperledger/[email protected]
+npx caliper bind --caliper-bind-sut fabric:fabric-gateway
+```
+
+Copy the connection profile created in test-network-k8s environment to Caliper environment.
+
+```shell
+cp <fabric-samples install dir>/test-network-k8s/build/fabric-rest-sample-config/HLF_CONNECTION_PROFILE_ORG1 networks/fabric/connection-profile.json
+```
+Replace `*.test-network.svc.cluster.local` with `*.localho.st` in "url" and "grpcOptions" section of connection-profile.json as below:
+
+```json
+    "peers": {
+        "org1-peers": {
+            "url": "grpcs://org1-peer1.localho.st:443",
+            "tlsCACerts": {
+                "pem": <contents of pem file>"
+            },
+            "grpcOptions": {
+                "ssl-target-name-override": "org1-peer1.localho.st",
+                "hostnameOverride": "org1-peer1.localho.st"
+            }
+        }
+    },
+```
+`*.localho.st` is wildcard domain defined for accessing K8s pod from external network via Nginx ingress controller. Please see [Working with Kubernetes](KUBERNETES.md) document for details.
+
+Open networks/fabric/test-network.yaml and edit it as below:
+
+
+```yaml
+name: Caliper Benchmarks
+version: "2.0.0"
+
+caliper:
+  blockchain: fabric
+
+channels:
+  # channelName of mychannel matches the name of the channel created by test network
+  - channelName: mychannel
+    # the chaincodeIDs of all the fabric chaincodes in caliper-benchmarks
+    contracts:
+    - id: fabcar
+    - id: fixed-asset
+    - id: marbles
+    - id: simple
+    - id: smallbank
+    - id: asset-transfer-basic
+
+organizations:
+  - mspid: Org1MSP
+    # Identities come from cryptogen created material for test-network
+    identities:
+      certificates:
+      - name: 'User1'
+        clientPrivateKey:
+          path: '<fabric-samples install absolute dir>/test-network-k8s/build/enrollments/org1/users/org1admin/msp/keystore/key.pem'
+
+        clientSignedCert:
+          path: '<fabric-samples install absolute dir>/test-network-k8s/build/enrollments/org1/users/org1admin/msp/signcerts/cert.pem'
+
+    connectionProfile:
+      path: 'networks/fabric/connection-profile.json'
+      discover: true
+```
+
+Currently, sample code for running a performance benchmark targeting asset-transfer-basic is not published on the Caliper repository. Therefore, use the sample code published in [Caliper's user manual to build a test workload.](https://hyperledger.github.io/caliper/vNext/fabric-tutorial/tutorials-fabric-existing/)
+
+
+As shown in "Step 3" of the above document, create workload/readAsset.js file and edit it as below:
+
+
+```javascript
+'use strict';
+
+const { WorkloadModuleBase } = require('@hyperledger/caliper-core');
+
+class MyWorkload extends WorkloadModuleBase {
+    constructor() {
+        super();
+    }
+
+    async initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext) {
+        await super.initializeWorkloadModule(workerIndex, totalWorkers, roundIndex, roundArguments, sutAdapter, sutContext);
+
+        for (let i=0; i<this.roundArguments.assets; i++) {
+            const assetID = `${this.workerIndex}_${i}`;
+            console.log(`Worker ${this.workerIndex}: Creating asset ${assetID}`);
+            const request = {
+                contractId: this.roundArguments.contractId,
+                contractFunction: 'CreateAsset',
+                invokerIdentity: 'User1',
+                contractArguments: [assetID,'blue','20','penguin','500'],
+                readOnly: false
+            };
+
+            await this.sutAdapter.sendRequests(request);
+        }
+    }
+
+    async submitTransaction() {
+        const randomId = Math.floor(Math.random()*this.roundArguments.assets);
+        const myArgs = {
+            contractId: this.roundArguments.contractId,
+            contractFunction: 'ReadAsset',
+            invokerIdentity: 'User1',
+            contractArguments: [`${this.workerIndex}_${randomId}`],
+            readOnly: true
+        };
+
+        await this.sutAdapter.sendRequests(myArgs);
+    }
+
+    async cleanupWorkloadModule() {
+        for (let i=0; i<this.roundArguments.assets; i++) {
+            const assetID = `${this.workerIndex}_${i}`;
+            console.log(`Worker ${this.workerIndex}: Deleting asset ${assetID}`);
+            const request = {
+                contractId: this.roundArguments.contractId,
+                contractFunction: 'DeleteAsset',
+                invokerIdentity: 'User1',
+                contractArguments: [assetID],
+                readOnly: false
+            };
+
+            await this.sutAdapter.sendRequests(request);
+        }
+    }
+}
+
+function createWorkloadModule() {
+    return new MyWorkload();
+}
+
+module.exports.createWorkloadModule = createWorkloadModule;
+```
+
+As shown in "Step 4" of the above document, create benchmarks/myAssetBenchmark.yaml file and edit it as below:
+
+
+```yaml
+test:
+    name: basic-contract-benchmark
+    description: test benchmark
+    workers:
+      number: 2
+    rounds:
+      - label: readAsset
+        description: Read asset benchmark
+        txDuration: 30
+        rateControl:
+          type: fixed-load
+          opts:
+            transactionLoad: 2
+        workload:
+          module: workload/readAsset.js
+          arguments:
+            assets: 10
+            contractId: asset-transfer-basic
+```
+
+After creating the above files, you can run a performance benchmark with the following command, which will create some assets and measure the time it takes to reference the assets.
+
+```shell
+
+npx caliper launch manager --caliper-workspace . --caliper-benchconfig benchmarks/myAssetBenchmark.yaml --caliper-networkconfig networks/fabric/test-network.yaml
+```
diff --git a/test-network-k8s/docs/README.md b/test-network-k8s/docs/README.md
@@ -44,3 +44,4 @@ _Chaincode-as-a-Service_ running in a shared Kubernetes namespace.
 - [Debugging Chaincode](CHAINCODE_AS_A_SERVICE.md)
 - [Working with Applications](APPLICATIONS.md)
 - [High Availability](HIGH_AVAILABILITY.md)
+- [Benchmarking the performance using Hyperledger Caliper](CALIPER.md)