From ab5972a15967d3d7a6340837dad15fb7afb68ad8 Mon Sep 17 00:00:00 2001
From: Manu Sridharan <msridhar@gmail.com>
Date: Wed, 12 Jun 2024 16:27:51 -0700
Subject: [PATCH] Reenable JMH benchmarking in a safer manner (#975)

Fixes #968

The key difference is now the benchmarking job only starts when the
label with the name `run-benchmarks` is added to a PR. According to [the
docs](https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels#applying-a-label)
only those with triage access to the repository can add or remove a
label. In contrast, anyone can comment on an issue, which made our
previous technique for kick-starting the benchmarks unsafe. Before
adding the `run-benchmarks` label, a PR should be reviewed to check for
malicious code.

It is impossible to test this workflow without first merging it to the
main branch. However, I did test it using a PR on my fork, and confirmed
it could comment back the benchmark results like before:

https://github.com/msridhar/NullAway/pull/12

After merging this PR we'll also have to add some credentials to the
main NullAway repo to make this work. But first we should review, land,
and then see that it fails as expected without the credentials.

Compared to [the earlier workflow
file](https://github.com/uber/NullAway/blob/87ec10d4f26630d4bb91aefe5ff7c0fc181f030a/.github/workflows/jmh-benchmark.yml),
beyond changing to use labeling, I updated the versions of some external
GitHub actions we are using.
---
 .github/workflows/jmh-benchmark.yml | 68 +++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
 create mode 100644 .github/workflows/jmh-benchmark.yml

diff --git a/.github/workflows/jmh-benchmark.yml b/.github/workflows/jmh-benchmark.yml
new file mode 100644
index 0000000000..9e59a448e1
--- /dev/null
+++ b/.github/workflows/jmh-benchmark.yml
@@ -0,0 +1,68 @@
+# This GitHub Actions workflow runs JMH benchmarks when requested
+name: Run JMH Benchmarks for Pull Request
+
+on:
+  pull_request: # Trigger when a label is added.  Only contributors have permissions to add labels
+    types: [labeled]
+
+# Only allow one instance of JMH benchmarking to be running at any given time
+concurrency: all
+
+jobs:
+  benchmarking:
+    # Only run this job if the label is 'run-benchmarks' and the PR in on the uber/NullAway repository
+    if: github.event.label.name == 'run-benchmarks' && github.repository == 'uber/NullAway'
+    runs-on: ubuntu-latest 
+    permissions: write-all 
+
+    steps:
+
+    - name: Checkout repository 
+      uses: actions/checkout@v4
+
+    - name: Set branch name
+      env:
+        GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} 
+      run: |
+        chmod +x ./.github/workflows/get_repo_details.sh 
+        ./.github/workflows/get_repo_details.sh "${{ secrets.GITHUB_TOKEN }}" "${{ github.event.number }}" "${{ github.repository }}"
+
+    - id: 'auth'
+      name: Authenticating 
+      uses: google-github-actions/auth@v2
+      with:
+        credentials_json: '${{ secrets.GCP_SA_KEY_1 }}' 
+
+    - name: Set up Google Cloud SDK 
+      uses: google-github-actions/setup-gcloud@v2
+      
+    - name: Start VM
+      run: gcloud compute instances start nullway-jmh --zone=us-central1-a
+      
+    - name: Run benchmarks 
+      run: |
+        chmod +x ./.github/workflows/run_gcp_benchmarks.sh
+        ./.github/workflows/run_gcp_benchmarks.sh 
+        
+    - name: Cleanup
+      # Delete the branch directory on the Google Cloud instance
+      if: always()
+      run: |
+        ./.github/workflows/gcloud_ssh.sh " export BRANCH_NAME=${BRANCH_NAME} && rm -r -f $BRANCH_NAME"
+
+    - name: Formatting Benchmark # Create a text file containing the benchmark results
+      run:  |
+        (echo 'Main Branch:'; echo '```' ; cat main_text.txt; echo '```'; echo 'With This PR:'; echo '```' ; cat pr_text.txt; echo '```') > benchmark.txt
+   
+    - name: Comment Benchmark
+      uses: mshick/add-pr-comment@v2
+      if: always() # This step is for adding the comment
+      with:         
+        message-path: benchmark.txt  # The path to the message file to leave as a comment
+        message-id: benchmark
+    - name: Stop VM
+      if: always()
+      run: gcloud compute instances stop nullway-jmh --zone=us-central1-a
+
+      
+