Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cla debugger docs #12160

Merged
merged 9 commits into from
Feb 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 63 additions & 24 deletions core/scripts/chaincli/DEBUGGING.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,46 @@
## Automation Debugging Script
# Automation Debugging Script

### Context
Use this script to debug and diagnose possible issues with registered upkeeps in Automation v2 registries. The script can debug custom logic upkeeps, log-trigger upkeeps, and upkeeps that use StreamsLookup.

The debugging script is a tool within ChainCLI designed to facilitate the debugging of upkeeps in Automation v21, covering both conditional and log-based scenarios.

### Setup
## Setup

Before starting, you will need:
1. Git clone this chainlink [repo](https://github.com/smartcontractkit/chainlink)
2. A working [Go](https://go.dev/doc/install) installation
2. Change directory to `core/scripts/chaincli` and create a `.env` file based on the example `.env.debugging.example`

### Configuration in `.env` File
- A registered [upkeep](https://docs.chain.link/chainlink-automation/overview/getting-started)
- A working [Go](https://go.dev/doc/install) installation, please use this Go [version](https://github.com/smartcontractkit/chainlink/blob/develop/go.mod#L3)

1. Clone the chainlink [repo](https://github.com/smartcontractkit/chainlink) and navigate to the `core/scripts/chaincli`
directory:
```
git clone https://github.com/smartcontractkit/chainlink.git && cd chainlink/core/scripts/chaincli
```
1. Create a `.env` file based on the example `.env.debugging.example`:

```
cp .env.debugging.example .env
```

#### Mandatory Fields
## Configuration

Ensure the following fields are provided in your `.env` file:
Fill in the values for these mandatory fields in your `.env` file:

- `NODE_URL`: Archival node URL
- `KEEPER_REGISTRY_ADDRESS`: Address of the Keeper Registry contract. Refer to the [Supported Networks](https://docs.chain.link/chainlink-automation/overview/supported-networks#configurations) doc for addresses.
- `NODE_URL`: Archival node URL for the network to "simulate" the upkeep. Use your own node or get an endpoint from Alchemy or Infura.
- `KEEPER_REGISTRY_ADDRESS`: Address of the registry where your upkeep is registered. Refer to the [Supported Networks](https://docs.chain.link/chainlink-automation/overview/supported-networks#configurations) doc for registry addresses.

For example
![Example_ENV_file](/core/scripts/chaincli/images/env_file_example.png "Example .ENV file")

#### Optional Fields (Streams Lookup)
#### StreamsLookup (optional)

If your targeted upkeep involves streams lookup, please provide the following details. If you are using Data Streams v0.3 (which is likely), only provide the DATA_STREAMS_URL. The DATA_STREAMS_LEGACY_URL is specifically for Data Streams v0.2.
If your targeted upkeep involves StreamsLookup, please provide the following details. If you are using Data Streams v0.3 (which is likely), only provide the `DATA_STREAMS_URL`. Ignore `DATA_STREAMS_LEGACY_URL`.

- `DATA_STREAMS_ID`
- `DATA_STREAMS_KEY`
- `DATA_STREAMS_LEGACY_URL`
- `DATA_STREAMS_URL`

#### Optional Fields (Tenderly Integration)
#### Tenderly integration (optional)

For detailed transaction simulation logs, set up Tenderly credentials. Refer to the [Tenderly Documentation](https://docs.tenderly.co/other/platform-access/how-to-generate-api-access-tokens) for creating an API key, account name, and project name.
For detailed transaction simulation logs, set up Tenderly credentials. Refer to the [Tenderly documentation](https://docs.tenderly.co/other/platform-access/how-to-generate-api-access-tokens) to learn how to create an API key, account name, and project name on Tenderly.

- `TENDERLY_KEY`
- `TENDERLY_ACCOUNT_NAME`
Expand All @@ -41,25 +50,26 @@ For detailed transaction simulation logs, set up Tenderly credentials. Refer to

Execute the following command based on your upkeep type:

- For conditional upkeep, if a block number is given we use that block, otherwise we use the latest block:
- For custom logic:

```bash
go run main.go keeper debug UPKEEP_ID [OPTIONAL BLOCK_NUMBER]
```
If you don't specify a block number, the debugging script uses the latest block.

- For log trigger upkeep:

```bash
go run main.go keeper debug UPKEEP_ID TX_HASH LOG_INDEX
```

### Checks Performed by the Debugging Script
### What the debugging script checks

1. **Fetch and Sanity Check Upkeep:**
1. The script runs these basic checks on all upkeeps based on the TX_HASH or BLOCK_NUMBER (if provided)
- Verify upkeep status: active, paused, or canceled
- Check upkeep balance

2. **For Conditional Upkeep:**
2. **For Custom Logic Upkeep:**
- Check conditional upkeep
- Simulate `performUpkeep`

Expand All @@ -74,7 +84,7 @@ Execute the following command based on your upkeep type:

- Simulate `performUpkeep`

### Examples
#### Examples
- Eligible and log trigger based and using mercury lookup v0.3:

```bash
Expand All @@ -92,4 +102,33 @@ Execute the following command based on your upkeep type:
```bash
go run main.go keeper debug 5591498142036749453487419299781783197030971023186134955311257372668222176389 0xc0686ae85d2a7a976ef46df6c613517b9fd46f23340ac583be4e44f5c8b7a186 1
```
---
### Common issues with Upkeeps and how to resolve them

#### All upkeeps

- Upkeep is underfunded
- Underfunded upkeeps will not perform. Fund your upkeep in the Automation [app](https://automation.chain.link/)
- Upkeep is paused
- Unpause your upkeep in the Automation [app](https://automation.chain.link/)
- Insufficient check gas
- There is a limit of 10,000,000 (as per Automation v2) on the amount of gas that can be used to "simulate" your `checkUpkeep` function.
- To diagnose if your upkeep is running out of check gas, you will need to enable the Tenderly options above and then open the simulation link once you run the script.
- ![Insufficient Check Gas](/core/scripts/chaincli/images/insufficient_check_gas.png "Open the Tenderly simulation and switch to debug mode")
- ![Out of Gas](/core/scripts/chaincli/images/tenderly_out_of_check_gas.png "Tenderly shows checkUpkeeps has consumed all available gas and is now out of gas")
- You will need to adjust your checkUpkeep to consume less gas than the limit
- Insufficient perform gas
- Your upkeep's perform transaction uses more gas than you specified
- ![Insufficient Perform Gas](/core/scripts/chaincli/images/insufficient_perform_gas.png "Insufficient perform gas")
- Use the Automation [app](https://automation.chain.link/) and increase the gas limit of your upkeep
- The maximum supported perform gas is 5,000,000

#### Log-trigger upkeeps

Log-trigger upkeeps require that you also supply the txn hash containing the log and the index of the log that would have triggered your upkeep. You can find both in the block scanner of the chain in question. For example the txn hash is in the URL and the block number in the green circle on the left.
![Txn Hash and Log Index Number](/core/scripts/chaincli/images/txnHash_and_index.png "Find txn hash and log index in block scanner")

- Log doesn't match the trigger config
- Log-trigger upkeeps come with a filter (aka trigger config), if the emitted log doesn't match the filter, the upkeep won't run.
- ![Log doesn't match](/core/scripts/chaincli/images/log_trigger_log_doesnt_match.png "Log doesn't match trigger config")
- Use the Automation [app](https://automation.chain.link/) to update the upkeep's trigger config to match the log.
---
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading