Feature/ks 460/heartbeat

[patrickhuie19]

Requires

Supports

[github-actions]

I see you updated files related to core. Please run pnpm changeset in the root directory to add a changeset as well as in the text include at least one of the following tags:

• #added For any new functionality added.
• #breaking_change For any functionality that requires manual action for the node to boot.
• #bugfix For bug fixes.
• #changed For any change to the existing functionality.
• #db_update For any feature that introduces updates to database schema.
• #deprecation_notice For any upcoming deprecation functionality.
• #internal For changesets that need to be excluded from the final changelog.
• #nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
• #removed For any functionality/config that is removed.
• #updated For any functionality that is updated.
• #wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

[github-actions]

AER Report: CI Core

aer_workflow , commit , Detect Changes , Scheduled Run Frequency , Clean Go Tidy & Generate , Flakeguard Root Project / Get Tests To Run , lint , Core Tests (go_core_tests) , Core Tests (go_core_tests_integration) , Core Tests (go_core_ccip_deployment_tests) , Core Tests (go_core_race_tests) , Core Tests (go_core_fuzz) , Flakeguard Deployment Project , Flakeguard Root Project / Run Tests , Flakeguard Root Project / Report , Flakey Test Detection , SonarQube Scan

1. Error return value not checked: [Golang Lint]

Source of Error:

core/services/chainlink/application.go:255:11: Error return value of `cme.Emit` is not checked (errcheck)
cme.Emit(engCtx, "heartbeat")

**Why**: The `cme.Emit` function call does not check for an error return value, which is a common best practice to ensure that any potential errors are handled appropriately.

Suggested fix: Add error handling for the cme.Emit function call to ensure any errors are properly managed.

2. Variable naming issue: [Golang Lint]

Source of Error:

core/services/chainlink/application.go:86:7: var-naming: don't use ALL_CAPS in Go names; use CamelCase (revive)
const APPLICATION_HEARTBEAT_SECONDS = 1

**Why**: The constant `APPLICATION_HEARTBEAT_SECONDS` uses all caps, which is against Go naming conventions that prefer CamelCase.

Suggested fix: Rename the constant to ApplicationHeartbeatSeconds to follow Go naming conventions.

3. Test output mismatch: [Run tests]

Source of Error:

health_controller_test.go:183: 
Error Trace: /home/runner/work/chainlink/chainlink/core/web/health_controller_test.go:183
Error: Not equal: 
expected: "<style>\n details {\n margin: 0.0em 0.0em 0.0em 0.4em;\n padding: 0.3em 0.0em 0.0em 0.4em;\n }\n pre {\n margin-left:1em;\n margin-top: 0;\n }\n summary {\n padding-bottom: 0.4em;\n }\n details {\n border: thin solid black;\n border-bottom-color: rgba(0,0,0,0);\n border-right-color: rgba(0,0,0,0);\n }\n .passing:after {\n color: blue;\n content: \" - (Passing)\";\n font-size:small;\n text-transform: uppercase;\n }\n .failing:after {\n color: red;\n content: \" - (Failing)\";\n font-weight: bold;\n font-size:small;\n text-transform: uppercase;\n }\n summary.noexpand::marker {\n color: rgba(100,101,10,0);\n }\n</style>\n<details open>\n <summary title=\"\"><span class=\"\">Cosmos</span></summary>\n <details open>\n <summary title=\"\"><span class=\"\">Foo</span></summary>\n <details open>\n <summary title=\"Cosmos.Foo.Chain\" class=\"noexpand\"><span class=\"passing\">Chain</span></summary>\n </details>\n <details open>\n <summary title=\"Cosmos.Foo.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"Cosmos.Foo.Txm\" class=\"noexpand\"><span class=\"passing\">Txm</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">EVM</span></summary>\n <details open>\n <summary title=\"EVM.0\"><span class=\"passing\">0</span></summary>\n <details open>\n <summary title=\"EVM.0.BalanceMonitor\" class=\"noexpand\"><span class=\"passing\">BalanceMonitor</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.HeadBroadcaster\" class=\"noexpand\"><span class=\"passing\">HeadBroadcaster</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.HeadTracker\"><span class=\"passing\">HeadTracker</span></summary>\n <details open>\n <summary title=\"EVM.0.HeadTracker.HeadListener\"><span class=\"failing\">HeadListener</span></summary>\n <pre>Listener is not connected</pre>\n </details>\n </details>\n <details open>\n <summary title=\"EVM.0.LogBroadcaster\" class=\"noexpand\"><span class=\"passing\">LogBroadcaster</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm\"><span class=\"passing\">Txm</span></summary>\n <details open>\n <summary title=\"EVM.0.Txm.BlockHistoryEstimator\" class=\"noexpand\"><span class=\"passing\">BlockHistoryEstimator</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.Broadcaster\" class=\"noexpand\"><span class=\"passing\">Broadcaster</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.Confirmer\" class=\"noexpand\"><span class=\"passing\">Confirmer</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.Finalizer\" class=\"noexpand\"><span class=\"passing\">Finalizer</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.WrappedEvmEstimator\" class=\"noexpand\"><span class=\"passing\">WrappedEvmEstimator</span></summary>\n </details>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"HeadReporter\" class=\"noexpand\"><span class=\"passing\">HeadReporter</span></summary>\n</details>\n<details open>\n <summary title=\"JobSpawner\" class=\"noexpand\"><span class=\"passing\">JobSpawner</span></summary>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">Mailbox</span></summary>\n <details open>\n <summary title=\"Mailbox.Monitor\" class=\"noexpand\"><span class=\"passing\">Monitor</span></summary>\n </details>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">Mercury</span></summary>\n <details open>\n <summary title=\"Mercury.WSRPCPool\"><span class=\"passing\">WSRPCPool</span></summary>\n <details open>\n <summary title=\"Mercury.WSRPCPool.CacheSet\" class=\"noexpand\"><span class=\"passing\">CacheSet</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"PipelineORM\" class=\"noexpand\"><span class=\"passing\">PipelineORM</span></summary>\n</details>\n<details open>\n <summary title=\"PipelineRunner\"><span class=\"passing\">PipelineRunner</span></summary>\n <details open>\n <summary title=\"PipelineRunner.BridgeCache\" class=\"noexpand\"><span class=\"passing\">BridgeCache</span></summary>\n </details>\n</details>\n<details open>\n <summary title=\"RetirementReportCache\" class=\"noexpand\"><span class=\"passing\">RetirementReportCache</span></summary>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">Solana</span></summary>\n <details open>\n <summary title=\"\"><span class=\"\">Bar</span></summary>\n <details open>\n <summary title=\"Solana.Bar.Chain\" class=\"noexpand\"><span class=\"passing\">Chain</span></summary>\n </details>\n <details open>\n <summary title=\"Solana.Bar.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"Solana.Bar.Txm\" class=\"noexpand\"><span class=\"passing\">Txm</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">StarkNet</span></summary>\n <details open>\n <summary title=\"\"><span class=\"\">Baz</span></summary>\n <details open>\n <summary title=\"StarkNet.Baz.Chain\" class=\"noexpand\"><span class=\"passing\">Chain</span></summary>\n </details>\n <details open>\n <summary title=\"StarkNet.Baz.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"StarkNet.Baz.Txm\" class=\"noexpand\"><span class=\"passing\">Txm</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"TelemetryManager\" class=\"noexpand\"><span class=\"passing\">TelemetryManager</span></summary>\n</details>\n<details open>\n <summary title=\"WorkflowDBStore\" class=\"noexpand\"><span class=\"passing\">WorkflowDBStore</span></summary>\n</details>"
actual : "<style>\n details {\n margin: 0.0em 0.0em 0.0em 0.4em;\n padding: 0.3em 0.0em 0.0em 0.4em;\n }\n pre {\n margin-left:1em;\n margin-top: 0;\n }\n summary {\n padding-bottom: 0.4em;\n }\n details {\n border: thin solid black;\n border-bottom-color: rgba(0,0,0,0);\n border-right-color: rgba(0,0,0,0);\n }\n .passing:after {\n color: blue;\n content: \" - (Passing)\";\n font-size:small;\n text-transform: uppercase;\n }\n .failing:after {\n color: red;\n content: \" - (Failing)\";\n font-weight: bold;\n font-size:small;\n text-transform: uppercase;\n }\n summary.noexpand::marker {\n color: rgba(100,101,10,0);\n }\n</style>\n<details open>\n <summary title=\"\"><span class=\"\">Cosmos</span></summary>\n <details open>\n <summary title=\"\"><span class=\"\">Foo</span></summary>\n <details open>\n <summary title=\"Cosmos.Foo.Chain\" class=\"noexpand\"><span class=\"passing\">Chain</span></summary>\n </details>\n <details open>\n <summary title=\"Cosmos.Foo.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"Cosmos.Foo.Txm\" class=\"noexpand\"><span class=\"passing\">Txm</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">EVM</span></summary>\n <details open>\n <summary title=\"EVM.0\"><span class=\"passing\">0</span></summary>\n <details open>\n <summary title=\"EVM.0.BalanceMonitor\" class=\"noexpand\"><span class=\"passing\">BalanceMonitor</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.HeadBroadcaster\" class=\"noexpand\"><span class=\"passing\">HeadBroadcaster</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.HeadTracker\"><span class=\"passing\">HeadTracker</span></summary>\n <details open>\n <summary title=\"EVM.0.HeadTracker.HeadListener\"><span class=\"failing\">HeadListener</span></summary>\n <pre>Listener is not connected</pre>\n </details>\n </details>\n <details open>\n <summary title=\"EVM.0.LogBroadcaster\" class=\"noexpand\"><span class=\"passing\">LogBroadcaster</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm\"><span class=\"passing\">Txm</span></summary>\n <details open>\n <summary title=\"EVM.0.Txm.BlockHistoryEstimator\" class=\"noexpand\"><span class=\"passing\">BlockHistoryEstimator</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.Broadcaster\" class=\"noexpand\"><span class=\"passing\">Broadcaster</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.Confirmer\" class=\"noexpand\"><span class=\"passing\">Confirmer</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.Finalizer\" class=\"noexpand\"><span class=\"passing\">Finalizer</span></summary>\n </details>\n <details open>\n <summary title=\"EVM.0.Txm.WrappedEvmEstimator\" class=\"noexpand\"><span class=\"passing\">WrappedEvmEstimator</span></summary>\n </details>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"HeadReporter\" class=\"noexpand\"><span class=\"passing\">HeadReporter</span></summary>\n</details>\n<details open>\n <summary title=\"JobSpawner\" class=\"noexpand\"><span class=\"passing\">JobSpawner</span></summary>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">Mailbox</span></summary>\n <details open>\n <summary title=\"Mailbox.Monitor\" class=\"noexpand\"><span class=\"passing\">Monitor</span></summary>\n </details>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">Mercury</span></summary>\n <details open>\n <summary title=\"Mercury.WSRPCPool\"><span class=\"passing\">WSRPCPool</span></summary>\n <details open>\n <summary title=\"Mercury.WSRPCPool.CacheSet\" class=\"noexpand\"><span class=\"passing\">CacheSet</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"NodeHeartbeat\" class=\"noexpand\"><span class=\"passing\">NodeHeartbeat</span></summary>\n</details>\n<details open>\n <summary title=\"PipelineORM\" class=\"noexpand\"><span class=\"passing\">PipelineORM</span></summary>\n</details>\n<details open>\n <summary title=\"PipelineRunner\"><span class=\"passing\">PipelineRunner</span></summary>\n <details open>\n <summary title=\"PipelineRunner.BridgeCache\" class=\"noexpand\"><span class=\"passing\">BridgeCache</span></summary>\n </details>\n</details>\n<details open>\n <summary title=\"RetirementReportCache\" class=\"noexpand\"><span class=\"passing\">RetirementReportCache</span></summary>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">Solana</span></summary>\n <details open>\n <summary title=\"\"><span class=\"\">Bar</span></summary>\n <details open>\n <summary title=\"Solana.Bar.Chain\" class=\"noexpand\"><span class=\"passing\">Chain</span></summary>\n </details>\n <details open>\n <summary title=\"Solana.Bar.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"Solana.Bar.Txm\" class=\"noexpand\"><span class=\"passing\">Txm</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"\"><span class=\"\">StarkNet</span></summary>\n <details open>\n <summary title=\"\"><span class=\"\">Baz</span></summary>\n <details open>\n <summary title=\"StarkNet.Baz.Chain\" class=\"noexpand\"><span class=\"passing\">Chain</span></summary>\n </details>\n <details open>\n <summary title=\"StarkNet.Baz.Relayer\" class=\"noexpand\"><span class=\"passing\">Relayer</span></summary>\n </details>\n <details open>\n <summary title=\"StarkNet.Baz.Txm\" class=\"noexpand\"><span class=\"passing\">Txm</span></summary>\n </details>\n </details>\n</details>\n<details open>\n <summary title=\"TelemetryManager\" class=\"noexpand\"><span class=\"passing\">TelemetryManager</span></summary>\n</details>\n<details open>\n <summary title=\"WorkflowDBStore\" class=\"noexpand\"><span class=\"passing\">WorkflowDBStore</span></summary>\n</details>"

**Why**: The test `TestHealthController_Health_body` is failing because the actual output does not match the expected output. The difference is the presence of `NodeHeartbeat` in the actual output, which is not in the expected output.

Suggested fix: Update the expected output in the test to include NodeHeartbeat if it is a valid and expected part of the output. Otherwise, investigate why NodeHeartbeat is appearing and adjust the code or test setup accordingly.

4. CSA key does not exist: [Run tests]

Source of Error:

chainlink/application.go:832	[Feeds Service] Failed to start CSA key does not exist

**Why**: The Feeds Service failed to start because the required CSA key does not exist.

Suggested fix: Ensure that the CSA key is generated and available before starting the Feeds Service. This might involve adding setup steps in the test or ensuring the key is present in the environment.