Insert time interval based heartbeats in BHS store

[kidambisrinivas]

Motivation

• VRF Incident SlowVRFv2Fulfillment: [POLYGON-MAINNET] [VRFv2] A randomness request with keyhash "0x.." has not been fulfilled for 28403 blocks. StalledVRFv2Fulfillment: [POLYGON-MAINNET] [VRFv2] A randomness request for "0x.." (0xgw..) has stalled, there has not been a fullfilment for 28403 blocks [Incident #649766]

Solution

• A redundant method by which we insert blockhashes into the BHS store at a regular basis irrespective of whether there are VRF requests going on or not, in case the log poller-based BHS has problems and can no longer read logs / the chain and we need to run a backwards mode.
• It seems that it might be best to have a wall-clock timer rather than a block-timer for this

Testing

$ cd chainlink
$ go test -count=1 ./core/services/blockhashstore/...
ok      github.com/smartcontractkit/chainlink/v2/core/services/blockhashstore   1.878s

$ go test -count=1 ./core/services/job/...
?       github.com/smartcontractkit/chainlink/v2/core/services/job/mocks        [no test files]
ok      github.com/smartcontractkit/chainlink/v2/core/services/job      3.379s

$ go test -count=1 ./core/web/presenters/...
ok      github.com/smartcontractkit/chainlink/v2/core/web/presenters    0.515s

$ go test -count=1 ./core/web/resolver/...
ok      github.com/smartcontractkit/chainlink/v2/core/web/resolver      1.129s

$ go test -count=1 -v -run="^TestStartHeartbeats$" ./core/services/vrf/v2/...
...
--- PASS: TestStartHeartbeats (6.37s)
    --- PASS: TestStartHeartbeats/bhs_feeder_startheartbeats_happy_path (6.14s)
PASS
ok      github.com/smartcontractkit/chainlink/v2/core/services/vrf/v2   7.248s

[github-actions]

I see that you haven't updated any README files. Would it make sense to do so?

[jinhoonbang]

I think we might need to update job ORM, migrations and web view layer. Here's an example commit that adds a new field (fromAddresses) to a BHS job spec - aeb2c16

[makramkd]

Note that you also need to add the appropriate database migration for the new field. See https://github.com/smartcontractkit/chainlink/pull/8487/files as an example for the BHS job.

[kidambisrinivas]

Note that you also need to add the appropriate database migration for the new field

Done

[jinhoonbang]

I think we need to changes to job/orm.go as well.

[cl-sonarqube-production]

SonarQube Quality Gate

0.0% Coverage on New Code (is less than 80%)

See analysis details on SonarQube