feat: improve scraper startup error handling

[kamiyaa]

Description

• handle errors and sets set chain metrics if scraper fails for chain during startup

Drive-by changes

• implement Clone for ScraperDb
  • this is because DbConn doesn't implement Clone when mock feature is enabled

Related issues

Fixes https://github.com/hyperlane-xyz/issues/issues/1408

Backward compatibility

Yes

Testing

Added unittest to test function. But don't have tests in place for testing the logic in Scraper::run()

[changeset-bot]

⚠️ No Changeset found

Latest commit: 6325c65

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.53%. Comparing base (9eb19ca) to head (6325c65).
Report is 12 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5191   +/-   ##
=======================================
  Coverage   77.53%   77.53%           
=======================================
  Files         103      103           
  Lines        2110     2110           
  Branches      190      190           
=======================================
  Hits         1636     1636           
  Misses        453      453           
  Partials       21       21           

Components	Coverage Δ
core	87.80% <ø> (ø)
hooks	79.39% <ø> (ø)
isms	83.68% <ø> (ø)
token	91.27% <ø> (ø)
middlewares	79.80% <ø> (ø)

[ameten]

I have a concern regarding possible situation when we have scraper tasks running for a domain while getting ChainConf and MetricsUpdater fails. We shall have a scraper running without proper observability.

[kamiyaa]

So would you recommend rearranging the order like so?

build chain_conf
build metrics_updater
build scraper task