subscription stats lost on shutdown

[BlinkyStitt]

2023-06-27T03:24:16.676481Z  WARN web3_proxy::compute_units: unknown method eth_subscribe(newHeads)
  at web3_proxy/src/compute_units.rs:100

^C  2023-06-27T03:24:17.474301Z  INFO web3_proxy_cli::proxyd: quiting from ctrl-c
  at web3_proxy/src/bin/web3_proxy_cli/proxyd.rs:176

2023-06-27T03:24:17.477851Z  INFO web3_proxy_cli::proxyd: frontend exited gracefully
  at web3_proxy/src/bin/web3_proxy_cli/proxyd.rs:215

2023-06-27T03:24:17.477940Z  INFO web3_proxy_cli::proxyd: waiting on 1 important background tasks
  at web3_proxy/src/bin/web3_proxy_cli/proxyd.rs:223

2023-06-27T03:24:17.478039Z  INFO web3_proxy::stats::stat_buffer: stat_loop shutting down
  at web3_proxy/src/stats/stat_buffer.rs:156

2023-06-27T03:24:17.485357Z  INFO web3_proxy::stats::stat_buffer: saved 1 pending relational stat(s)
  at web3_proxy/src/stats/stat_buffer.rs:167

2023-06-27T03:24:17.488919Z  INFO web3_proxy::stats::stat_buffer: saved 2 pending tsdb stat(s)
  at web3_proxy/src/stats/stat_buffer.rs:171

2023-06-27T03:24:17.488972Z  INFO web3_proxy::stats::stat_buffer: accounting and stat save loop complete
  at web3_proxy/src/stats/stat_buffer.rs:173

2023-06-27T03:24:17.489101Z  INFO web3_proxy_cli::proxyd: finished
  at web3_proxy/src/bin/web3_proxy_cli/proxyd.rs:247

2023-06-27T03:24:17.490177Z ERROR web3_proxy::frontend::authorization: failed sending stat RpcQuery(RpcQueryStats { chain_id: 137, authorization: Authorization { checks: AuthorizationChecks { user_id: 0, latest_balance: RwLock { data: Balance { total_deposit: 0, total_spend: 0 } }, rpc_secret_key: None, rpc_secret_key_id: None, max_requests_per_period: None, max_concurrent_requests: None, allowed_origins: None, allowed_referers: None, allowed_user_agents: None, allowed_ips: None, log_revert_chance: 100, private_txs: false, proxy_mode: Best }, db_conn: Some(SqlxMySqlPoolConnection), ip: 127.0.0.1, origin: None, referer: None, user_agent: Some(UserAgent("llamanodes_web3_proxy/v0.31.0")), authorization_type: Internal }, method: "eth_getBlockByNumber", archive_request: true, error_response: true, request_bytes: 87, backend_rpcs_used: [Web3Rpc { name: "nodeifi_05", blocks: 64, .. }], response_bytes: 0, response_millis: 25, response_timestamp: 1687836257, compute_unit_cost: 0.000015999999999999990000 }): "SendError(..)"

i think the subscriptions close after the stats buffer exits.

• need the websockets to select on the shutdown sender.
• need the stats sender to stay open until all the websockets have closed