Tracking: Support indexer/caching service in Zebra

[arya2]

Motivation

The indexer needs to notify clients when there are changes to the mempool or best chain with affected transactions.

This is more efficient than polling to check for changes, especially from the indexer's perspective, and has better usability for clients.

Design

Add a method to ChainTipChange in zebra-state for getting changes to the nullifier set, and a tonic IndexerRpcServer server for streaming responses to clients with server-sent events.

Implementation Plan

Preperation:

• feat(zebra-state): Send a reference to latest non-finalized state in TipAction::Reset #7927
• Add a tonic RPC server to zebra-rpc with a read state service #8618

Indexing:

• Add a column family to zebra-state tracking [spent Outpoint] -> [transaction::Hash that spent the output] #8837
• Add hashes of transactions that reveal a nullifier in the {sprout, sapling, orchard}_nullifiers column families as values #8838

RPC methods:

• Add a chain_tip_change() RPC method for listening to chain tip changes #8619
• Add a mempool_transactions() RPC method for listening to changes in the mempool #8620
• Add a nullifiers() RPC method for getting new nullifier for a block hash or height #8621
• Copy Zebra RPC methods that provide required information to the tonic/indexer RPC server #8622

Index Server / Cacher:

• Add an init function for a standalone ReadStateService #8520
• Add a function for syncing blocks from Zebra's non-finalized best chain to a ReadStateService in another process #8576
• Add a current_tip() or similar to ReadState #8580
• Add a zebra-indexer binary crate that starts a read state with a syncer from the sync module in zebra-rpc and starts a tonic RPC server #8623
• Use the new RPC methods to track changes to the mempool in zebra-indexer #8628

Optional:

• Update TrustedChainSync to use the new RPC method to be notified when there’s a chain tip change or a change to the mempool
• Update zebra-scan to use RPC methods instead of a synced read state

[conradoplg]

I think this might be missing some of the notifications we need, namely when transactions are available and blocks are validated. See zcash/librustzcash#1395 (comment)