feat: migration("re-indexing"), backfilling and diasgnostics tooling for the ChainIndexer

CHANGELOG.md

@@ -1,8 +1,10 @@
 # Lotus changelog
 
 # UNRELEASED
+ (`ChainIndexer`) to index Filecoin chain state such as tipsets, messages, events and ETH transactions for accurate and faster RPC responses. The `ChainIndexer` replaces the existing `MsgIndex`, `EthTxIndex` and `EventIndex` implementations in Lotus, which [suffer from a multitude of known problems](https://github.com/filecoin-project/lotus/issues/12293).  If you are an RPC provider/node operator, please refer to the [ChainIndexer documentation for RPC providers](TODO: URL) for information on how to enable, configure and use the new Indexer.
 Add `EthGetBlockReceipts` RPC method to retrieve transaction receipts for a specified block. This method allows users to obtain Ethereum format receipts of all transactions included in a given tipset as specified by its Ethereum block equivalent. ([filecoin-project/lotus#12478](https://github.com/filecoin-project/lotus/pull/12478))
 
+
 ## ☢️ Upgrade Warnings ☢️
 
 - Minimum go-version been updated to v1.22.7 ([filecoin-project/lotus#12459](https://github.com/filecoin-project/lotus/pull/12459))

api/api_full.go

@@ -63,6 +63,36 @@ type FullNode interface {
 	Common
 	Net
 
+	// MethodGroup: ChainIndexer
+	// The ChainIndexer method group contains methods for interacting with the chain indexer.
+	//
+	// The `ChainValidateIndex` API serves multiple purposes:
+	//
+	// 1. Validates the chain index at a specific epoch:
+	//   - Ensures consistency between indexed data and actual chain state
+	//   - Reports any errors found during validation (i.e. the indexed data does not match the actual chain state, missing data, etc.)
+	//
+	// 2. Optionally backfills missing data:
+	//   - Backfills data if the index is missing information for the specified epoch
+	//   - Backfilling only occurs when the `backfill` parameter is set to `true`
+	//
+	// 3. Detects "holes" in the index:
+	//   - If `backfill` is `false` and the index lacks data for the specified epoch, the API returns an error indicating missing data
+	//
+	// ChainValidateIndex validates the integrity of the chain index at a specified epoch and also optionally backfills missing data.
+	//
+	// Parameters:
+	//   - epoch: The specific chain epoch for which to validate/backfill the index.
+	//   - backfill: A boolean flag indicating whether to attempt backfilling of missing data if the index does not have data for the
+	//               specified epoch.
+	//
+	// Returns:
+	//   - *types.IndexValidation: A pointer to an IndexValidation struct containing the results of the validation/backfill.
+	//   - error: An error object if the validation/backfill fails. The error message will contain details about the index
+	//            corruption if the call fails because of an incosistency between indexed data and the actual chain state.
+	//            Note: The API returns an error if the index does not have data for the specified epoch and backfill is set to false.
+	ChainValidateIndex(ctx context.Context, epoch abi.ChainEpoch, backfill bool) (*types.IndexValidation, error) //perm:write
+
 	// MethodGroup: Chain
 	// The Chain method group contains methods for interacting with the
 	// blockchain, but that do not require any form of state computation.