v1.6.3

Substreams fixes

• Fix "cannot resolve 'old cursor' from files in passthrough mode" error on some requests with an old cursor
• Fix handling of 'special case' substreams module with only "params" as its input: should not skip this execution (used in graph-node for head tracking)
  -> empty files in module cache with hash d3b1920483180cbcd2fd10abcabbee431146f4c8 should be deleted for consistency
• Fix bug where some invalid cursors may be sent (with 'LIB' being above the block being sent) and add safeguard/loggin if the bug appears again
• Fix panic in the whole tier2 process when stores go above the size limit while being read from "kvops" cached changes

Core fixes

• fix: reader-node-stdin not shutting down after receiving an EOF

v1.6.2

Core

• [Operator] The flag --advertise-block-id-encoding now accepts shorter form: hex, base64, etc. The older longer form BLOCK_ID_ENCODING_HEX is still supported but we suggested using the shorter form from now on.

Substreams v1.10.2

Note Since a bug that affected substreams with "skipping blocks" was corrected in this release, any previously produced substreams cache should be considered as possibly corrupted and be eventually replaced

• Substreams: fix bad handling of modules with multiple inputs when only one of them is filtered, resulting in bad outputs in production-mode.
• Substreams: fix stalling on some substreams with stores and mappers with different start block numbers on the same stage
• Substreams: fix 'development mode' and LIVE mode executing some modules that should be skipped

v1.6.1

• Bump substreams to v1.10.0: Version 1.10.0 adds a new EndpointInfo/Info endpoint, introduces a 3-minute default execution timeout per block, updates metering metrics with a deprecation warning, enhances substreams init commands, and improves wasm module caching and Prometheus tool flexibility. Full changelog: https://github.com/streamingfast/substreams/releases/tag/v1.10.0
• Metering update: more detailed metering with addition of new metrics. DEPRECATION WARNING: bytes_read and bytes_written metrics will be removed in the future, please use the new metrics for metering instead

v1.6.0

• Add sf.firehose.v2.EndpointInfo/Info service on Firehose and sf.substreams.rpc.v2.EndpointInfo/Info to Substreams endpoints. This involves the following new flags:

  • advertise-chain-name Canonical name of the chain according to https://thegraph.com/docs/en/developing/supported-networks/ (required, unless it is in the "well-known" list)
  • advertise-chain-aliases Alternate names for that chain (optional)
  • advertise-block-features List of features describing the blocks (optional)
  • advertise-block-id-encoding Encoding format of the block ID [BLOCK_ID_ENCODING_BASE58, BLOCK_ID_ENCODING_BASE64, BLOCK_ID_ENCODING_BASE64URL, BLOCK_ID_ENCODING_HEX, BLOCK_ID_ENCODING_0X_HEX] (required, unless the block type is in the "well-known" list)
  • ignore-advertise-validation Runtime checks of chain name/features/encoding against the genesis block will no longer cause server to wait or fail.

• Add a well-known list of chains (hard-coded in wellknown/chains.go to help automatically determine the 'advertise' flag values). Users are encouraged to propose Pull Requests to add more chains to the list.

• The new info endpoint adds a mandatory fetching of the first streamable block on startup, with a failure if no block can be fetched after 3 minutes and you are running firehose or substreams-tier1 service.
  It validates the following on a well-known chain:

  • if the first-streamable-block Num/ID match the genesis block of a known chain, e.g. matic, it will refuse another value for advertise-chain-name than matic or one of its aliases (polygon)
  • If the first-streamable-block does not match any known chain, it will require the advertise-chain-name to be non-empty
  • If the first-streamable-block type is unknown (i.e. not ethereum, solana, near, cosmos, bitcoin...), it will require the user to provide advertise-chain-name as well as advertise-block-id-encoding

• Substreams: add --common-tmp-dir flag to activate local caching of pre-compiled WASM modules through wazero v1.8.0 feature (performance improvement on WASM compilation)

• Substreams: revert module hash calculation from v1.5.5, when using a non-zero firstStreamableBlock. Hashes will now be the same even if the chain's first streamable block affects the initialBlock of a module.

• Substreams: add --substreams-block-execution-timeout flag (default 3 minutes) to prevent requests stalling

v1.5.7

• Bump substreams to v1.9.3: fix high CPU usage on tier1 caused by a bad error handling

v1.5.6

• Bump substreams to v1.9.2: Prevent Noop handler from sending outputs with 'Stalled' step in cursor (which breaks substreams-sink-kv)
• add --reader-node-line-buffer-size flag and bump default value from 100M to 200M to go over crazy block 278208000 on Solana

v1.5.5

• added well known type for starknet and cosmos
• Fixed a bug in substreams where chains with non-zero first-streamable-block would cause some substreams to hang. Solution changes the 'cached' hashes for those substreams.

v1.5.4

Substreams bumped to v1.9.0

Important Substreams BUG FIX

• Fix a bug introduced in v1.6.0 that could result in corrupted store "state" file if all
  the "outputs" were already cached for a module in a given segment (rare occurence)
• We recommend clearing your substreams cache after this upgrade and re-processing or
  validating your data if you use stores.

Added

• Expose a new intrinsic to modules: skip_empty_output, which causes the module output to be skipped if it has zero bytes. (Watch out, a protobuf object with all its default values will have zero bytes)
• Improve schedule order (faster time to first block) for substreams with multiple stages when starting mid-chain

v1.5.3

• fix "hub" not recovering on certain disconnections in relayer/firehose/substreams (scenarios requiring full restart)

v1.5.2

Substreams changes

• Added substreams back-filler to populate cache for live requests when the blocks become final
• Fixed: truncate very long details on error messages to prevent them from disappearing when behind a (misbehaving) load-balancer