trie-db: Panic on database missing expected key

[lexnv]

Kusama validator panics on:

polkadot-sdk/substrate/primitives/state-machine/src/ext.rs

Lines 168 to 174 in 3906c57

	fn storage(&mut self, key: &[u8]) -> Option<StorageValue> {
	let _guard = guard();
	let result = self
	.overlay
	.storage(key)
	.map(|x| x.map(|x| x.to_vec()))
	.unwrap_or_else(|| self.backend.storage(key).expect(EXT_NOT_ALLOWED_TO_FAIL));

Version deployed: version 1.16.1-ca8beaed148

Possibly related log

	
2024-11-21 13:19:44.659  WARN tokio-runtime-worker trie-cache: Timeout while trying to acquire a write lock for the shared trie cache

Details

Grafana logs.

Thread 'tokio-runtime-worker' panicked at 'Externalities not allowed to fail within runtime: "Trie lookup error: Database missing expected key: 0xd2a41f821b81b2c30f351ace095dedd755f8b7c6bf189b58905b9903f16c1074"', /builds/parity/mirrors/polkadot-sdk/substrate/primitives/state-machine/src/ext.rs:174
This is a bug. Please report it at:
	https://github.com/paritytech/polkadot-sdk/issues/new

@paritytech/sdk-node

[Anastasiia-Khab]

Same here #6547