cardano-testnet: Waiting for blocks using EpochStateView.

[carbolymer]

Description

This PR provides new functions allowing to extract slot number and block number from epoch state view, and additionally wait for epoch state view updates. This cuts down the test execution time a bit. Waiting for an epoch in a plutus test takes ~70s, where one block wait is around 20s.

This PR is stacked on top of #5835

watchEpochStateUpdate was inspired by @palas 's watchEpochStateUpdate function in #5830 , but it uses TChan for waiting for an update, instead of polling.

Checklist

• Commit sequence broadly makes sense and commits have useful messages
• New tests are added if needed and existing tests are updated. These may include:
  • golden tests
  • property tests
  • roundtrip tests
  • integration tests
    See Runnings tests for more details
• Any changes are noted in the CHANGELOG.md for affected package
• The version bounds in .cabal files are updated
• CI passes. See note on CI. The following CI checks are required:
  • Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
  • Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
  • Code builds on Linux, MacOS and Windows for ghc-8.10.7 and ghc-9.2.7
• Self-reviewed the diff

Note on CI

If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.

[palas]

Nice! This will make tests much simpler. I just added a couple of ideas for small simplifications

[palas]

👍

[Jimbo4350]

What was wrong with the IORef? If we are going to use TChans and TMVars there needs to be good justification for doing so. I don't immediately see the benefit of this increased complexity in an already tedious testing environment.

[carbolymer]

The goal here is to listen to changes of EpochStateView. You can't do this just using IORef. You could do this using MVar, and having watching threads block on empty MVar and continue when MVar gets filled. But then you still need additional IORef for functions checking for current EpochStateView without blocking. The problem with using MVar and IORef at the same time, is that you don't have any guarantees about the order of operations in the listening thread. In other words, If you update IORef and MVar in order in one thread, the other threads may see it in a different order. This means that listeners waiting on MVar () may see stale IORef value.

I've chosen STM which gives you a single view on both of variables, but you don't have MVar analogue for waiting in multiple threads for an update of a single MVar. (TMVar is only single-wakeup).

If we want to avoid STM here, we can define view and lock:

, wakeLock :: !(MVar (AnyNewEpochState, SlotNo, BlockNo))
, epochStateView :: !(IORef (AnyNewEpochState, SlotNo, BlockNo))

assuming we won't use one value to check the other, because they can be out-of-sync.

[Jimbo4350]

Why do we need to notify the listeners about changes? I could understand wanting to notify listeners of changes if the changes are short lived but they are not (or at least I can't think of any that are). What problem is this solving? Why specifically do we need this level of granularity?

[carbolymer]

It's an alternative to watchEpochStateView which is polling every 100ms EpochStateView. Sometimes new epoch state is updated much slower, like once every few seconds. So it saves some CPU on needlessly computing the check again and again on the same input.

[Jimbo4350]

I think the additional complexity is not worth it. We should be using the atomic* versions of the IORef functions to avoid race conditions/partial writes etc.

[carbolymer]

But you can't implement blocking and waiting or multi-wakeup using just IORef.

[carbolymer]

If you feel that MVars are too much, I can rewrite this PR using polling in watchEpochStateView.

[Jimbo4350]

But you can't implement blocking and waiting or multi-wakeup using just IORef.

Why do we need this? If the goal is to "listen to changes of EpochStateView" and your reason is:

It's an alternative to watchEpochStateView which is polling every 100ms EpochStateView. Sometimes new epoch state is updated much slower, like once every few seconds. So it saves some CPU on needlessly computing the check again and again on the same input.

I say the complexity is not worth it. The changes we are interested in don't require this level of granularity. Saving a little bit of CPU does not justify the additional complexity.

[carbolymer]

Waiting on MVar is not that much more complex than polling. I'll rewrite the PR using polling then.

[palas]

The one I wrote already uses polling, you can use that one, or tune it

[carbolymer]

Superseded by #5843