Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to the testing suite #506

Open
1 of 3 tasks
igamigo opened this issue Aug 28, 2024 · 0 comments
Open
1 of 3 tasks

Improvements to the testing suite #506

igamigo opened this issue Aug 28, 2024 · 0 comments
Milestone

Comments

@igamigo
Copy link
Collaborator

igamigo commented Aug 28, 2024

Right now, the testing suite is pretty complete and tests pretty much all aspects of the client and protocol. However, there are some inconveniences in terms of usability. For example, the integration tests check is lasting upwards of 20 minutes after the latest base and VM upgrades. Writing CLI tests can be somewhat cumbersome as well. Part of this is a consequence of not updating old tests that could very well make use of newer features (such as consuming more than one note on one transaction).
The good news is that there are a number of things that we can do to improve this situation (at least, to some degree):

  • Remove mock module #455: If we can remove the mock module and utilize the testing feature of the Miden dependencies, we might be able to test more functionality with unit tests using mocked models, especially when there is no need to really test integration with the node for a specific feature (one example of this may be testing validations for TransactionRequest, for example).
  • Coalesce/refactor integration tests. Some tests run are in one or more of these situations:
    • They run very similar setup code which involves creating unnecessary accounts
    • They run multiple mint transactions before really testing what the integration test is meant to test
    • They test a subset of another test's target functionality
    • They do not make use of newer features (as a result of being written before these features were available). These features encompass:
      • Consuming multiple notes in one transaction (requires multiple transactions for consuming notes)
      • Minting more than one note in a single transaction
      • Consuming notes as Note instead of InputNote (ie, not having to wait for the note to be included in a block)
      • Including more than one transaction for a single account in a block. The consequence of this is that we don't necessarily have to wait for a new block between transactions for the same account (which we do in a couple of tests, I believe)
      • Something else that could be explored here is creating more public accounts (and notes?) and sharing state between tests, although this could have its own complications
  • Writing and reading CLI tests could be improved somewhat by better wrapping functionality into structs/functions. For example, there could be a CliClient struct that handles a lot of the boilerplate for creating the client's files, running commands and reading outputs. Additionally, there are a couple of CLI features that we are not testing (for instance, the token mapping file functionality). As the CLI gets more complex, we will want to have it better tested so as to avoid some of the things we ran into in the past, like some commands that stop working due to an upstream change (like printing keys, etc.).

Surely there are some other avenues that could be explored (something that comes to mind is configuring how tests are executed to see if we can improve runtimes by increasing concurrency, etc.), but I think these could be a good starting point.

@bobbinth bobbinth added this to the v0.6 milestone Aug 29, 2024
@igamigo igamigo modified the milestones: v0.6.0, v0.7.0 Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants