Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up memory + trace output comparison for contracts run in starknet-devnet tests #174

Open
wants to merge 58 commits into
base: main
Choose a base branch
from

Conversation

fmoletta
Copy link
Contributor

@fmoletta fmoletta commented Dec 1, 2022

Add a Makefile target compare_memory_devnet that runs the relevant starknet-devnet tests with cairo-rs-py runner and the original runner and compares the relocated memory + trace outputs
Also add a workflow that runs this target
NOTE: Current CI time for this new workflow is over 1 hour, some of it can be reduced by using cache

Steps:

  • Add patched envs build script
  • Expose write_binary_memory in PyCairoRunner
  • Add script/target to run each test in both envs and compare memory outputs after each run
  • Filter failing tests
  • Add Makefile target compare-memory-devnet
  • Integrate into CI

Future plans:

  • Add trace comparison
  • Produce different named files then running contracts so that we can compare every output instead of the last sets
  • Change target and file names to reflect addition of trace comparison
  • Run contracts outside of devnet tests
  • Target other projects that use starknet

@fmoletta fmoletta changed the base branch from main to memory-comparison-devnet December 1, 2022 22:05
@fmoletta fmoletta changed the title Add trace comparison to memory comparison target + workflow Add trace comparison to devote memory comparison target + workflow Dec 1, 2022
@fmoletta fmoletta changed the title Add trace comparison to devote memory comparison target + workflow Add trace comparison to devnet memory comparison target + workflow Dec 1, 2022
@fmoletta fmoletta marked this pull request as ready for review December 7, 2022 20:15
@fmoletta fmoletta changed the base branch from memory-comparison-devnet to main December 7, 2022 20:33
@fmoletta fmoletta changed the title Add trace comparison to devnet memory comparison target + workflow Set up memory + trace output comparison for contracts run in starknet-devnet tests Dec 7, 2022
scripts/memory_comparator/build_envs.sh Outdated Show resolved Hide resolved


if __name__ == '__main__':
main()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\n

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants