Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add integration tests #331
feat: add integration tests #331
Changes from 4 commits
dac1871
dcee7d8
14e3e77
bf4a14f
5dd6748
c4d48da
37d5c39
a667906
df52cce
6272d47
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would be the easiest way to maintain this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can use something like the following:
or
That could test that we can use the Grader functions we want.
I added more context to the code to cover a little more in the tests, using some examples from the unit tests. However, this may not be necessary since it has already been tested in the unit tests. We can test only the fact that we can use the graders. I think this could be totally up to @blarghmatey.
@blarghmatey, do you prefer a code with some examples but with more maintenance or only a test that we can use the graders, which is the main functionality, with less maintenance, and leave the rest to the unit test?
Thanks, @mariajgrimaldi, for that observation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have been thinking about this, and the best approach is to test with the import and leave the details for the unit tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd suggest running just a MatrixGrader. That's the class that uses the most external dependencies (scipy, numpy). I agree that so long as those are working, the unit tests can handle the rest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that makes sense. The main purpose for the integration test is to make sure that we can be aware of any regressions due to underlying dependency changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@blarghmatey, for dependency changes, the unit test we are running in python3.8 y 3.11 should be enough because we are using in the test the
edx-sandbox
requirements that codejail uses for redwood and sumac, respectively.We are running an environment for this integration test to see if we can use the library as expected in a production environment.