-
Notifications
You must be signed in to change notification settings - Fork 653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replaced numpy.testing.assert_almost_equal to numpy.testing.assert_allclose #4438
Conversation
…tent floating point comparisons e.g. # assert_almost_equal(rms.rmsd(first_frame, first_frame), 0.0, 5, # err_msg="error: rmsd(X,X) should be 0") assert_allclose(rms.rmsd(first_frame, first_frame), 0.0, rtol=0, atol=1.5*1e-5, err_msg="error: rmsd(X,X) should be 0") 1.5 factor in atol bcz,in assert_almost_equal The test verifies that the elements of actual and desired satisfy. abs(desired-actual) < float64(1.5 * 10**(-decimal)) So, to keep behaviour consistent in assert_allclose introduced factor 1.5 in atol
Hello @aditya292002! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:
Comment last updated at 2024-02-24 09:35:19 UTC |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello there first time contributor! Welcome to the MDAnalysis community! We ask that all contributors abide by our Code of Conduct and that first time contributors introduce themselves on GitHub Discussions so we can get to know you. You can learn more about participating here. Please also add yourself to package/AUTHORS
as part of this PR.
Linter Bot Results:Hi @aditya292002! Thanks for making this PR. We linted your code and found the following: Some issues were found with the formatting of your code.
Please have a look at the Please note: The |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #4438 +/- ##
===========================================
- Coverage 93.38% 93.37% -0.01%
===========================================
Files 171 183 +12
Lines 21744 22823 +1079
Branches 4014 4014
===========================================
+ Hits 20305 21311 +1006
- Misses 952 1025 +73
Partials 487 487 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had one or two comments, apart from those most cases seem to match this idiom:
Before: abs(desired-actual) < float64(1.5 * 10**(-decimal))
New: abs(desired-actual) < atol + rtol * abs(desired)
with 0
for rtol
reducing to abs(desired-actual) < atol
, so you just set atol
to 1.5 * 10**(-decimal)
to match the original behavior.
I suppose that's hard to argue with in terms of being equivalent. There probably are some cases where you do want to allow an rtol
, but I don't have a huge problem with the literal translations for now I suppose.
@@ -167,7 +167,8 @@ def test_subselection_alignto(self, universe, reference, subselection, expectati | |||
|
|||
with expectation: | |||
rmsd = align.alignto(universe, reference, subselection=subselection) | |||
assert_almost_equal(rmsd[1], 0.0, decimal=9) | |||
# assert_almost_equal(rmsd[1], 0.0, decimal=9) | |||
assert_allclose(rmsd[1], 0.0, rtol=0, atol=1.5*1e-9) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can remove the commented old version
|
||
def test_rmsd_custom_mass_weights(self, universe, reference): | ||
last_atoms_weight = universe.atoms.masses | ||
A = universe.trajectory[0] | ||
B = reference.trajectory[-1] | ||
rmsd = align.alignto(universe, reference, | ||
weights=reference.atoms.masses) | ||
weights=reference.atoms.masses) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some of the whitespace changes don't seem useful?
err_msg="assert_allclose failed for positions") | ||
assert_allclose(avg.results.rmsd, rmsd, | ||
rtol=0, atol=1.5*(1e-4), | ||
err_msg="assert_allclose failed for rmsd") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fwiw, I don't think these custom messages are worth it in this context, but that may just be based on the style of the projects I usually maintain
rtol=0, atol=1.5*(1e-4), | ||
err_msg="assert_allclose failed for positions") | ||
assert_allclose(avg.results.rmsd, rmsd, | ||
rtol=0, atol=1.5*(1e-4), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't you want 1e-7
since assert_almost_equal
is using default arguments in the original here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that's a mistake. I will correct this.
- correction - removed unnecessary error messages
Hi, @tylerjereddy, I've made all the suggested changes. Should I proceed with applying the same modifications to all the files? There are nearly 840 occurrences of just the 'assert_almost_close'. I'm thinking of creating a Python script to automate these changes. Should I go ahead with that approach? Also, could you please explain why not all the test cases are passing? I didn't make any changes to the '/mdanalysis/testsuite/MDAnalysisTests/parallelism/test_multiprocessing.py' file, but the 'def test_creating_multiple_universe_without_offset(temp_xtc, ncopies=3)' test failed for Ubuntu with Python 3.12. Thank you. |
@aditya292002 this issue is really meant to be taken in small chunks and not all on one go. I would suggest doing a handful of files so that it's both manageable to yourself and reviewers. Regarding the test failures, this is a known issue and not related to this PR. |
@IAlibay, I've made additional changes to the files: test_base, test_bat, test_contracts, test_dihedrals, and test_encore. Please review these modifications and merge this pull request for the time being. I'd like to further explore this project and address any other issues. |
|
@aditya292002 - whilst I appreciate your work here, I would like to ask for your patience. The MDAnalysis developers are not full time hired on the project and have their own separate jobs / lives. I will add this to my task list and will attempt to review, althoughtit may take at least a few days (there are already a few other reviews pending my attention). |
Apologies for any impatience displayed. I am currently a university student, and this marks my initial involvement in contributing to an open-source project. Rest assured, I will take note of this experience for future contributions. |
I won't be able to get to reviewing this any time soon - can one of the current @MDAnalysis/coredevs take this on instead please? |
@tylerjereddy would you be able review this PR, given that you already had some eyes on it? |
@aditya292002 please remove the After you check that your diff only has the changes that are needed, feel free to ping me again and I'll take another look. |
@tylerjereddy, I have removed the specified bytecode file. |
assert_almost_equal(rmsd, desired, decimal=5, | ||
err_msg="frame {0:d} of fit does not have " | ||
"expected RMSD".format(frame)) | ||
fitted.trajectory[frame] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You still have unwanted whitespace changes here and elsewhere, please re-read the entire diff line-by-line and make sure those changes are not present (since I'm spending time doing the same).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Please fix any whitespace issues: if you didn't touch a line then it shouldn't change in the diff.
- Write floats in engineering notation and not as products (see comment).
- Remove
assert_almost_equal
from theimport
line in any file where you replaced it withassert_allclose
.
np.testing.assert_almost_equal(array_cube_vertex_distances_from_centroid.min(), | ||
array_cube_vertex_distances_from_centroid.max(), decimal=4, | ||
np.testing.assert_allclose(array_cube_vertex_distances_from_centroid.min(), | ||
array_cube_vertex_distances_from_centroid.max(), rtol=0, atol=1.5*(1e-4), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Write floats as 1.5e-4
and not as 1.5*(1e-4)
.
This applies to all floats with powers of ten throughout.
|
||
self._assert_rmsd(reference, fitted, 0, 0.0, | ||
weights=universe.atoms.masses) | ||
weights=universe.atoms.masses) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
undo indent
self._assert_rmsd(reference, fitted, -1, 6.929083032629219, | ||
weights=universe.atoms.masses) | ||
weights=universe.atoms.masses) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
undo indent
@@ -27,7 +27,7 @@ | |||
|
|||
import numpy as np | |||
|
|||
from numpy.testing import assert_equal, assert_almost_equal | |||
from numpy.testing import assert_equal, assert_almost_equal, assert_allclose |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove assert_almost_equal: the whole point of the PR is to get rid of it :-)
assert_almost_equal(result_value, expected_value, decimal=-3, | ||
err_msg="Unexpected value for Harmonic Ensemble Similarity: {0:f}. Expected {1:f}.".format(result_value, expected_value)) | ||
assert_allclose(result_value, expected_value, rtol=0, atol=1.5*(1e3), | ||
err_msg="Unexpected value for Harmonic Ensemble Similarity: {0:f}. Expected {1:f}.".format(result_value, expected_value)) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please check if I have got this right
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's a very loose test!
err_msg="Unexpected average value for bootstrapped samples in Clustering Ensemble similarity") | ||
assert_almost_equal(stdev, expected_stdev, decimal=0, | ||
|
||
assert_almost_equal(stdev, expected_stdev, decimal=0, | ||
err_msg="Unexpected standard daviation for bootstrapped samples in Clustering Ensemble similarity") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for this, "assert_almost_equal" statement
the value of parameter decimal = 0,
which means abs(x,y) should be < 1.5 (which i found by running some examples),
but, for assert_allclose(x,y, rtol=0, atol=0) means the abs(x-y) = 0,
To resolve this should I go ahead and replace the assert_almost_equal which a if condition.
Please suggest some alternate ways to handle it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The equivalent for decimal=0
is atol=1.5e0
which is just atol=1.5
:
assert_allclose(stdev, expected_stdev, rtol=0, atol=1.5)
Does the code above work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Btw, can you please fix the spelling mistake: "daviation" --> "deviation"?
removed all the extra widespaces and correct the format for atol=1.5e-x
@tylerjereddy, could you please take a look at the changes I made? |
@orbeckst , could you please take a look at the changes I made? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks almost done:
-
fix the last
assert_almost_equal
withdecimal=0
(see comment) -
add yourself to the AUTHORS file
-
add an entry to CHANGELOG:
"updated tests that used assert_almost_equal(..., decimal={N}) with equivalent assert_allclose(..., rtol=0, atol=1.5e-{N}) (issue modernize testing code #3743, PR Replaced numpy.testing.assert_almost_equal to numpy.testing.assert_allclose #4438)"
(Normally we don't include changes to the tests in CHANGELOG, but this one may make some tests fail and we want to have documentation that alerts us to the fact that we changed test evaluation.)
assert_equal, | ||
assert_array_equal, | ||
assert_array_almost_equal | ||
assert_array_almost_equal, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert_array_almost_equal()
could also be replaced with assert_allclose()
.
assert_almost_equal(result_value, expected_value, decimal=-3, | ||
err_msg="Unexpected value for Harmonic Ensemble Similarity: {0:f}. Expected {1:f}.".format(result_value, expected_value)) | ||
assert_allclose(result_value, expected_value, rtol=0, atol=1.5*(1e3), | ||
err_msg="Unexpected value for Harmonic Ensemble Similarity: {0:f}. Expected {1:f}.".format(result_value, expected_value)) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's a very loose test!
err_msg="Unexpected average value for bootstrapped samples in Clustering Ensemble similarity") | ||
assert_almost_equal(stdev, expected_stdev, decimal=0, | ||
|
||
assert_almost_equal(stdev, expected_stdev, decimal=0, | ||
err_msg="Unexpected standard daviation for bootstrapped samples in Clustering Ensemble similarity") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The equivalent for decimal=0
is atol=1.5e0
which is just atol=1.5
:
assert_allclose(stdev, expected_stdev, rtol=0, atol=1.5)
Does the code above work?
and - changed assert_array_almost_equal with assert_allclose
…equal to assert_allclose
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please fix CHANGELOG, otherwise looking good!
UPDATE: There are now tests that are failing. Please have a look at them and propose adjustments to the tolerances to make them pass again.
package/CHANGELOG
Outdated
24/02/23 aditya292002 | ||
* "updated tests that used assert_almost_equal(..., decimal={N}) with equivalent assert_allclose(..., rtol=0, atol=1.5e-{N}) (issue modernize testing code #3743, PR Replaced numpy.testing.assert_almost_equal to numpy.testing.assert_allclose #4438)" | ||
|
||
/workspaces/mdanalysis/package/CHANGELOG |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove this line!
package/CHANGELOG
Outdated
24/02/23 aditya292002 | ||
* "updated tests that used assert_almost_equal(..., decimal={N}) with equivalent assert_allclose(..., rtol=0, atol=1.5e-{N}) (issue modernize testing code #3743, PR Replaced numpy.testing.assert_almost_equal to numpy.testing.assert_allclose #4438)" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add information to the top of the file under 2.8.0.
- add your handle after
??/??/?? IAlibay, HeetVekariya, marinegor, lilyminium
- add the text under "Changes"
@orbeckst Do I need to do anything more in this PR. |
Congratulations on your first contribution @aditya292002 ! 🎉 |
Fixes #
Changes made in this Pull Request:
PR Checklist
Developers certificate of origin
📚 Documentation preview 📚: https://mdanalysis--4438.org.readthedocs.build/en/4438/