-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pyudunits2 #1118
base: develop
Are you sure you want to change the base?
Pyudunits2 #1118
Changes from all commits
fb6301a
145ced5
3b11d09
fd04d25
0e7ef12
4fdaf8f
648affc
f503710
c765a73
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
import pyudunits2 | ||
|
||
try: | ||
import cf_units | ||
except ImportError: | ||
cf_units = False | ||
|
||
UT_SYSTEM = None | ||
|
||
|
||
class PyUdunits2: | ||
"""Workaround for the differences in pyudunits2 and cf-units. | ||
|
||
NB: Some of these may change and/or get implemented upstream. Pyudunits2 is new and in-flux. | ||
|
||
1/4 Raise the same ValueError to match cf-unit errors. | ||
2/4 Creates an empty unit from None to mimic cf-unit's Unit('unknown') | ||
ocefpaf marked this conversation as resolved.
Show resolved
Hide resolved
|
||
3/4 Add a definition object that is ust units.expanded() | ||
""" | ||
|
||
def __init__(self, units: str | None): | ||
"""Keep unit system so we can convert from string later.""" | ||
global UT_SYSTEM | ||
if UT_SYSTEM is None: | ||
UT_SYSTEM = pyudunits2.UnitSystem.from_udunits2_xml() | ||
|
||
self.ut_system = UT_SYSTEM | ||
|
||
if units is None: | ||
units = "" | ||
|
||
try: | ||
self.units = self.ut_system.unit(units) | ||
except (SyntaxError, pyudunits2.UnresolvableUnitException) as err: | ||
raise ValueError from err | ||
self.definition = self.units.expanded() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would avoid doing this until you have to. On the It looks like this isn't actually used presently anyway? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My memory is failing me here now but I believe I added this to create a correspondence with cf_units. However, if the changes above solve the slowdowns I created here with my messy code, I'll probably just ditch cf_units and use only pyudunits2. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Some very crude benchmarks points to this as the source of slowdowns.
vs
I believe all the pieces to return a faster There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for pointing out where it is used! That check intentionally doesn't support I guess we need to think about what equality of units means - I think With such a definition, I would be able to add hashability to the
I think all of this needs to go into some documentation in All of the above doesn't solve the performance difference. The numbers are really useful, so thank you! |
||
|
||
def __eq__(self, other): | ||
return self.units == other | ||
|
||
def is_dimensionless(self): | ||
return self.units.is_dimensionless() | ||
|
||
def is_time_reference(self): | ||
return isinstance(self.units, pyudunits2.DateUnit) | ||
|
||
def is_convertible(self, other): | ||
if isinstance(other, str): | ||
other = self.ut_system.unit(other) | ||
elif isinstance(other, (PyUdunits2)): | ||
other = other.units | ||
else: | ||
msg = f"Expected valid unit string or pyudunits2 unit object. Got {other}." | ||
raise ValueError(msg) | ||
|
||
# FIXME: cf-units Workaround 1/4 -> cf_units.Unit(None) -> Unit('unknown'). | ||
if "" in (self.units.expanded(), other.expanded()): | ||
return False | ||
|
||
convertible = self.units.is_convertible_to(other) | ||
# FIXME: cf-units Workaround 2/4 -> time is not convertible to time reference. | ||
|
||
# Both are time reference confirm. | ||
if self.is_time_reference() and isinstance(other, pyudunits2.DateUnit): | ||
convertible = True | ||
|
||
return convertible | ||
|
||
|
||
if cf_units: | ||
PyUdunits2 = cf_units.Unit | ||
|
||
|
||
class Unit(PyUdunits2): | ||
def __init__(self, units): | ||
super().__init__(units) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,7 +9,7 @@ requires = [ | |
[project] | ||
name = "compliance-checker" | ||
description = "Checks Datasets and SOS endpoints for standards compliance" | ||
readme = "README.md" | ||
readme = { file = "README.md", content-type = "text/markdown" } | ||
license = { text = "Apache-2.0" } | ||
maintainers = [ | ||
{ name = "Dave Foster", email = "[email protected]" }, | ||
|
@@ -38,6 +38,7 @@ dynamic = [ | |
"dependencies", | ||
"version", | ||
] | ||
optional-dependencies.extras = [ "cf-units>=2" ] | ||
urls.documentation = "https://ioos.github.io/compliance-checker" | ||
urls.homepage = "https://compliance.ioos.us/index.html" | ||
urls.repository = "https://github.com/ioos/compliance-checker" | ||
|
@@ -79,7 +80,6 @@ compliance_checker = [ | |
dependencies = { file = [ | ||
"requirements.txt", | ||
] } | ||
readme = { file = "README.md", content-type = "text/markdown" } | ||
|
||
[tool.setuptools_scm] | ||
write_to = "compliance_checker/_version.py" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be interested to measure the performance of this vs for example:
(note the private API usage for now - would like to expose symbol in a friendly form ASAP)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your suggestion is quite faster (again, this is poor's man benchmark):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for measuring! I totally forgot that I actually made this easier:
Performance shouldn't be any different.