-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add upsampling to the client #145
base: v0.x.x
Are you sure you want to change the base?
Conversation
b78d433
to
f761662
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't look into the resampling code.
py/frequenz/client/weather/_types.py
Outdated
raise ValueError( | ||
f"Original period {original_period} must be positive non-zero" | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
raise ValueError( | |
f"Original period {original_period} must be positive non-zero" | |
) | |
raise ValueError( | |
f"Original period {original_period} must be greater than zero" | |
) |
py/frequenz/client/weather/_types.py
Outdated
f"number of features ({len(features)})" | ||
) | ||
|
||
# Validate periods are positive non-zero |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# Validate periods are positive non-zero | |
# Validate periods are greater than zero |
py/frequenz/client/weather/_types.py
Outdated
# Validate target period divides evenly into original period | ||
if original_period.total_seconds() % target_period.total_seconds() != 0: | ||
raise ValueError( | ||
f"Target period {target_period} must divide evenly into " | ||
f"original period {original_period}" | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe like this?
# Validate target period divides evenly into original period | |
if original_period.total_seconds() % target_period.total_seconds() != 0: | |
raise ValueError( | |
f"Target period {target_period} must divide evenly into " | |
f"original period {original_period}" | |
) | |
# Validate target period is an exact divisor of the original period | |
if original_period.total_seconds() % target_period.total_seconds() != 0: | |
raise ValueError( | |
f"Target period {target_period} must be an exact divisor " | |
f"of the original period {original_period}." | |
) |
Signed-off-by: Talwinder Singh <[email protected]>
Signed-off-by: Talwinder Singh <[email protected]>
Signed-off-by: Talwinder Singh <[email protected]>
Signed-off-by: Talwinder Singh <[email protected]>
array: np.ndarray[ | ||
tuple[typing.Any, typing.Any, typing.Any], np.dtype[np.float64] | ||
], | ||
target_period: dt.timedelta, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of the target period, you can pass in the requested timestamps. That simplifies the code below, allows arbitrarily sampled data and would align with what the user requests from the client. Namely they would request weather forecasts for specific datetime objects which are 15 min resampled. In this approach you first extract the target period (which could fail if it's not equidistant) and then re-construct the timestamps below for resampling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Afterwards I think this whole function could be simplified roughly like the following:
First resampling
def resample(forecasts, validity_times, new_timestamps, axis=0):
vts = np.array([t.timestamp() for t in validity_times])
nts = np.array([t.timestamp() for t in new_timestamps])
resampled = np.apply_along_axis(
lambda arr: np.interp(nts, vts, arr),
axis=axis,
arr=forecasts
)
return resampled
Then step shifting the solar feature indices (idxs):
def shift_idxs(arr, steps, idxs):
# Modifies in place and bfills
arr[steps:, :, idxs] = arr[:-steps, :, idxs]
arr[:steps, :, idxs] = arr[steps:steps+1, :, idxs]
return arr
This PR introduces upsampling to the client and adds test to check this functionality.