Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add raise_if_computes for testing #545

Closed
wants to merge 1 commit into from
Closed

Conversation

tomwhite
Copy link
Member

@tomwhite tomwhite commented Aug 8, 2024

Fixes #310

This is useful for xarray testing.

I had hoped to be able to implement this by writing an executor that raises when its execute_dag method is called - but that doesn't work. The problem is that the second part of this test fails, since b's spec object contains the raise-if-computes executor since it was created within the context manager. Even though compute is called outside the context manager, the executor is still set on the spec, so that is what is used.

def test_raise_if_computes():
    # shouldn't raise since compute has not been called
    with raise_if_computes():
        a = xp.ones((3, 3), chunks=(2, 2))
        b = xp.negative(a)

    # shouldn't raise since we are outside the context manager
    assert_array_equal(b.compute(), -np.ones((3, 3)))

I'm not sure that changing this is the right thing to do, hence this PR which does things a bit more directly.

@tomwhite tomwhite added the xarray-integration Uses or required for cubed-xarray integration label Aug 8, 2024
@TomNicholas
Copy link
Member

TomNicholas commented Aug 8, 2024

This is definitely needed, but the current solution seems a little ugly - modifying the business logic just for testing purposes.

Could you pull a trick using a dedicated callback instead? How does dask do this?

Another failure mode that it would be nice to check for automatically is cubed-xarray accidentally wrapping a cubed array with a dask array. Isinstance checks would catch that but that has to be manually added to each test then. See Raphael's rechunking issue for an example.

@tomwhite
Copy link
Member Author

I'm not too fond of this either, but it will probably need some more fundamental changes to how the spec/config is used to make the special executor/callback approach work. That would be a better solution though.

@tomwhite tomwhite marked this pull request as draft August 12, 2024 08:31
@tomwhite
Copy link
Member Author

tomwhite commented Oct 8, 2024

Closing this for the time being

@tomwhite tomwhite closed this Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
xarray-integration Uses or required for cubed-xarray integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide a way to check computations are lazy
2 participants