Y-axis flipped when reading data with Xarray #650
Comments
|
Did you try the |
|
@dcherian Sorry I realized I should have posted this in rasterio repo. I was confused rasterio and rioxarray! I will try them out and let you know soon, but you may close the issue if you think this doesn't fit here. |
|
No! This is the place. Just try |
|
Actually, |
|
Oh that's confusing! In that case, i guess it's calling |
|
So I tried open_rasterio with NSIDC output, but the results are the same.
|
|
If you do this: rds = rioxarray.open_rasterio(os.path.join(input_path, fn_NSIDC_output), group="Geophysical_Data")or this: rds = xarray.open_dataset(os.path.join(input_path, fn_NSIDC_output), group="Geophysical_Data", engine="rasterio")The data plots correctly with rasterio. EDIT: This does not fix the issue. The plot has the correct orientation. However, the data is missing coordinates. |
|
If you do this, the lat/lon data looks correct: ds_NSIDC_root = xarray.open_dataset(data_path, variable=["cell_lon", "cell_lat"], engine='rasterio')
ds_NSIDC_root.cell_lat.plot() |
|
This appears to be a rasterio/GDAL issue: import rasterio
import netCDF4
import numpy
with netCDF4.Dataset(data_path) as nfh:
nc_data = nfh['Geophysical_Data']['precipitation_total_surface_flux'][:]
with rasterio.open('netcdf:SMAP_L4_SM_gph_20150331T013000_Vv7032_001_HEGOUT.nc:/Geophysical_Data/precipitation_total_surface_flux') as rfh:
rio_data = rfh.read(1, masked=True)numpy.array_equal(nc_data, rio_data)numpy.array_equal(nc_data, numpy.flip(rio_data, 0))This is not something that can be resolved in rioxarray. |
|
Hi, I'm just found this issue while searching for a related problem. Thank you for documenting it so well. I'm actually working on a tutorial for NSIDC to demonstrate working with SMAP data. So this is quite timely. I do not think it is a software/tool problem but how you are trying to read the data. I'm using the original HDF5 file from NSIDC but the result is the same. I'm guessing you used a reprojection service to get the dataset you posted here. TLDR: I think the problem is that I'm using the following packages The SMAP datasets you are using are either HDF5 (the original file from NSIDC) or NetCDF4 and follow most of the CF-Conventions, at least for the Level-4 data. So you can use Or These both return an xarray dataset like Note that the What I think is happening when you use either the This returns a bunch of warnings about not being able to determine georeferencing Printing I suspect that Plotting
Using If you want to take advantage of the accessors and methods provided by
|
|
@snowman2 I'm seeing the same behaviour notebook: https://gist.github.com/vincentsarago/81f1e849ca8cef938a342ac49a2a1d49 I'm wondering if there is something going on with how the transform is calculated here rioxarray/rioxarray/rioxarray.py Lines 702 to 709 in f3ad955 to me it seems that the transform should be src_resolution_x, src_resolution_y = da.rio.resolution()
src_left, _, _, src_top = da.rio.bounds(recalc=True)
transform = Affine.translation(src_left, src_top) * Affine.scale(
src_resolution_x, -src_resolution_y
)at least for the file we are using the |
|
@vincentsarago I believe that the transform in rioxarray should be equivalent. Do you have an example file you can share? |
|
@snowman2 you can get files using import earthaccess
import xarray
earthaccess.login()
results = earthaccess.search_data(
concept_id="C2036881735-POCLOUD", count=1, temporal=("2024-09-20", "2024-09-20")
)
fp = earthaccess.download(results, "earthaccess_data")[0]
ds = xarray.open_dataset(fp, decode_coords="all")you can also check this file (Note this file has also the import rioxarray
import xarray
import fsspec
src_path = "https://dap.ceda.ac.uk/neodc/esacci/land_surface_temperature/data/MULTISENSOR_IRCDR/L3S/0.01/v2.00/monthly/2020/11/ESACCI-LST-L3S-LST-IRCDR_-0.01deg_1MONTHLY_DAY-20201101000000-fv2.00.nc"
filesystem = fsspec.filesystem("https")
fp = filesystem.open(src_path)
ds = xarray.open_dataset(fp, decode_coords="all", engine="h5netcdf")
da = ds["lst"]
print(da.rio.bounds())
print(da.rio._unordered_bounds())
print(da.rio.transform())
(-179.99999511705187, -90.00000274631076, 179.99999511705187, 89.9999874875217)
(-179.99999511705187, 89.9999874875217, 179.99999511705187, -90.00000274631076)
| 0.01, 0.00,-180.00|
| 0.00, 0.01,-90.00|
| 0.00, 0.00, 1.00|note we've added a |
That sounds like a good solution to me 👍 |
|
Thank you all for joining the discussions and coming up with the solutions!! Glad to hear that it was helpful for developing tutorials too. |
Update
Update on March 15, 2023 based on discussion here pydata/xarray#7621
First posted on March 13, 2023
Description
When reading data with Xarray using a rasterio engine, the Y axis is flipped. This happens with a specific dataset, and the summary is as follows.
i.e., The issue happens with SMAP L4 data, pre-processed on the NSIDC server, read with rasterio engine. The issue does not happen with NetCDF4 engine. It did not happen with the one directly downloaded from NASA Earth Data.
When reading data with Xarray using both the netcdf4 engine and the rasterio engine, the Y axis is flipped. However, when reading the same data with the netcdf4 package, the issue doesn't seem to be occurring.It's unclear whether the issue is with Xarray or the data itself.
Details
I have created a Jupyter notebook with the details of the issue, including the code used to analyze the data, which is available here
Data
for different versions of the dataset andfiles for any dates or times. I tested it with two different versions of datasets.The data can be downloaded from the links provided by NASA and NSIDC (the links are specified in the notebook), or from my Google Drive.
Environment Information
(But I remember I've encountered this issues since a few years ago for SMAP data with ver 3.9 on Win 10)
Installation method
Please let me know if you need any further information or have any suggestions on resolving this issue.
I am cross-posting this Xarray issues.
Please let me know if you need any further information or have any suggestions on resolving this issue.
Thank you!
The text was updated successfully, but these errors were encountered: