Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

artefacts in FCI RGBs using 3.8 µm #3009

Open
gerritholl opened this issue Dec 5, 2024 · 8 comments · May be fixed by #3013
Open

artefacts in FCI RGBs using 3.8 µm #3009

gerritholl opened this issue Dec 5, 2024 · 8 comments · May be fixed by #3013

Comments

@gerritholl
Copy link
Member

gerritholl commented Dec 5, 2024

Describe the bug

The 3.8 µm channel and RGB that use it have artefacts during the night, such as black pixels. This includes the convection RGB and night microphysics.

To Reproduce

import hdf5plugin
import os
from sattools.io import plotdir
from satpy import Scene
from satpy.utils import debug_on; debug_on()
from glob import glob
fci_files_fdhsi = glob("/media/nas/x23352/MTG/FCI/L1c-cases/202412050600-north-sea/W*FDHSI*BODY*.nc")
fci_files_hrfi = glob("/media/nas/x23352/MTG/FCI/L1c-cases/202412050600-north-sea/W*HRFI*BODY*.nc")
sc = Scene(filenames=fci_files_fdhsi+fci_files_hrfi, reader="fci_l1c_nc")
sc.load(["convection"])
ls = sc.resample("scan")
ls.save_datasets(
        filename=os.fspath(plotdir(create=True) /
        "{start_time:%Y%m%d%H%M}-{platform_name}-{sensor}-{area.area_id}-{name}.tif"),
        fill_value=0)

Expected behavior

I expect an image without black pixels.

Actual results

I get an image with black pixels.

Screenshots

202412060700-gwenview-convection-black-pixels

Environment Info:

  • OS: openSUSE Leap 15.3
  • Satpy Version: v0.53.0-43-g9b817a431

Additional context

I'm aware that the convection composite uses the difference between two solar channels, so it is not suitable to use during the night. But that still shouldn't make pixels black. The problem also occurs with night_microphysics and with an internal DWD RGB where the night part is similar to cloudtop.

@ameraner
Copy link
Member

ameraner commented Dec 5, 2024

My first suspect is the ir_38 that has NaN brightness temperatures over cold clouds at night (due to extra low/negative radiances mostly generated by the ESL correction). This happens more often in the 1km version, while it gets mostly absorbed in the 2km version, but it can happen there as well... An example of this happening on the 2km version is today 05/12 00:00:
image

@gerritholl
Copy link
Member Author

gerritholl commented Dec 9, 2024

Indeed. Here is IR 3.8 for 2024-12-09 04:00 in the North Atlantic:

202409120400-ninjo-north-atlantic-ir38

This is what happens to night microphysics:

202409120400-ninjo-north-atlantic-night-microphysics

Or with our internal RGB, similar to Satpys cloudtop:

202409120400-ninjo-north-atlantic-rgb-clouds-day-night-small

Interestingly, this is IR 3.8 with the nir_emissive modifier applied:

202409120400-ninjo-north-atlantic-ir38t

@gerritholl gerritholl changed the title black pixels in FCI convection composite when combining FDHSI and HRFI artefacts in FCI RGBs using 3.8 µm Dec 9, 2024
@simonrp84
Copy link
Member

Can something like this: #2806
Be implemented for FCI? That would "fix" the problem for qualitative use cases...

@gerritholl
Copy link
Member Author

Thanks @simonrp84, that looks interesting. Negative radiances are clipped to the lowest non-negative radiance before calibrating to brightness temperature?

@simonrp84
Copy link
Member

Yes. As long as it's an optional feature then it should solve the problem nicely without impacting other uses.

@gerritholl
Copy link
Member Author

Another MCVE:

import hdf5plugin
from satpy import Scene
from pyresample import create_area_def
from glob import glob
fci_files = glob("/media/nas/x23352/MTG/FCI/L1c-cases/202412090100-atlantic/*BODY*O_0008_003[56].nc")
ar = create_area_def("ofz", 4087, description="oceanographer fracture zone", area_extent=[-4230000, 4675000, -3562000, 5232000], resolution=750)
sc = Scene(filenames={"fci_l1c_nc": fci_files})
sc.load(["cloudtop"])
ls = sc.resample(ar)
ls.save_datasets()

Looks like there are both white and transparent pixels.

cloudtop_20241209_011000

As shown in irfanview:

image

@gerritholl gerritholl linked a pull request Dec 9, 2024 that will close this issue
3 tasks
@ameraner
Copy link
Member

ameraner commented Dec 9, 2024

The root cause of both types of pixels is the same: already-low radiance values over cold clouds that get even lower due to the ESL correction and/or noise. Likely, the white pixels are very low radiance values that still have a valid BT (which is going to be extremely cold), and the black pixels are negative radiance pixels that cannot be converted to BTs and hence become NaN/transparent.

@gerritholl
Copy link
Member Author

gerritholl commented Dec 10, 2024

For pixels to become white, we'd need extremely cold radiances in all three channels, each of 3.8 µm, 10.5 µm, 12.3 µm. However, on closer inspection, I only see this for 3.8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants