Bug Report: Landsat 5 images are all being filtered out

[2320sharon]

Bug Report

Description

There is a bug with the image filtering system in the repository that is causing Landsat 5 imagery to be filtered out incorrectly. The current behavior is that all Landsat 5 imagery is being filtered out as bad, regardless of its size. The expected behavior should be to only filter out Landsat 5 imagery as bad if the area of the image does not match the expected ROI area.

Additionally, there is a suspicion that the cloud threshold filter may be broken, as images that are completely covered in clouds are making it past the filter. It is also possible that this is related to a known glitch with Google Earth Engine, but it's worth investigating.

Tasks

• 1. Recreate the image filtering bug with Landsat 5.
• 2. Identify the cause of Landsat 5 being filtered out incorrectly.
• 3. Check if the cloud threshold filter is working as expected.
• 4. Examine the cloud masks on the images that were completely covered in clouds to determine if this is related to a Google Earth Engine Cloud masking glitch.

Steps to Reproduce

To recreate the issue, please follow these steps using the provided config files:

• Run the image filtering system with Landsat 5 imagery.

Expected Behavior

The image filtering system should only flag Landsat 5 imagery as bad if the area of the image does not match the expected ROI area.

Actual Behavior

All Landsat 5 imagery is being filtered out as bad, regardless of its size.

Config Files

Attach or provide links to the relevant configuration files that can help in reproducing the bug.

ROI2.zip

island_ROI1.zip

[2320sharon]

So I figured out why all the LandSat 5 imagery was getting filtered out. The algorithm I used was filtering them out because they exceeded the max size.
Here is how the algorithm to determine the size of the jpg works

1. Get the width and heigh of the image in pixels

• ex. 300 pixels x 300 pixels

3. Get the conversion factor for pixels to meters for that satellite

• L5 has 30 meters per pixel

5. Convert the pixels to meters

• 300 x30 x 300x30 = 81,000,000 meters^2 or 81km^2

7. Then convert to km^2

• 81km^2

The landsat 5 imagery should be about 167 pixels x 167 pixels to have an area of 25km^2.

I checked both the RGB and the ms tif that were downloaded and they each have a size of 300x300 pixels. I'm guessing some kind of upscaling of the landsat 5 image is to blame or maybe my math is wrong. I found this math to work for all the LandSat imagery and S2.

I'm going to dig into the coastsat_package code. I feel like I'm missing something important

[2320sharon]

A simple solution to this issue would be to remove the max image size filter altogether, but I want to determine the root cause of this size discrepancy between the pixel area of the image and the actual area of the image in meters.

[2320sharon]

Figured it out:
Landsat 5 imagery (and only landsat 5 imagery was resampled from 30mx30m per pixel resolution to 15mx15m. Its tricky to catch because only in this call double_res=True in the download function in coastsat is the image resolution changed. When I was calculating the resolution of the image I didn't know the resolution had changed hence the error.

warp_image_to_target(
                        fn_in,
                        fn_out,
                        fn_target,
                        double_res=True,
                        resampling_method="bilinear",
                    )
                

[2320sharon]

• Remove large image filter
• modify the per pixel ratio in filter_images

New Code

    pixel_size_per_satellite = {
        "S2": 10,
        "L7": 15,
        "L8": 15,
        "L9": 15,
        "L5": 15,  # coastsat modifies the per pixel resolution from 30m to 15m for L5
    }

[2320sharon]

Closed in coastseg 1.1.15