Create instrument swath outlines as shapefiles #93

adybbroe · 2024-11-19T15:14:44Z

This PR adds a script to create Satellite instrument swath outlines in shapefile format, e.g. for use in an WMS/WFS service.

Also some re-factoring has been done in this PR.

Closes Should be possible to store swath outlines as shapefiles #90
Tests added
Tests passed
Passes git diff origin/main **/*py | flake8 --diff
Fully documented

…shapefile support Signed-off-by: Adam.Dybbroe <[email protected]>

Signed-off-by: Adam.Dybbroe <[email protected]>

…ine-shapefile # Conflicts: # trollsched/tests/test_satpass.py

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe · 2024-11-20T14:26:03Z

Here an example of how the script can be run:
create_shapefiles_from_schedule -s NOAA-21 NOAA-20 -t ./tle-202409060015.txt -x ./2024-09-06-00-45-08-acquisition-schedule-confirmation-nrk.xml -o /data/pass_outlines

TAlonglong · 2024-11-20T14:36:32Z

Could you add something like that example you gave to the docs?

Signed-off-by: Adam.Dybbroe <[email protected]>

codecov · 2024-11-20T19:49:31Z

Codecov Report

Attention: Patch coverage is 81.38528% with 43 lines in your changes missing coverage. Please review.

Project coverage is 60.49%. Comparing base (fccd0a5) to head (086d579).
Report is 12 commits behind head on main.

Files with missing lines	Patch %	Lines
trollsched/combine.py	11.11%	16 Missing ⚠️
trollsched/drawing.py	0.00%	13 Missing ⚠️
trollsched/shapefiles_from_schedule.py	89.39%	7 Missing ⚠️
trollsched/logger.py	77.77%	4 Missing ⚠️
trollsched/utils.py	81.81%	2 Missing ⚠️
trollsched/helper_functions.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #93      +/-   ##
==========================================
+ Coverage   55.71%   60.49%   +4.77%     
==========================================
  Files          14       18       +4     
  Lines        2107     2278     +171     
==========================================
+ Hits         1174     1378     +204     
+ Misses        933      900      -33

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe · 2024-11-22T12:51:56Z

Could you add something like that example you gave to the docs?

Good point, absolutely. I have added a bare minimum at least.

adybbroe · 2024-11-22T13:02:14Z

Here just showing an example of a QGIS display of one shapefile produced by the script above:

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe · 2024-11-25T15:51:30Z

Test closing, then opening, to trigger RTD pages build on PRs

adybbroe · 2024-11-26T16:06:06Z

@TAlonglong @pnuu @mraspaud
Any opinion or can I merge it?

pnuu

Some suggestions and requests inline.

trollsched/pass_scheduling_utils.py

trollsched/satpass.py

trollsched/shapefiles_from_schedule.py

Co-authored-by: Panu Lahtinen <[email protected]>

Signed-off-by: Adam.Dybbroe <[email protected]>

…schedule into swathoutline-shapefile

Signed-off-by: Adam.Dybbroe <[email protected]>

trollsched/pass_scheduling_utils.py

trollsched/shapefiles_from_schedule.py

Co-authored-by: Panu Lahtinen <[email protected]>

adybbroe · 2024-12-03T10:20:25Z

I think I should have actually removed the main block in shapefiles_from_schedule.py do you agree @pnuu ?

Signed-off-by: Adam.Dybbroe <[email protected]>

mraspaud

Wow, thanks a lot for the huge work in cleaning this package up! Looks good in general, but I have a few comments and suggestions/requests inline

.github/workflows/ci.yaml

.stickler.yml

docs/usage.rst

.bumpversion.cfg

pyproject.toml

trollsched/graph.py

trollsched/logger.py

trollsched/schedule.py

mraspaud · 2024-12-03T11:07:56Z

trollsched/shapefiles_from_schedule.py

+                continue
+
+            if platform_name not in satellites:
+                continue
+            try:
+                overpass = create_pass(platform_name, instrument,
+                                       datetime.strptime(child.attrib["start-time"],
+                                                         "%Y-%m-%d-%H:%M:%S"),
+                                       datetime.strptime(child.attrib["end-time"],
+                                                         "%Y-%m-%d-%H:%M:%S"),
+                                       tle_filename=tle_file)
+            except KeyError:
+                continue


imo these continues should be replaced by log messages.

Not all of them!? Not "replaced"? I don't want to try create a pass if the instrument is None or the platform-name not requested...
How do you mean?

I mean that if one of these continues is reached, something is not going as planned, right? so shouldn’t we log about it?

I believe this is fixed now

I think we need to figure out if the the KeyError part can be tested and when/why that actually might happen?

trollsched/tests/test_pass_outlines.py

…errors Signed-off-by: Adam.Dybbroe <[email protected]>

Co-authored-by: Martin Raspaud <[email protected]>

Signed-off-by: Adam.Dybbroe <[email protected]>

…schedule into swathoutline-shapefile

…nd line args Signed-off-by: Adam.Dybbroe <[email protected]>

Signed-off-by: Adam.Dybbroe <[email protected]>

mraspaud · 2024-12-05T12:31:45Z

docs/environment.yaml

+  - sphinx_rtd_theme
+  - sphinxcontrib-apidoc
+  - trollsift
+  - xarray


do we need xarray here?

Adam.Dybbroe added 8 commits October 3, 2024 17:16

Refactor and use pyproject.toml for intallation - first draft add of …

34ac1d2

…shapefile support Signed-off-by: Adam.Dybbroe <[email protected]>

Version file is now generated upon installation

3609a71

Signed-off-by: Adam.Dybbroe <[email protected]>

Merge branch 'main' into swathoutline-shapefile

23c0888

Merge branch 'bugfix-slstr-not-yet-properly-supported' into swathoutl…

03af390

…ine-shapefile # Conflicts: # trollsched/tests/test_satpass.py

Merge branch 'main' into swathoutline-shapefile

692eb25

Add a first script to create the shapefiles from the XML schedule file

5d14d72

Signed-off-by: Adam.Dybbroe <[email protected]>

Add a basic logger module

29e7576

Signed-off-by: Adam.Dybbroe <[email protected]>

Bugfix

9682861

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe added the enhancement label Nov 19, 2024

adybbroe self-assigned this Nov 19, 2024

Adam.Dybbroe added 5 commits November 20, 2024 13:38

Attempt to fix the ruff/linter stuf in the pyproject toml file

d1533c2

Signed-off-by: Adam.Dybbroe <[email protected]>

Try fix pyproject.toml

a78d7f4

Signed-off-by: Adam.Dybbroe <[email protected]>

Trt fix the CI and github actions

8ed05a8

Signed-off-by: Adam.Dybbroe <[email protected]>

Bugfix - we do not try to run CI tests with Basemap support

c364f6e

Signed-off-by: Adam.Dybbroe <[email protected]>

Fix so the Code-factor bot is happy

e675cd2

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe requested review from mraspaud, pnuu and TAlonglong November 20, 2024 14:24

Adam.Dybbroe added 3 commits November 20, 2024 16:25

Update pre-commit config -copying from Satpy

e2268f8

Signed-off-by: Adam.Dybbroe <[email protected]>

Test add back rye

7a37ec3

Signed-off-by: Adam.Dybbroe <[email protected]>

Fix cut'n'paste error

8150fdd

Signed-off-by: Adam.Dybbroe <[email protected]>

Adam.Dybbroe added 2 commits November 22, 2024 13:46

Add a minimum amount of documentation for the new script

972f4a2

Signed-off-by: Adam.Dybbroe <[email protected]>

Bugfix

8e82f37

Signed-off-by: Adam.Dybbroe <[email protected]>

Adam.Dybbroe added 2 commits November 22, 2024 16:10

Add tests and fix style issues

472c1e0

Signed-off-by: Adam.Dybbroe <[email protected]>

Fix codefactor complaints

354ff71

adybbroe marked this pull request as ready for review November 22, 2024 15:46

Bugfix and minor clean up

2b1e276

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe closed this Nov 25, 2024

adybbroe reopened this Nov 25, 2024

pnuu requested changes Dec 2, 2024

View reviewed changes

adybbroe and others added 5 commits December 2, 2024 20:45

Update trollsched/satpass.py

dfebf59

Co-authored-by: Panu Lahtinen <[email protected]>

Fix more meaningful doc strings

22f689e

Signed-off-by: Adam.Dybbroe <[email protected]>

Merge branch 'swathoutline-shapefile' of github.com:adybbroe/pytroll-…

ba59c38

…schedule into swathoutline-shapefile

Add more test coverage

9df160b

Signed-off-by: Adam.Dybbroe <[email protected]>

Fix style issues

ba5f81e

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe requested a review from pnuu December 3, 2024 08:40

pnuu reviewed Dec 3, 2024

View reviewed changes

trollsched/pass_scheduling_utils.py Outdated Show resolved Hide resolved

trollsched/shapefiles_from_schedule.py Outdated Show resolved Hide resolved

Update trollsched/pass_scheduling_utils.py

e545647

Co-authored-by: Panu Lahtinen <[email protected]>

Remove __main__ from module. It uses the run part for the script

c5d3e40

Signed-off-by: Adam.Dybbroe <[email protected]>

mraspaud requested changes Dec 3, 2024

View reviewed changes

Adam.Dybbroe and others added 9 commits December 4, 2024 10:44

Clean up, removing unneccessary/unused stuff, and fixing cut'n'paste …

26c8f3b

…errors Signed-off-by: Adam.Dybbroe <[email protected]>

Update trollsched/schedule.py

754c607

Co-authored-by: Martin Raspaud <[email protected]>

Update trollsched/graph.py

bdc004c

Co-authored-by: Martin Raspaud <[email protected]>

Add more debug logging

38330e2

Signed-off-by: Adam.Dybbroe <[email protected]>

Restructure imports

2b4ef25

Signed-off-by: Adam.Dybbroe <[email protected]>

Merge branch 'swathoutline-shapefile' of github.com:adybbroe/pytroll-…

c949c58

…schedule into swathoutline-shapefile

Fix test to include the testing of running the main script with comma…

5243dbe

…nd line args Signed-off-by: Adam.Dybbroe <[email protected]>

Re-introduce some of the debug printouts using the logging module

a5a2e77

Signed-off-by: Adam.Dybbroe <[email protected]>

Update the example config with current satellites

086d579

Signed-off-by: Adam.Dybbroe <[email protected]>

adybbroe requested review from pnuu and mraspaud December 4, 2024 21:50

mraspaud reviewed Dec 5, 2024

View reviewed changes

docs/environment.yaml

- sphinx_rtd_theme

- sphinxcontrib-apidoc

- trollsift

- xarray

Copy link

Member

mraspaud Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need xarray here?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create instrument swath outlines as shapefiles #93

Create instrument swath outlines as shapefiles #93

adybbroe commented Nov 19, 2024 •

edited

Loading

adybbroe commented Nov 20, 2024

TAlonglong commented Nov 20, 2024

codecov bot commented Nov 20, 2024 •

edited

Loading

adybbroe commented Nov 22, 2024

adybbroe commented Nov 22, 2024

adybbroe commented Nov 25, 2024

adybbroe commented Nov 26, 2024

pnuu left a comment

adybbroe commented Dec 3, 2024

mraspaud left a comment

mraspaud Dec 3, 2024

adybbroe Dec 4, 2024

mraspaud Dec 4, 2024

adybbroe Dec 4, 2024

adybbroe Dec 4, 2024

mraspaud Dec 5, 2024

Create instrument swath outlines as shapefiles #93

Are you sure you want to change the base?

Create instrument swath outlines as shapefiles #93

Conversation

adybbroe commented Nov 19, 2024 • edited Loading

adybbroe commented Nov 20, 2024

TAlonglong commented Nov 20, 2024

codecov bot commented Nov 20, 2024 • edited Loading

Codecov Report

adybbroe commented Nov 22, 2024

adybbroe commented Nov 22, 2024

adybbroe commented Nov 25, 2024

adybbroe commented Nov 26, 2024

pnuu left a comment

Choose a reason for hiding this comment

adybbroe commented Dec 3, 2024

mraspaud left a comment

Choose a reason for hiding this comment

mraspaud Dec 3, 2024

Choose a reason for hiding this comment

adybbroe Dec 4, 2024

Choose a reason for hiding this comment

mraspaud Dec 4, 2024

Choose a reason for hiding this comment

adybbroe Dec 4, 2024

Choose a reason for hiding this comment

adybbroe Dec 4, 2024

Choose a reason for hiding this comment

mraspaud Dec 5, 2024

Choose a reason for hiding this comment

adybbroe commented Nov 19, 2024 •

edited

Loading

codecov bot commented Nov 20, 2024 •

edited

Loading