Best-in-time-and-space performance improvement approach

[dhardestylewis]

Improve performance of workflow by eliminating tiles which don't intersect any other projects whatsoever

[dhardestylewis]

Roughly 70% performance savings

based on area of intersecting projects and area of Texas

[dhardestylewis]

@andycarter-pe this involved generating that golden vector image which will tell you where the best available point-cloud imagery exists throughout Texas

[dhardestylewis]

best-in-time-and-space algorithm

Pseudo-code to improve performance of terrain_aggregator workflow described in #44

1. ✅ dissolve each bucket of tiles separated by the characteristics of the table described in Draft selection table for best-in-time-and-space version of the terrain_aggregator workflow #44 into a single polygon per bucket, creating table of buckets of tiles

2. ✅ ensure buckets are ordered similarly to how described in this table, although I will be making year first priority characteristic after hearing feedback from TNRIS

3. ✅ insert polygon 1 (ie bucket 1's dissolved polygon) as row at top of table best-in-time-and-space by bucket

4. as a loop, starting from the 2nd bucket:

   1. ✅ take polygons 1,...,i

   2. ✅ polygon i minus union(polygons 1,...,i-1) = polygon i's remaining tiles as a single polygon

   3. ⏳ store polygon i's remaining tiles as a single polygon as a new row in table best-in-time-and-space by bucket

table best-in-time-and-space by bucket is a set of polygons that cover Texas without gaps and without intersection and describe the highest quality bucket of tiles at every location in Texas.

table best-in-time-and-space by bucket can be merged against the table of buckets of tiles and intersected against the original table of TNRIS Lidar tiles to pull back out a table of best-in-time-and-space tiles at every location in Texas.

[dhardestylewis]

Best-in-time-and-space result can be used to generate highest quality elevation data directly from Lidar point cloud

[dhardestylewis]

Link to @andycarter-pe's approach in Python.

• implement in PostgreSQL

https://github.com/andycarter-pe/TNRIS_Terrain_Harvest/blob/master/02_jupyter_notebook/cwe_get_tnris_lidar_collections_20220805.ipynb