Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to i_overlay ~~1.8~~ 1.9 #1275

Merged
merged 3 commits into from
Dec 1, 2024
Merged

Upgrade to i_overlay ~~1.8~~ 1.9 #1275

merged 3 commits into from
Dec 1, 2024

Conversation

michaelkirk
Copy link
Member

@michaelkirk michaelkirk commented Nov 26, 2024

  • I agree to follow the project's code of conduct.
  • I added an entry to CHANGES.md if knowledge of this change could be valuable to users.

This is was a draft because it's a little bit slower (see "Performance"). It might solve a problem #1273 (?), in which case, it's probably worth merging this and taking the modest perf hit.

There's actually an even newer release (1.9) (see my branch: mkirk/i_overlay_1.9) but it has some more serious performance regressions. I'll open an issue upstream about that. updated: here it is: iShape-Rust/iOverlay#13

update:

I now think we should update all the way to 1.9.

@urschrei has verified that both 1.8 and 1.9 solve his crash and I have at least a probable explanation for the performance regression seen in 1.9 - see iShape-Rust/iOverlay#13

(I've also verified the crash is fixed with my branch mkirk/cascaded_union)

Further, the perf regression is with our benchmarks which use synthetic geometries. I think it's more interesting that @urschrei is seeing moderate performance improvements with his real world data. Plus, I'd rather leave any performance twiddling as a responsibility of upstream rather than having our own customizations optimized for our benchmarks.


Description of Changes

i_overlay 1.8 introduced a new trait based approach which allows us to
remove some of our own trait juggling.

However, our old friend, the orphan trait rule, prevents this from
happening easily. For now, I'm creating wrapper structs in geo for this
algo.

i_overlay 1.8 also introduced a new method signature for doing bool_ops,
removing the old ones. This caused our trait juggling to infinitely
recurse: See #1270

Now that we're using the new methods, the problem is avoided.

Note that I've also opened iShape-Rust/iOverlay#12 to discuss using semver with i_overlay for future releases. For now though, until we hear otherwise, I think we should assume that breaking changes will occur in minor release - the Y in X.Y.Z.

FIXES #1270

Performance

Bench output is a little mixed, but mostly a bit slower:

bench output with i_overlay 1.8
$ cargo bench --bench=boolean_ops -- --baseline=main                                                                                                  
                                                                                                                                                      
   Compiling geo v0.29.2 (/Users/mkirk/src/georust/geo/geo)                                                                                           
   Compiling jts-test-runner v0.1.0 (/Users/mkirk/src/georust/geo/jts-test-runner)                                                                    
   Compiling geo-bool-ops-benches v0.1.0 (/Users/mkirk/src/georust/geo/geo-bool-ops-benches)                                                          
    Finished `bench` profile [optimized] target(s) in 4.23s                                                                                           
     Running benches/boolean_ops.rs (target/release/deps/boolean_ops-e4f1a59740387ff9)                                                                
Gnuplot not found, using plotters backend                                                                                                             
Circular polygon boolean-ops/bops::intersection/256                                                                                                   
                        time:   [456.73 µs 460.23 µs 464.58 µs]                                                                                       
                        change: [-3.5765% -3.1482% -2.6752%] (p = 0.00 < 0.05)                                                                        
                        Performance has improved.                                                                                                     
Found 1 outliers among 10 measurements (10.00%)                                                                                                       
  1 (10.00%) high severe                                                                                                                              
Circular polygon boolean-ops/bops::union/256                                                                                                          
                        time:   [466.67 µs 468.54 µs 472.49 µs]                                                                                       
                        change: [-2.0193% -1.4161% -0.7974%] (p = 0.00 < 0.05)                                                                        
                        Change within noise threshold.                                                                                                
Circular polygon boolean-ops/bops::intersection/512                                                                                                   
                        time:   [1.4295 ms 1.4399 ms 1.4487 ms]                                                                                       
                        change: [+3.6040% +4.3318% +5.0995%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::union/512                                                                                                          
                        time:   [1.4589 ms 1.4682 ms 1.4806 ms]                                                                                       
                        change: [+2.0346% +3.1412% +4.1578%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.
Circular polygon boolean-ops/bops::intersection/1024                                                                                                  
                        time:   [4.4927 ms 4.5238 ms 4.5523 ms]                                                                                       
                        change: [+5.2243% +6.2935% +7.2200%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::union/1024                                                                                                         
                        time:   [4.5848 ms 4.6039 ms 4.6151 ms]                                                                                       
                        change: [+6.8274% +7.5795% +8.3237%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::intersection/2048                                                                                                  
                        time:   [13.835 ms 13.969 ms 14.052 ms]                                                                                       
                        change: [+7.9433% +9.2997% +10.496%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::union/2048                                                                                                         
                        time:   [13.962 ms 13.988 ms 14.025 ms]                                                                                       
                        change: [+10.815% +12.081% +13.461%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::intersection/4096                                                                                                  
                        time:   [50.950 ms 51.183 ms 51.395 ms]                                                                                       
                        change: [+14.277% +15.022% +15.799%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::union/4096                                                                                                         
                        time:   [50.868 ms 51.255 ms 51.602 ms]                                                                                       
                        change: [+13.931% +14.919% +15.835%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Found 2 outliers among 10 measurements (20.00%)                                                                                                       
  1 (10.00%) low mild                                                                                                                                 
  1 (10.00%) high mild                                                                                                                                
Circular polygon boolean-ops/bops::intersection/8192                                                                                                  
                        time:   [188.90 ms 189.52 ms 190.23 ms]                                                                                       
                        change: [+12.707% +15.285% +16.939%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Circular polygon boolean-ops/bops::union/8192                                                                                                         
                        time:   [188.93 ms 189.41 ms 189.97 ms]                                                                                       
                        change: [+13.351% +14.859% +16.227%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Found 1 outliers among 10 measurements (10.00%)                                                                                                       
  1 (10.00%) high mild

bench output with i_overlay 1.9
georust/geo$ cargo bench --bench=boolean_ops -- --baseline=main
Circular polygon boolean-ops/bops::intersection/256                                                                                                   
                        time:   [438.45 µs 440.68 µs 445.14 µs]                                                                                       
                        change: [-7.0558% -6.1120% -4.8985%] (p = 0.00 < 0.05)                                                                        
                        Performance has improved.                                                                                                     
Circular polygon boolean-ops/bops::union/256                                                                                                          
                        time:   [448.53 µs 450.78 µs 454.71 µs]                                                                                       
                        change: [-6.2725% -5.7614% -5.0363%] (p = 0.00 < 0.05)                                                                        
                        Performance has improved.                                                                                                     
Found 2 outliers among 10 measurements (20.00%)                                                                                                       
  1 (10.00%) high mild                                                                                                                                
  1 (10.00%) high severe                                                                                                                              
Circular polygon boolean-ops/bops::intersection/512                                                                                                   
                        time:   [1.4381 ms 1.4409 ms 1.4442 ms]                                                                                       
                        change: [+4.0769% +4.3027% +4.4898%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Found 1 outliers among 10 measurements (10.00%)                                                                                                       
  1 (10.00%) low mild                                                                                                                                 
Circular polygon boolean-ops/bops::union/512                                                                                                          
                        time:   [1.4642 ms 1.4880 ms 1.5130 ms]                                                                                       
                        change: [+1.8288% +3.2331% +4.8314%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Found 1 outliers among 10 measurements (10.00%)                                                                                                       
  1 (10.00%) high mild                                                                                                                                
Circular polygon boolean-ops/bops::intersection/1024                                                                                                  
                        time:   [4.3295 ms 4.3493 ms 4.3684 ms]                                                                                       
                        change: [+1.1227% +2.1430% +3.0016%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.                                                                                                    
Found 1 outliers among 10 measurements (10.00%)                                                                                                       
  1 (10.00%) high mild                                                                                                                                
Circular polygon boolean-ops/bops::union/1024                                                                                                         
                        time:   [4.3625 ms 4.3922 ms 4.4093 ms]                                                                                       
                        change: [+1.7493% +2.4971% +3.2026%] (p = 0.00 < 0.05)                                                                        
                        Performance has regressed.
Circular polygon boolean-ops/bops::intersection/2048
                        time:   [14.057 ms 14.115 ms 14.157 ms]
                        change: [+9.0224% +10.438% +11.658%] (p = 0.00 < 0.05)
                        Performance has regressed.
Circular polygon boolean-ops/bops::union/2048
                        time:   [14.012 ms 14.071 ms 14.134 ms]
                        change: [+11.212% +12.141% +13.317%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
Circular polygon boolean-ops/bops::intersection/4096
                        time:   [360.66 ms 361.18 ms 361.83 ms]
                        change: [+709.38% +713.12% +716.34%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high severe
Circular polygon boolean-ops/bops::union/4096
                        time:   [361.35 ms 361.71 ms 362.13 ms]
                        change: [+710.58% +713.76% +716.59%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking Circular polygon boolean-ops/bops::intersection/8192: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 15.4s.
Circular polygon boolean-ops/bops::intersection/8192
                        time:   [1.5360 s 1.5404 s 1.5452 s]
                        change: [+816.16% +837.03% +850.36%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking Circular polygon boolean-ops/bops::union/8192: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 15.4s.
Circular polygon boolean-ops/bops::union/8192
                        time:   [1.5359 s 1.5375 s 1.5393 s]
                        change: [+820.30% +832.33% +843.12%] (p = 0.00 < 0.05)
                        Performance has regressed.

i_overlay 1.8 introduced a new trait based approach which allows us to
remove some of our own trait juggling.

However, our old friend, the orphan trait rule, prevents this from
happening easily. For now, I'm creating wrapper structs in geo for this
algo.

i_overlay 1.8 also introduced a new method signature for doing bool_ops,
removing the old ones. This caused our trait juggling to infinitely
recurse: See #1270

Now that we're using the new methods, the problem is avoided.
@urschrei
Copy link
Member

As noted in #1273, I'm seeing a perf improvement for a large workload.

Note there is a serious performance regression for larger geometries in our benchmarks vs. 1.8.

    cargo bench --bench=boolean_ops -- --baseline=main
    ... small ones are fine, but big ones have huge regression
    Circular polygon boolean-ops/bops::union/1024
                            time:   [4.3625 ms 4.3922 ms 4.4093 ms]
                            change: [+1.7493% +2.4971% +3.2026%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Circular polygon boolean-ops/bops::intersection/2048
                            time:   [14.057 ms 14.115 ms 14.157 ms]
                            change: [+9.0224% +10.438% +11.658%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Circular polygon boolean-ops/bops::union/2048
                            time:   [14.012 ms 14.071 ms 14.134 ms]
                            change: [+11.212% +12.141% +13.317%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 1 outliers among 10 measurements (10.00%)
      1 (10.00%) high severe
    Circular polygon boolean-ops/bops::intersection/4096
                            time:   [360.66 ms 361.18 ms 361.83 ms]
                            change: [+709.38% +713.12% +716.34%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 2 outliers among 10 measurements (20.00%)
      2 (20.00%) high severe
    Circular polygon boolean-ops/bops::union/4096
                            time:   [361.35 ms 361.71 ms 362.13 ms]
                            change: [+710.58% +713.76% +716.59%] (p = 0.00 < 0.05)
                            Performance has regressed.

Likely this is because there is a new heuristic for "solver" strategies
based on how big your input is - the cliff likely indicates the switch
to one of the solvers used for larger inputs.

See iShape-Rust/iOverlay#13

There are significant gains with some "real world" data, e.g. urshrei's cascaded
union tests:
#1273 (comment)

I'm more inclined to trust that more realistic data over the synthetic
geometries we use in our benchmarks.

Plus, I'd prefer to leave in performance twiddling to upstream and
contribute changes there rather than fine-tuning things at our own
call sites.
@michaelkirk michaelkirk changed the title Upgrade to i_overlay 1.8 Upgrade to i_overlay ~~1.8~~ 1.9 Nov 26, 2024
@michaelkirk michaelkirk marked this pull request as ready for review November 26, 2024 23:32
@michaelkirk
Copy link
Member Author

update:

I now think we should update all the way to 1.9.

@urschrei has verified that both 1.8 and 1.9 solve his crash and I have at least a probable explanation for the performance regression seen in 1.9 - see iShape-Rust/iOverlay#13

(I've also verified the crash is fixed with my branch mkirk/cascaded_union)

Further, the perf regression is with our benchmarks which use synthetic geometries. I think it's more interesting that @urschrei is seeing moderate performance improvements with his real world data. Plus, I'd rather leave any performance twiddling as a responsibility of upstream rather than having our own customizations optimized for our benchmarks.

@urschrei
Copy link
Member

I agree that 1.9 is the way to go. iShape-Rust/iOverlay#13 (comment) is interesting – it'll become clearer in time whether there's a need for an additional API with a selectable solver in geo, or whether a heuristic approach in i_overlay will provide good enough perf for most workloads. Thanks for your detective work on this @michaelkirk

@frewsxcv frewsxcv added this pull request to the merge queue Dec 1, 2024
Merged via the queue into main with commit 5cc40ad Dec 1, 2024
18 checks passed
@frewsxcv frewsxcv deleted the mkirk/i_overlay_1.8 branch December 1, 2024 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Infinite recursion with i_overlay 1.8
3 participants