Adds support for MOI.AbstractOptimizer that do not support MOI.NLPBlock #381

ValentinKaisermayer · 2022-10-09T15:36:15Z

This adds a caching interface for MOI solvers as well as simple (only linear and quadratic objective and constraints with possible integer variables) support for MOI solvers that do not support the NLBlock interface, e.g. Gurobi or HiGHS.

needs changes interface to use cache SciMLBase.jl#286
For the reinit! to work properly (later) Keep pars in expr ModelingToolkit.jl#1901
closes Support for MathOptInterface.AbstractOptimizer that do not support MathOptInterface.NLPBlock #369

using ModelingToolkit
import Gurobi
import Optimization
import OptimizationMOI

@parameters c = 0.0
@variables x[1:2] = [0.0, 0.0] [bounds = (c, Inf)]
@variables y = 0.0
@variables z = 0.0
@parameters a = 1.0
@parameters b = 1.0
@parameters d = 1.0
@named sys = OptimizationSystem(
    x[1] + x[2], [x..., y, z], [a, b, c, d]; 
    constraints=[
        1.0 ≲ a * x[1]^2 * b + x[2]^2, # non-convex constraint
        x[1]^2 + a*b/d*x[2]^2 ≲ 2.0, 
        a * y - a*z*y + b^2*z*y ~ 1.0, # some dummy constraint
    ]
)
prob = OptimizationProblem(sys, [x[1] => 2.0, x[2] => 0.0], []; grad=true, hess=true)
cache = Optimization.init(prob, Gurobi.Optimizer(), NonConvex=2)
sol = Optimization.solve(cache)
sol.u # ≈ [1, 0]

Optimal solution found (tolerance 1.00e-04)
Best objective 9.999997337719e-01, best bound 9.999993908614e-01, gap 0.0000%

User-callback calls 192, time in user-callback 0.00 sec
4-element Vector{Float64}:
 0.99999973377192
 0.0
 1.0
 0.0

Codecov Report

Merging #381 (1b199e6) into master (672d7ce) will decrease coverage by 4.45%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #381      +/-   ##
==========================================
- Coverage   27.89%   23.44%   -4.45%     
==========================================
  Files          34       36       +2     
  Lines        1678     2026     +348     
==========================================
+ Hits          468      475       +7     
- Misses       1210     1551     +341

Impacted Files	Coverage Δ
lib/OptimizationMOI/src/OptimizationMOI.jl	`1.14% <0.00%> (+1.14%)`	⬆️
lib/OptimizationMOI/src/moi.jl	`0.00% <0.00%> (ø)`
lib/OptimizationMOI/src/nlp.jl	`0.00% <0.00%> (ø)`
src/function/function.jl	`89.18% <ø> (ø)`
src/function/mtk.jl	`81.25% <ø> (ø)`

... and 4 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

ValentinKaisermayer · 2022-10-20T15:32:49Z

builds on #400

lib/OptimizationMOI/src/nlp.jl

lib/OptimizationMOI/src/OptimizationMOI.jl

ChrisRackauckas · 2022-12-02T20:59:46Z

This is pretty awesome. @Vaibhavdixit02 are you reviewing this?

Vaibhavdixit02 · 2022-12-03T04:42:52Z

Yes

ValentinKaisermayer · 2022-12-08T16:16:48Z

No idea why the docs fail. There is no such @assert in the code. Maybe the docs do not build on this branch?

ChrisRackauckas · 2022-12-09T03:21:47Z

It does run the branch

ValentinKaisermayer · 2022-12-09T19:26:20Z

It does run the branch

I think it is not. I added the exact same code to the tests and they pass but the docs not.

ValentinKaisermayer · 2023-01-10T17:50:10Z

Still, I think the docs are not built with this branch. The only place the error can be from is this: https://github.com/SciML/Optimization.jl/blob/master/lib/OptimizationMOI/src/OptimizationMOI.jl#L219
There is no such @assert in my branch.

zsz00 · 2023-01-13T03:58:31Z

what's status now?
@ValentinKaisermayer @ChrisRackauckas

ValentinKaisermayer · 2023-02-14T05:43:16Z

It needs review and the conflicts need to be resolved.

Vaibhavdixit02 · 2023-02-14T06:48:54Z

lib/OptimizationMOI/Project.toml

@@ -5,9 +5,11 @@ version = "0.1.7"

 [deps]
 MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
+ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"


Do we need this dependecy?

Vaibhavdixit02 · 2023-02-14T06:50:36Z

@ValentinKaisermayer this seems to based off on the caching PR but that's marked as draft still, can you clean that up and we will merge that and then this?

ValentinKaisermayer · 2023-02-14T08:35:31Z

lib/OptimizationMOI/src/moi.jl

+    expr_map = get_expr_map(prob.f.sys)
+    expr = repl_getindex!(convert_to_expr(MTK.subs_constants(MTK.objective(prob.f.sys)),
+                                          prob.f.sys; expand_expr = false, expr_map))
+
+    cons = MTK.constraints(prob.f.sys)
+    cons_expr = Vector{Expr}(undef, length(cons))
+    Threads.@sync for i in eachindex(cons)
+        Threads.@spawn cons_expr[i] = repl_getindex!(convert_to_expr(Symbolics.canonical_form(MTK.subs_constants(cons[i])),
+                                                                     prob.f.sys;
+                                                                     expand_expr = false,
+                                                                     expr_map))


MTK is needed here

Although both are in the prob.expr and prob.cons_expr, the parameters are replaced in MTK:
https://github.com/SciML/ModelingToolkit.jl/blob/master/src/systems/optimization/optimizationsystem.jl#L497-L498

zsz00 · 2023-03-17T07:50:59Z

what's status now? can merge this ?
@ValentinKaisermayer @ChrisRackauckas

ChrisRackauckas · 2023-03-17T12:45:32Z

It doesn't pass tests if I'm not mistaken.

src/function/mtk.jl

Vaibhavdixit02 · 2023-04-06T18:38:04Z

@ValentinKaisermayer can you take a look at the Codecov warnings, it seems to be a lot so I am worried (also kind of making reviewing the MOI file painful)

robustifies MOI term collection fix fix fix format do the expression expansion ourselfs adds returncode handling format make sure that simplify returns eventually expand quad expressions dont call on operator changes @asserts to meaningfull exceptions error early if NaN or Inf is in an expression adds test adds opt fix test use process_p_u0_symbolic formatting

ValentinKaisermayer · 2023-04-13T14:38:18Z

I'm pretty certain that the codevov is broken. It marks essentially every line as not hit, but the tests pass, that is not possible.

ChrisRackauckas · 2023-04-13T18:17:51Z

Yes, Codecov has issues that won't be fixed here. That shouldn't get in the way of the review.

lib/OptimizationMOI/src/moi.jl

ValentinKaisermayer · 2023-04-17T11:50:07Z

I took it from here, but ok.
https://github.com/SciML/SciMLBase.jl/blob/ae464bdd7b9cfc299ae8018e217be063edc2cbeb/src/remake.jl#L52

ChrisRackauckas · 2023-04-17T11:54:01Z

That should probably also using nothings.

ChrisRackauckas · 2023-04-17T11:54:37Z

Oh wait, there the reason is that nothing is an allowed answer for a lot of those values.

ChrisRackauckas · 2023-04-17T11:55:31Z

A concretized cache can have a nothing initial condition if u0 is not used by the optimizer (some global optimizers), so maybe that is appropriate here.

ChrisRackauckas · 2023-04-17T11:55:57Z

Also p = nothing is an allowed value. So yes keep it missing.

Vaibhavdixit02 · 2023-04-17T13:18:47Z

Okay that makes sense, I'll dismiss my comments above.

ChrisRackauckas · 2023-04-17T13:20:09Z

🎉

Very happy this is in.

Though what's missing is tutorials on these new features. This and the integer support is not documented anywhere. The docs really need to get up to speed.

Vaibhavdixit02 · 2023-04-17T13:20:39Z

Really great work @ValentinKaisermayer!

ValentinKaisermayer mentioned this pull request Oct 15, 2022

Adds field indicating integrality of the opt vars SciML/SciMLBase.jl#271

Merged

ValentinKaisermayer force-pushed the vk-moi-no-nlp-block branch from a6d69cf to a143979 Compare October 20, 2022 15:22

ChrisRackauckas requested a review from Vaibhavdixit02 October 20, 2022 15:25

ValentinKaisermayer commented Oct 20, 2022

View reviewed changes

lib/OptimizationMOI/src/nlp.jl Outdated Show resolved Hide resolved

ValentinKaisermayer mentioned this pull request Oct 22, 2022

Splits MOI solve into init and solve #400

Closed

ValentinKaisermayer marked this pull request as ready for review October 22, 2022 16:58

ValentinKaisermayer marked this pull request as draft October 25, 2022 15:04

ValentinKaisermayer force-pushed the vk-moi-no-nlp-block branch from 230fe1d to 37f38f2 Compare October 26, 2022 08:28

ValentinKaisermayer marked this pull request as ready for review October 26, 2022 17:15

ValentinKaisermayer commented Oct 26, 2022

View reviewed changes

lib/OptimizationMOI/src/OptimizationMOI.jl Outdated Show resolved Hide resolved

ValentinKaisermayer mentioned this pull request Oct 28, 2022

Implements init dispatch SciML/SciMLBase.jl#294

Merged

Vaibhavdixit02 closed this Oct 29, 2022

Vaibhavdixit02 reopened this Oct 29, 2022

ValentinKaisermayer mentioned this pull request Oct 31, 2022

init interface for reduced overhead in repeated solves #352

Closed

ValentinKaisermayer force-pushed the vk-moi-no-nlp-block branch from 1aa0395 to 74da67d Compare October 31, 2022 17:14

ValentinKaisermayer mentioned this pull request Oct 31, 2022

Keep pars in expr SciML/ModelingToolkit.jl#1901

Draft

ValentinKaisermayer force-pushed the vk-moi-no-nlp-block branch 2 times, most recently from f4d9cb5 to 5606df3 Compare November 27, 2022 11:33

ValentinKaisermayer force-pushed the vk-moi-no-nlp-block branch from 8de3788 to 95db4c4 Compare December 9, 2022 17:46

ValentinKaisermayer mentioned this pull request Jan 10, 2023

Allow to process caches as well SciML/ModelingToolkit.jl#2024

Merged

Vaibhavdixit02 reviewed Feb 14, 2023

View reviewed changes

ValentinKaisermayer commented Feb 14, 2023

View reviewed changes

ValentinKaisermayer mentioned this pull request Feb 20, 2023

Support for MathOptInterface.AbstractOptimizer that do not support MathOptInterface.NLPBlock #369

Closed

Vaibhavdixit02 reviewed Apr 6, 2023

View reviewed changes

src/function/mtk.jl Outdated Show resolved Hide resolved

ValentinKaisermayer closed this Apr 12, 2023

ValentinKaisermayer reopened this Apr 12, 2023

ValentinKaisermayer added 2 commits April 13, 2023 16:02

fix

322c625

ValentinKaisermayer force-pushed the vk-moi-no-nlp-block branch from 5f4b05e to 322c625 Compare April 13, 2023 14:03

adds tests

d9205d9

fix and formatting

1b199e6

ValentinKaisermayer requested a review from Vaibhavdixit02 April 16, 2023 09:24

Vaibhavdixit02 approved these changes Apr 17, 2023

View reviewed changes

lib/OptimizationMOI/src/moi.jl Show resolved Hide resolved

lib/OptimizationMOI/src/moi.jl Show resolved Hide resolved

lib/OptimizationMOI/src/moi.jl Show resolved Hide resolved

lib/OptimizationMOI/src/moi.jl Show resolved Hide resolved

Vaibhavdixit02 merged commit 2639ecb into SciML:master Apr 17, 2023

Vaibhavdixit02 mentioned this pull request May 11, 2023

Document problems of more types than non-linear optimization problems only #525

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds support for MOI.AbstractOptimizer that do not support MOI.NLPBlock #381

Adds support for MOI.AbstractOptimizer that do not support MOI.NLPBlock #381

ValentinKaisermayer commented Oct 9, 2022 •

edited

Loading

codecov bot commented Oct 9, 2022 •

edited

Loading

ValentinKaisermayer commented Oct 20, 2022

ChrisRackauckas commented Dec 2, 2022

Vaibhavdixit02 commented Dec 3, 2022

ValentinKaisermayer commented Dec 8, 2022

ChrisRackauckas commented Dec 9, 2022

ValentinKaisermayer commented Dec 9, 2022

ValentinKaisermayer commented Jan 10, 2023

zsz00 commented Jan 13, 2023 •

edited

Loading

ValentinKaisermayer commented Feb 14, 2023

Vaibhavdixit02 Feb 14, 2023

Vaibhavdixit02 commented Feb 14, 2023

ValentinKaisermayer Feb 14, 2023

ValentinKaisermayer Feb 14, 2023

zsz00 commented Mar 17, 2023

ChrisRackauckas commented Mar 17, 2023

Vaibhavdixit02 commented Apr 6, 2023

ValentinKaisermayer commented Apr 13, 2023

ChrisRackauckas commented Apr 13, 2023

ValentinKaisermayer commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

Vaibhavdixit02 commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

Vaibhavdixit02 commented Apr 17, 2023

Adds support for MOI.AbstractOptimizer that do not support MOI.NLPBlock #381

Adds support for MOI.AbstractOptimizer that do not support MOI.NLPBlock #381

Conversation

ValentinKaisermayer commented Oct 9, 2022 • edited Loading

codecov bot commented Oct 9, 2022 • edited Loading

Codecov Report

ValentinKaisermayer commented Oct 20, 2022

ChrisRackauckas commented Dec 2, 2022

Vaibhavdixit02 commented Dec 3, 2022

ValentinKaisermayer commented Dec 8, 2022

ChrisRackauckas commented Dec 9, 2022

ValentinKaisermayer commented Dec 9, 2022

ValentinKaisermayer commented Jan 10, 2023

zsz00 commented Jan 13, 2023 • edited Loading

ValentinKaisermayer commented Feb 14, 2023

Vaibhavdixit02 Feb 14, 2023

Choose a reason for hiding this comment

Vaibhavdixit02 commented Feb 14, 2023

ValentinKaisermayer Feb 14, 2023

Choose a reason for hiding this comment

ValentinKaisermayer Feb 14, 2023

Choose a reason for hiding this comment

zsz00 commented Mar 17, 2023

ChrisRackauckas commented Mar 17, 2023

Vaibhavdixit02 commented Apr 6, 2023

ValentinKaisermayer commented Apr 13, 2023

ChrisRackauckas commented Apr 13, 2023

ValentinKaisermayer commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

Vaibhavdixit02 commented Apr 17, 2023

ChrisRackauckas commented Apr 17, 2023

Vaibhavdixit02 commented Apr 17, 2023

ValentinKaisermayer commented Oct 9, 2022 •

edited

Loading

codecov bot commented Oct 9, 2022 •

edited

Loading

zsz00 commented Jan 13, 2023 •

edited

Loading