Add a conversion mechanism between NLPModel to OptimizationProblem

[alonsoC1s]

Checklist

• Appropriate tests were added
• Any code changes were done in a way that does not break public API
• All documentation related to code changes were updated
• The new code follows the
  contributor guidelines, in particular the SciML Style Guide and
  COLPRAC.
• Any new documentation only uses public API

Additional context

Would close #790 by adding a new constructor OptimizationProblem(model::AbstractNLPModel, ...) that makes the problem defined as an NLPModel available as an OptimizationProblem and thus, makes it available to the rest of the SciML ecosystem

So far the appropriate tests are still missing and the public API documentation hasn't been updated

[alonsoC1s]

@Vaibhavdixit02 I'm working on the tests and I ran into an issue. Since OptimizationFunction doesn't define all fields (e.g. cons_h), it stays as missing and throws an error deeper down in the solvers. Those fields are not directly available from the NLPModel API. How should I compute them? Re-introduce the ADType?

[Vaibhavdixit02]

Can you show the code you ran and the error?

[alonsoC1s]

I was manually running through the tests

    hs5f(u, p) = sin(u[1] + u[2]) + (u[1] - u[2])^2 - (3 / 2) * u[1] + (5 / 2)u[2] + 1
    f = Optimization.OptimizationFunction((u, p) -> hs5f(u, p), Optimization.AutoForwardDiff())
    lb = [-1.5; -3]
    ub = [4; 3]
    u0 = [0.0; 0.0]
    oprob = Optimization.OptimizationProblem(
        f, u0, lb = lb, ub = ub, sense = Optimization.MinSense)
    nlpmo = NLPModelsTest.HS5()

    converted = OptimizationNLPModels.OptimizationProblem(nlpmo)

solve(converted, Optim.IPNewton())

And the full error is

julia> solve(converted, Optim.IPNewton())
ERROR: MethodError: objects of type Nothing are not callable
Stacktrace:
  [1] (::OptimizationOptimJL.var"#29#36"{OptimizationCache{…}})(h::Matrix{Float64}, θ::Vector{Float64}, λ::Vector{Float64})
    @ OptimizationOptimJL ~/.julia/packages/OptimizationOptimJL/hDX5k/src/OptimizationOptimJL.jl:418
  [2] hessianI!(h::Matrix{…}, x::Vector{…}, constraints::TwiceDifferentiableConstraints{…}, λcI::Vector{…}, μ::Int64)
    @ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/solvers/constrained/ipnewton/interior.jl:462
  [3] hessianI(x::Vector{…}, constraints::TwiceDifferentiableConstraints{…}, λcI::Vector{…}, μ::Int64)
    @ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/solvers/constrained/ipnewton/interior.jl:476
  [4] initial_state(method::IPNewton{…}, options::Optim.Options{…}, d::TwiceDifferentiable{…}, constraints::TwiceDifferentiableConstraints{…}, initial_x::Vector{…})
    @ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/solvers/constrained/ipnewton/ipnewton.jl:167
  [5] optimize(d::TwiceDifferentiable{…}, constraints::TwiceDifferentiableConstraints{…}, initial_x::Vector{…}, method::IPNewton{…}, options::Optim.Options{…})
    @ Optim ~/.julia/packages/Optim/ZhuZN/src/multivariate/solvers/constrained/ipnewton/interior.jl:229
  [6] __solve(cache::OptimizationCache{…})
    @ OptimizationOptimJL ~/.julia/packages/OptimizationOptimJL/hDX5k/src/OptimizationOptimJL.jl:443
  [7] solve!(cache::OptimizationCache{…})
    @ SciMLBase ~/.julia/packages/SciMLBase/vhP5T/src/solve.jl:188
  [8] solve(::SciMLBase.OptimizationProblem{…}, ::IPNewton{…}; kwargs::@Kwargs{})
    @ SciMLBase ~/.julia/packages/SciMLBase/vhP5T/src/solve.jl:96
  [9] solve(::SciMLBase.OptimizationProblem{…}, ::IPNewton{…})
    @ SciMLBase ~/.julia/packages/SciMLBase/vhP5T/src/solve.jl:93
 [10] top-level scope
    @ REPL[102]:1
Some type information was truncated. Use `show(err)` to see complete types.

The actual error is not that informative, the line where it breaks is

        cache.f.cons_h(res, θ)

[Vaibhavdixit02]

Okay so the issue is because IPNewton from Optim expects cons_h to be defined as you said, the conversion doesn't run instantiate_function so the fields are not populated by us either. So this is not really an issue but basically incompatibility of the cutest models with IPNewton.

You can add a adtype argument to the conversion so it can populate the missing fields using the AD backend choice.

Alternatively just use a different optimizer here, Ipopt should be able to work with the lagrangian directly.

[Vaibhavdixit02]

@alonsoC1s please pull the branch before pushing the next commit since I made some changes above

[alonsoC1s]

@Vaibhavdixit02 I tried using LBFGS as in the changes you made but I was getting some errors that looked unrelated. That's why I had changed it back. Let me investigate further

[Vaibhavdixit02]

Yeah sorry about that, this should work now

[Vaibhavdixit02]

Looks like it's missing the new packages in the docs environment can you add that, and also run the formatter. We should be done then!

[Vaibhavdixit02]

@alonsoC1s the documentation is failing and looks like a real failure can you take a look?

[Vaibhavdixit02]

I think this should fix it

[amontoison]

Great work!!!
We should maybe add an entry about it in the README.md?

cc @tmigot

[tmigot]

That's cool! Thanks @alonsoC1s @Vaibhavdixit02
We should probably think of a wrapper the other way around too

[Vaibhavdixit02]

@amontoison yeah I'll update the readme it's a bit outdated now

@tmigot sure it shouldn't be a lot of work to do it the other way around too. Just to clarify our main motivation for this PR was to access Cutest benchmarks with Optimization.jl

[alonsoC1s]

@tmigot That sounds interesting and useful. I think Optimization.jl wraps more optimizers, so having the mechanism to switch between could be nice