Update documentation

SciML · Dec 7, 2023 · c86413c · c86413c
1 parent ad93e39
commit c86413c
Show file tree

Hide file tree

Showing 25 changed files with 691 additions and 452 deletions.
diff --git a/Project.toml b/Project.toml
@@ -66,7 +66,7 @@ NonlinearProblemLibrary = "0.1"
 Pkg = "1"
 PrecompileTools = "1"
 Printf = "<0.0.1, 1"
-Random = "1"
+Random = "<0.0.1, 1"
 RecursiveArrayTools = "2"
 Reexport = "0.2, 1"
 SafeTestsets = "0.1"

diff --git a/docs/Project.toml b/docs/Project.toml
@@ -9,6 +9,7 @@ LinearSolve = "7ed4a6bd-45f5-4d41-b270-4a48e9bafcae"
 ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"
 NonlinearSolve = "8913a72c-1f9b-4ce2-8d82-65094dcecaec"
 NonlinearSolveMINPACK = "c100e077-885d-495a-a2ea-599e143bf69d"
+Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 SciMLBase = "0bca4576-84f4-4d90-8ffe-ffa030f20462"
 SciMLNLSolve = "e9a6253c-8580-4d32-9898-8661bb511710"
 SimpleNonlinearSolve = "727e6d20-b764-4bd8-a329-72de5adea6c7"
@@ -26,12 +27,13 @@ Documenter = "1"
 IncompleteLU = "0.2"
 LinearSolve = "2"
 ModelingToolkit = "8"
-NonlinearSolve = "1, 2"
+NonlinearSolve = "3"
 NonlinearSolveMINPACK = "0.1"
+Random = "<0.0.1, 1"
 SciMLBase = "2.4"
 SciMLNLSolve = "0.1"
 SimpleNonlinearSolve = "1"
 StaticArrays = "1"
-SteadyStateDiffEq = "1.10, 2"
+SteadyStateDiffEq = "2"
 Sundials = "4.11"
 Symbolics = "4, 5"
diff --git a/docs/src/api/fastlevenbergmarquardt.md b/docs/src/api/fastlevenbergmarquardt.md
@@ -0,0 +1,19 @@
+# FastLevenbergMarquardt.jl
+
+This is a wrapper package for importing solvers from FastLevenbergMarquardt.jl into the
+SciML interface. Note that these solvers do not come by default, and thus one needs to
+install the package before using these solvers:
+
+```julia
+using Pkg
+Pkg.add("FastLevenbergMarquardt")
+using FastLevenbergMarquardt
+```
+
+These methods can be used independently of the rest of NonlinearSolve.jl
+
+## Solver API
+
+```@docs
+FastLevenbergMarquardtJL
+```
diff --git a/docs/src/api/leastsquaresoptim.md b/docs/src/api/leastsquaresoptim.md
@@ -0,0 +1,19 @@
+# LeastSquaresOptim.jl
+
+This is a wrapper package for importing solvers from LeastSquaresOptim.jl into the SciML
+interface. Note that these solvers do not come by default, and thus one needs to install
+the package before using these solvers:
+
+```julia
+using Pkg
+Pkg.add("LeastSquaresOptim")
+using LeastSquaresOptim
+```
+
+These methods can be used independently of the rest of NonlinearSolve.jl
+
+## Solver API
+
+```@docs
+LeastSquaresOptimJL
+```
diff --git a/docs/src/api/minpack.md b/docs/src/api/minpack.md
@@ -1,8 +1,8 @@
 # MINPACK.jl
 
 This is a wrapper package for importing solvers from Sundials into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
-the package before using these solvers:
+Note that these solvers do not come by default, and thus one needs to install the package
+before using these solvers:
 
 ```julia
 using Pkg

diff --git a/docs/src/api/nlsolve.md b/docs/src/api/nlsolve.md
@@ -1,8 +1,8 @@
 # NLsolve.jl
 
 This is a wrapper package for importing solvers from NLsolve.jl into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
-the package before using these solvers:
+Note that these solvers do not come by default, and thus one needs to install the package
+before using these solvers:
 
 ```julia
 using Pkg

diff --git a/docs/src/api/simplenonlinearsolve.md b/docs/src/api/simplenonlinearsolve.md
@@ -6,7 +6,8 @@ These methods can be used independently of the rest of NonlinearSolve.jl
 
 ### Interval Methods
 
-These methods are suited for interval (scalar) root-finding problems, i.e. `IntervalNonlinearProblem`.
+These methods are suited for interval (scalar) root-finding problems,
+i.e. `IntervalNonlinearProblem`.
 
 ```@docs
 ITP
@@ -18,14 +19,15 @@ Brent
 
 ### General Methods
 
-These methods are suited for any general nonlinear root-finding problem, i.e. `NonlinearProblem`.
+These methods are suited for any general nonlinear root-finding problem, i.e.
+`NonlinearProblem`.
 
 ```@docs
 SimpleNewtonRaphson
-Broyden
+SimpleBroyden
 SimpleHalley
-Klement
+SimpleKlement
 SimpleTrustRegion
 SimpleDFSane
-LBroyden
+SimpleLimitedMemoryBroyden
 ```
diff --git a/docs/src/api/steadystatediffeq.md b/docs/src/api/steadystatediffeq.md
@@ -1,8 +1,8 @@
 # SteadyStateDiffEq.jl
 
 This is a wrapper package for using ODE solvers from
-[DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/) into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
+[DifferentialEquations.jl](https://docs.sciml.ai/DiffEqDocs/stable/) into the SciML
+interface. Note that these solvers do not come by default, and thus one needs to install
 the package before using these solvers:
 
 ```julia

diff --git a/docs/src/api/sundials.md b/docs/src/api/sundials.md
@@ -1,8 +1,8 @@
 # Sundials.jl
 
 This is a wrapper package for importing solvers from Sundials into the SciML interface.
-Note that these solvers do not come by default, and thus one needs to install
-the package before using these solvers:
+Note that these solvers do not come by default, and thus one needs to install the package
+before using these solvers:
 
 ```julia
 using Pkg

diff --git a/docs/src/basics/FAQ.md b/docs/src/basics/FAQ.md
@@ -8,7 +8,7 @@ On the test example:
 ```@example
 using NonlinearSolve, BenchmarkTools
 
-N = 100_000;
+const N = 100_000;
 levels = 1.5 .* rand(N);
 out = zeros(N);
 myfun(x, lv) = x * sin(x) - lv
@@ -31,8 +31,62 @@ end
 @btime f2(out, levels, 1.0)
 ```
 
-MATLAB 2022a achieves 1.66s. Try this code yourself: we receive 0.06 seconds, or a 28x speedup.
-This example is still not optimized in the Julia code, and we expect an improvement in a near
-future version.
+MATLAB 2022a achieves 1.66s. Try this code yourself: we receive 0.009 seconds, or a 184x
+speedup.
 
 For more information on performance of SciML, see the [SciMLBenchmarks](https://docs.sciml.ai/SciMLBenchmarksOutput/stable/).
+
+## The solver tried to set a Dual Number in my Vector of Floats.How do I fix that?
+
+This is a common problem that occurs if the code was not written to be generic based on the
+input types. For example, consider this example taken from
+[this issue](https://github.com/SciML/NonlinearSolve.jl/issues/298)
+
+```@example dual_error_faq
+using NonlinearSolve, Random
+
+function fff_incorrect(var, p)
+    v_true = [1.0, 0.1, 2.0, 0.5]
+    xx = [1.0, 2.0, 3.0, 4.0]
+    xx[1] = var[1] - v_true[1]
+    return var - v_true
+end
+
+v_true = [1.0, 0.1, 2.0, 0.5]
+v_init = v_true .+ randn!(similar(v_true)) * 0.1
+
+prob_oop = NonlinearLeastSquaresProblem{false}(fff_incorrect, v_init)
+try
+    sol = solve(prob_oop, LevenbergMarquardt(); maxiters = 10000, abstol = 1e-8)
+catch e
+    @error e
+end
+```
+
+Essentially what happened was, NonlinearSolve checked that we can use ForwardDiff.jl to
+differentiate the function based on the input types. However, this function has
+`xx = [1.0, 2.0, 3.0, 4.0]` followed by a `xx[1] = var[1] - v_true[1]` where `var` might
+be a Dual number. This causes the error. To fix it:
+
+ 1. Specify the `autodiff` to be `AutoFiniteDiff`
+
+```@example dual_error_faq
+sol = solve(prob_oop, LevenbergMarquardt(; autodiff = AutoFiniteDiff()); maxiters = 10000,
+    abstol = 1e-8)
+```
+
+This worked but, Finite Differencing is not the recommended approach in any scenario.
+Instead, rewrite the function to use
+[PreallocationTools.jl](https://github.com/SciML/PreallocationTools.jl) or write it as
+
+```@example dual_error_faq
+function fff_correct(var, p)
+    v_true = [1.0, 0.1, 2.0, 0.5]
+    xx = eltype(var)[1.0, 2.0, 3.0, 4.0]
+    xx[1] = var[1] - v_true[1]
+    return xx - v_true
+end
+
+prob_oop = NonlinearLeastSquaresProblem{false}(fff_correct, v_init)
+sol = solve(prob_oop, LevenbergMarquardt(); maxiters = 10000, abstol = 1e-8)
+```
diff --git a/docs/src/basics/NonlinearFunctions.md b/docs/src/basics/NonlinearFunctions.md
@@ -1,10 +1,10 @@
 # [NonlinearFunctions and Jacobian Types](@id nonlinearfunctions)
 
 The SciML ecosystem provides an extensive interface for declaring extra functions
-associated with the differential equation's data. In traditional libraries, there
-is usually only one option: the Jacobian. However, we allow for a large array
-of pre-computed functions to speed up the calculations. This is offered via the
-`NonlinearFunction` types, which can be passed to the problems.
+associated with the differential equation's data. In traditional libraries, there is usually
+only one option: the Jacobian. However, we allow for a large array of pre-computed functions
+to speed up the calculations. This is offered via the `NonlinearFunction` types, which can
+be passed to the problems.
 
 ## Function Type Definitions
 

diff --git a/docs/src/basics/NonlinearProblem.md b/docs/src/basics/NonlinearProblem.md
@@ -1,13 +1,17 @@
 # [Nonlinear Problems](@id problems)
 
-## The Three Types of Nonlinear Problems
+## The Four Types of Nonlinear Problems
 
-NonlinearSolve.jl tackles three related types of nonlinear systems:
+NonlinearSolve.jl tackles four related types of nonlinear systems:
 
- 1. Interval rootfinding problems. I.e., find the ``t \in [t_0, t_f]`` such that ``f(t) = 0``.
+ 1. Interval rootfinding problems. I.e., find the ``t \in [t_0, t_f]`` such that
+    ``f(t) = 0``.
  2. Systems of nonlinear equations, i.e., find the ``u`` such that ``f(u) = 0``.
  3. Steady state problems, i.e., find the ``u`` such that ``u' = f(u,t)`` has reached steady state,
     i.e., ``0 = f(u, ∞)``.
+ 4. The nonlinear least squares problem, which is an under/over-constrained nonlinear system
+    which might not be satisfiable, i.e. there may be no `u` such that `f(u) = 0`, and thus
+    we find the `u` which minimizes `||f(u)||` in the least squares sense.
 
 The first is for solving scalar rootfinding problems, i.e., finding a single number, and
 requires that a bracketing interval is known. For a bracketing interval, one must have that
@@ -16,9 +20,9 @@ interval.
 
 !!! note
 
-    Interval rootfinding problems allow for `f` to return an array, in which case the interval
-    rootfinding problem is interpreted as finding the first `t` such that any of the components
-    of the array hit zero.
+    Interval rootfinding problems allow for `f` to return an array, in which case the
+    interval rootfinding problem is interpreted as finding the first `t` such that any of
+    the components of the array hit zero.
 
 The second type of nonlinear system can be multidimensional, and thus no ordering nor
 boundaries are assumed to be known. For a system of nonlinear equations, `f` can return
@@ -43,4 +47,5 @@ ODE `u' = f(u,t)`.
 SciMLBase.IntervalNonlinearProblem
 SciMLBase.NonlinearProblem
 SciMLBase.SteadyStateProblem
+SciMLBase.NonlinearLeastSquaresProblem
 ```
diff --git a/docs/src/basics/solve.md b/docs/src/basics/solve.md
@@ -6,28 +6,28 @@ solve(prob::SciMLBase.NonlinearProblem, args...; kwargs...)
 
 ## General Controls
 
-- `alias_u0::Bool`: Whether to alias the initial condition or use a copy.
-  Defaults to `false`.
-- `internal_norm::Function`: The norm used by the solver. Default depends on algorithm
-  choice.
+  - `alias_u0::Bool`: Whether to alias the initial condition or use a copy.
+    Defaults to `false`.
+  - `internal_norm::Function`: The norm used by the solver. Default depends on algorithm
+    choice.
 
 ## Iteration Controls
 
-- `maxiters::Int`: The maximum number of iterations to perform. Defaults to `1000`.
-- `abstol::Number`: The absolute tolerance.
-- `reltol::Number`: The relative tolerance.
-- `termination_condition`: Termination Condition from DiffEqBase. Defaults to
-  `AbsSafeBestTerminationMode()` for `NonlinearSolve.jl` and `AbsTerminateMode()` for
-  `SimpleNonlinearSolve.jl`.
+  - `maxiters::Int`: The maximum number of iterations to perform. Defaults to `1000`.
+  - `abstol::Number`: The absolute tolerance.
+  - `reltol::Number`: The relative tolerance.
+  - `termination_condition`: Termination Condition from DiffEqBase. Defaults to
+    `AbsSafeBestTerminationMode()` for `NonlinearSolve.jl` and `AbsTerminateMode()` for
+    `SimpleNonlinearSolve.jl`.
 
 ## Tracing Controls
 
 These are exclusively available for native `NonlinearSolve.jl` solvers.
 
-- `show_trace`: Must be `Val(true)` or `Val(false)`. This controls whether the trace is
-  displayed to the console. (Defaults to `Val(false)`)
-- `trace_level`: Needs to be one of Trace Objects: [`TraceMinimal`](@ref),
-  [`TraceWithJacobianConditionNumber`](@ref), or [`TraceAll`](@ref). This controls the
-  level of detail of the trace. (Defaults to `TraceMinimal()`)
-- `store_trace`: Must be `Val(true)` or `Val(false)`. This controls whether the trace is
-  stored in the solution object. (Defaults to `Val(false)`)
+  - `show_trace`: Must be `Val(true)` or `Val(false)`. This controls whether the trace is
+    displayed to the console. (Defaults to `Val(false)`)
+  - `trace_level`: Needs to be one of Trace Objects: [`TraceMinimal`](@ref),
+    [`TraceWithJacobianConditionNumber`](@ref), or [`TraceAll`](@ref). This controls the
+    level of detail of the trace. (Defaults to `TraceMinimal()`)
+  - `store_trace`: Must be `Val(true)` or `Val(false)`. This controls whether the trace is
+    stored in the solution object. (Defaults to `Val(false)`)
diff --git a/docs/src/index.md b/docs/src/index.md
@@ -1,20 +1,19 @@
 # NonlinearSolve.jl: High-Performance Unified Nonlinear Solvers
 
-NonlinearSolve.jl is a unified interface for the nonlinear solving packages of
-Julia. The package includes its own high-performance nonlinear solvers which include the
-ability to swap out to fast direct and iterative linear solvers, along with the
-ability to use sparse automatic differentiation for Jacobian construction and
-Jacobian-vector products. NonlinearSolve.jl interfaces with other packages of the Julia ecosystem
-to make it easy to test alternative solver packages and pass small types to
-control algorithm swapping. It also interfaces with the
-[ModelingToolkit.jl](https://docs.sciml.ai/ModelingToolkit/stable/) world of symbolic
-modeling to allow for automatically generating high-performance code.
-
-Performance is key: the current methods are made to be highly performant on
-scalar and statically sized small problems, with options for large-scale systems.
-If you run into any performance issues, please file an issue.
-Consult the [NonlinearSystemSolvers](@ref nonlinearsystemsolvers) page for
-information on how to import solvers from different packages.
+NonlinearSolve.jl is a unified interface for the nonlinear solving packages of Julia. The
+package includes its own high-performance nonlinear solvers which include the ability to
+swap out to fast direct and iterative linear solvers, along with the ability to use sparse
+automatic differentiation for Jacobian construction and Jacobian-vector products.
+NonlinearSolve.jl interfaces with other packages of the Julia ecosystem to make it easy to
+test alternative solver packages and pass small types to control algorithm swapping. It also
+interfaces with the [ModelingToolkit.jl](https://docs.sciml.ai/ModelingToolkit/stable/)
+world of symbolic modeling to allow for automatically generating high-performance code.
+
+Performance is key: the current methods are made to be highly performant on scalar and
+statically sized small problems, with options for large-scale systems. If you run into any
+performance issues, please file an issue. Consult the
+[NonlinearSystemSolvers](@ref nonlinearsystemsolvers) page for information on how to import
+solvers from different packages.
 
 ## Installation
 
@@ -27,17 +26,17 @@ Pkg.add("NonlinearSolve")
 
 ## Contributing
 
-- Please refer to the
-  [SciML ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://github.com/SciML/ColPrac/blob/master/README.md)
-  for guidance on PRs, issues, and other matters relating to contributing to SciML.
-
-- See the [SciML Style Guide](https://github.com/SciML/SciMLStyle) for common coding practices and other style decisions.
-- There are a few community forums:
-
-  - The #diffeq-bridged and #sciml-bridged channels in the [Julia Slack](https://julialang.org/slack/)
-  - The #diffeq-bridged and #sciml-bridged channels in the [Julia Zulip](https://julialang.zulipchat.com/#narrow/stream/279055-sciml-bridged)
-  - On the [Julia Discourse forums](https://discourse.julialang.org)
-  - See also [SciML Community page](https://sciml.ai/community/)
+  - Please refer to the
+    [SciML ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://github.com/SciML/ColPrac/blob/master/README.md)
+    for guidance on PRs, issues, and other matters relating to contributing to SciML.
+
+  - See the [SciML Style Guide](https://github.com/SciML/SciMLStyle) for common coding practices and other style decisions.
+  - There are a few community forums:
+    
+      + The #diffeq-bridged and #sciml-bridged channels in the [Julia Slack](https://julialang.org/slack/)
+      + The #diffeq-bridged and #sciml-bridged channels in the [Julia Zulip](https://julialang.zulipchat.com/#narrow/stream/279055-sciml-bridged)
+      + On the [Julia Discourse forums](https://discourse.julialang.org)
+      + See also [SciML Community page](https://sciml.ai/community/)
 
 ## Reproducibility