The SciML common interface ties together the numerical solvers of the Julia package ecosystem into a single unified interface. It is designed for maximal efficiency and parallelism, while incorporating essential features for large-scale scientific machine learning such as differentiability, composability, and sparsity.
This documentation is made to pool together the docs of the various SciML libraries to paint the overarching picture, establish development norms, and document the shared/common functionality.
The SciML common interface covers the following domains:
- Linear systems (
LinearProblem)- Direct methods for dense and sparse
- Iterative solvers with preconditioning
- Nonlinear Systems (
NonlinearProblem)- Rootfinding for systems of nonlinear equations
- Interval Nonlinear Systems
- Bracketing rootfinders for nonlinear equations with interval bounds
- Integrals (quadrature) (
IntegralProblem) - Differential Equations
- Discrete equations (function maps, discrete stochastic (Gillespie/Markov) simulations) (
DiscreteProblem) - Ordinary differential equations (ODEs) (
ODEProblem) - Split and Partitioned ODEs (Symplectic integrators, IMEX Methods) (
SplitODEProblem) - Stochastic ordinary differential equations (SODEs or SDEs) (
SDEProblem) - Stochastic differential-algebraic equations (SDAEs) (
SDEProblem with mass matrices) - Random differential equations (RODEs or RDEs) (
RODEProblem) - Differential algebraic equations (DAEs) (
DAEProblem and ODEProblem with mass matrices) - Delay differential equations (DDEs) (
DDEProblem) - Neutral, retarded, and algebraic delay differential equations (NDDEs, RDDEs, and DDAEs)
- Stochastic delay differential equations (SDDEs) (
SDDEProblem) - Experimental support for stochastic neutral, retarded, and algebraic delay differential equations (SNDDEs, SRDDEs, and SDDAEs)
- Mixed discrete and continuous equations (Hybrid Equations, Jump Diffusions) (
AbstractDEProblems with callbacks)
- Optimization (
OptimizationProblem)- Nonlinear (constrained) optimization
- (Stochastic/Delay/Differential-Algebraic) Partial Differential Equations (
PDESystem)- Finite difference and finite volume methods
- Interfaces to finite element methods
- Physics-Informed Neural Networks (PINNs)
- Integro-Differential Equations
- Fractional Differential Equations
The SciML common interface also includes ModelingToolkit.jl for defining such systems symbolically, allowing for optimizations like automated generation of parallel code, symbolic simplification, and generation of sparsity patterns.
In addition to the purely numerical representations of mathematical objects, there are also sets of problem types associated with common mathematical algorithms. These are:
- Data-driven modeling
- Discrete-time data-driven dynamical systems (
DiscreteDataDrivenProblem) - Continuous-time data-driven dynamical systems (
ContinuousDataDrivenProblem) - Symbolic regression (
DirectDataDrivenProblem)
- Uncertainty quantification and expected values (
ExpectationProblem)
We note that parameter estimation and inverse problems are solved directly on their constituent problem types using tools like DiffEqFlux.jl. Thus for example, there is no ODEInverseProblem, and instead ODEProblem is used to find the parameters p that solve the inverse problem.
The SciML interface is common as the usage of arguments is standardized across all of the problem domains. Underlying high level ideas include:
- All domains use the same interface of defining a
AbstractSciMLProblem which is then solved via solve(prob,alg;kwargs), where alg is a AbstractSciMLAlgorithm. The keyword argument namings are standardized across the organization. AbstractSciMLProblems are generally defined by a SciMLFunction which can define extra details about a model function, such as its analytical Jacobian, its sparsity patterns and so on.- There is an organization-wide method for defining linear and nonlinear solvers used within other solvers, giving maximum control of performance to the user.
- Types used within the packages are defined by the input types. For example, packages attempt to internally use the type of the initial condition as the type for the state within differential equation solvers.
solve calls should be thread-safe and parallel-safe.init(prob,alg;kwargs) returns an iterator which allows for directly iterating over the solution process- High performance is key. Any performance that is not at the top level is considered a bug and should be reported as such.
- All functions have an in-place and out-of-place form, where the in-place form is made to utilize mutation for high performance on large-scale problems and the out-of-place form is for compatibility with tooling like static arrays and some reverse-mode automatic differentiation systems.
- DifferentialEquations.jl
- Multi-package interface of high performance numerical solvers of differential equations
- ModelingToolkit.jl
- The symbolic modeling package which implements the SciML symbolic common interface.
- LinearSolve.jl
- Multi-package interface for specifying linear solvers (direct, sparse, and iterative), along with tools for caching and preconditioners for use in large-scale modeling.
- NonlinearSolve.jl
- High performance numerical solving of nonlinear systems.
- Integrals.jl
- Multi-package interface for high performance, batched, and parallelized numerical quadrature.
- Optimization.jl
- Multi-package interface for numerical solving of optimization problems.
- NeuralPDE.jl
- Physics-Informed Neural Network (PINN) package for transforming partial differential equations into optimization problems.
- DiffEqOperators.jl
- Automated finite difference method (FDM) package for transforming partial differential equations into nonlinear problems and ordinary differential equations.
- DiffEqFlux.jl
- High level package for scientific machine learning applications, such as neural and universal differential equations, solving of inverse problems, parameter estimation, nonlinear optimal control, and more.
- DataDrivenDiffEq.jl
- Multi-package interface for data-driven modeling, Koopman dynamic mode decomposition, symbolic regression/sparsification, and automated model discovery.
- SciMLExpectations.jl
- Extension to the dynamical modeling tools for calculating expectations.
- SciMLBase.jl
- The core package defining the interface which is consumed by the modeling and solver packages.
- DiffEqBase.jl
- The core package defining the extended interface which is consumed by the differential equation solver packages.
- SciMLSensitivity.jl
- A package which pools together the definition of derivative overloads to define the common
sensealg automatic differentiation interface.
- DiffEqNoiseProcess.jl
- A package which defines the stochastic
AbstractNoiseProcess interface for the SciML ecosystem.
- RecursiveArrayTools.jl
- A package which defines the underlying
AbstractVectorOfArray structure used as the output for all time series results.
- ArrayInterface.jl
- The package which defines the extended
AbstractArray interface employed throughout the SciML ecosystem.
There are too many to name here and this will be populated when there is time!
The following example showcases how the pieces of the common interface connect to solve a problem that mixes inference, symbolics, and numerics.
- diffeqr
- Solving differential equations in R using DifferentialEquations.jl with ModelingToolkit for JIT compilation and GPU-acceleration
- diffeqpy
- Solving differential equations in Python using DifferentialEquations.jl
There are too many to name here. Check out the SciML Organization Github Page for details.
The documentation of this SciML package was built using these direct dependencies,
Status `~/work/SciMLBase.jl/SciMLBase.jl/docs/Project.toml`
[e30172f5] Documenter v1.1.0
[961ee093] ModelingToolkit v8.71.2
- [0bca4576] SciMLBase v2.1.0 `~/work/SciMLBase.jl/SciMLBase.jl`
and using this machine and Julia version.
Julia Version 1.9.3
+ [0bca4576] SciMLBase v2.2.0 `~/work/SciMLBase.jl/SciMLBase.jl`
and using this machine and Julia version.
Julia Version 1.9.3
Commit bed2cd540a1 (2023-08-24 14:43 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
- CPU: 2 × Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+ CPU: 2 × Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
- LLVM: libLLVM-14.0.6 (ORCJIT, skylake-avx512)
+ LLVM: libLLVM-14.0.6 (ORCJIT, icelake-server)
Threads: 1 on 2 virtual cores
A more complete overview of all dependencies and their versions is also provided.
Status `~/work/SciMLBase.jl/SciMLBase.jl/docs/Manifest.toml`
[47edcb42] ADTypes v0.2.4
[a4c015fc] ANSIColoredPrinters v0.0.1
@@ -41,7 +41,7 @@
[9a962f9c] DataAPI v1.15.0
[864edb3b] DataStructures v0.18.15
[e2d170a0] DataValueInterfaces v1.0.0
- [2b5f629d] DiffEqBase v6.131.0
+ [2b5f629d] DiffEqBase v6.132.0
[459566f4] DiffEqCallbacks v2.33.0
[163ba53b] DiffResults v1.1.0
[b552c78f] DiffRules v1.15.1
@@ -95,7 +95,7 @@
[0e77f7df] LazilyInitializedFields v1.2.1
[50d2b5c4] Lazy v0.15.1
[d3d80556] LineSearches v7.2.0
- [7ed4a6bd] LinearSolve v2.9.1
+ [7ed4a6bd] LinearSolve v2.9.2
[2ab3a3ac] LogExpFunctions v0.3.26
[bdcacae8] LoopVectorization v0.12.165
[d8e11817] MLStyle v0.4.17
@@ -110,11 +110,11 @@
[d41bc354] NLSolversBase v7.8.3
[2774e3e8] NLsolve v4.5.1
[77ba4419] NaNMath v1.0.2
- [8913a72c] NonlinearSolve v2.0.1
+ [8913a72c] NonlinearSolve v2.1.0
[6fe1bfb0] OffsetArrays v1.12.10
[bac558e1] OrderedCollections v1.6.2
- [1dea7af3] OrdinaryDiffEq v6.57.1
- [90014a1f] PDMats v0.11.24
+ [1dea7af3] OrdinaryDiffEq v6.58.0
+ [90014a1f] PDMats v0.11.25
[65ce6f38] PackageExtensionCompat v1.0.2
[d96e819e] Parameters v0.12.3
[69de0a69] Parsers v2.7.2
@@ -139,7 +139,7 @@
[fdea26ae] SIMD v3.4.5
[94e857df] SIMDTypes v0.1.0
[476501e8] SLEEFPirates v0.6.39
- [0bca4576] SciMLBase v2.1.0 `~/work/SciMLBase.jl/SciMLBase.jl`
+ [0bca4576] SciMLBase v2.2.0 `~/work/SciMLBase.jl/SciMLBase.jl`
[e9a6253c] SciMLNLSolve v0.1.9
[c0aeaf25] SciMLOperators v0.3.6
[efcf1570] Setfield v1.1.1
@@ -227,4 +227,4 @@
[83775a58] Zlib_jll v1.2.13+0
[8e850b90] libblastrampoline_jll v5.8.0+0
[8e850ede] nghttp2_jll v1.48.0+0
- [3f19e933] p7zip_jll v17.4.0+0
You can also download the manifest file and the project file.