From f857a116e1824ab92e1d275b5b83fbf3ede0070d Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Sun, 19 Jan 2025 16:09:10 +0000 Subject: [PATCH] build based on d8bfd03 --- dev/.documenter-siteinfo.json | 2 +- dev/api.html | 18 +++++++++--------- dev/gp.html | 16 ++++++++-------- dev/index.html | 2 +- dev/tutorials.html | 2 +- dev/tutorials/callbacks.html | 2 +- dev/tutorials/decomposition.html | 2 +- dev/tutorials/greedy_mcmc.html | 2 +- dev/tutorials/iclaims.html | 2 +- dev/tutorials/overview.html | 2 +- dev/utils.html | 4 ++-- versions.js | 7 ------- 12 files changed, 27 insertions(+), 34 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 1a1c55c..c362af5 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.8.5","generation_timestamp":"2025-01-19T04:07:43","documenter_version":"1.8.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.8.5","generation_timestamp":"2025-01-19T16:09:02","documenter_version":"1.8.0"}} \ No newline at end of file diff --git a/dev/api.html b/dev/api.html index e079523..de490d0 100644 --- a/dev/api.html +++ b/dev/api.html @@ -37,11 +37,11 @@ B &:= \texttt{Linear} \mid \texttt{Periodic} \mid \texttt{GammaExponential} \mid \dots \\ \oplus &:= \texttt{+} \mid \texttt{*} \mid \texttt{ChangePoint} \\ k &:= B \mid \texttt{(}k_1 \oplus k_2\texttt{)}. -\end{aligned}\]

and $d(k)$ is the number of parameters in expression $k$. Given data $\{(t_i,y_i)\}_{i=1}^n$, AutoGP infers likely values of the symbolic structure of the covariance kernel $k$, the kernel parameters $\theta$, and the observation noise variance $\eta$.

The ability to automatically synthesize covariance kernels in AutoGP is in contrast to existing Gaussian process libraries such as GaussianProcess.jl sklearn.gaussian_process, GPy, and GPyTorch, which all require users to manually specify $k$.

After model fitting, users can query the models to generate forecasts, compute probabilities, and inspect qualitative structure.

AutoGP — Module

Main module.

Exports

source

Model Initialization

AutoGP.GPModel — Type
struct GPModel

A GPModel contains covariance kernel structures and parameters for modeling data.

Fields

  • pf_state::Gen.ParticleFilterState: Internal particle set.
  • config::GP.GPConfig=GP.GPConfig(): User-specific customization, refer to GP.GPConfig.
  • ds::IndexType: Observed time points.
  • y::Vector{<:Real}: Observed time series values.
  • ds_transform::Transforms.LinearTransform: Transformation of time to direct space.
  • y_transform::Transforms.LinearTransform: Transformation of observations to direct space.

Constructors

model = GPModel(
+\end{aligned}\]
and $d(k)$ is the number of parameters in expression $k$. Given data $\{(t_i,y_i)\}_{i=1}^n$, AutoGP infers likely values of the symbolic structure of the covariance kernel $k$, the kernel parameters $\theta$, and the observation noise variance $\eta$.
The ability to automatically synthesize covariance kernels in AutoGP is in contrast to existing Gaussian process libraries such as GaussianProcess.jl sklearn.gaussian_process, GPy, and GPyTorch, which all require users to manually specify $k$.
After model fitting, users can query the models to generate forecasts, compute probabilities, and inspect qualitative structure.
Model Initialization
AutoGP.GPModel — Type
struct GPModel
A GPModel contains covariance kernel structures and parameters for modeling data.
Fields
  • pf_state::Gen.ParticleFilterState: Internal particle set.
  • config::GP.GPConfig=GP.GPConfig(): User-specific customization, refer to GP.GPConfig.
  • ds::IndexType: Observed time points.
  • y::Vector{<:Real}: Observed time series values.
  • ds_transform::Transforms.LinearTransform: Transformation of time to direct space.
  • y_transform::Transforms.LinearTransform: Transformation of observations to direct space.
Constructors
model = GPModel(
     ds::IndexType,
     y::Vector{<:Real};
     n_particles::Integer=8,
-    config::GP.GPConfig=GP.GPConfig())
See also
To perform learning given the data, refer to
source
AutoGP.seed! — Function
seed!(seed)
Set the random seed of the global random number generator.
source
AutoGP.IndexType — Type
IndexType = Union{Vector{<:Real}, Vector{<:Dates.TimeType}}
Permitted Julia types for Gaussian process time points. Real numbers are ingested directly, treated as time points.. Instances of Dates.TimeType are converted to numeric time points by using Dates.datetime2unix.
source
End-to-End Model Fitting
AutoGP.seed! — Function
seed!(seed)
Set the random seed of the global random number generator.
source
AutoGP.IndexType — Type
IndexType = Union{Vector{<:Real}, Vector{<:Dates.TimeType}}
Permitted Julia types for Gaussian process time points. Real numbers are ingested directly, treated as time points.. Instances of Dates.TimeType are converted to numeric time points by using Dates.datetime2unix.
source
End-to-End Model Fitting
AutoGP.fit_smc! — Function
fit_smc!(
     model::GPModel;
     schedule::Vector{<:Integer},
     n_mcmc::Int,
@@ -53,20 +53,20 @@
     hmc_config::Dict=Dict(),
     verbose::Bool=false,
     check::Bool=false,
-    callback_fn::Function=(; kwargs...) -> nothing)
Infer the structure and parameters of an appropriate Gaussian process covariance kernel for modeling the observed data. Inference is performed using sequential Monte Carlo.
Arguments
  • model::GPModel: Instance of the GPModel to use.
  • schedule::Vector{<:Integer}: Schedule for incorporating data for SMC, refer to Schedule.
  • n_mcmc::Int: Number of involutive MCMC rejuvenation steps.
  • n_hmc::Int: Number of HMC steps per accepted involutive MCMC step.
  • biased::Bool:   Whether to bias the proposal to produce "short" structures.
  • shuffle::Bool=true: Whether to shuffle indexes ds or incorporate data in the given order.
  • adaptive_resampling::Bool=true: If true resamples based on ESS threshold, else at each step.
  • adaptive_rejuvenation::Bool=false: If true rejuvenates only if resampled, else at each step.
  • hmc_config::Dict: Configuration for HMC inference on numeric parameters. Allowable keys are:
    • n_exit::Integer=1: Number of successive rejections after which HMC loop is terminated.
    • L_param::Integer=10 Number of leapfrog steps for kernel parameters.
    • L_noise::Integer=10: Number of leapfrog steps for noise parameter.
    • eps_param::Float64=0.02: Step size for kernel parameters.
    • eps_noise::Float64=0.02: Step size for noise parameter.
  • verbose::Bool=false: Report progress to stdout.
  • check::Bool=false: Perform dynamic correctness checks during inference.
  • config::GP.GPConfig=GP.GPConfig(): User-specific customization, refer to GP.GPConfig.
  • callback_fn: A callback for monitoring inference, must be generated by AutoGP.Callbacks.make_smc_callback.
source
AutoGP.fit_mcmc! — Function
fit_mcmc!(
+    callback_fn::Function=(; kwargs...) -> nothing)
Infer the structure and parameters of an appropriate Gaussian process covariance kernel for modeling the observed data. Inference is performed using sequential Monte Carlo.
Arguments
  • model::GPModel: Instance of the GPModel to use.
  • schedule::Vector{<:Integer}: Schedule for incorporating data for SMC, refer to Schedule.
  • n_mcmc::Int: Number of involutive MCMC rejuvenation steps.
  • n_hmc::Int: Number of HMC steps per accepted involutive MCMC step.
  • biased::Bool:   Whether to bias the proposal to produce "short" structures.
  • shuffle::Bool=true: Whether to shuffle indexes ds or incorporate data in the given order.
  • adaptive_resampling::Bool=true: If true resamples based on ESS threshold, else at each step.
  • adaptive_rejuvenation::Bool=false: If true rejuvenates only if resampled, else at each step.
  • hmc_config::Dict: Configuration for HMC inference on numeric parameters. Allowable keys are:
    • n_exit::Integer=1: Number of successive rejections after which HMC loop is terminated.
    • L_param::Integer=10 Number of leapfrog steps for kernel parameters.
    • L_noise::Integer=10: Number of leapfrog steps for noise parameter.
    • eps_param::Float64=0.02: Step size for kernel parameters.
    • eps_noise::Float64=0.02: Step size for noise parameter.
  • verbose::Bool=false: Report progress to stdout.
  • check::Bool=false: Perform dynamic correctness checks during inference.
  • config::GP.GPConfig=GP.GPConfig(): User-specific customization, refer to GP.GPConfig.
  • callback_fn: A callback for monitoring inference, must be generated by AutoGP.Callbacks.make_smc_callback.
source
AutoGP.fit_mcmc! — Function
fit_mcmc!(
     model::GPModel;
     n_mcmc::Integer,
     n_hmc::Integer,
     biased::Bool=false,
     verbose::Bool=false,
     check::Bool=false,
-    callback_fn::Function=(; kwargs...) -> nothing)
Perform n_mcmc steps of involutive MCMC on the structure, with n_hmc steps of Hamiltonian Monte Carlo sampling on the parameters per accepted involutive MCMC move.
A callback_fn can be provided to monitor the progress each MCMC step for which at least one particle (i.e, chain) accepted a transition. Its signature must contain a single varargs specifier, which will be populated with keys :model, :step, :elapsed.
Warning
The callback_fn imposes a roughly 2x runtime overhead as compared to the equivalent mcmc_structure! method, because parallel execution must be synchronized across the particles to invoke the callback at each step. The :elapsed variable provided to the callback function will still reflect an accurate estimate of the inference runtime without this overhead. If no callback is required, use mcmc_structure! instead.
source
AutoGP.fit_greedy! — Function
function fit_greedy!(
+    callback_fn::Function=(; kwargs...) -> nothing)
Perform n_mcmc steps of involutive MCMC on the structure, with n_hmc steps of Hamiltonian Monte Carlo sampling on the parameters per accepted involutive MCMC move.
A callback_fn can be provided to monitor the progress each MCMC step for which at least one particle (i.e, chain) accepted a transition. Its signature must contain a single varargs specifier, which will be populated with keys :model, :step, :elapsed.
Warning
The callback_fn imposes a roughly 2x runtime overhead as compared to the equivalent mcmc_structure! method, because parallel execution must be synchronized across the particles to invoke the callback at each step. The :elapsed variable provided to the callback function will still reflect an accurate estimate of the inference runtime without this overhead. If no callback is required, use mcmc_structure! instead.
source
AutoGP.fit_greedy! — Function
function fit_greedy!(
         model::GPModel;
         max_depth::Integer=model.config.max_depth,
         verbose::Bool=false,
         check::Bool=false,
-        callback_fn::Function=(; kwargs...) -> nothing)
Infer the structure and parameters of an appropriate Gaussian process covariance kernel for modeling the observed data. Inference is performed using greedy search, as described in Algorithm 2 of Kim and Teh, 2018. It is an error if max_depth is not a finite positive number.
A callback_fn can be provided to monitor the search progress at each stage. Its signature must contain a single varargs specifier, which will be populated with keys :model, :step, :elapsed at each step of the greedy search.
source
Incremental Model Fitting
AutoGP.add_data! — Function
add_data!(model::GPModel, ds::IndexType, y::Vector{<:Real})
Incorporate new observations (ds, y) into model.
source
AutoGP.maybe_resample! — Function
maybe_resample!(model::GPModel, ess_threshold::Real)
Resample the particle collection in model if ESS is below ess_threshold. Setting ess_threshold = AutoGP.num_particles(model) + 1 will ensure that resampling always takes place, since the ESS is upper bounded by the number of particles.
source
AutoGP.mcmc_structure! — Function
mcmc_structure!(model::GPModel, n_mcmc::Integer, n_hmc::Integer;
-    biased::Bool=false, verbose::Bool=false, check::Bool=false)
Perform n_mcmc steps of involutive MCMC on the structure, with n_hmc steps of Hamiltonian Monte Carlo sampling on the parameters per accepted involutive MCMC move.
source
AutoGP.mcmc_parameters! — Function
mcmc_parameters!(model::GPModel, n_hmc::Integer; verbose::Bool=false, check::Bool=false)
Perform n_hmc steps of Hamiltonian Monte Carlo sampling on the parameters.
source
Model Querying
AutoGP.predict — Function
predictions = predict(
+        callback_fn::Function=(; kwargs...) -> nothing)
Infer the structure and parameters of an appropriate Gaussian process covariance kernel for modeling the observed data. Inference is performed using greedy search, as described in Algorithm 2 of Kim and Teh, 2018. It is an error if max_depth is not a finite positive number.
A callback_fn can be provided to monitor the search progress at each stage. Its signature must contain a single varargs specifier, which will be populated with keys :model, :step, :elapsed at each step of the greedy search.
source
Incremental Model Fitting
AutoGP.add_data! — Function
add_data!(model::GPModel, ds::IndexType, y::Vector{<:Real})
Incorporate new observations (ds, y) into model.
source
AutoGP.maybe_resample! — Function
maybe_resample!(model::GPModel, ess_threshold::Real)
Resample the particle collection in model if ESS is below ess_threshold. Setting ess_threshold = AutoGP.num_particles(model) + 1 will ensure that resampling always takes place, since the ESS is upper bounded by the number of particles.
source
AutoGP.mcmc_structure! — Function
mcmc_structure!(model::GPModel, n_mcmc::Integer, n_hmc::Integer;
+    biased::Bool=false, verbose::Bool=false, check::Bool=false)
Perform n_mcmc steps of involutive MCMC on the structure, with n_hmc steps of Hamiltonian Monte Carlo sampling on the parameters per accepted involutive MCMC move.
source
AutoGP.mcmc_parameters! — Function
mcmc_parameters!(model::GPModel, n_hmc::Integer; verbose::Bool=false, check::Bool=false)
Perform n_hmc steps of Hamiltonian Monte Carlo sampling on the parameters.
source
Model Querying
AutoGP.predict — Function
predictions = predict(
     model::GPModel,
     ds::IndexType;
     quantiles::Vector{Float64}=Float64[],
@@ -83,7 +83,7 @@
    5 │ 2020-01-01         3  0.0748059    -43638.6   65083.0    10722.2
    6 │ 2020-01-02         3  0.0748059    -43662.0   65074.7    10706.4
    7 │ 2020-01-01         4  0.60809      -17582.2   30762.4     6590.06
-   8 │ 2020-01-02         4  0.60809      -17588.0   30771.5     6591.78
source
AutoGP.predict_proba — Function
function predict_proba(model::GPModel, ds::IndexType, y::Vector{<:Real})
Compute predictive probability of data y at time points ds under model.
Example
julia> ds = [Dates.Date(2020,1,1), Dates.Date(2020,1,2)] # Dates to query
+   8 │ 2020-01-02         4  0.60809      -17588.0   30771.5     6591.78
source
AutoGP.predict_proba — Function
function predict_proba(model::GPModel, ds::IndexType, y::Vector{<:Real})
Compute predictive probability of data y at time points ds under model.
Example
julia> ds = [Dates.Date(2020,1,1), Dates.Date(2020,1,2)] # Dates to query
 julia> y = [0.1, .0.5] # values to query
 julia> GPModel.predict(model, ds, y)
 7×3 DataFrame
@@ -96,6 +96,6 @@
    4 │        4  0.00164846   -59.5311
    5 │        5  0.215255     -61.066
    6 │        6  0.134198     -64.4041
-   7 │        7  0.000201078  -68.462
source
AutoGP.predict_mvn — Function
dist = predict_mvn(model::GPModel, ds::IndexType; noise_pred::Union{Nothing,Float64}=nothing)
Return an instance of Distributions.MixtureModel representing the overall posterior predictive distribution for data at index points ds. By default, the noise_pred of the new data is equal to the inferred noise of the observed data within each particle in model; using noise_pred=0. returns the posterior distribution over the noiseless function values.
The returned dist has precisely num_particles(model) components, each of type Distributions.MvNormal, with weights particle_weights(model). These objects can be retrieved using Distributions.components and Distributions.probs, respectively.
source
AutoGP.predict_mvn — Function
dist = predict_mvn(model::GPModel, ds::IndexType; noise_pred::Union{Nothing,Float64}=nothing)
Return an instance of Distributions.MixtureModel representing the overall posterior predictive distribution for data at index points ds. By default, the noise_pred of the new data is equal to the inferred noise of the observed data within each particle in model; using noise_pred=0. returns the posterior distribution over the noiseless function values.
The returned dist has precisely num_particles(model) components, each of type Distributions.MvNormal, with weights particle_weights(model). These objects can be retrieved using Distributions.components and Distributions.probs, respectively.
source
AutoGP.predict_quantile — Function
(x::Vector, success::Bool) = predict_quantile(
     model::GPModel, ds::IndexType, q::Real;
-    noise_pred::Union{Nothing,Float64}=nothing, tol=1e-5, max_iter=1e6)
Evaluates the inverse cumulative distribution function (CDF) of the multivariate Gaussian mixture model returned by predict_mvn at q (between 0 and 1, exclusive) separately for each dimension. The returned vector x has the same length as the index points ds.
Note
The inverse CDF is numerically estimated using a binary search algorithm. The keyword arguments tol and max_iter  correspond to the desired absolute tolerance of the estimate and the maximum number of binary search iterations, respectively. The returned Boolean variable success indicates whether the returned value x has been located to within the specified error tolerance.
See also
source
AutoGP.decompose — Function
function decompose(model::GPModel)
Decompose each particle within model into its constituent kernels. Supposing that num_particles(model) equals $k$, the return value models::Vector{GPModel} of decompose is a length-$k$ vector of GPModel instances.
Therefore, models[i] is a GPModel that represents the decomposition of particle i in model into its constituent kernels. Each particle in models[i] corresponds to a kernel fragment in the covariance for particle i of model (i.e., one particle for each GP.Node in the covariance kernel).
The weights of models[i] are arbitrary and have no meaningful value.
This function is particularly useful for visualizing the individual time series structures that make up each particle of model.
source
+ noise_pred::Union{Nothing,Float64}=nothing, tol=1e-5, max_iter=1e6)

Evaluates the inverse cumulative distribution function (CDF) of the multivariate Gaussian mixture model returned by predict_mvn at q (between 0 and 1, exclusive) separately for each dimension. The returned vector x has the same length as the index points ds.

Note

The inverse CDF is numerically estimated using a binary search algorithm. The keyword arguments tol and max_iter correspond to the desired absolute tolerance of the estimate and the maximum number of binary search iterations, respectively. The returned Boolean variable success indicates whether the returned value x has been located to within the specified error tolerance.

See also

source
AutoGP.log_marginal_likelihood_estimate — Function
log_marginal_likelihood_estimate(model::GPModel)

Return estimate of marginal likelihood of data (in log space).

source
AutoGP.particle_weights — Function
particle_weights(model::GPModel)

Return vector of normalized particle weights.

source
AutoGP.effective_sample_size — Function
effective_sample_size(model::GPModel)

Return effective sample size (ESS) of weighted particle collection.

source
AutoGP.covariance_kernels — Function
covariance_kernels(model::GPModel)

Return Gaussian process covariance kernels in model.

source
AutoGP.observation_noise_variances — Function
observation_noise_variances(model::GPModel)

Return list of observation noise variances for each particle in model.

source
AutoGP.decompose — Function
function decompose(model::GPModel)

Decompose each particle within model into its constituent kernels. Supposing that num_particles(model) equals $k$, the return value models::Vector{GPModel} of decompose is a length-$k$ vector of GPModel instances.

Therefore, models[i] is a GPModel that represents the decomposition of particle i in model into its constituent kernels. Each particle in models[i] corresponds to a kernel fragment in the covariance for particle i of model (i.e., one particle for each GP.Node in the covariance kernel).

The weights of models[i] are arbitrary and have no meaningful value.

This function is particularly useful for visualizing the individual time series structures that make up each particle of model.

source
diff --git a/dev/gp.html b/dev/gp.html index a3373b1..be58425 100644 --- a/dev/gp.html +++ b/dev/gp.html @@ -1,19 +1,19 @@ -Gaussian Process Library · AutoGP
GitHub

Gaussian Process Library

This section describes a library for Gaussian process time series models. A technical overview of key concepts can be found in the following references.

Roberts S, Osborne M, Ebden M, Reece S, Gibson N, Aigrain S. 2013. Gaussian processes for time-series modelling. Phil Trans R Soc A 371: 20110550. http://dx.doi.org/10.1098/rsta.2011.0550

Rasmussen C, Williams C. 2006. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA. http://gaussianprocess.org/gpml/chapters/

Covariance Kernels

AutoGP.GP.BinaryOpNode — Type
abstract type BinaryOpNode <: Node end

Abstract class for [composite covariance kernels](@ref (@ref gpcovkernel_comp).

source
Base.size — Function
Base.size(node::Node)
+Gaussian Process Library · AutoGP
GitHub
Gaussian Process Library
This section describes a library for Gaussian process time series models. A technical overview of key concepts can be found in the following references.
Roberts S, Osborne M, Ebden M, Reece S, Gibson N, Aigrain S. 2013. Gaussian processes for time-series modelling. Phil Trans R Soc A 371: 20110550. http://dx.doi.org/10.1098/rsta.2011.0550
Rasmussen C, Williams C. 2006. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA. http://gaussianprocess.org/gpml/chapters/
Covariance Kernels
AutoGP.GP.BinaryOpNode — Type
abstract type BinaryOpNode <: Node end
Abstract class for [composite covariance kernels](@ref (@ref gpcovkernel_comp).
source
Base.size — Function
Base.size(node::Node)
 Base.size(node::LeafNode) = 1
-Base.size(a::LeafNode, b::Node) = size(a) + size(b)
Return the total number of subexpressions in a Node, as defined above.
source
AutoGP.GP.eval_cov — Function
eval_cov(node::Node, t1::Real, t2::Real)
-eval_cov(node::Node, ts::Vector{Float64})
Evaluate the covariance function node at the given time indexes. The first form returns a Real number and the second form returns a covariance Matrix.
source
AutoGP.GP.compute_cov_matrix_vectorized — Function
compute_cov_matrix_vectorized(node::Node, noise, ts)
Compute covariance matrix by evaluating node on all pair of ts. The noise is added to the diagonal of the covariance matrix, which means that if ts[i] == ts[j], then X[ts[i]] and Xs[ts[j]] are i.i.d. samples of the true function at ts[i] plus mean zero Gaussian noise.
source
Primitive Kernels
Notation. In this section, generic parameters (e.g., $\theta$, $\theta_1$, $\theta_2$), are used to denote fieldnames of the corresponding Julia structs in the same order as they appear in the constructors.
AutoGP.GP.WhiteNoise — Type
WhiteNoise(value)
White noise covariance kernel.
\[k(t, t') = \mathbf{I}[t = t'] \theta\]
The random variables $X[t]$ and $X[t']$ are perfectly correlated whenever $t = t'$ and independent otherwise. This kernel cannot be used to represent the joint distribution of multiple i.i.d. measurements of $X[t]$, instead see compute_cov_matrix_vectorized.
source
AutoGP.GP.Constant — Type
Constant(value)
Constant covariance kernel.
\[k(t,t') = \theta\]
Draws from this kernel are horizontal lines, where $\theta_1$ determines the variance of the constant value around the mean (typically zero).
source
AutoGP.GP.Linear — Type
Linear(intercept[, bias=1, amplitude=1])
Linear covariance kernel.
\[k(t, t') = \theta_2 + \theta_3 (t - \theta_1)(t'-\theta_1)\]
Draws from this kernel are sloped lines in the 2D plane. The time intercept is $\theta_1$. The variance around the time intercept is $\theta_2$. The scale factor, which dictates the slope, is $\theta_3$.
source
AutoGP.GP.SquaredExponential — Type
SquaredExponential(lengthscale[, amplitude=1])
Squared Exponential covariance kernel.
\[k(t,t') = \theta_2 \exp\left(-1/2|t-t'|/\theta_2)^2 \right)\]
Draws from this kernel are smooth functions.
source
AutoGP.GP.GammaExponential — Type
GammaExponential(lengthscale, gamma[, amplitude=1])
Gamma Exponential covariance kernel.
\[k(t,t') = \theta_3 \exp(-(|t-t'|/\theta_1)^{\theta_2})\]
Requires 0 < gamma <= 2. Recovers the SquaredExponential kernel when gamma = 2.
source
AutoGP.GP.Periodic — Type
Periodic(lengthscale, period[, amplitude=1])
Periodic covariance kernel.
\[k(t,t') = \exp\left( (-2/\theta_1^2) \sin^2((\pi/\theta_2) |t-t'|) \right)\]
The lengthscale determines how smooth the periodic function is within each period. Heuristically, the periodic kernel can be understood as:
  1. Sampling $[X(t), t \in [0,p]] \sim \mathrm{GP}(0, \mathrm{SE}(\theta_1))$.
  2. Repeating this fragment for all intervals $[jp, (j+1)p], j \in \mathbb{Z}$.
source
Composite Kernels
AutoGP.GP.Times — Type
Times(left::Node, right::Node)
-Base.:*(left::Node, right::Node)
Covariance kernel obtained by multiplying two covariance kernels pointwise.
\[k(t,t') = k_{\rm left}(t,t') \times k_{\rm right}(t,t')\]
source
AutoGP.GP.Plus — Type
Plus(left::Node, right::Node)
-Base.:+(left::Node, right::Node)
Covariance kernel obtained by summing two covariance kernels pointwise.
\[k(t,t') = k_{\rm left}(t,t') + k_{\rm right}(t,t')\]
source
AutoGP.GP.ChangePoint — Type
ChangePoint(left::Node, right::Node, location::Real, scale::Real)
Covariance kernel obtained by switching between two kernels at location.
\[\begin{aligned}
+Base.size(a::LeafNode, b::Node) = size(a) + size(b)

Return the total number of subexpressions in a Node, as defined above.

source
AutoGP.GP.eval_cov — Function
eval_cov(node::Node, t1::Real, t2::Real)
+eval_cov(node::Node, ts::Vector{Float64})

Evaluate the covariance function node at the given time indexes. The first form returns a Real number and the second form returns a covariance Matrix.

source
AutoGP.GP.compute_cov_matrix_vectorized — Function
compute_cov_matrix_vectorized(node::Node, noise, ts)

Compute covariance matrix by evaluating node on all pair of ts. The noise is added to the diagonal of the covariance matrix, which means that if ts[i] == ts[j], then X[ts[i]] and Xs[ts[j]] are i.i.d. samples of the true function at ts[i] plus mean zero Gaussian noise.

source

Primitive Kernels

Notation. In this section, generic parameters (e.g., $\theta$, $\theta_1$, $\theta_2$), are used to denote fieldnames of the corresponding Julia structs in the same order as they appear in the constructors.

AutoGP.GP.WhiteNoise — Type
WhiteNoise(value)

White noise covariance kernel.

\[k(t, t') = \mathbf{I}[t = t'] \theta\]

The random variables $X[t]$ and $X[t']$ are perfectly correlated whenever $t = t'$ and independent otherwise. This kernel cannot be used to represent the joint distribution of multiple i.i.d. measurements of $X[t]$, instead see compute_cov_matrix_vectorized.

source
AutoGP.GP.Constant — Type
Constant(value)

Constant covariance kernel.

\[k(t,t') = \theta\]

Draws from this kernel are horizontal lines, where $\theta_1$ determines the variance of the constant value around the mean (typically zero).

source
AutoGP.GP.Linear — Type
Linear(intercept[, bias=1, amplitude=1])

Linear covariance kernel.

\[k(t, t') = \theta_2 + \theta_3 (t - \theta_1)(t'-\theta_1)\]

Draws from this kernel are sloped lines in the 2D plane. The time intercept is $\theta_1$. The variance around the time intercept is $\theta_2$. The scale factor, which dictates the slope, is $\theta_3$.

source
AutoGP.GP.SquaredExponential — Type
SquaredExponential(lengthscale[, amplitude=1])

Squared Exponential covariance kernel.

\[k(t,t') = \theta_2 \exp\left(-1/2|t-t'|/\theta_2)^2 \right)\]

Draws from this kernel are smooth functions.

source
AutoGP.GP.GammaExponential — Type
GammaExponential(lengthscale, gamma[, amplitude=1])

Gamma Exponential covariance kernel.

\[k(t,t') = \theta_3 \exp(-(|t-t'|/\theta_1)^{\theta_2})\]

Requires 0 < gamma <= 2. Recovers the SquaredExponential kernel when gamma = 2.

source
AutoGP.GP.Periodic — Type
Periodic(lengthscale, period[, amplitude=1])

Periodic covariance kernel.

\[k(t,t') = \exp\left( (-2/\theta_1^2) \sin^2((\pi/\theta_2) |t-t'|) \right)\]

The lengthscale determines how smooth the periodic function is within each period. Heuristically, the periodic kernel can be understood as:

  1. Sampling $[X(t), t \in [0,p]] \sim \mathrm{GP}(0, \mathrm{SE}(\theta_1))$.
  2. Repeating this fragment for all intervals $[jp, (j+1)p], j \in \mathbb{Z}$.
source

Composite Kernels

AutoGP.GP.Times — Type
Times(left::Node, right::Node)
+Base.:*(left::Node, right::Node)

Covariance kernel obtained by multiplying two covariance kernels pointwise.

\[k(t,t') = k_{\rm left}(t,t') \times k_{\rm right}(t,t')\]

source
AutoGP.GP.Plus — Type
Plus(left::Node, right::Node)
+Base.:+(left::Node, right::Node)

Covariance kernel obtained by summing two covariance kernels pointwise.

\[k(t,t') = k_{\rm left}(t,t') + k_{\rm right}(t,t')\]

source
AutoGP.GP.ChangePoint — Type
ChangePoint(left::Node, right::Node, location::Real, scale::Real)

Covariance kernel obtained by switching between two kernels at location.

\[\begin{aligned} k(t,t') &= [\sigma_1 \cdot k_{\rm left}(t, t') \cdot \sigma_2] + [(1 - \sigma_1) \cdot k_{\rm right}(t, t') \cdot (1-\sigma_2)] \\ \mathrm{where}\, \sigma_1 &= (1 + \tanh((t - \theta_1) / \theta_2))/2, \\ \sigma_2 &= (1 + \tanh((t' - \theta_1) / \theta_2))/2. -\end{aligned}\]

The location parameter $\theta_1$ denotes the time point at which the change occurs. The scale parameter $\theta_2$ is a nonnegative number that controls the rate of change; its behavior can be understood by analyzing the two extreme values:

  • If location=0 then $k_{\rm left}$ is active and $k_{\rm right}$ is inactive for all times less than location; $k_{\rm right}$ is active and $k_{\rm left}$ is inactive for all times greater than location; and $X[t] \perp X[t']$ for all $t$ and $t'$ on opposite sides of location.

  • If location=Inf then $k_{\rm left}$ and $k_{\rm right}$ have equal effect for all time points, and $k(t,t') = 1/2 (k_{\rm left}(t,'t) + k_{\rm right}(t,t'))$, which is equivalent to a Plus kernel scaled by a factor of $1/2$.

source

Prediction Utilities

Distributions.MvNormal — Type
dist = Distributions.MvNormal(
+\end{aligned}\]
The location parameter $\theta_1$ denotes the time point at which the change occurs.  The scale parameter $\theta_2$ is a nonnegative number that controls the rate of change; its behavior can be understood by analyzing the two extreme values:
  • If location=0 then $k_{\rm left}$ is active and $k_{\rm right}$ is inactive for all times less than location; $k_{\rm right}$ is active and $k_{\rm left}$ is inactive for all times greater than location; and $X[t] \perp X[t']$ for all $t$ and $t'$ on opposite sides of location.
  • If location=Inf then $k_{\rm left}$ and $k_{\rm right}$ have equal effect for all time points, and $k(t,t') = 1/2 (k_{\rm left}(t,'t) + k_{\rm right}(t,t'))$, which is equivalent to a Plus kernel scaled by a factor of $1/2$.
source

Prediction Utilities

Distributions.MvNormal — Type
dist = Distributions.MvNormal(
         node::Node,
         noise::Float64,
         ts::Vector{Float64},
         xs::Vector{Float64},
         ts_pred::Vector{Float64};
-        noise_pred::Union{Nothing,Float64}=nothing)

Return MvNormal posterior predictive distribution over xs_pred at time indexes ts_pred, given noisy observations [ts, xs] and covariance function node with given level of observation noise.

By default, the observation noise (noise_pred) of the new data is equal to the noise of the observed data; use noise_pred = 0. to obtain the predictive distribution over noiseless future values.

See also

source
Statistics.quantile — Function
Distributions.quantile(dist::Distributions.MvNormal, p)

Compute quantiles of marginal distributions of dist.

Examples

Distributions.quantile(Distributions.MvNormal([0,1,2,3], LinearAlgebra.I(4)), .5)
-Distributions.quantile(Distributions.MvNormal([0,1,2,3], LinearAlgebra.I(4)), [[.1, .5, .9]])
source

Prior Configuration

AutoGP.GP.GPConfig — Type
config = GPConfig(kwargs...)

Configuration of prior distribution over Gaussian process kernels, i.e., an instance of Node. The main kwargs (all optional) are:

  • node_dist_leaf::Vector{Real}: Prior distribution over LeafNode kernels; default is uniform.

  • node_dist_nocp::Vector{Real}: Prior distribution over BinaryOpNode kernels; only used if changepoints=false.

  • node_dist_cp::Vector{Real}: Prior distribution over BinaryOpNode kernels; only used if changepoints=true.

  • max_depth::Integer: Maximum depth of covariance node; default is -1 for unbounded.

  • changepoints::Bool: Whether to permit ChangePoint compositions; default is true.

  • noise::Union{Nothing,Float64}: Whether to use a fixed observation noise; default is nothing to infer automatically.

source
+ noise_pred::Union{Nothing,Float64}=nothing)

Return MvNormal posterior predictive distribution over xs_pred at time indexes ts_pred, given noisy observations [ts, xs] and covariance function node with given level of observation noise.

By default, the observation noise (noise_pred) of the new data is equal to the noise of the observed data; use noise_pred = 0. to obtain the predictive distribution over noiseless future values.

See also

source
Statistics.quantile — Function
Distributions.quantile(dist::Distributions.MvNormal, p)

Compute quantiles of marginal distributions of dist.

Examples

Distributions.quantile(Distributions.MvNormal([0,1,2,3], LinearAlgebra.I(4)), .5)
+Distributions.quantile(Distributions.MvNormal([0,1,2,3], LinearAlgebra.I(4)), [[.1, .5, .9]])
source

Prior Configuration

AutoGP.GP.GPConfig — Type
config = GPConfig(kwargs...)

Configuration of prior distribution over Gaussian process kernels, i.e., an instance of Node. The main kwargs (all optional) are:

  • node_dist_leaf::Vector{Real}: Prior distribution over LeafNode kernels; default is uniform.

  • node_dist_nocp::Vector{Real}: Prior distribution over BinaryOpNode kernels; only used if changepoints=false.

  • node_dist_cp::Vector{Real}: Prior distribution over BinaryOpNode kernels; only used if changepoints=true.

  • max_depth::Integer: Maximum depth of covariance node; default is -1 for unbounded.

  • changepoints::Bool: Whether to permit ChangePoint compositions; default is true.

  • noise::Union{Nothing,Float64}: Whether to use a fixed observation noise; default is nothing to infer automatically.

source
diff --git a/dev/index.html b/dev/index.html index 6f335b5..1a3bd58 100644 --- a/dev/index.html +++ b/dev/index.html @@ -8,4 +8,4 @@ pages = {29473--29489}, year = {2023}, publisher = {PMLR}, -}

License

AutoGP.jl is licensed under the Apache License, Version 2.0; refer to LICENSE.

+}

License

AutoGP.jl is licensed under the Apache License, Version 2.0; refer to LICENSE.

diff --git a/dev/tutorials.html b/dev/tutorials.html index f523bb4..493f162 100644 --- a/dev/tutorials.html +++ b/dev/tutorials.html @@ -2,4 +2,4 @@ Tutorials · AutoGP
GitHub

Tutorials

To run the tutorials interactively, first install IJulia and PyPlot. Then run the following commands from the terminal, making sure to replace </path/to/>AutoGP.jl with the actual path to the directory:

$ export JULIA_NUM_THREADS=$(nproc)
 $ export JULIA_PROJECT=</path/to/>AutoGP.jl
 $ cd ${JULIA_PROJECT}/docs/src/tutorials
-$ julia -e 'using IJulia; notebook(dir=".")'

The notebook server will be available in the browser at https://localhost:8888.

+$ julia -e 'using IJulia; notebook(dir=".")'

The notebook server will be available in the browser at https://localhost:8888.

diff --git a/dev/tutorials/callbacks.html b/dev/tutorials/callbacks.html index c2fc90a..65b3706 100644 --- a/dev/tutorials/callbacks.html +++ b/dev/tutorials/callbacks.html @@ -144,4 +144,4 @@ │ └── PER(0.57, 0.20; 0.23) └── × ├── LIN(0.12; 0.19, 0.40) - └── LIN(0.10; 0.28, 1.18) + └── LIN(0.10; 0.28, 1.18) diff --git a/dev/tutorials/decomposition.html b/dev/tutorials/decomposition.html index 8cdd81b..37556b7 100644 --- a/dev/tutorials/decomposition.html +++ b/dev/tutorials/decomposition.html @@ -421,4 +421,4 @@ return fig, axes end
plot_decomposition (generic function with 1 method)
idx = 13
 fig, ax = plot_decomposition(decomposed_models[idx]);
-fig.suptitle("Decomposition of Learned Model $(idx)", fontsize=18, va="center", y=1);

png

+fig.suptitle("Decomposition of Learned Model $(idx)", fontsize=18, va="center", y=1);

png

diff --git a/dev/tutorials/greedy_mcmc.html b/dev/tutorials/greedy_mcmc.html index c5aa63c..86958ed 100644 --- a/dev/tutorials/greedy_mcmc.html +++ b/dev/tutorials/greedy_mcmc.html @@ -77,4 +77,4 @@ plt.close(fig) end
fn_callback (generic function with 1 method)
AutoGP.fit_mcmc!(model; n_mcmc=45, n_hmc=10, callback_fn=fn_callback)

Let's now show some plots that were collected by the callback during MCMC inference. The final plot shows an interesting example of how MCMC learning can reflect posterior uncertainty over the covariance structure.

display(figures[1])
 display(figures[10])
-display(figures[end])

png

png

png

MCMC vs SMC

MCMC sampling using AutoGP.fit_mcmc! invokes the same transition kernels over structure and parameters as those used in particle rejuvenation step of AutoGP.fit_smc!. The main difference is that AutoGP.fit_smc! anneals the posterior over subsets of data at each step, whereas AutoGP.fit_mcmc! uses the full data at each step.

The benefits of SMC include

  1. Runtime efficiency in cases where the structure can be quickly inferred using a subset of the data; whereas MCMC always conditions on the full data at each step yielding more expensive likelihood evaluations.
  2. Principled online inference, by using AutoGP.add_data!; whereas MCMC is an offline method.
  3. Particle resampling, by using AutoGP.maybe_resample!, to redirect computational effort to more promising structures and parameters; whereas MCMC iterates independent particles (i.e., chains).
  4. The availability of unbiased marginal likelihood estimates (via AutoGP.log_marginal_likelihood_estimate; whereas the marginal likelihood estimate obtained from MCMC is essentially importance sampling the posterior using the prior as a proposal.
+display(figures[end])

png

png

png

MCMC vs SMC

MCMC sampling using AutoGP.fit_mcmc! invokes the same transition kernels over structure and parameters as those used in particle rejuvenation step of AutoGP.fit_smc!. The main difference is that AutoGP.fit_smc! anneals the posterior over subsets of data at each step, whereas AutoGP.fit_mcmc! uses the full data at each step.

The benefits of SMC include

  1. Runtime efficiency in cases where the structure can be quickly inferred using a subset of the data; whereas MCMC always conditions on the full data at each step yielding more expensive likelihood evaluations.
  2. Principled online inference, by using AutoGP.add_data!; whereas MCMC is an offline method.
  3. Particle resampling, by using AutoGP.maybe_resample!, to redirect computational effort to more promising structures and parameters; whereas MCMC iterates independent particles (i.e., chains).
  4. The availability of unbiased marginal likelihood estimates (via AutoGP.log_marginal_likelihood_estimate; whereas the marginal likelihood estimate obtained from MCMC is essentially importance sampling the posterior using the prior as a proposal.
diff --git a/dev/tutorials/iclaims.html b/dev/tutorials/iclaims.html index 5e0c852..8871c85 100644 --- a/dev/tutorials/iclaims.html +++ b/dev/tutorials/iclaims.html @@ -225,4 +225,4 @@ ax.plot(ds_query, exp.(Distributions.mean(mvn)), color="k", label="Incorrect Mean Forecasts in Direct Space") ax.axvline(df_test.ds[end], color="k", linestyle="--") ax.legend()

png

PyObject <matplotlib.legend.Legend object at 0x7f5b068d02e0>

The difference between the blue and black curves is too small to observe on the scale above; let us plot the bias that arises from doing a naive transformation of the predictive mean.

fig, ax = plt.subplots()
-ax.plot(ds_query, Distributions.mean(log_mvn) - exp.(Distributions.mean(mvn)));

png

+ax.plot(ds_query, Distributions.mean(log_mvn) - exp.(Distributions.mean(mvn)));

png

diff --git a/dev/tutorials/overview.html b/dev/tutorials/overview.html index 9ce531c..d2ef6e0 100644 --- a/dev/tutorials/overview.html +++ b/dev/tutorials/overview.html @@ -464,4 +464,4 @@ ax.fill_between( subdf.ds, subdf[!,"y_0.025"], subdf[!,"y_0.975"]; color="tab:blue", alpha=0.05) -end

png

+end

png

diff --git a/dev/utils.html b/dev/utils.html index 3f2eb8c..b0cc2cd 100644 --- a/dev/utils.html +++ b/dev/utils.html @@ -1,7 +1,7 @@ -Utilities · AutoGP
GitHub

Utilities

AutoGP.Callbacks

AutoGP.Callbacks.make_smc_callback — Method
make_smc_callback(fn::Function, model::AutoGP.GPModel; kwargs...)

Convert fn into a callback for AutoGP.fit_smc!(model, ...).

The function fn must have a signature of the form fn(; [<opt>,] kw...), where kw... is a varargs specifier and <opt> denotes a (possibly empty) collection of required and optional keyword arguments.

For example fn(; a, b=2, kw...) is valid because fn takes no named arguments and the varargs specifier kw... is present. If fn includes named keyword arguments (i.e. a and b), then all required keyword arguments (i.e., a) must be provided in the call to make_smc_callback and optional keyword arguments (i.e., b) may be provided, as shown in the examples below.

model = AutoGP.GPModel(...)
+Utilities · AutoGP
GitHub
Utilities
AutoGP.Callbacks
AutoGP.Callbacks.make_smc_callback — Method
make_smc_callback(fn::Function, model::AutoGP.GPModel; kwargs...)
Convert fn into a callback for AutoGP.fit_smc!(model, ...).
The function fn must have a signature of the form fn(; [<opt>,] kw...), where kw... is a varargs specifier and <opt> denotes a (possibly empty) collection of required and optional keyword arguments.
For example  fn(; a, b=2, kw...) is valid because fn takes no named arguments and the varargs specifier kw... is present. If fn includes named keyword arguments (i.e. a and b), then all required keyword arguments (i.e., a) must be provided in the call to make_smc_callback and optional keyword arguments (i.e., b) may be provided, as shown in the examples below.
model = AutoGP.GPModel(...)
 fn = (; a, b=2, kw...) -> ...
 make_smc_callback(fn, model; a=2)       # valid, `a` specified, `b` optional.
 make_smc_callback(fn, model; a=1, b=2)  # valid, `a` and `b` are specified.
 make_smc_callback(fn, model)            # invalid, `a` required but not specified.
-make_smc_callback(fn, model; b=2)       # invalid, `a` required but not specified.
The callback return by make_smc_callback is guaranteed to receive in its varargs (which we called kw) an "SMC" state which. The following variables can be accessed in the body of fn by indexing kw:
  • kw[:step]::Integer: Current SMC step.
  • kw[:model]::AutoGP.GPModel: Inferred model at current SMC step.
  • kw[:ds_next]::AutoGP.IndexType: Future ds (time points) to be observed in later SMC rounds.
  • kw[:y_next]::Vector{<:Real}:  Future y (observations) to be observed in later SMC rounds.
  • kw[:rejuvenated]::Bool: Did rejuvenation occur at current step?
  • kw[:resampled]:Bool: Did resampling occur at current step?
  • kw[:elapsed]::Float64: Wall-clock inference runtime elapsed.
  • kw[:verbose]::Bool: Verbose setting.
  • kw[:schedule]::Vector{Integer}: The SMC data annealing schedule.
  • kw[:permutation]::Vector{Integer}: Indexes used to shuffle (model.ds, model.y).
source
AutoGP.Schedule
AutoGP.Schedule.logarithmic_schedule — Method
logarithmic_schedule(n::Integer, base::Integer, start::Integer)
The first step adds start observations (must be positive). At step i, start⋅baseⁱ new observations are added.
source
AutoGP.Transforms

+make_smc_callback(fn, model; b=2)       # invalid, `a` required but not specified.

The callback return by make_smc_callback is guaranteed to receive in its varargs (which we called kw) an "SMC" state which. The following variables can be accessed in the body of fn by indexing kw:

  • kw[:step]::Integer: Current SMC step.
  • kw[:model]::AutoGP.GPModel: Inferred model at current SMC step.
  • kw[:ds_next]::AutoGP.IndexType: Future ds (time points) to be observed in later SMC rounds.
  • kw[:y_next]::Vector{<:Real}: Future y (observations) to be observed in later SMC rounds.
  • kw[:rejuvenated]::Bool: Did rejuvenation occur at current step?
  • kw[:resampled]:Bool: Did resampling occur at current step?
  • kw[:elapsed]::Float64: Wall-clock inference runtime elapsed.
  • kw[:verbose]::Bool: Verbose setting.
  • kw[:schedule]::Vector{Integer}: The SMC data annealing schedule.
  • kw[:permutation]::Vector{Integer}: Indexes used to shuffle (model.ds, model.y).
source

AutoGP.Schedule

AutoGP.Schedule.logarithmic_schedule — Method
logarithmic_schedule(n::Integer, base::Integer, start::Integer)

The first step adds start observations (must be positive). At step i, start⋅baseⁱ new observations are added.

source

AutoGP.Transforms

diff --git a/versions.js b/versions.js index 51d88aa..d903b67 100644 --- a/versions.js +++ b/versions.js @@ -1,13 +1,6 @@ var DOC_VERSIONS = [ "stable", "v0.1", - "v0.1.7", - "v0.1.6", - "v0.1.5", - "v0.1.4", - "v0.1.3", - "v0.1.1", - "v0.1.0", "dev", ]; var DOCUMENTER_NEWEST = "v0.1.7";