diff --git a/previews/PR1000/404.html b/previews/PR1000/404.html index 965aa1fae8..48d019e3d8 100644 --- a/previews/PR1000/404.html +++ b/previews/PR1000/404.html @@ -5,10 +5,10 @@ 404 | Lux.jl Docs - + - + @@ -25,7 +25,7 @@
- + \ No newline at end of file diff --git a/previews/PR1000/api/Accelerator_Support/MLDataDevices.html b/previews/PR1000/api/Accelerator_Support/MLDataDevices.html index e2866f3915..c6f1e6abac 100644 --- a/previews/PR1000/api/Accelerator_Support/MLDataDevices.html +++ b/previews/PR1000/api/Accelerator_Support/MLDataDevices.html @@ -5,14 +5,14 @@ MLDataDevices | Lux.jl Docs - + - + - + - + @@ -30,10 +30,10 @@
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

MLDataDevices ​

MLDataDevices.jl is a lightweight package defining rules for transferring data across devices.

Preferences ​

MLDataDevices.gpu_backend! Function
julia
gpu_backend!() = gpu_backend!("")
 gpu_backend!(backend) = gpu_backend!(string(backend))
 gpu_backend!(backend::AbstractGPUDevice)
-gpu_backend!(backend::String)

Creates a LocalPreferences.toml file with the desired GPU backend.

If backend == "", then the gpu_backend preference is deleted. Otherwise, backend is validated to be one of the possible backends and the preference is set to backend.

If a new backend is successfully set, then the Julia session must be restarted for the change to take effect.

source

Data Transfer ​

MLDataDevices.cpu_device Function
julia
cpu_device() -> CPUDevice()

Return a CPUDevice object which can be used to transfer data to CPU.

source

MLDataDevices.gpu_device Function
julia
gpu_device(device_id::Union{Nothing, Integer}=nothing;
-    force::Bool=false) -> AbstractDevice

Selects GPU device based on the following criteria:

  1. If gpu_backend preference is set and the backend is functional on the system, then that device is selected.

  2. Otherwise, an automatic selection algorithm is used. We go over possible device backends in the order specified by supported_gpu_backends() and select the first functional backend.

  3. If no GPU device is functional and force is false, then cpu_device() is invoked.

  4. If nothing works, an error is thrown.

Arguments

  • device_id::Union{Nothing, Integer}: The device id to select. If nothing, then we return the last selected device or if none was selected then we run the autoselection and choose the current device using CUDA.device() or AMDGPU.device() or similar. If Integer, then we select the device with the given id. Note that this is 1-indexed, in contrast to the 0-indexed CUDA.jl. For example, id = 4 corresponds to CUDA.device!(3).

Warning

device_id is only applicable for CUDA and AMDGPU backends. For Metal, oneAPI and CPU backends, device_id is ignored and a warning is printed.

Warning

gpu_device won't select a CUDA device unless both CUDA.jl and cuDNN.jl are loaded. This is to ensure that deep learning operations work correctly. Nonetheless, if cuDNN is not loaded you can still manually create a CUDADevice object and use it (e.g. dev = CUDADevice()).

Keyword Arguments

  • force::Bool: If true, then an error is thrown if no functional GPU device is found.

source

MLDataDevices.xla_device Function
julia
xla_device(; force::Bool=false) -> Union{XLADevice, CPUDevice}

Return a XLADevice object if functional. Otherwise, throw an error if force is true. Falls back to CPUDevice if force is false.

Danger

This is an experimental feature and might change without deprecations

source

Miscellaneous ​

MLDataDevices.reset_gpu_device! Function
julia
reset_gpu_device!()

Resets the selected GPU device. This is useful when automatic GPU selection needs to be run again.

source

MLDataDevices.supported_gpu_backends Function
julia
supported_gpu_backends() -> Tuple{String, ...}

Return a tuple of supported GPU backends.

Warning

This is not the list of functional backends on the system, but rather backends which MLDataDevices.jl supports.

source

MLDataDevices.default_device_rng Function
julia
default_device_rng(::AbstractDevice)

Returns the default RNG for the device. This can be used to directly generate parameters and states on the device using WeightInitializers.jl.

source

MLDataDevices.get_device Function
julia
get_device(x) -> dev::AbstractDevice | Exception | Nothing

If all arrays (on the leaves of the structure) are on the same device, we return that device. Otherwise, we throw an error. If the object is device agnostic, we return nothing.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

  • nothing – denotes that the object is device agnostic. For example, scalar, abstract range, etc.

  • UnknownDevice() – denotes that the device type is unknown

See also get_device_type for a faster alternative that can be used for dispatch based on device type.

source

MLDataDevices.get_device_type Function
julia
get_device_type(x) -> Type{<:AbstractDevice} | Exception | Type{Nothing}

Similar to get_device but returns the type of the device instead of the device itself. This value is often a compile time constant and is recommended to be used instead of get_device where ever defining dispatches based on the device type.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

  • Nothing – denotes that the object is device agnostic. For example, scalar, abstract range, etc.

  • UnknownDevice – denotes that the device type is unknown

source

MLDataDevices.loaded Function
julia
loaded(x::AbstractDevice) -> Bool
-loaded(::Type{<:AbstractDevice}) -> Bool

Checks if the trigger package for the device is loaded. Trigger packages are as follows:

  • CUDA.jl and cuDNN.jl (or just LuxCUDA.jl) for NVIDIA CUDA Support.

  • AMDGPU.jl for AMD GPU ROCM Support.

  • Metal.jl for Apple Metal GPU Support.

  • oneAPI.jl for Intel oneAPI GPU Support.

source

MLDataDevices.functional Function
julia
functional(x::AbstractDevice) -> Bool
-functional(::Type{<:AbstractDevice}) -> Bool

Checks if the device is functional. This is used to determine if the device can be used for computation. Note that even if the backend is loaded (as checked via MLDataDevices.loaded), the device may not be functional.

Note that while this function is not exported, it is considered part of the public API.

source

MLDataDevices.isleaf Function
julia
isleaf(x) -> Bool

Returns true if x is a leaf node in the data structure.

Defining MLDataDevices.isleaf(x::T) = true for custom types can be used to customize the behavior the data movement behavior when an object with nested structure containing the type is transferred to a device.

Adapt.adapt_structure(::AbstractDevice, x::T) or Adapt.adapt_structure(::AbstractDevice, x::T) will be called during data movement if isleaf(x::T) == true.

If MLDataDevices.isleaf(x::T) is not defined, then it will fall back to Functors.isleaf(x).

source

Multi-GPU Support ​

MLDataDevices.set_device! Function
julia
set_device!(T::Type{<:AbstractDevice}, dev_or_id)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

  • T::Type{<:AbstractDevice}: The device type to set.

  • dev_or_id: Can be the device from the corresponding package. For example for CUDA it can be a CuDevice. If it is an integer, it is the device id to set. This is 1-indexed.

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

julia
set_device!(T::Type{<:AbstractDevice}, ::Nothing, rank::Integer)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

  • T::Type{<:AbstractDevice}: The device type to set.

  • rank::Integer: Local Rank of the process. This is applicable for distributed training and must be 0-indexed.

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

Iteration ​

MLDataDevices.DeviceIterator Type
julia
DeviceIterator(dev::AbstractDevice, iterator)

Create a DeviceIterator that iterates through the provided iterator via iterate. Upon each iteration, the current batch is copied to the device dev, and the previous iteration is marked as freeable from GPU memory (via unsafe_free!) (no-op for a CPU device).

The conversion follows the same semantics as dev(<item from iterator>).

Similarity to CUDA.CuIterator

The design inspiration was taken from CUDA.CuIterator and was generalized to work with other backends and more complex iterators (using Functors).

MLUtils.DataLoader

Calling dev(::MLUtils.DataLoader) will automatically convert the dataloader to use the same semantics as DeviceIterator. This is generally preferred over looping over the dataloader directly and transferring the data to the device.

Examples

The following was run on a computer with an NVIDIA GPU.

julia
julia> using MLDataDevices, MLUtils
+gpu_backend!(backend::String)

Creates a LocalPreferences.toml file with the desired GPU backend.

If backend == "", then the gpu_backend preference is deleted. Otherwise, backend is validated to be one of the possible backends and the preference is set to backend.

If a new backend is successfully set, then the Julia session must be restarted for the change to take effect.

source

Data Transfer ​

MLDataDevices.cpu_device Function
julia
cpu_device() -> CPUDevice()

Return a CPUDevice object which can be used to transfer data to CPU.

source

MLDataDevices.gpu_device Function
julia
gpu_device(device_id::Union{Nothing, Integer}=nothing;
+    force::Bool=false) -> AbstractDevice

Selects GPU device based on the following criteria:

  1. If gpu_backend preference is set and the backend is functional on the system, then that device is selected.

  2. Otherwise, an automatic selection algorithm is used. We go over possible device backends in the order specified by supported_gpu_backends() and select the first functional backend.

  3. If no GPU device is functional and force is false, then cpu_device() is invoked.

  4. If nothing works, an error is thrown.

Arguments

  • device_id::Union{Nothing, Integer}: The device id to select. If nothing, then we return the last selected device or if none was selected then we run the autoselection and choose the current device using CUDA.device() or AMDGPU.device() or similar. If Integer, then we select the device with the given id. Note that this is 1-indexed, in contrast to the 0-indexed CUDA.jl. For example, id = 4 corresponds to CUDA.device!(3).

Warning

device_id is only applicable for CUDA and AMDGPU backends. For Metal, oneAPI and CPU backends, device_id is ignored and a warning is printed.

Warning

gpu_device won't select a CUDA device unless both CUDA.jl and cuDNN.jl are loaded. This is to ensure that deep learning operations work correctly. Nonetheless, if cuDNN is not loaded you can still manually create a CUDADevice object and use it (e.g. dev = CUDADevice()).

Keyword Arguments

  • force::Bool: If true, then an error is thrown if no functional GPU device is found.

source

MLDataDevices.xla_device Function
julia
xla_device(; force::Bool=false) -> Union{XLADevice, CPUDevice}

Return a XLADevice object if functional. Otherwise, throw an error if force is true. Falls back to CPUDevice if force is false.

Danger

This is an experimental feature and might change without deprecations

source

Miscellaneous ​

MLDataDevices.reset_gpu_device! Function
julia
reset_gpu_device!()

Resets the selected GPU device. This is useful when automatic GPU selection needs to be run again.

source

MLDataDevices.supported_gpu_backends Function
julia
supported_gpu_backends() -> Tuple{String, ...}

Return a tuple of supported GPU backends.

Warning

This is not the list of functional backends on the system, but rather backends which MLDataDevices.jl supports.

source

MLDataDevices.default_device_rng Function
julia
default_device_rng(::AbstractDevice)

Returns the default RNG for the device. This can be used to directly generate parameters and states on the device using WeightInitializers.jl.

source

MLDataDevices.get_device Function
julia
get_device(x) -> dev::AbstractDevice | Exception | Nothing

If all arrays (on the leaves of the structure) are on the same device, we return that device. Otherwise, we throw an error. If the object is device agnostic, we return nothing.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

  • nothing – denotes that the object is device agnostic. For example, scalar, abstract range, etc.

  • UnknownDevice() – denotes that the device type is unknown

See also get_device_type for a faster alternative that can be used for dispatch based on device type.

source

MLDataDevices.get_device_type Function
julia
get_device_type(x) -> Type{<:AbstractDevice} | Exception | Type{Nothing}

Similar to get_device but returns the type of the device instead of the device itself. This value is often a compile time constant and is recommended to be used instead of get_device where ever defining dispatches based on the device type.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

  • Nothing – denotes that the object is device agnostic. For example, scalar, abstract range, etc.

  • UnknownDevice – denotes that the device type is unknown

source

MLDataDevices.loaded Function
julia
loaded(x::AbstractDevice) -> Bool
+loaded(::Type{<:AbstractDevice}) -> Bool

Checks if the trigger package for the device is loaded. Trigger packages are as follows:

  • CUDA.jl and cuDNN.jl (or just LuxCUDA.jl) for NVIDIA CUDA Support.

  • AMDGPU.jl for AMD GPU ROCM Support.

  • Metal.jl for Apple Metal GPU Support.

  • oneAPI.jl for Intel oneAPI GPU Support.

source

MLDataDevices.functional Function
julia
functional(x::AbstractDevice) -> Bool
+functional(::Type{<:AbstractDevice}) -> Bool

Checks if the device is functional. This is used to determine if the device can be used for computation. Note that even if the backend is loaded (as checked via MLDataDevices.loaded), the device may not be functional.

Note that while this function is not exported, it is considered part of the public API.

source

MLDataDevices.isleaf Function
julia
isleaf(x) -> Bool

Returns true if x is a leaf node in the data structure.

Defining MLDataDevices.isleaf(x::T) = true for custom types can be used to customize the behavior the data movement behavior when an object with nested structure containing the type is transferred to a device.

Adapt.adapt_structure(::AbstractDevice, x::T) or Adapt.adapt_structure(::AbstractDevice, x::T) will be called during data movement if isleaf(x::T) == true.

If MLDataDevices.isleaf(x::T) is not defined, then it will fall back to Functors.isleaf(x).

source

Multi-GPU Support ​

MLDataDevices.set_device! Function
julia
set_device!(T::Type{<:AbstractDevice}, dev_or_id)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

  • T::Type{<:AbstractDevice}: The device type to set.

  • dev_or_id: Can be the device from the corresponding package. For example for CUDA it can be a CuDevice. If it is an integer, it is the device id to set. This is 1-indexed.

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

julia
set_device!(T::Type{<:AbstractDevice}, ::Nothing, rank::Integer)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

  • T::Type{<:AbstractDevice}: The device type to set.

  • rank::Integer: Local Rank of the process. This is applicable for distributed training and must be 0-indexed.

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

Iteration ​

MLDataDevices.DeviceIterator Type
julia
DeviceIterator(dev::AbstractDevice, iterator)

Create a DeviceIterator that iterates through the provided iterator via iterate. Upon each iteration, the current batch is copied to the device dev, and the previous iteration is marked as freeable from GPU memory (via unsafe_free!) (no-op for a CPU device).

The conversion follows the same semantics as dev(<item from iterator>).

Similarity to CUDA.CuIterator

The design inspiration was taken from CUDA.CuIterator and was generalized to work with other backends and more complex iterators (using Functors).

MLUtils.DataLoader

Calling dev(::MLUtils.DataLoader) will automatically convert the dataloader to use the same semantics as DeviceIterator. This is generally preferred over looping over the dataloader directly and transferring the data to the device.

Examples

The following was run on a computer with an NVIDIA GPU.

julia
julia> using MLDataDevices, MLUtils
 
 julia> X = rand(Float64, 3, 33);
 
@@ -51,8 +51,8 @@
        end
 (i, summary(x)) = (1, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
 (i, summary(x)) = (2, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
-(i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

- +(i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Building_Blocks/LuxCore.html b/previews/PR1000/api/Building_Blocks/LuxCore.html index df167098c7..c1344ea7d3 100644 --- a/previews/PR1000/api/Building_Blocks/LuxCore.html +++ b/previews/PR1000/api/Building_Blocks/LuxCore.html @@ -5,14 +5,14 @@ LuxCore | Lux.jl Docs - + - + - + - + @@ -27,8 +27,8 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

LuxCore.AbstractLuxLayer Type
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

  • initialparameters(rng::AbstractRNG, layer::CustomAbstractLuxLayer) – This returns a NamedTuple containing the trainable parameters for the layer.

  • initialstates(rng::AbstractRNG, layer::CustomAbstractLuxLayer) – This returns a NamedTuple containing the current state for the layer. For most layers this is typically empty. Layers that would potentially contain this include BatchNorm, LSTM, GRU, etc.

Optionally:

  • parameterlength(layer::CustomAbstractLuxLayer) – These can be automatically calculated, but it is recommended that the user defines these.

  • statelength(layer::CustomAbstractLuxLayer) – These can be automatically calculated, but it is recommended that the user defines these.

See also AbstractLuxContainerLayer

source

LuxCore.AbstractLuxWrapperLayer Type
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

LuxCore.AbstractLuxContainerLayer Type
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

General ​

LuxCore.apply Function
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

LuxCore.stateless_apply Function
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

LuxCore.check_fmap_condition Function
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

  • cond - A function that takes a single argument and returns a Bool.

  • tmatch - A shortcut to check if x is of type tmatch. Can be disabled by passing nothing.

  • x - The structure to check.

Returns

A Boolean Value

source

LuxCore.contains_lux_layer Function
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

LuxCore.display_name Function
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

LuxCore.replicate Function
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

LuxCore.setup Function
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

Parameters ​

LuxCore.initialparameters Function
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

LuxCore.parameterlength Function
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

States ​

LuxCore.initialstates Function
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

LuxCore.statelength Function
julia
statelength(layer)

Return the total number of states of the layer l.

source

LuxCore.testmode Function
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

LuxCore.trainmode Function
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

LuxCore.update_state Function
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

Layer size ​

LuxCore.outputsize Function
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

- +
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

LuxCore.AbstractLuxLayer Type
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

  • initialparameters(rng::AbstractRNG, layer::CustomAbstractLuxLayer) – This returns a NamedTuple containing the trainable parameters for the layer.

  • initialstates(rng::AbstractRNG, layer::CustomAbstractLuxLayer) – This returns a NamedTuple containing the current state for the layer. For most layers this is typically empty. Layers that would potentially contain this include BatchNorm, LSTM, GRU, etc.

Optionally:

  • parameterlength(layer::CustomAbstractLuxLayer) – These can be automatically calculated, but it is recommended that the user defines these.

  • statelength(layer::CustomAbstractLuxLayer) – These can be automatically calculated, but it is recommended that the user defines these.

See also AbstractLuxContainerLayer

source

LuxCore.AbstractLuxWrapperLayer Type
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

LuxCore.AbstractLuxContainerLayer Type
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

General ​

LuxCore.apply Function
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

LuxCore.stateless_apply Function
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

LuxCore.check_fmap_condition Function
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

  • cond - A function that takes a single argument and returns a Bool.

  • tmatch - A shortcut to check if x is of type tmatch. Can be disabled by passing nothing.

  • x - The structure to check.

Returns

A Boolean Value

source

LuxCore.contains_lux_layer Function
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

LuxCore.display_name Function
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

LuxCore.replicate Function
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

LuxCore.setup Function
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

Parameters ​

LuxCore.initialparameters Function
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

LuxCore.parameterlength Function
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

States ​

LuxCore.initialstates Function
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

LuxCore.statelength Function
julia
statelength(layer)

Return the total number of states of the layer l.

source

LuxCore.testmode Function
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

LuxCore.trainmode Function
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

LuxCore.update_state Function
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

Layer size ​

LuxCore.outputsize Function
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Building_Blocks/LuxLib.html b/previews/PR1000/api/Building_Blocks/LuxLib.html index e2ca8d87ff..bdfc848c4e 100644 --- a/previews/PR1000/api/Building_Blocks/LuxLib.html +++ b/previews/PR1000/api/Building_Blocks/LuxLib.html @@ -5,14 +5,14 @@ LuxLib | Lux.jl Docs - + - + - + - + @@ -27,20 +27,20 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

LuxLib ​

Backend for Lux.jl

Apply Activation ​

LuxLib.API.fast_activation Function
julia
fast_activation(σ::F, x::AbstractArray) where {F}

Compute σ.(x) with the best possible implementation available. On CPUs we unroll the loop and use LoopVectorization.jl to vectorize the computation. On GPUs we use simply use broadcasting.

Note

This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.

Arguments

  • σ: Activation function

  • x: Input array

Returns

  • Output Array with the same size as x

source

LuxLib.API.fast_activation!! Function
julia
fast_activation!!(σ::F, x::AbstractArray) where {F}

Compute σ.(x) with the best possible implementation available. If it is possible to rewrite x in-place, it does so. If x is an immutable array, it falls back to the generic implementation.

Note

This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.

Load SLEEFPirates.jl to get faster activations

Certain activation functions are replaced with specialized implementations from SLEEFPirates.jl for FP32. This might lead to faster performance but can cause slight decrease in accuracy (in the floating point limit).

Arguments

  • σ: Activation function

  • x: Input array

Returns

  • Output Array with the same size as x

source

Batched Operations ​

LuxLib.API.batched_matmul Function
julia
batched_matmul(x, y)

Computes the batched matrix multiplication of x and y. For more details see the NNlib documentation on NNlib.batched_mul. This function is mostly a wrapper around batched_mul but attempts to be faster on CPUs.

Load LoopVectorization.jl to get faster batched matrix multiplication

On CPUs loading LoopVectorization adds faster implementations of batched matrix multiplication.

source

Bias Activation ​

LuxLib.API.bias_activation Function
julia
bias_activation(σ, x, bias)

Applies the activation function σ elementwise to the result of broadcasted addition of x and bias along the penultimate dimension. A vector x is treated as a matrix with a single last dimension.

Arguments

  • σ: Activation function

  • x: Input to be transformed

  • bias: Bias to be added. Can be nothing.

See also bias_activation!!, fast_activation.

source

LuxLib.API.bias_activation!! Function
julia
bias_activation!!(σ, x, bias)

Same as bias_activation but might update x in-place if possible. Users should not rely on x being mutated, it is recommended to use it like y = bias_activation!!(σ, x, bias). If x is updated in-place, y aliases x.

See also bias_activation, fast_activation!!.

source

Convolutional Layers ​

LuxLib.API.fused_conv_bias_activation Function
julia
fused_conv_bias_activation(σ::F, weight::AbstractArray, x::AbstractArray,
-    b::Optional{<:AbstractVector}, cdims::ConvDims) where {F}

Computes σ.(conv(x, weight, cdims) .+ b) (b is not exactly broadcasted like this, rather it is reshaped and broadcasted to the penultimate dimension) with the best possible implementation available. This operation fuses operations into a single kernel if possible, and minimizes reallocations by reusing the output buffer for multiple operations.

Arguments

  • σ: Activation function

  • weight: Weight tensor

  • x: Input tensor

  • b: Bias tensor (can be nothing)

  • cdims: ConvDims object

Notes on implementation

  • For CUDA Arrays, this uses fused CUDNN kernels when the activation is identity or relu. For other activations, it tries to fuse the operations on the Julia side.

  • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.

  • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.

  • For Mixed-Precision Inputs on GPU, we type promote the inputs to the highest precision, with a warning.

source

Dropout ​

LuxLib.API.alpha_dropout Function
julia
alpha_dropout(rng::AbstractRNG, x, p, training)
-alpha_dropout(rng::AbstractRNG, x, p, training, α, A, B)

Alpha Dropout: Dropout ensuring that the mean and variance of the output remains same as the input. For details see [1]. Use the second call signature to avoid recomputing the constants for a fixed dropout probability.

Arguments

  • rng: Random number generator

  • x: Input Array

  • p: Probability of an element to be dropped out

  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context`

  • α: -1.7580993408473766. Computed at limit x tends to infinity, selu(x) = -λβ = α

  • A: Scaling factor for the mean

  • B: Scaling factor for the variance

Returns

  • Output Array after applying alpha dropout

  • Updated state for the random number generator

References

[1] Klambauer, Günter, et al. "Self-normalizing neural networks." Advances in neural information processing systems 30 (2017).

source

LuxLib.API.dropout Function
julia
dropout(rng::AbstractRNG, x, p, training, invp, dims)
+    
Lux.jl Docs
Appearance

 Trusted by 
Scientific Computing
SciML.ai
Machine Learning
LuxLib ​
Backend for Lux.jl
Apply Activation ​
LuxLib.API.fast_activation Function
julia
fast_activation(σ::F, x::AbstractArray) where {F}
Compute σ.(x) with the best possible implementation available. On CPUs we unroll the loop and use LoopVectorization.jl to vectorize the computation. On GPUs we use simply use broadcasting.
Note
This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.
Arguments
  • σ: Activation function
  • x: Input array
Returns
  • Output Array with the same size as x
source
LuxLib.API.fast_activation!! Function
julia
fast_activation!!(σ::F, x::AbstractArray) where {F}
Compute σ.(x) with the best possible implementation available. If it is possible to rewrite x in-place, it does so. If x is an immutable array, it falls back to the generic implementation.
Note
This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.
Load SLEEFPirates.jl to get faster activations
Certain activation functions are replaced with specialized implementations from SLEEFPirates.jl for FP32. This might lead to faster performance but can cause slight decrease in accuracy (in the floating point limit).
Arguments
  • σ: Activation function
  • x: Input array
Returns
  • Output Array with the same size as x
source
Batched Operations ​
LuxLib.API.batched_matmul Function
julia
batched_matmul(x, y)
Computes the batched matrix multiplication of x and y. For more details see the NNlib documentation on NNlib.batched_mul. This function is mostly a wrapper around batched_mul but attempts to be faster on CPUs.
Load LoopVectorization.jl to get faster batched matrix multiplication
On CPUs loading LoopVectorization adds faster implementations of batched matrix multiplication.
source
Bias Activation ​
LuxLib.API.bias_activation Function
julia
bias_activation(σ, x, bias)
Applies the activation function σ elementwise to the result of broadcasted addition of x and bias along the penultimate dimension. A vector x is treated as a matrix with a single last dimension.
Arguments
  • σ: Activation function
  • x: Input to be transformed
  • bias: Bias to be added. Can be nothing.
See also bias_activation!!, fast_activation.
source
LuxLib.API.bias_activation!! Function
julia
bias_activation!!(σ, x, bias)
Same as bias_activation but might update x in-place if possible. Users should not rely on x being mutated, it is recommended to use it like y = bias_activation!!(σ, x, bias). If x is updated in-place, y aliases x.
See also bias_activation, fast_activation!!.
source
Convolutional Layers ​
LuxLib.API.fused_conv_bias_activation Function
julia
fused_conv_bias_activation(σ::F, weight::AbstractArray, x::AbstractArray,
+    b::Optional{<:AbstractVector}, cdims::ConvDims) where {F}
Computes σ.(conv(x, weight, cdims) .+ b) (b is not exactly broadcasted like this, rather it is reshaped and broadcasted to the penultimate dimension) with the best possible implementation available. This operation fuses operations into a single kernel if possible, and minimizes reallocations by reusing the output buffer for multiple operations.
Arguments
  • σ: Activation function
  • weight: Weight tensor
  • x: Input tensor
  • b: Bias tensor (can be nothing)
  • cdims: ConvDims object
Notes on implementation
  • For CUDA Arrays, this uses fused CUDNN kernels when the activation is identity or relu. For other activations, it tries to fuse the operations on the Julia side.
  • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
  • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
  • For Mixed-Precision Inputs on GPU, we type promote the inputs to the highest precision, with a warning.
source
Dropout ​
LuxLib.API.alpha_dropout Function
julia
alpha_dropout(rng::AbstractRNG, x, p, training)
+alpha_dropout(rng::AbstractRNG, x, p, training, α, A, B)
Alpha Dropout: Dropout ensuring that the mean and variance of the output remains same as the input. For details see [1]. Use the second call signature to avoid recomputing the constants for a fixed dropout probability.
Arguments
  • rng: Random number generator
  • x: Input Array
  • p: Probability of an element to be dropped out
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context`
  • α: -1.7580993408473766. Computed at limit x tends to infinity, selu(x) = -λβ = α
  • A: Scaling factor for the mean
  • B: Scaling factor for the variance
Returns
  • Output Array after applying alpha dropout
  • Updated state for the random number generator
References
[1] Klambauer, Günter, et al. "Self-normalizing neural networks." Advances in neural information processing systems 30 (2017).
source
LuxLib.API.dropout Function
julia
dropout(rng::AbstractRNG, x, p, training, invp, dims)
 dropout(rng::AbstractRNG, x, mask, p, training, update_mask::Union{Val, StaticBool},
-    invp, dims)
Dropout: Simple Way to prevent Neural Networks for Overfitting. For details see [1].
Arguments
  • rng: Random number generator
  • x: Input Array
  • mask: Dropout Mask. If not used then it is constructed automatically
  • p: Probability of an element to be dropped out
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • update_mask: If Val(true) or True() then the mask is generated and used. Else, the mask provided is directly used
  • invp: Inverse multiplied to the mask. Calculated as invp = 1 / (1 - p).
Returns
  • Output Array after applying dropout
  • Dropout Mask (if training == false, the returned value is meaningless)
  • Updated state for the random number generator
References
[1] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The journal of machine learning research 15.1 (2014): 1929-1958.
source
Fully Connected Layers ​
LuxLib.API.fused_dense_bias_activation Function
julia
fused_dense_bias_activation(σ::F, weight::AbstractMatrix, x::AbstractMatrix,
+    invp, dims)
Dropout: Simple Way to prevent Neural Networks for Overfitting. For details see [1].
Arguments
  • rng: Random number generator
  • x: Input Array
  • mask: Dropout Mask. If not used then it is constructed automatically
  • p: Probability of an element to be dropped out
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • update_mask: If Val(true) or True() then the mask is generated and used. Else, the mask provided is directly used
  • invp: Inverse multiplied to the mask. Calculated as invp = 1 / (1 - p).
Returns
  • Output Array after applying dropout
  • Dropout Mask (if training == false, the returned value is meaningless)
  • Updated state for the random number generator
References
[1] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The journal of machine learning research 15.1 (2014): 1929-1958.
source
Fully Connected Layers ​
LuxLib.API.fused_dense_bias_activation Function
julia
fused_dense_bias_activation(σ::F, weight::AbstractMatrix, x::AbstractMatrix,
     b::Optional{<:AbstractVector}) where {F}
Compute σ.(weight * x .+ b) with the best possible implementation available. Currently this implementation attempts to minimize reallocations by reusing the output buffer for multiple operations.
Arguments
  • σ: Activation function
  • weight: Weight matrix
  • x: Input matrix
  • b: Bias vector (can be nothing)
Notes on implementation
  • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
  • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
  • For CUDA Arrays, this uses a special fused implementation via cuBLASLt.
  • For small CPU Arrays, we use LoopVectorization.jl. On x86_64 we use Octavian for medium sized matrices. This is overridden if special BLAS implementations are loaded (currently MKL, AppleAccelerate, and BLISBLAS).
!!! tip "Load Octavian.jl
Loading `Octavian.jl` enables a polyalgorithm that uses different backends based on the
-input sizes.
source
Normalization ​
LuxLib.API.batchnorm Function
julia
batchnorm(x, scale, bias, running_mean, running_var, training,
-    σ=identity, momentum = 0.1f0, epsilon = eps(eltype(x)) ^ (5 // 7))
Batch Normalization. For details see [1].
Batch Normalization computes the mean and variance for each D1×...×DN−2×1×DN input slice and normalises the input accordingly.
Arguments
  • x: Input to be Normalized
  • scale: Scale factor (γ) (can be nothing)
  • bias: Bias factor (β) (can be nothing)
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
Returns
Normalized Array of same size as x. And a Named Tuple containing the updated running mean and variance.
References
[1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International conference on machine learning. PMLR, 2015.
source
LuxLib.API.groupnorm Function
julia
groupnorm(x, scale, bias, groups::Int, σ::F=identity,
-    epsilon::Real=eps(eltype(x)) ^ (5 // 7))
Group Normalization. For details see [1].
This op is similar to batch normalization, but statistics are shared across equally-sized groups of channels and not shared across batch dimension. Thus, group normalization does not depend on the batch composition and does not require maintaining internal state for storing statistics.
Arguments
  • x: Input to be Normalized
  • scale: Scale factor (γ) (can be nothing)
  • bias: Bias factor (β) (can be nothing)
  • groups: Number of groups
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
Returns
The normalized array is returned.
References
[1] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European conference on computer vision (ECCV). 2018.
source
LuxLib.API.instancenorm Function
julia
instancenorm(x, scale, bias, training, act, epsilon = eps(eltype(x)) ^ (5 // 7))
+input sizes.
source
Normalization ​
LuxLib.API.batchnorm Function
julia
batchnorm(x, scale, bias, running_mean, running_var, training,
+    σ=identity, momentum = 0.1f0, epsilon = eps(eltype(x)) ^ (5 // 7))
Batch Normalization. For details see [1].
Batch Normalization computes the mean and variance for each D1×...×DN−2×1×DN input slice and normalises the input accordingly.
Arguments
  • x: Input to be Normalized
  • scale: Scale factor (γ) (can be nothing)
  • bias: Bias factor (β) (can be nothing)
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
Returns
Normalized Array of same size as x. And a Named Tuple containing the updated running mean and variance.
References
[1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International conference on machine learning. PMLR, 2015.
source
LuxLib.API.groupnorm Function
julia
groupnorm(x, scale, bias, groups::Int, σ::F=identity,
+    epsilon::Real=eps(eltype(x)) ^ (5 // 7))
Group Normalization. For details see [1].
This op is similar to batch normalization, but statistics are shared across equally-sized groups of channels and not shared across batch dimension. Thus, group normalization does not depend on the batch composition and does not require maintaining internal state for storing statistics.
Arguments
  • x: Input to be Normalized
  • scale: Scale factor (γ) (can be nothing)
  • bias: Bias factor (β) (can be nothing)
  • groups: Number of groups
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
Returns
The normalized array is returned.
References
[1] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European conference on computer vision (ECCV). 2018.
source
LuxLib.API.instancenorm Function
julia
instancenorm(x, scale, bias, training, act, epsilon = eps(eltype(x)) ^ (5 // 7))
 instancenorm(x, scale, bias, running_mean, running_var, training, act, momentum,
-    epsilon = eps(eltype(x)) ^ (5 // 7))
Instance Normalization. For details see [1].
Instance Normalization computes the mean and variance for each D1×...×DN−2×1×1 input slice and normalises the input accordingly.
Arguments
  • x: Input to be Normalized (must be atleast 3D)
  • scale: Scale factor (γ) (can be nothing)
  • bias: Bias factor (β) (can be nothing)
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
Returns
Normalized Array of same size as x. And a Named Tuple containing the updated running mean and variance.
References
[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).
source
LuxLib.API.layernorm Function
julia
layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
-    epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}
Layer Normalization. For details see [1].
Given an input array x, this layer computes
y=x−E[x]Var[x]+ϵ∗γ+β
and applies the activation function σ elementwise to y.
Arguments
  • x: Input to be Normalized
  • scale: Scale factor (γ) (can be nothing)
  • bias: Bias factor (β) (can be nothing)
  • σ: Activation function (default: identity)
  • dims: Dimensions along which the mean and std of x is computed. If nothing is passed, the dims are inferred based on the dimensions of scale and bias. For example, if x is N dimensional and scale and bias are M dimensional, then the dims will be 1:(N - M).
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
Returns
Normalized Array of same size as x.
References
[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).
source
Helper Functions ​
LuxLib.internal_operation_mode Function
julia
internal_operation_mode(xs::Tuple)
-internal_operation_mode(x::AbstractArray)
Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.
Currently supported modes are:
  • GenericBroadcastOp: This is the fallback for most types. For the following types this is the preferred mode:
    • Arrays with fast_scalar_indexing set to False.
    • Static Arrays
    • ReverseDiff Arrays
    • Tracker Arrays
    • ForwardDiff.Dual Arrays
  • GPUBroadcastOp{dev}: GPU Arrays where dev is obtained from get_device_type(xs). This option dispatches should preferably use KernelAbstractions or specialized vendor dispatches.
  • LoopedArrayOp: CPU arrays that can be optimized using SIMD Loops, ideally using LoopVectorization.jl or Polyester.jl.
source

-    
+    epsilon = eps(eltype(x)) ^ (5 // 7))

Instance Normalization. For details see [1].

Instance Normalization computes the mean and variance for each D1×...×DN−2×1×1 input slice and normalises the input accordingly.

Arguments

  • x: Input to be Normalized (must be atleast 3D)

  • scale: Scale factor (γ) (can be nothing)

  • bias: Bias factor (β) (can be nothing)

  • running_mean: Running mean (can be nothing)

  • running_var: Running variance (can be nothing)

  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context

  • σ: Activation function (default: identity)

  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))

  • momentum: Momentum for updating running mean and variance (default: 0.1f0)

Returns

Normalized Array of same size as x. And a Named Tuple containing the updated running mean and variance.

References

[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

source

LuxLib.API.layernorm Function
julia
layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
+    epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}

Layer Normalization. For details see [1].

Given an input array x, this layer computes

y=x−E[x]Var[x]+ϵ∗γ+β

and applies the activation function σ elementwise to y.

Arguments

  • x: Input to be Normalized

  • scale: Scale factor (γ) (can be nothing)

  • bias: Bias factor (β) (can be nothing)

  • σ: Activation function (default: identity)

  • dims: Dimensions along which the mean and std of x is computed. If nothing is passed, the dims are inferred based on the dimensions of scale and bias. For example, if x is N dimensional and scale and bias are M dimensional, then the dims will be 1:(N - M).

  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))

Returns

Normalized Array of same size as x.

References

[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).

source

Helper Functions ​

LuxLib.internal_operation_mode Function
julia
internal_operation_mode(xs::Tuple)
+internal_operation_mode(x::AbstractArray)

Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.

Currently supported modes are:

  • GenericBroadcastOp: This is the fallback for most types. For the following types this is the preferred mode:

    • Arrays with fast_scalar_indexing set to False.

    • Static Arrays

    • ReverseDiff Arrays

    • Tracker Arrays

    • ForwardDiff.Dual Arrays

  • GPUBroadcastOp{dev}: GPU Arrays where dev is obtained from get_device_type(xs). This option dispatches should preferably use KernelAbstractions or specialized vendor dispatches.

  • LoopedArrayOp: CPU arrays that can be optimized using SIMD Loops, ideally using LoopVectorization.jl or Polyester.jl.

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Building_Blocks/WeightInitializers.html b/previews/PR1000/api/Building_Blocks/WeightInitializers.html index b3c8a7825c..a3132e7f17 100644 --- a/previews/PR1000/api/Building_Blocks/WeightInitializers.html +++ b/previews/PR1000/api/Building_Blocks/WeightInitializers.html @@ -5,14 +5,14 @@ WeightInitializers | Lux.jl Docs - + - + - + - + @@ -27,9 +27,9 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

WeightInitializers ​

This package is a light dependency providing common weight initialization schemes for deep learning models.

Supported RNG Types ​

RNG Type / PackageReturned Array TypeUnsupported Functions
Random.jlArray
StableRNGs.jlArray
CUDA.default_rng()CuArray
GPUArrays.default_rng(CuArray)CuArray
AMDGPU.rocrand_rng()ROCArray
AMDGPU.gpuarrays_rng()ROCArray
GPUArrays.default_rng(ROCArray)ROCArray
Metal.gpuarrays_rng()MtlArrayorthogonal, truncated_normal
GPUArrays.default_rng(MtlArray)MtlArrayorthogonal, truncated_normal
oneAPI.gpuarrays_rng()oneArrayorthogonal, truncated_normal
GPUArrays.default_rng(oneArray)oneArrayorthogonal, truncated_normal

API Reference ​

Main Functions ​

WeightInitializers.glorot_normal Function
julia
glorot_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = 1) -> AbstractArray{T, length(size)}

Return an AbstractArray{T} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.

References

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.

source

WeightInitializers.glorot_uniform Function
julia
glorot_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = 1) -> AbstractArray{T, length(size)}

Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [−x,x], where x = gain * sqrt(6 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.

References

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.

source

WeightInitializers.identity_init Function
julia
identity_init([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; gain::Number=1,
+    
Lux.jl Docs
Appearance

 Trusted by 
Scientific Computing
SciML.ai
Machine Learning
WeightInitializers ​
This package is a light dependency providing common weight initialization schemes for deep learning models.
Supported RNG Types ​
RNG Type / PackageReturned Array TypeUnsupported Functions
Random.jlArray
StableRNGs.jlArray
CUDA.default_rng()CuArray
GPUArrays.default_rng(CuArray)CuArray
AMDGPU.rocrand_rng()ROCArray
AMDGPU.gpuarrays_rng()ROCArray
GPUArrays.default_rng(ROCArray)ROCArray
Metal.gpuarrays_rng()MtlArrayorthogonal
GPUArrays.default_rng(MtlArray)MtlArrayorthogonal
oneAPI.gpuarrays_rng()oneArrayorthogonal, truncated_normal
GPUArrays.default_rng(oneArray)oneArrayorthogonal, truncated_normal
API Reference ​
Main Functions ​
WeightInitializers.glorot_normal Function
julia
glorot_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = 1) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.
References
[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.
source
WeightInitializers.glorot_uniform Function
julia
glorot_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = 1) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [−x,x], where x = gain * sqrt(6 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.
References
[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.
source
WeightInitializers.identity_init Function
julia
identity_init([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; gain::Number=1,
     shift::Union{Integer, Tuple{Integer, Integer}}=0) -> AbstractArray{T}
Constructs an array that aims to provide an identity mapping when used as parameters in most layers of a neural network. The identity mapping is scaled by the gain parameter.
Behavior
  • 1D: Returns a Vector of zeros (useful for biases in layers where input_size == output_size).
  • 2D: Returns an identity matrix (useful for fully connected layers with equal input and output sizes).
  • More than 2D: Returns a tensor where the central slice along the last two dimensions is an identity matrix, and the rest are zeros (useful for convolutional layers, simulating an identity convolution).
Caveats
  • Not all layers will result in an identity mapping when using this initializer. Exceptions include recurrent and normalization layers.
  • Layers must have input_size == output_size for a perfect identity mapping. In cases where this condition is not met, the function pads extra dimensions with zeros.
  • For convolutional layers to achieve an identity mapping, kernel sizes must be odd, and appropriate padding must be applied to ensure the output feature maps are the same size as the input feature maps.
Arguments
  • rng::AbstractRNG: An optional random number generator, included for consistency with other initializers but ignored since the output is deterministic.
  • T::Type{<:Number}: The numeric type of the array elements.
  • size...: The dimensions of the array to be initialized.
  • gain::Number=1: A scaling factor applied to the identity mapping.
  • shift::Union{Integer, Tuple{Integer, Integer}}=0: An integer or a tuple specifying the circular shift applied to the output array.
Returns
  • AbstractArray{T}: An array initialized to represent an identity mapping, scaled by gain and optionally shifted by shift.
Examples
julia
julia> identity_init(Xoshiro(123), Float32, 5, 5)
 5×5 Matrix{Float32}:
  1.0  1.0  1.0  1.0  1.0
@@ -43,43 +43,43 @@
 [:, :, 1, 1] =
  0.0  0.0  0.0
  0.0  1.5  0.0
- 0.0  0.0  0.0
source
WeightInitializers.kaiming_normal Function
julia
kaiming_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = √T(2)) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size containing random numbers taken from a normal distribution standard deviation gain / sqrt(fan_in)
References
[1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
source
WeightInitializers.kaiming_uniform Function
julia
kaiming_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = √T(2)) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [-x, x], where x = gain * sqrt(3/fan_in).
References
[1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
source
WeightInitializers.sparse_init Function
julia
sparse_init([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
+ 0.0  0.0  0.0
source
WeightInitializers.kaiming_normal Function
julia
kaiming_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = √T(2)) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size containing random numbers taken from a normal distribution standard deviation gain / sqrt(fan_in)
References
[1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
source
WeightInitializers.kaiming_uniform Function
julia
kaiming_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = √T(2)) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [-x, x], where x = gain * sqrt(3/fan_in).
References
[1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
source
WeightInitializers.sparse_init Function
julia
sparse_init([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
     sparsity::Number, std::Number=0.01) -> AbstractArray{T}
Creates a sparsely initialized weight matrix with a specified proportion of zeroed elements, using random numbers drawn from a normal distribution for the non-zero elements. This method was introduced in [1].
Note
The sparsity parameter controls the proportion of the matrix that will be zeroed. For example, a sparsity of 0.3 means that approximately 30% of the elements will be set to zero. The non-zero elements are distributed according to a normal distribution, scaled by the std parameter.
Arguments
  • rng::AbstractRNG: The random number generator to use.
  • T::Type{<:Number}: The numeric type of the elements in the returned array.
  • dims::Integer...: The dimensions of the weight matrix to be generated.
  • sparsity::Number: The proportion of elements to be zeroed. Must be between 0 and 1.
  • std::Number=0.01: The standard deviation of the normal distribution before applying gain.
Returns
  • AbstractArray{T}: A sparsely initialized weight matrix of dimensions dims and type T.
Examples
julia
julia> y = sparse_init(Xoshiro(123), Float32, 5, 5; sparsity=0.3, std=0.01);
 
 julia> y isa Matrix{Float32}
 true
 
 julia> size(y) == (5, 5)
-true
References
[1] Martens, J, "Deep learning via Hessian-free optimization" Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.
source
WeightInitializers.truncated_normal Function
julia
truncated_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; mean = 0,
-    std = 1, lo = -2, hi = 2) -> AbstractArray{T, length(size)}
Return an AbstractArray{T} of the given size where each element is drawn from a truncated normal distribution. The numbers are distributed like filter(x -> lo ≤ x ≤ hi, mean .+ std .* randn(100)).
source
WeightInitializers.orthogonal Function
julia
orthogonal([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
-    gain = 1)  -> AbstractArray{T, length(dims)}
Return an AbstractArray{T} of the given dimensions (dims) which is a (semi) orthogonal matrix, as described in [1].
The function constructs an orthogonal or semi-orthogonal matrix depending on the specified dimensions. For two dimensions, it returns a matrix where dims = (rows, cols). For more than two dimensions, it computes an orthogonal matrix of size prod(dims[1:(end - 1)]) by dims[end] before reshaping it to the original dimensions.
Cannot construct a vector, i.e., length(dims) == 1 is forbidden.
Arguments
  • rng::AbstractRNG: Random number generator.
  • T::Type{<:Real}: The type of the elements in the array.
  • dims::Integer...: The dimensions of the array.
  • gain::Number: Scaling factor for the elements of the orthogonal matrix.
References
[1] Saxe, McClelland, Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", ICLR 2014, https://arxiv.org/abs/1312.6120
source
Other Convenience Functions ​
Beware
Unlike the other functions these ones don't take a type argument.
WeightInitializers.zeros16 Function
julia
zeros16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
Return an AbstractArray{Float16} of the given size containing an AbstractArray of zeros.
source
WeightInitializers.ones16 Function
julia
ones16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
Return an AbstractArray{Float16} of the given size containing an AbstractArray of ones.
source
WeightInitializers.rand16 Function
julia
rand16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
Return an AbstractArray{Float16} of the given size containing random numbers from a uniform distribution.
source
WeightInitializers.randn16 Function
julia
randn16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
Return an AbstractArray{Float16} of the given size containing random numbers from a standard normal distribution.
source
WeightInitializers.zeros32 Function
julia
zeros32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
Return an AbstractArray{Float32} of the given size containing an AbstractArray of zeros.
source
WeightInitializers.ones32 Function
julia
ones32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
Return an AbstractArray{Float32} of the given size containing an AbstractArray of ones.
source
WeightInitializers.rand32 Function
julia
rand32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
Return an AbstractArray{Float32} of the given size containing random numbers from a uniform distribution.
source
WeightInitializers.randn32 Function
julia
randn32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
Return an AbstractArray{Float32} of the given size containing random numbers from a standard normal distribution.
source
WeightInitializers.zeros64 Function
julia
zeros64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
Return an AbstractArray{Float64} of the given size containing an AbstractArray of zeros.
source
WeightInitializers.ones64 Function
julia
ones64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
Return an AbstractArray{Float64} of the given size containing an AbstractArray of ones.
source
WeightInitializers.rand64 Function
julia
rand64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
Return an AbstractArray{Float64} of the given size containing random numbers from a uniform distribution.
source
WeightInitializers.randn64 Function
julia
randn64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
Return an AbstractArray{Float64} of the given size containing random numbers from a standard normal distribution.
source
WeightInitializers.zerosC16 Function
julia
zerosC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of zeros.
source
WeightInitializers.onesC16 Function
julia
onesC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of ones.
source
WeightInitializers.randC16 Function
julia
randC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
Return an AbstractArray{ComplexF16} of the given size containing random numbers from a uniform distribution.
source
WeightInitializers.randnC16 Function
julia
randnC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
Return an AbstractArray{ComplexF16} of the given size containing random numbers from a standard normal distribution.
source
WeightInitializers.zerosC32 Function
julia
zerosC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of zeros.
source
WeightInitializers.onesC32 Function
julia
onesC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of ones.
source
WeightInitializers.randC32 Function
julia
randC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
Return an AbstractArray{ComplexF32} of the given size containing random numbers from a uniform distribution.
source
WeightInitializers.randnC32 Function
julia
randnC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
Return an AbstractArray{ComplexF32} of the given size containing random numbers from a standard normal distribution.
source
WeightInitializers.zerosC64 Function
julia
zerosC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of zeros.
source
WeightInitializers.onesC64 Function
julia
onesC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.
source
WeightInitializers.randC64 Function
julia
randC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.
source
WeightInitializers.randnC64 Function
julia
randnC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.
source

-    
+true

References

[1] Martens, J, "Deep learning via Hessian-free optimization" Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.

source

WeightInitializers.truncated_normal Function
julia
truncated_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; mean = 0,
+    std = 1, lo = -2, hi = 2) -> AbstractArray{T, length(size)}

Return an AbstractArray{T} of the given size where each element is drawn from a truncated normal distribution. The numbers are distributed like filter(x -> lo ≤ x ≤ hi, mean .+ std .* randn(100)).

source

WeightInitializers.orthogonal Function
julia
orthogonal([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
+    gain = 1)  -> AbstractArray{T, length(dims)}

Return an AbstractArray{T} of the given dimensions (dims) which is a (semi) orthogonal matrix, as described in [1].

The function constructs an orthogonal or semi-orthogonal matrix depending on the specified dimensions. For two dimensions, it returns a matrix where dims = (rows, cols). For more than two dimensions, it computes an orthogonal matrix of size prod(dims[1:(end - 1)]) by dims[end] before reshaping it to the original dimensions.

Cannot construct a vector, i.e., length(dims) == 1 is forbidden.

Arguments

  • rng::AbstractRNG: Random number generator.

  • T::Type{<:Real}: The type of the elements in the array.

  • dims::Integer...: The dimensions of the array.

  • gain::Number: Scaling factor for the elements of the orthogonal matrix.

References

[1] Saxe, McClelland, Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", ICLR 2014, https://arxiv.org/abs/1312.6120

source

Other Convenience Functions ​

Beware

Unlike the other functions these ones don't take a type argument.

WeightInitializers.zeros16 Function
julia
zeros16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

Return an AbstractArray{Float16} of the given size containing an AbstractArray of zeros.

source

WeightInitializers.ones16 Function
julia
ones16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

Return an AbstractArray{Float16} of the given size containing an AbstractArray of ones.

source

WeightInitializers.rand16 Function
julia
rand16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

Return an AbstractArray{Float16} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randn16 Function
julia
randn16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

Return an AbstractArray{Float16} of the given size containing random numbers from a standard normal distribution.

source

WeightInitializers.zeros32 Function
julia
zeros32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

Return an AbstractArray{Float32} of the given size containing an AbstractArray of zeros.

source

WeightInitializers.ones32 Function
julia
ones32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

Return an AbstractArray{Float32} of the given size containing an AbstractArray of ones.

source

WeightInitializers.rand32 Function
julia
rand32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

Return an AbstractArray{Float32} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randn32 Function
julia
randn32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

Return an AbstractArray{Float32} of the given size containing random numbers from a standard normal distribution.

source

WeightInitializers.zeros64 Function
julia
zeros64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

Return an AbstractArray{Float64} of the given size containing an AbstractArray of zeros.

source

WeightInitializers.ones64 Function
julia
ones64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

Return an AbstractArray{Float64} of the given size containing an AbstractArray of ones.

source

WeightInitializers.rand64 Function
julia
rand64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

Return an AbstractArray{Float64} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randn64 Function
julia
randn64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

Return an AbstractArray{Float64} of the given size containing random numbers from a standard normal distribution.

source

WeightInitializers.zerosC16 Function
julia
zerosC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of zeros.

source

WeightInitializers.onesC16 Function
julia
onesC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of ones.

source

WeightInitializers.randC16 Function
julia
randC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

Return an AbstractArray{ComplexF16} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randnC16 Function
julia
randnC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

Return an AbstractArray{ComplexF16} of the given size containing random numbers from a standard normal distribution.

source

WeightInitializers.zerosC32 Function
julia
zerosC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of zeros.

source

WeightInitializers.onesC32 Function
julia
onesC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of ones.

source

WeightInitializers.randC32 Function
julia
randC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

Return an AbstractArray{ComplexF32} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randnC32 Function
julia
randnC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

Return an AbstractArray{ComplexF32} of the given size containing random numbers from a standard normal distribution.

source

WeightInitializers.zerosC64 Function
julia
zerosC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of zeros.

source

WeightInitializers.onesC64 Function
julia
onesC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.

source

WeightInitializers.randC64 Function
julia
randC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.

source

WeightInitializers.randnC64 Function
julia
randnC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Lux/autodiff.html b/previews/PR1000/api/Lux/autodiff.html index b21d908fde..f09db65c7b 100644 --- a/previews/PR1000/api/Lux/autodiff.html +++ b/previews/PR1000/api/Lux/autodiff.html @@ -5,14 +5,14 @@ Automatic Differentiation Helpers | Lux.jl Docs - + - + - + - + @@ -27,8 +27,8 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Automatic Differentiation Helpers ​

JVP & VJP Wrappers ​

Lux.jacobian_vector_product Function
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)

Compute the Jacobian-Vector Product (∂f∂x)u. This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the JVP.

  • x: The input to the function.

  • u: An object of the same structure as x.

Returns

  • v: The Jacobian Vector Product.

source

Lux.vector_jacobian_product Function
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)

Compute the Vector-Jacobian Product (∂f∂x)Tu. This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the VJP.

  • x: The input to the function.

  • u: An object of the same structure as f(x).

Returns

  • v: The Vector Jacobian Product.

source

Batched AD ​

Lux.batched_jacobian Function
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the jacobian.

  • x: The input to the function. Must have ndims(x) ≥ 2.

Returns

  • J: The Jacobian of f with respect to x. This will be a 3D Array. If the dimensions of x are (N₁, N₂, ..., Nₙ, B) and of y are (M₁, M₂, ..., Mₘ, B), then J will be a ((M₁ × M₂ × ... × Mₘ), (N₁ × N₂ × ... × Nₙ), B) Array.

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

Nested 2nd Order AD ​

Consult the manual page on Nested AD for information on nested automatic differentiation.

- +
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Automatic Differentiation Helpers ​

JVP & VJP Wrappers ​

Lux.jacobian_vector_product Function
julia
jacobian_vector_product(f, backend::AbstractADType, x, u)

Compute the Jacobian-Vector Product (∂f∂x)u. This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the JVP.

  • x: The input to the function.

  • u: An object of the same structure as x.

Returns

  • v: The Jacobian Vector Product.

source

Lux.vector_jacobian_product Function
julia
vector_jacobian_product(f, backend::AbstractADType, x, u)

Compute the Vector-Jacobian Product (∂f∂x)Tu. This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD.

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl

Warning

Gradient wrt u in the reverse pass is always dropped.

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the VJP.

  • x: The input to the function.

  • u: An object of the same structure as f(x).

Returns

  • v: The Vector Jacobian Product.

source

Batched AD ​

Lux.batched_jacobian Function
julia
batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

  1. ndims(y) ≥ 2

  2. size(y, ndims(y)) == size(x, ndims(x))

Backends & AD Packages

Supported BackendsPackages Needed
AutoForwardDiff
AutoZygoteZygote.jl

Arguments

  • f: The function to compute the jacobian of.

  • backend: The backend to use for computing the jacobian.

  • x: The input to the function. Must have ndims(x) ≥ 2.

Returns

  • J: The Jacobian of f with respect to x. This will be a 3D Array. If the dimensions of x are (N₁, N₂, ..., Nₙ, B) and of y are (M₁, M₂, ..., Mₘ, B), then J will be a ((M₁ × M₂ × ... × Mₘ), (N₁ × N₂ × ... × Nₙ), B) Array.

Danger

f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

source

Nested 2nd Order AD ​

Consult the manual page on Nested AD for information on nested automatic differentiation.

+ \ No newline at end of file diff --git a/previews/PR1000/api/Lux/contrib.html b/previews/PR1000/api/Lux/contrib.html index 44040af151..0077487a8d 100644 --- a/previews/PR1000/api/Lux/contrib.html +++ b/previews/PR1000/api/Lux/contrib.html @@ -5,14 +5,14 @@ Experimental Features | Lux.jl Docs - + - + - + - + @@ -28,8 +28,8 @@
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Experimental Features ​

All features listed on this page are experimental which means:

  1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.

  2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.

  3. None of the features are exported.

Parameter Freezing ​

Lux.Experimental.FrozenLayer Type
julia
FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})

Freeze the parameters with name which_params of the layer l.

Use Lux.Experimental.freeze instead

It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.

No checks for which_params

There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.

Arguments

  • l: Lux AbstractLuxLayer.

  • which_params: Parameter Names to be Frozen. Can be set to nothing, in which case all parameters are frozen.

Extended Help

Parameters

  • Parameters of the layer l excluding which_params.

States

  • frozen_params: Parameters that are frozen, i.e., which_params.

  • states: The state of the inner layer l.

Note on Internal Layer Implementation

The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.

Example

julia
julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
-FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

source

Lux.Experimental.freeze Function
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

source

julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
-    which_params::Optional{Tuple} = nothing)

Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

source

Lux.Experimental.unfreeze Function
julia
unfreeze(l::FrozenLayer)

Unfreezes the layer l.

source

julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)

Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

source

For detailed usage example look at the manual page.

Map over Layer ​

Lux.Experimental.layer_map Function
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

KeyPath provided to the function

The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

Call Signature for f

  • Must take 4 inputs – AbstractLuxLayer, Corresponding Parameters, Corresponding States, and the Functors.KeyPath to the layer.

  • Must return a tuple of 3 elements – AbstractLuxLayer, new parameters and the new states.

Extended Help

Example

julia
julia> using Lux, Random
+FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

source

Lux.Experimental.freeze Function
julia
freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

source

julia
freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
+    which_params::Optional{Tuple} = nothing)

Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

source

Lux.Experimental.unfreeze Function
julia
unfreeze(l::FrozenLayer)

Unfreezes the layer l.

source

julia
unfreeze(l::FrozenLayer, ps, st::NamedTuple)

Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

source

For detailed usage example look at the manual page.

Map over Layer ​

Lux.Experimental.layer_map Function
julia
layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

KeyPath provided to the function

The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

Call Signature for f

  • Must take 4 inputs – AbstractLuxLayer, Corresponding Parameters, Corresponding States, and the Functors.KeyPath to the layer.

  • Must return a tuple of 3 elements – AbstractLuxLayer, new parameters and the new states.

Extended Help

Example

julia
julia> using Lux, Random
 
 julia> c = Parallel(
            +; chain=Chain(; dense_1=Dense(2 => 3), bn=BatchNorm(3), dense_2=Dense(3 => 5)),
@@ -56,10 +56,10 @@
 julia> all(iszero, (ps_new.chain.dense_1.weight, ps_new.chain.dense_1.bias,
                     ps_new.chain.dense_2.weight, ps_new.chain.dense_2.bias,
                     ps_new.dense_3.weight, ps_new.dense_3.bias))
-true

source

Debugging Functionality ​

Model not working properly! Here are some functionalities to help you debug you Lux model.

Lux.Experimental.@debug_mode Macro
julia
@debug_mode layer kwargs...

Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

source

Lux.Experimental.DebugLayer Type
julia
DebugLayer(layer::AbstractLuxLayer;
+true

source

Debugging Functionality ​

Model not working properly! Here are some functionalities to help you debug you Lux model.

Lux.Experimental.@debug_mode Macro
julia
@debug_mode layer kwargs...

Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

source

Lux.Experimental.DebugLayer Type
julia
DebugLayer(layer::AbstractLuxLayer;
     nan_check::Union{Symbol, StaticSymbol, Val}=static(:both),
     error_check::Union{StaticBool, Bool, Val{true}, Val{false}}=True(),
-    location::KeyPath=KeyPath())

A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

Arguments

  • layer: The layer to be wrapped.

Extended Help

Keyword Arguments

  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.

  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.

  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.

Input / Output

Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

ChainRules Compatible Reverse Mode AD Tools

nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

Disable After Debugging

This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

See Lux.Experimental.@debug_mode to construct this layer.

source

Tied Parameters ​

Lux.Experimental.share_parameters Function
julia
share_parameters(ps, sharing)
+    location::KeyPath=KeyPath())

A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

Arguments

  • layer: The layer to be wrapped.

Extended Help

Keyword Arguments

  • nan_check: Whether to check for NaNs in the input, parameters, and states. Can be :both, :forward, :backward, or :none.

  • error_check: Whether to check for errors in the layer. If true, will throw an error if the layer fails.

  • location: The location of the layer. Use Lux.Experimental.@debug_mode to construct this layer to populate this value correctly.

Input / Output

Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

ChainRules Compatible Reverse Mode AD Tools

nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

Disable After Debugging

This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

See Lux.Experimental.@debug_mode to construct this layer.

source

Tied Parameters ​

Lux.Experimental.share_parameters Function
julia
share_parameters(ps, sharing)
 share_parameters(ps, sharing, new_parameters)

Updates the parameters in ps with a common set of parameters new_parameters that are shared between each list in the nested list sharing. (That was kind of a mouthful, the example should make it clear).

Arguments

  • ps: Original parameters.

  • sharing: A nested list of lists of accessors of ps which need to shate the parameters (See the example for details). (Each list in the list must be disjoint)

  • new_parameters: If passed the length of new_parameters must be equal to the length of sharing. For each vector in sharing the corresponding parameter in new_parameters will be used. (If not passed, the parameters corresponding to the first element of each vector in sharing will be used).

Returns

Updated Parameters having the same structure as ps.

Example

julia
julia> model = Chain(; d1=Dense(2 => 4, tanh),
            d3=Chain(; l1=Dense(4 => 2), l2=Dense(2 => 4)), d2=Dense(4 => 2))
 Chain(
@@ -81,8 +81,8 @@
            ps.d3.l2.bias === ps.d1.bias &&
            ps.d2.weight === ps.d3.l1.weight &&
            ps.d2.bias === ps.d3.l1.bias
-true

ComponentArrays

ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

source

- +true

ComponentArrays

ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Lux/distributed_utils.html b/previews/PR1000/api/Lux/distributed_utils.html index f0b34febca..8c7ebb8d69 100644 --- a/previews/PR1000/api/Lux/distributed_utils.html +++ b/previews/PR1000/api/Lux/distributed_utils.html @@ -5,14 +5,14 @@ Distributed Utils | Lux.jl Docs - + - + - + - + @@ -27,11 +27,11 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

Lux.MPIBackend Type
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

Lux.NCCLBackend Type
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

Initialization ​

Lux.DistributedUtils.initialize Function
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

source

Lux.DistributedUtils.initialized Function
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

Lux.DistributedUtils.get_distributed_backend Function
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

Helper Functions ​

Lux.DistributedUtils.local_rank Function
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

Lux.DistributedUtils.total_workers Function
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

Communication Primitives ​

Lux.DistributedUtils.allreduce! Function
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
-allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.bcast! Function
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
-bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

Lux.DistributedUtils.reduce! Function
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
-reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.synchronize!! Function
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

Optimizers.jl Integration ​

Lux.DistributedUtils.DistributedOptimizer Type
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

  • optimizer: An Optimizer compatible with the Optimisers.jl package

source

MLUtils.jl Integration ​

Lux.DistributedUtils.DistributedDataContainer Type
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

- +
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Distributed Utils ​

Note

These functionalities are available via the Lux.DistributedUtils module.

Backends ​

Lux.MPIBackend Type
julia
MPIBackend(comm = nothing)

Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

source

Lux.NCCLBackend Type
julia
NCCLBackend(comm = nothing, mpi_backend = nothing)

Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

source

Initialization ​

Lux.DistributedUtils.initialize Function
julia
initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

Possible values for backend are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

source

Lux.DistributedUtils.initialized Function
julia
initialized(backend::Type{<:AbstractLuxDistributedBackend})

Check if the given backend is initialized.

source

Lux.DistributedUtils.get_distributed_backend Function
julia
get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

Get the distributed backend for the given backend type. Possible values are:

  • MPIBackend: MPI backend for distributed training. Requires MPI.jl to be installed.

  • NCCLBackend: NCCL backend for CUDA distributed training. Requires CUDA.jl, MPI.jl, and NCCL.jl to be installed. This also wraps MPI backend for non-CUDA communications.

Danger

initialize(backend; kwargs...) must be called before calling this function.

source

Helper Functions ​

Lux.DistributedUtils.local_rank Function
julia
local_rank(backend::AbstractLuxDistributedBackend)

Get the local rank for the given backend.

source

Lux.DistributedUtils.total_workers Function
julia
total_workers(backend::AbstractLuxDistributedBackend)

Get the total number of workers for the given backend.

source

Communication Primitives ​

Lux.DistributedUtils.allreduce! Function
julia
allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
+allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.bcast! Function
julia
bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
+bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

source

Lux.DistributedUtils.reduce! Function
julia
reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
+reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

op allows a special DistributedUtils.avg operation that averages the result across all workers.

source

Lux.DistributedUtils.synchronize!! Function
julia
synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

source

Optimizers.jl Integration ​

Lux.DistributedUtils.DistributedOptimizer Type
julia
DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

Arguments

  • optimizer: An Optimizer compatible with the Optimisers.jl package

source

MLUtils.jl Integration ​

Lux.DistributedUtils.DistributedDataContainer Type
julia
DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

Load MLUtils.jl

MLUtils.jl must be installed and loaded before using this.

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Lux/interop.html b/previews/PR1000/api/Lux/interop.html index f86eb5529c..6187a19676 100644 --- a/previews/PR1000/api/Lux/interop.html +++ b/previews/PR1000/api/Lux/interop.html @@ -5,14 +5,14 @@ Interoperability between Lux and other packages | Lux.jl Docs - + - + - + - + @@ -27,7 +27,7 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Interoperability between Lux and other packages ​

Switching from older frameworks ​

Flux Models to Lux Models ​

Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.

Tip

Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.

Adapt.adapt Method
julia
Adapt.adapt(from::FromFluxAdaptor, L)

Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.

source

Lux.FromFluxAdaptor Type
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)

Convert a Flux Model to Lux Model.

active field

This always ignores the active field of some of the Flux layers. This is almost never going to be supported.

Keyword Arguments

  • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.

  • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.

Example

julia
julia> import Flux
+    
Lux.jl Docs
Appearance

 Trusted by 
Scientific Computing
SciML.ai
Machine Learning
Interoperability between Lux and other packages ​
Switching from older frameworks ​
Flux Models to Lux Models ​
Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.
Tip
Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.
Adapt.adapt Method
julia
Adapt.adapt(from::FromFluxAdaptor, L)
Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.
source
Lux.FromFluxAdaptor Type
julia
FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)
Convert a Flux Model to Lux Model.
active field
This always ignores the active field of some of the Flux layers. This is almost never going to be supported.
Keyword Arguments
  • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.
  • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.
Example
julia
julia> import Flux
 
 julia> using Adapt, Lux, Random
 
@@ -40,7 +40,7 @@
 julia> ps, st = Lux.setup(Random.default_rng(), m2);
 
 julia> size(first(m2(x, ps, st)))
-(2, 32)
source
Lux.FluxLayer Type
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
Adapt.adapt Method
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
Lux.ToSimpleChainsAdaptor Type
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
+(2, 32)
source
Lux.FluxLayer Type
julia
FluxLayer(layer)
Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
Warning
Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
Warning
Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
Arguments
  • layer: Flux layer
Parameters
  • p: Flattened parameters of the layer
source
Using a different backend for Lux ​
Lux Models to Simple Chains ​
SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
Tip
Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
Adapt.adapt Method
julia
Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
source
Lux.ToSimpleChainsAdaptor Type
julia
ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
Warning
There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
Warning
Any kind of initialization function is not preserved when converting to SimpleChains.jl.
Arguments
  • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
  • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
Example
julia
julia> import SimpleChains
 
 julia> using Adapt, Lux, Random
 
@@ -57,9 +57,9 @@
 julia> x = randn(Float32, 28, 28, 1, 1);
 
 julia> size(first(simple_chains_model(x, ps, st)))
-(10, 1)
source
Lux.SimpleChainsLayer Type
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
-SimpleChainsLayer(layer, lux_layer, to_array)
Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.
ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.
Arguments
  • layer: SimpleChains layer
  • lux_layer: Potentially equivalent Lux layer that is used for printing
source

-    
+(10, 1)

source

Lux.SimpleChainsLayer Type
julia
SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
+SimpleChainsLayer(layer, lux_layer, to_array)

Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.

ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.

Arguments

  • layer: SimpleChains layer

  • lux_layer: Potentially equivalent Lux layer that is used for printing

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Lux/layers.html b/previews/PR1000/api/Lux/layers.html index 27ba7b1750..954b4a22cc 100644 --- a/previews/PR1000/api/Lux/layers.html +++ b/previews/PR1000/api/Lux/layers.html @@ -5,14 +5,14 @@ Built-In Layers | Lux.jl Docs - + - + - + - + @@ -34,7 +34,7 @@ layer_2 = NoOpLayer(), layer_3 = NoOpLayer(), ) # Total: 0 parameters, - # plus 0 states.

source

Lux.Chain Type
julia
Chain(layers...; name=nothing)
+          #        plus 0 states.

source

Lux.Chain Type
julia
Chain(layers...; name=nothing)
 Chain(; layers..., name=nothing)

Collects multiple layers / functions to be called in sequence on a given input.

Arguments

Extended Help

Inputs

Input x is passed sequentially to each layer, and must conform to the input requirements of the internal layers.

Returns

Parameters

States

Miscellaneous Properties

Example

julia
julia> Chain(Dense(2, 3, relu), BatchNorm(3), Dense(3, 2))
 Chain(
     layer_1 = Dense(2 => 3, relu),      # 9 parameters
@@ -49,7 +49,7 @@
     layer_2 = BatchNorm(3, affine=true, track_stats=true),  # 6 parameters, plus 7
     layer_3 = Dense(3 => 2),            # 8 parameters
 )         # Total: 23 parameters,
-          #        plus 7 states.

source

Lux.PairwiseFusion Type
julia
PairwiseFusion(connection, layers...; name=nothing)
+          #        plus 7 states.

source

Lux.PairwiseFusion Type
julia
PairwiseFusion(connection, layers...; name=nothing)
 PairwiseFusion(connection; name=nothing, layers...)
 PairwiseFusion(; connection, layers..., name=nothing)
x1 → layer1 → y1 ↘
                   connection → layer2 → y2 ↘
@@ -60,7 +60,7 @@
 end
  1. Any other kind of input
julia
y = x
 for i in 1:N
     y = connection(x, layers[i](y))
-end

Returns

Parameters

States

source

Lux.Parallel Type
julia
Parallel(connection, layers...; name=nothing)
+end

Returns

Parameters

States

source

Lux.Parallel Type
julia
Parallel(connection, layers...; name=nothing)
 Parallel(connection; name=nothing, layers...)
 Parallel(; connection, layers..., name=nothing)

Create a layer which passes an input to each path in layers, before reducing the output with connection.

Arguments

Extended Help

Inputs

Returns

Parameters

States

See also SkipConnection which is Parallel with one identity.

Example

julia
julia> model = Parallel(nothing, Dense(2, 1), Dense(2, 1))
 Parallel(
@@ -76,37 +76,37 @@
        x2 = randn(rng, Float32, 2);
 
 julia> size.(first(model((x1, x2), ps, st)))
-((1,), (1,))

source

Lux.SkipConnection Type
julia
SkipConnection(layers, connection; name=nothing)
-SkipConnection(; layers, connection, name=nothing)

Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

The simplest "ResNet"-type connection is just SkipConnection(layer, +).

Arguments

Extended Help

Inputs

Returns

Parameters

States

See Parallel for a more general implementation.

source

Lux.RepeatedLayer Type
julia
RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

julia
res = x
+((1,), (1,))

source

Lux.SkipConnection Type
julia
SkipConnection(layers, connection; name=nothing)
+SkipConnection(; layers, connection, name=nothing)

Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

The simplest "ResNet"-type connection is just SkipConnection(layer, +).

Arguments

Extended Help

Inputs

Returns

Parameters

States

See Parallel for a more general implementation.

source

Lux.RepeatedLayer Type
julia
RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

julia
res = x
 for i in 1:repeats
     res, st = model(res, ps, st)
 end
julia
res = x
 for i in 1:repeats
     res, st = model((res, x), ps, st)
-end

It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Convolutional Layers ​

Lux.Conv Type
julia
Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+end

It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Convolutional Layers ​

Lux.Conv Type
julia
Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
      activation=identity; init_weight=nothing, init_bias=nothing, stride=1,
-     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

Standard convolutional layer.

Conv 2D

Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

Warning

Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

Arguments

Extended Help

Keyword Arguments

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

Parameters

source

Lux.ConvTranspose Type
julia
ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

Standard convolutional layer.

Conv 2D

Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

Warning

Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

Arguments

Extended Help

Keyword Arguments

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

Parameters

source

Lux.ConvTranspose Type
julia
ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
               activation=identity; init_weight=glorot_uniform, init_bias=zeros32,
               stride=1, pad=0, outpad=0, dilation=1, groups=1, use_bias=True(),
-              cross_correlation=False())

Standard convolutional transpose layer.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Dropout Layers ​

Lux.AlphaDropout Type
julia
AlphaDropout(p::Real)

AlphaDropout layer.

Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also Dropout, VariationalHiddenDropout

source

Lux.Dropout Type
julia
Dropout(p; dims=:)

Dropout layer.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, VariationalHiddenDropout

source

Lux.VariationalHiddenDropout Type
julia
VariationalHiddenDropout(p; dims=:)

VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, Dropout

source

Pooling Layers ​

Lux.AdaptiveLPPool Type
julia
AdaptiveLPPool(output_size; p=2)

Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.AdaptiveMaxPool Type
julia
AdaptiveMaxPool(output_size)

Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.AdaptiveMeanPool Type
julia
AdaptiveMeanPool(output_size)

Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.GlobalLPPool Type
julia
GlobalLPPool(; p=2)

Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.GlobalMaxPool Type
julia
GlobalMaxPool()

Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.GlobalMeanPool Type
julia
GlobalMeanPool()

Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.LPPool Type
julia
LPPool(window; stride=window, pad=0, dilation=1, p=2)

LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

Arguments

Keyword Arguments

GPU Support

This layer is currently only supported on CPU.

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MaxPool Type
julia
MaxPool(window; stride=window, pad=0, dilation=1)

Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MeanPool Type
julia
MeanPool(window; stride=window, pad=0, dilation=1)

Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Recurrent Layers ​

Lux.GRUCell Type
julia
GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
-        init_weight=nothing, init_bias=nothing, init_state=zeros32)

Gated Recurrent Unit (GRU) Cell

r=σ(Wir×x+bir+Whr×hprev+bhr)z=σ(Wiz×x+biz+Whz×hprev+bhz)n=tanh⁡(Win×x+bin+r⋅(Whn×hprev+bhn))hnew=(1−z)⋅n+z⋅hprev

Arguments

Inputs

Returns

Parameters

States

source

Lux.LSTMCell Type
julia
LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
+              cross_correlation=False())

Standard convolutional transpose layer.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Dropout Layers ​

Lux.AlphaDropout Type
julia
AlphaDropout(p::Real)

AlphaDropout layer.

Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also Dropout, VariationalHiddenDropout

source

Lux.Dropout Type
julia
Dropout(p; dims=:)

Dropout layer.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, VariationalHiddenDropout

source

Lux.VariationalHiddenDropout Type
julia
VariationalHiddenDropout(p; dims=:)

VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

Arguments

Keyword Arguments

Inputs

Returns

States

Call Lux.testmode to switch to test mode.

See also AlphaDropout, Dropout

source

Pooling Layers ​

Lux.AdaptiveLPPool Type
julia
AdaptiveLPPool(output_size; p=2)

Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.AdaptiveMaxPool Type
julia
AdaptiveMaxPool(output_size)

Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.AdaptiveMeanPool Type
julia
AdaptiveMeanPool(output_size)

Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

Arguments

Inputs

Returns

source

Lux.GlobalLPPool Type
julia
GlobalLPPool(; p=2)

Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

GPU Support

This layer is currently only supported on CPU.

Inputs

Returns

source

Lux.GlobalMaxPool Type
julia
GlobalMaxPool()

Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.GlobalMeanPool Type
julia
GlobalMeanPool()

Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

Inputs

Returns

source

Lux.LPPool Type
julia
LPPool(window; stride=window, pad=0, dilation=1, p=2)

LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

Arguments

Keyword Arguments

GPU Support

This layer is currently only supported on CPU.

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MaxPool Type
julia
MaxPool(window; stride=window, pad=0, dilation=1)

Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Lux.MeanPool Type
julia
MeanPool(window; stride=window, pad=0, dilation=1)

Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Oi=⌊Ii+pi+p(i+N)%|p|−di×(ki−1)si+1⌋

source

Recurrent Layers ​

Lux.GRUCell Type
julia
GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
+        init_weight=nothing, init_bias=nothing, init_state=zeros32)

Gated Recurrent Unit (GRU) Cell

r=σ(Wir×x+bir+Whr×hprev+bhr)z=σ(Wiz×x+biz+Whz×hprev+bhz)n=tanh⁡(Win×x+bin+r⋅(Whn×hprev+bhn))hnew=(1−z)⋅n+z⋅hprev

Arguments

Inputs

Returns

Parameters

States

source

Lux.LSTMCell Type
julia
LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
          train_memory::Bool=false, init_weight=nothing, init_bias=nothing,
-         init_state=zeros32, init_memory=zeros32)

Long Short-Term (LSTM) Cell

i=σ(Wii×x+Whi×hprev+bi)f=σ(Wif×x+Whf×hprev+bf)g=tanh(Wig×x+Whg×hprev+bg)o=σ(Wio×x+Who×hprev+bo)cnew=f⋅cprev+i⋅ghnew=o⋅tanh(cnew)

Arguments

Inputs

Returns

Parameters

States

source

Lux.RNNCell Type
julia
RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
-    init_bias=nothing, init_weight=nothing, init_state=zeros32)

An Elman RNNCell cell with activation (typically set to tanh or relu).

hnew=activation(weightih×x+biasih+weighthh×hprev+biashh)

Arguments

Inputs

Returns

Parameters

States

source

Lux.Recurrence Type
julia
Recurrence(cell;
+         init_state=zeros32, init_memory=zeros32)

Long Short-Term (LSTM) Cell

i=σ(Wii×x+Whi×hprev+bi)f=σ(Wif×x+Whf×hprev+bf)g=tanh(Wig×x+Whg×hprev+bg)o=σ(Wio×x+Who×hprev+bo)cnew=f⋅cprev+i⋅ghnew=o⋅tanh(cnew)

Arguments

Inputs

Returns

Parameters

States

source

Lux.RNNCell Type
julia
RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
+    init_bias=nothing, init_weight=nothing, init_state=zeros32)

An Elman RNNCell cell with activation (typically set to tanh or relu).

hnew=activation(weightih×x+biasih+weighthh×hprev+biashh)

Arguments

Inputs

Returns

Parameters

States

source

Lux.Recurrence Type
julia
Recurrence(cell;
     ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex(),
     return_sequence::Bool=false)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.

Relation to Flux.Recur

This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Tip

Frameworks like Tensorflow have special implementation of StackedRNNCells to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.

Chain(
     Recurrence(RNNCell(inputsize => latentsize); return_sequence=true),
     Recurrence(RNNCell(latentsize => latentsize); return_sequence=true),
     :
     x -> stack(x; dims=2)
-)

For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

source

Lux.StatefulRecurrentCell Type
julia
StatefulRecurrentCell(cell)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

Arguments

Inputs

Returns

States

source

Lux.BidirectionalRNN Type
julia
BidirectionalRNN(cell::AbstractRecurrentCell,
+)

For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

source

Lux.StatefulRecurrentCell Type
julia
StatefulRecurrentCell(cell)

Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

Arguments

Inputs

Returns

States

source

Lux.BidirectionalRNN Type
julia
BidirectionalRNN(cell::AbstractRecurrentCell,
     backward_cell::Union{AbstractRecurrentCell, Nothing}=nothing;
     merge_mode::Union{Function, Nothing}=vcat,
-    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

Bidirectional RNN wrapper.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Linear Layers ​

Lux.Bilinear Type
julia
Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
+    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

Bidirectional RNN wrapper.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

source

Linear Layers ​

Lux.Bilinear Type
julia
Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
          init_bias=nothing, use_bias=True())
 Bilinear(in12_dims => out, activation=identity; init_weight=nothing,
-         init_bias=nothing, use_bias=True())

Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

z[i] = activation(x' * W[i, :, :] * y + bias[i])

If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Dense Type
julia
Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
-      init_bias=nothing, use_bias=True())

Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Embedding Type
julia
Embedding(in_dims => out_dims; init_weight=rand32)

A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

This layer is often used to store word embeddings and retrieve them using indices.

Arguments

Keyword Arguments

Input

Returns

source

Lux.Scale Type
julia
Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Misc. Helper Layers ​

Lux.FlattenLayer Type
julia
FlattenLayer(; N = nothing)

Flattens the passed array into a matrix.

Keyword Arguments

Inputs

Returns

Example

julia
julia> model = FlattenLayer()
+         init_bias=nothing, use_bias=True())

Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

z[i] = activation(x' * W[i, :, :] * y + bias[i])

If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Dense Type
julia
Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
+      init_bias=nothing, use_bias=True())

Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Lux.Embedding Type
julia
Embedding(in_dims => out_dims; init_weight=rand32)

A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

This layer is often used to store word embeddings and retrieve them using indices.

Arguments

Keyword Arguments

Input

Returns

source

Lux.Scale Type
julia
Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

Arguments

Keyword Arguments

Input

Returns

Parameters

source

Misc. Helper Layers ​

Lux.FlattenLayer Type
julia
FlattenLayer(; N = nothing)

Flattens the passed array into a matrix.

Keyword Arguments

Inputs

Returns

Example

julia
julia> model = FlattenLayer()
 FlattenLayer{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -116,9 +116,9 @@
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(8, 2)

source

Lux.Maxout Type
julia
Maxout(layers...)
+(8, 2)

source

Lux.Maxout Type
julia
Maxout(layers...)
 Maxout(; layers...)
-Maxout(f::Function, n_alts::Int)

This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

See also Parallel to reduce with other operators.

Arguments

Extended Help

Inputs

Returns

Parameters

States

References

[1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

source

Lux.NoOpLayer Type
julia
NoOpLayer()

As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

Example

julia
julia> model = NoOpLayer()
+Maxout(f::Function, n_alts::Int)

This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

See also Parallel to reduce with other operators.

Arguments

Extended Help

Inputs

Returns

Parameters

States

References

[1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

source

Lux.NoOpLayer Type
julia
NoOpLayer()

As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

Example

julia
julia> model = NoOpLayer()
 NoOpLayer()
 
 julia> rng = Random.default_rng();
@@ -128,7 +128,7 @@
 1
 
 julia> y, st_new = model(x, ps, st)
-(1, NamedTuple())

source

Lux.ReshapeLayer Type
julia
ReshapeLayer(dims)

Reshapes the passed array to have a size of (dims..., :)

Arguments

Inputs

Returns

Example

julia
julia> model = ReshapeLayer((2, 2))
+(1, NamedTuple())

source

Lux.ReshapeLayer Type
julia
ReshapeLayer(dims)

Reshapes the passed array to have a size of (dims..., :)

Arguments

Inputs

Returns

Example

julia
julia> model = ReshapeLayer((2, 2))
 ReshapeLayer(output_dims = (2, 2, :))
 
 julia> rng = Random.default_rng();
@@ -138,7 +138,7 @@
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(2, 2, 3)

source

Lux.SelectDim Type
julia
SelectDim(dim, i)

Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

Arguments

Inputs

Returns

source

Lux.WrappedFunction Type
julia
WrappedFunction(f)

Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

Arguments

Inputs

Returns

source

Lux.ReverseSequence Type
julia
ReverseSequence(dim = nothing)

Reverse the specified dimension dims of the passed array

Arguments

Inputs

Returns

Example

julia
julia> model = ReverseSequence()
+(2, 2, 3)

source

Lux.SelectDim Type
julia
SelectDim(dim, i)

Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

Arguments

Inputs

Returns

source

Lux.WrappedFunction Type
julia
WrappedFunction(f)

Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

Arguments

Inputs

Returns

source

Lux.ReverseSequence Type
julia
ReverseSequence(dim = nothing)

Reverse the specified dimension dims of the passed array

Arguments

Inputs

Returns

Example

julia
julia> model = ReverseSequence()
 ReverseSequence{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -147,7 +147,7 @@
        x = [1.0, 2.0, 3.0];
 
 julia> y, st_new = model(x, ps, st)
-([3.0, 2.0, 1.0], NamedTuple())

source

Normalization Layers ​

Lux.BatchNorm Type
julia
BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+([3.0, 2.0, 1.0], NamedTuple())

source

Normalization Layers ​

Lux.BatchNorm Type
julia
BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
           affine=True(), track_stats=True(), epsilon=1f-5, momentum=0.1f0)

Batch Normalization layer.

BatchNorm computes the mean and variance for each D1×...×DN−2×1×DN input slice and normalises the input accordingly.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

Use Lux.testmode during inference.

Example

julia
julia> Chain(Dense(784 => 64), BatchNorm(64, relu), Dense(64 => 10), BatchNorm(10))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -155,7 +155,7 @@
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = BatchNorm(10, affine=true, track_stats=true),  # 20 parameters, plus 21
 )         # Total: 51_038 parameters,
-          #        plus 150 states.

Warning

Passing a batch size of 1, during training will result in an error.

See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.GroupNorm Type
julia
GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
+          #        plus 150 states.

Warning

Passing a batch size of 1, during training will result in an error.

See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.GroupNorm Type
julia
GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
           init_scale=ones32, affine=true, epsilon=1f-5)

Group Normalization layer.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

Use Lux.testmode during inference.

Example

julia
julia> Chain(Dense(784 => 64), GroupNorm(64, 4, relu), Dense(64 => 10), GroupNorm(10, 5))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -163,7 +163,7 @@
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = GroupNorm(10, 5, affine=true),  # 20 parameters
 )         # Total: 51_038 parameters,
-          #        plus 0 states.

See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.InstanceNorm Type
julia
InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+          #        plus 0 states.

See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

source

Lux.InstanceNorm Type
julia
InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
              affine=False(), track_stats=False(), epsilon=1f-5, momentum=0.1f0)

Instance Normalization. For details see [1].

Instance Normalization computes the mean and variance for each D1×...×DN−2×1×1` input slice and normalises the input accordingly.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

States

Use Lux.testmode during inference.

Example

julia
julia> Chain(Dense(784 => 64), InstanceNorm(64, relu; affine=true), Dense(64 => 10),
            InstanceNorm(10, relu; affine=true))
 Chain(
@@ -172,11 +172,11 @@
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = InstanceNorm(10, relu, affine=true, track_stats=false),  # 20 parameters, plus 1
 )         # Total: 51_038 parameters,
-          #        plus 2 states.

References

[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

source

Lux.LayerNorm Type
julia
LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
-          affine=true, init_bias=zeros32, init_scale=ones32)

Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

Given an input array x, this layer computes

y=x−E[x]Var[x]+ϵ∗γ+β

where γ & β are trainable parameters if affine=true.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Lux.WeightNorm Type
julia
WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
-           dims::Union{Tuple, Nothing}=nothing)

Applies weight normalization to a parameter in the given layer.

w=gv∥v∥

Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

Arguments

Inputs

Returns

Parameters

States

source

Upsampling ​

Lux.PixelShuffle Type
julia
PixelShuffle(r::Int)

Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

See NNlib.pixel_shuffle for more details.

Arguments

Inputs

Returns

source

Lux.Upsample Type
julia
Upsample(mode = :nearest; [scale, size, align_corners=false])
-Upsample(scale, mode = :nearest)

Upsampling Layer.

Layer Construction

Option 1

Exactly one of two keywords must be specified:

Option 2

Currently supported upsampling modes and corresponding NNlib's methods are:

Extended Help

Other Keyword Arguments

Inputs

Returns

source

- + # plus 2 states.

References

[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

source

Lux.LayerNorm Type
julia
LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
+          affine=true, init_bias=zeros32, init_scale=ones32)

Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

Given an input array x, this layer computes

y=x−E[x]Var[x]+ϵ∗γ+β

where γ & β are trainable parameters if affine=true.

Arguments

Keyword Arguments

Extended Help

Inputs

Returns

Parameters

source

Lux.WeightNorm Type
julia
WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
+           dims::Union{Tuple, Nothing}=nothing)

Applies weight normalization to a parameter in the given layer.

w=gv∥v∥

Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

Arguments

Inputs

Returns

Parameters

States

source

Upsampling ​

Lux.PixelShuffle Type
julia
PixelShuffle(r::Int)

Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

See NNlib.pixel_shuffle for more details.

Arguments

Inputs

Returns

source

Lux.Upsample Type
julia
Upsample(mode = :nearest; [scale, size, align_corners=false])
+Upsample(scale, mode = :nearest)

Upsampling Layer.

Layer Construction

Option 1

Exactly one of two keywords must be specified:

Option 2

Currently supported upsampling modes and corresponding NNlib's methods are:

Extended Help

Other Keyword Arguments

Inputs

Returns

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Lux/utilities.html b/previews/PR1000/api/Lux/utilities.html index 555872258c..29de8a6531 100644 --- a/previews/PR1000/api/Lux/utilities.html +++ b/previews/PR1000/api/Lux/utilities.html @@ -5,14 +5,14 @@ Utilities | Lux.jl Docs - + - + - + - + @@ -27,13 +27,13 @@ -
Lux.jl Docs

Appearance


Trusted by

Scientific Computing

SciML.ai

Machine Learning

Utilities ​

Training API ​

Helper Functions making it easier to train Lux.jl models.

Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.

Lux.Training.TrainState Type
julia
TrainState

Training State containing:

  • model: Lux model.

  • parameters: Trainable Variables of the model.

  • states: Non-trainable Variables of the model.

  • optimizer: Optimizer from Optimisers.jl.

  • optimizer_state: Optimizer State.

  • step: Number of updates of the parameters made.

Internal fields:

  • cache: Cached values. Implementations are free to use this for whatever they want.

  • objective_function: Objective function might be cached.

Warning

Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.

source

Lux.Training.compute_gradients Function
julia
compute_gradients(ad::AbstractADType, objective_function::Function, data,
-    ts::TrainState)

Compute the gradients of the objective function wrt parameters stored in ts.

Backends & AD Packages

Supported BackendsPackages Needed
AutoZygoteZygote.jl
AutoReverseDiff(; compile)ReverseDiff.jl
AutoTrackerTracker.jl
AutoEnzymeEnzyme.jl

Arguments

  • ad: Backend (from ADTypes.jl) used to compute the gradients.

  • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.

  • data: Data used to compute the gradients.

  • ts: Current Training State. See TrainState.

Return

A 4-Tuple containing:

  • grads: Computed Gradients.

  • loss: Loss from the objective function.

  • stats: Any computed statistics from the objective function.

  • ts: Updated Training State.

Known Limitations

  • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.

Aliased Gradients

grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.

source

Lux.Training.apply_gradients Function
julia
apply_gradients(ts::TrainState, grads)

Update the parameters stored in ts using the gradients grads.

Arguments

  • ts: TrainState object.

  • grads: Gradients of the loss function wrt ts.params.

Returns

Updated TrainState object.

source

Lux.Training.apply_gradients! Function
julia
apply_gradients!(ts::TrainState, grads)

Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.

Arguments

  • ts: TrainState object.

  • grads: Gradients of the loss function wrt ts.params.

Returns

Updated TrainState object.

source

Lux.Training.single_train_step Function
julia
single_train_step(backend, obj_fn::F, data, ts::TrainState)

Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.

In most cases you should use single_train_step! instead of this function.

Return

Returned values are the same as compute_gradients.

source

Lux.Training.single_train_step! Function
julia
single_train_step!(backend, obj_fn::F, data, ts::TrainState)

Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.

Return

Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).

source

Loss Functions ​

Loss Functions Objects take 2 forms of inputs:

  1. ŷ and y where ŷ is the predicted output and y is the target output.

  2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.

Warning

When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.

Lux.GenericLossFunction Type
julia
GenericLossFunction(loss_fn; agg = mean)

Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.

julia
julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
+    
Lux.jl Docs
Appearance

 Trusted by 
Scientific Computing
SciML.ai
Machine Learning
Utilities ​
Training API ​
Helper Functions making it easier to train Lux.jl models.
Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.
Lux.Training.TrainState Type
julia
TrainState
Training State containing:
  • model: Lux model.
  • parameters: Trainable Variables of the model.
  • states: Non-trainable Variables of the model.
  • optimizer: Optimizer from Optimisers.jl.
  • optimizer_state: Optimizer State.
  • step: Number of updates of the parameters made.
Internal fields:
  • cache: Cached values. Implementations are free to use this for whatever they want.
  • objective_function: Objective function might be cached.
Warning
Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.
source
Lux.Training.compute_gradients Function
julia
compute_gradients(ad::AbstractADType, objective_function::Function, data,
+    ts::TrainState)
Compute the gradients of the objective function wrt parameters stored in ts.
Backends & AD Packages
Supported BackendsPackages Needed
AutoZygoteZygote.jl
AutoReverseDiff(; compile)ReverseDiff.jl
AutoTrackerTracker.jl
AutoEnzymeEnzyme.jl
Arguments
  • ad: Backend (from ADTypes.jl) used to compute the gradients.
  • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.
  • data: Data used to compute the gradients.
  • ts: Current Training State. See TrainState.
Return
A 4-Tuple containing:
  • grads: Computed Gradients.
  • loss: Loss from the objective function.
  • stats: Any computed statistics from the objective function.
  • ts: Updated Training State.
Known Limitations
  • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.
Aliased Gradients
grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.
source
Lux.Training.apply_gradients Function
julia
apply_gradients(ts::TrainState, grads)
Update the parameters stored in ts using the gradients grads.
Arguments
  • ts: TrainState object.
  • grads: Gradients of the loss function wrt ts.params.
Returns
Updated TrainState object.
source
Lux.Training.apply_gradients! Function
julia
apply_gradients!(ts::TrainState, grads)
Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.
Arguments
  • ts: TrainState object.
  • grads: Gradients of the loss function wrt ts.params.
Returns
Updated TrainState object.
source
Lux.Training.single_train_step Function
julia
single_train_step(backend, obj_fn::F, data, ts::TrainState)
Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.
In most cases you should use single_train_step! instead of this function.
Return
Returned values are the same as compute_gradients.
source
Lux.Training.single_train_step! Function
julia
single_train_step!(backend, obj_fn::F, data, ts::TrainState)
Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.
Return
Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).
source
Loss Functions ​
Loss Functions Objects take 2 forms of inputs:
  1. ŷ and y where ŷ is the predicted output and y is the target output.
  2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.
Warning
When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.
Lux.GenericLossFunction Type
julia
GenericLossFunction(loss_fn; agg = mean)
Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.
julia
julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> mseloss(y_model, 1:3) ≈ 0.01
-true
Special Note
This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
source
Lux.BinaryCrossEntropyLoss Type
julia
BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
+true
Special Note
This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
source
Lux.BinaryCrossEntropyLoss Type
julia
BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
     label_smoothing::Union{Nothing, Real}=nothing,
     logits::Union{Bool, Val}=Val(false))
Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
Returns the binary cross entropy loss computed as:
  • If logits is either false or Val(false):
agg(−y~∗log⁡(y^+ϵ)−(1−y~)∗log⁡(1−y^+ϵ))
  • If logits is true or Val(true):
agg((1−y~)∗y^−logσ(y^))
The value of y~ is computed using label smoothing. If label_smoothing is nothing, then no label smoothing is applied. If label_smoothing is a real number ∈[0,1], then the value of y~ is:
y~=(1−α)∗y+α∗0.5
where α is the value of label_smoothing.
Extended Help
Example
julia
julia> bce = BinaryCrossEntropyLoss();
 
@@ -61,7 +61,7 @@
 julia> logitbce_ls = BinaryCrossEntropyLoss(label_smoothing=0.1, logits=Val(true));
 
 julia> logitbce_ls(y_model, y_bin) > logitbce(y_model, y_bin)
-true
source
Lux.BinaryFocalLoss Type
julia
BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
Return the binary focal loss [1]. The model input, y^, is expected to be normalized (i.e. softmax output).
For γ=0 this is equivalent to BinaryCrossEntropyLoss.
Example
julia
julia> y = [0  1  0
+true
source
Lux.BinaryFocalLoss Type
julia
BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
Return the binary focal loss [1]. The model input, y^, is expected to be normalized (i.e. softmax output).
For γ=0 this is equivalent to BinaryCrossEntropyLoss.
Example
julia
julia> y = [0  1  0
             1  0  1];
 
 julia> ŷ = [0.268941  0.5  0.268941
@@ -71,7 +71,7 @@
 true
 
 julia> BinaryFocalLoss(gamma=0)(ŷ, y) ≈ BinaryCrossEntropyLoss()(ŷ, y)
-true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.CrossEntropyLoss Type
julia
CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
+true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.CrossEntropyLoss Type
julia
CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
     label_smoothing::Union{Nothing, Real}=nothing)
Return the cross entropy loss which is used in multi-class classification tasks. The input, y^, is expected to be normalized (i.e. softmax output) if logits is false or Val(false).
The loss is calculated as:
agg(−∑y~log⁡(y^+ϵ))
where ϵ is added for numerical stability. The value of y~ is computed using label smoothing. If label_smoothing is nothing, then no label smoothing is applied. If label_smoothing is a real number ∈[0,1], then the value of y~ is calculated as:
y~=(1−α)∗y+α∗size along dim
where α is the value of label_smoothing.
Extended Help
Example
julia
julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
@@ -93,7 +93,7 @@
 true
 
 julia> CrossEntropyLoss(label_smoothing=0.15)(y_model, y) ≈ 1.5776052f0
-true
source
Lux.DiceCoeffLoss Type
julia
DiceCoeffLoss(; smooth = true, agg = mean)
Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
agg(1−2∑yy^+α∑y2+∑y^2+α)
where α is the smoothing factor (smooth).
Example
julia
julia> y_pred = [1.1, 2.1, 3.1];
+true
source
Lux.DiceCoeffLoss Type
julia
DiceCoeffLoss(; smooth = true, agg = mean)
Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
agg(1−2∑yy^+α∑y2+∑y^2+α)
where α is the smoothing factor (smooth).
Example
julia
julia> y_pred = [1.1, 2.1, 3.1];
 
 julia> DiceCoeffLoss()(y_pred, 1:3)  ≈ 0.000992391663909964
 true
@@ -102,7 +102,7 @@
 true
 
 julia> DiceCoeffLoss()(reshape(y_pred, 3, 1), reshape(1:3, 3, 1)) ≈ 0.000992391663909964
-true
References
[1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
source
Lux.FocalLoss Type
julia
FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, y^, is expected to be normalized (i.e. softmax output).
The modulating factor γ, controls the down-weighting strength. For γ=0 this is equivalent to CrossEntropyLoss.
Example
julia
julia> y = [1  0  0  0  1
+true
References
[1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
source
Lux.FocalLoss Type
julia
FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, y^, is expected to be normalized (i.e. softmax output).
The modulating factor γ, controls the down-weighting strength. For γ=0 this is equivalent to CrossEntropyLoss.
Example
julia
julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
 3×5 Matrix{Int64}:
@@ -117,20 +117,20 @@
  0.665241   0.665241   0.665241   0.665241   0.665241
 
 julia> FocalLoss()(ŷ, y) ≈ 1.1277556f0
-true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.HingeLoss Function
julia
HingeLoss(; agg = mean)
Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^))
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = HingeLoss();
+true
References
[1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
source
Lux.HingeLoss Function
julia
HingeLoss(; agg = mean)
Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^))
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = HingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.55
-true
source
Lux.HuberLoss Function
julia
HuberLoss(; delta = 1, agg = mean)
Returns the Huber loss, calculated as:
L={0.5∗|y−y^|2if |y−y^|≤δδ∗(|y−y^|−0.5∗δ)otherwise
where δ is the delta parameter.
Example
julia
julia> y_model = [1.1, 2.1, 3.1];
+true
source
Lux.HuberLoss Function
julia
HuberLoss(; delta = 1, agg = mean)
Returns the Huber loss, calculated as:
L={0.5∗|y−y^|2if |y−y^|≤δδ∗(|y−y^|−0.5∗δ)otherwise
where δ is the delta parameter.
Example
julia
julia> y_model = [1.1, 2.1, 3.1];
 
 julia> HuberLoss()(y_model, 1:3) ≈ 0.005000000000000009
 true
 
 julia> HuberLoss(delta=0.05)(y_model, 1:3) ≈ 0.003750000000000005
-true
source
Lux.KLDivergenceLoss Type
julia
KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
Return the Kullback-Leibler Divergence loss between the predicted distribution y^ and the true distribution y:
The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
For epsilon and label_smoothing, see CrossEntropyLoss.
Example
julia
julia> p1 = [1 0; 0 1]
+true
source
Lux.KLDivergenceLoss Type
julia
KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
Return the Kullback-Leibler Divergence loss between the predicted distribution y^ and the true distribution y:
The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
For epsilon and label_smoothing, see CrossEntropyLoss.
Example
julia
julia> p1 = [1 0; 0 1]
 2×2 Matrix{Int64}:
  1  0
  0  1
@@ -150,44 +150,44 @@
 0.0
 
 julia> KLDivergenceLoss(; epsilon=0)(p1, p2)
-Inf
source
Lux.MAELoss Function
julia
MAELoss(; agg = mean)
Returns the loss corresponding to mean absolute error:
agg(|y^−y|)
Example
julia
julia> loss = MAELoss();
+Inf
source
Lux.MAELoss Function
julia
MAELoss(; agg = mean)
Returns the loss corresponding to mean absolute error:
agg(|y^−y|)
Example
julia
julia> loss = MAELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.1
-true
source
Lux.MSELoss Function
julia
MSELoss(; agg = mean)
Returns the loss corresponding to mean squared error:
agg((y^−y)2)
Example
julia
julia> loss = MSELoss();
+true
source
Lux.MSELoss Function
julia
MSELoss(; agg = mean)
Returns the loss corresponding to mean squared error:
agg((y^−y)2)
Example
julia
julia> loss = MSELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.01
-true
source
Lux.MSLELoss Function
julia
MSLELoss(; agg = mean, epsilon = nothing)
Returns the loss corresponding to mean squared logarithmic error:
agg((log⁡(y^+ϵ)−log⁡(y+ϵ))2)
epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
Example
julia
julia> loss = MSLELoss();
+true
source
Lux.MSLELoss Function
julia
MSLELoss(; agg = mean, epsilon = nothing)
Returns the loss corresponding to mean squared logarithmic error:
agg((log⁡(y^+ϵ)−log⁡(y+ϵ))2)
epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
Example
julia
julia> loss = MSLELoss();
 
 julia> loss(Float32[1.1, 2.2, 3.3], 1:3) ≈ 0.009084041f0
 true
 
 julia> loss(Float32[0.9, 1.8, 2.7], 1:3) ≈ 0.011100831f0
-true
source
Lux.PoissonLoss Function
julia
PoissonLoss(; agg = mean, epsilon = nothing)
Return how much the predicted distribution y^ diverges from the expected Poisson distribution y, calculated as:
agg(y^−y∗log⁡(y^))
Example
julia
julia> y_model = [1, 3, 3];  # data should only take integral values
+true
source
Lux.PoissonLoss Function
julia
PoissonLoss(; agg = mean, epsilon = nothing)
Return how much the predicted distribution y^ diverges from the expected Poisson distribution y, calculated as:
agg(y^−y∗log⁡(y^))
Example
julia
julia> y_model = [1, 3, 3];  # data should only take integral values
 
 julia> PoissonLoss()(y_model, 1:3) ≈ 0.502312852219817
-true
source
Lux.SiameseContrastiveLoss Function
julia
SiameseContrastiveLoss(; margin = true, agg = mean)
Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
agg((1−y)y^2+y∗max(0,margin−y^)2)
Specify margin to set the baseline for distance at which pairs are dissimilar.
Example
julia
julia> ŷ = [0.5, 1.5, 2.5];
+true
source
Lux.SiameseContrastiveLoss Function
julia
SiameseContrastiveLoss(; margin = true, agg = mean)
Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
agg((1−y)y^2+y∗max(0,margin−y^)2)
Specify margin to set the baseline for distance at which pairs are dissimilar.
Example
julia
julia> ŷ = [0.5, 1.5, 2.5];
 
 julia> SiameseContrastiveLoss()(ŷ, 1:3) ≈ -4.833333333333333
 true
 
 julia> SiameseContrastiveLoss(margin=2)(ŷ, 1:3) ≈ -4.0
-true
References
[1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
source
Lux.SquaredHingeLoss Function
julia
SquaredHingeLoss(; agg = mean)
Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^)2)
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = SquaredHingeLoss();
+true
References
[1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
source
Lux.SquaredHingeLoss Function
julia
SquaredHingeLoss(; agg = mean)
Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
agg(max(0,1−yy^)2)
Usually used with classifiers like Support Vector Machines.
Example
julia
julia> loss = SquaredHingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.625
-true
source
LuxOps Module ​
Lux.LuxOps Module
julia
LuxOps
This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
source
Lux.LuxOps.eachslice Function
julia
eachslice(x, dims::Val)
Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
source
Lux.LuxOps.foldl_init Function
julia
foldl_init(op, x)
-foldl_init(op, x, init)
Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
source
Lux.LuxOps.getproperty Function
julia
getproperty(x, ::Val{v})
-getproperty(x, ::StaticSymbol{v})
Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
source
Lux.LuxOps.xlogx Function
julia
xlogx(x::Number)
Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
source
Lux.LuxOps.xlogy Function
julia
xlogy(x::Number, y::Number)
Return x * log(y) for y > 0, and zero when x == 0.
source
Lux.LuxOps.istraining Function
julia
istraining(::Val{training})
+true
source
LuxOps Module ​
Lux.LuxOps Module
julia
LuxOps
This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
source
Lux.LuxOps.eachslice Function
julia
eachslice(x, dims::Val)
Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
source
Lux.LuxOps.foldl_init Function
julia
foldl_init(op, x)
+foldl_init(op, x, init)
Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
source
Lux.LuxOps.getproperty Function
julia
getproperty(x, ::Val{v})
+getproperty(x, ::StaticSymbol{v})
Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
source
Lux.LuxOps.xlogx Function
julia
xlogx(x::Number)
Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
source
Lux.LuxOps.xlogy Function
julia
xlogy(x::Number, y::Number)
Return x * log(y) for y > 0, and zero when x == 0.
source
Lux.LuxOps.istraining Function
julia
istraining(::Val{training})
 istraining(::StaticBool)
 istraining(::Bool)
-istraining(st::NamedTuple)
Returns true if training is true or if st contains a training field with value true. Else returns false.
source
Lux.LuxOps.multigate Function
julia
multigate(x::AbstractArray, ::Val{N})
Split up x into N equally sized chunks (along dimension 1).
source
Recursive Operations ​
Lux.recursive_map Function
julia
recursive_map(f, x, args...)
Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
How this works?
For the following types it directly defines recursion rules:
  1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
  2. Tuple/NamedTuple: We recurse on the values.
  3. Number/Val/Nothing: We directly apply f.
  4. For all other types, we recurse on the fields using Functors.fmap.
Note
In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
source
Lux.recursive_add!! Function
julia
recursive_add!!(x, y)
Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
Any leaves of x that are arrays and allow in-place addition will be modified in place.
source
Lux.recursive_copyto! Function
julia
recursive_copyto!(x, y)
Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
source
Lux.recursive_eltype Function
julia
recursive_eltype(x, unwrap_ad_types = Val(false))
Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
For ambiguous inputs like nothing and Val types we return Bool as the eltype.
If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
source
Lux.recursive_make_zero Function
julia
recursive_make_zero(x)
Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
See also Lux.recursive_make_zero!!.
source
Lux.recursive_make_zero!! Function
julia
recursive_make_zero!!(x)
Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
See also Lux.recursive_make_zero for fully out-of-place version.
source
Updating Floating Point Precision ​
By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.
Lux.f16 Function
julia
f16(m)
Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
source
Lux.f32 Function
julia
f32(m)
Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
source
Lux.f64 Function
julia
f64(m)
Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
source
Element Type Matching ​
Lux.match_eltype Function
julia
match_eltype(layer, ps, st, args...)
Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
Extended Help
Controlling the Behavior via Preferences
Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
  • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
  • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
  • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
  • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
Warning
We print the warning for type-mismatch only once.
Element Type Conversions
For "convert" only the following conversions are done:
Element Type of parameters/statesElement Type of args...Converted to
Float64IntegerFloat64
Float32Float64Float32
Float32IntegerFloat32
Float16Float64Float16
Float16Float32Float16
Float16IntegerFloat16
source
Stateful Layer ​
Lux.StatefulLuxLayer Type
julia
StatefulLuxLayer{FT}(model, ps, st)
Warning
This is not a Lux.AbstractLuxLayer
A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
Usecases
  • Internal implementation of @compact heavily uses this layer.
  • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
  • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
Static Parameters
  • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
  • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
Arguments
  • model: A Lux layer
  • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
  • st: The state of the layer
Inputs
  • x: The input to the layer
  • ps: The parameters of the layer. Optional, defaults to s.ps
Outputs
  • y: The output of the layer
source
Compact Layer ​
Lux.@compact Macro
julia
@compact(kw...) do x
+istraining(st::NamedTuple)
Returns true if training is true or if st contains a training field with value true. Else returns false.
source
Lux.LuxOps.multigate Function
julia
multigate(x::AbstractArray, ::Val{N})
Split up x into N equally sized chunks (along dimension 1).
source
Recursive Operations ​
Lux.recursive_map Function
julia
recursive_map(f, x, args...)
Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
How this works?
For the following types it directly defines recursion rules:
  1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
  2. Tuple/NamedTuple: We recurse on the values.
  3. Number/Val/Nothing: We directly apply f.
  4. For all other types, we recurse on the fields using Functors.fmap.
Note
In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
source
Lux.recursive_add!! Function
julia
recursive_add!!(x, y)
Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
Any leaves of x that are arrays and allow in-place addition will be modified in place.
source
Lux.recursive_copyto! Function
julia
recursive_copyto!(x, y)
Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
source
Lux.recursive_eltype Function
julia
recursive_eltype(x, unwrap_ad_types = Val(false))
Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
For ambiguous inputs like nothing and Val types we return Bool as the eltype.
If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
source
Lux.recursive_make_zero Function
julia
recursive_make_zero(x)
Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
See also Lux.recursive_make_zero!!.
source
Lux.recursive_make_zero!! Function
julia
recursive_make_zero!!(x)
Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
See also Lux.recursive_make_zero for fully out-of-place version.
source
Updating Floating Point Precision ​
By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.
Lux.f16 Function
julia
f16(m)
Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
source
Lux.f32 Function
julia
f32(m)
Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
source
Lux.f64 Function
julia
f64(m)
Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
source
Element Type Matching ​
Lux.match_eltype Function
julia
match_eltype(layer, ps, st, args...)
Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
Extended Help
Controlling the Behavior via Preferences
Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
  • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
  • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
  • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
  • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
Warning
We print the warning for type-mismatch only once.
Element Type Conversions
For "convert" only the following conversions are done:
Element Type of parameters/statesElement Type of args...Converted to
Float64IntegerFloat64
Float32Float64Float32
Float32IntegerFloat32
Float16Float64Float16
Float16Float32Float16
Float16IntegerFloat16
source
Stateful Layer ​
Lux.StatefulLuxLayer Type
julia
StatefulLuxLayer{FT}(model, ps, st)
Warning
This is not a Lux.AbstractLuxLayer
A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
Usecases
  • Internal implementation of @compact heavily uses this layer.
  • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
  • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
Static Parameters
  • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
  • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
Arguments
  • model: A Lux layer
  • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
  • st: The state of the layer
Inputs
  • x: The input to the layer
  • ps: The parameters of the layer. Optional, defaults to s.ps
Outputs
  • y: The output of the layer
source
Compact Layer ​
Lux.@compact Macro
julia
@compact(kw...) do x
     ...
     @return y # optional (but recommended for best performance)
 end
@@ -303,7 +303,7 @@
 true
You may also specify a name for the model, which will be used instead of the default printout, which gives a verbatim representation of the code used to construct the model:
julia
julia> model = @compact(w=rand(3), name="Linear(3 => 1)") do x
            @return sum(w .* x)
        end
-Linear(3 => 1)      # 3 parameters
This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
Type Stability
If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
Parameter Count
Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
source
Lux.@init_fn Macro
julia
@init_fn(fn, kind::Symbol = :parameter)
Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
Arguments
  • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
  • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
Examples
julia
julia> using Lux, Random
+Linear(3 => 1)      # 3 parameters
This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
Type Stability
If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
Parameter Count
Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
source
Lux.@init_fn Macro
julia
@init_fn(fn, kind::Symbol = :parameter)
Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
Arguments
  • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
  • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
Examples
julia
julia> using Lux, Random
 
 julia> r = @compact(w=@init_fn(rng->randn32(rng, 3, 2)),
            b=@init_fn(rng->randn32(rng, 3), :state)) do x
@@ -319,7 +319,7 @@
 (3,)
 
 julia> size(r([1, 2], ps, st)[1])
-(3,)
source
Lux.@non_trainable Macro
julia
@non_trainable(x)
Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
Arguments
  • x: The value to be marked as non-trainable.
Examples
julia
julia> using Lux, Random
+(3,)
source
Lux.@non_trainable Macro
julia
@non_trainable(x)
Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
Arguments
  • x: The value to be marked as non-trainable.
Examples
julia
julia> using Lux, Random
 
 julia> r = @compact(w=ones(3), w_fixed=@non_trainable(rand(3))) do x
            @return sum(w .* x .+ w_fixed)
@@ -339,9 +339,9 @@
 true
 
 julia> res isa Number
-true
source
Miscellaneous ​
Lux.set_dispatch_doctor_preferences! Function
julia
set_dispatch_doctor_preferences!(mode::String)
-set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")
Set the dispatch doctor preference for LuxCore and LuxLib packages.
mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.
If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.
source

-    
+true

source

Miscellaneous ​

Lux.set_dispatch_doctor_preferences! Function
julia
set_dispatch_doctor_preferences!(mode::String)
+set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")

Set the dispatch doctor preference for LuxCore and LuxLib packages.

mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.

If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.

source

+ \ No newline at end of file diff --git a/previews/PR1000/api/Testing_Functionality/LuxTestUtils.html b/previews/PR1000/api/Testing_Functionality/LuxTestUtils.html index 53739339e7..a68134bc05 100644 --- a/previews/PR1000/api/Testing_Functionality/LuxTestUtils.html +++ b/previews/PR1000/api/Testing_Functionality/LuxTestUtils.html @@ -5,14 +5,14 @@ LuxTestUtils | Lux.jl Docs - + - + - + - + @@ -34,12 +34,12 @@ julia> @jet sum(1, 1) target_modules=(Base, Core) opt_broken=true call_broken=true Test Broken - Expression: #= REPL[21]:1 =# JET.@test_opt target_modules = (Base, Core) sum(1, 1)

source

LuxTestUtils.jet_target_modules! Function
julia
jet_target_modules!(list::Vector{String}; force::Bool=false)

This sets target_modules for all JET tests when using @jet.

source

Gradient Correctness ​

LuxTestUtils.test_gradients Function
julia
test_gradients(f, args...; skip_backends=[], broken_backends=[], kwargs...)

Test the gradients of f with respect to args using the specified backends.

BackendADTypeCPUGPUNotes
Zygote.jlAutoZygote()✔✔
Tracker.jlAutoTracker()✔✔
ReverseDiff.jlAutoReverseDiff()✔✖
ForwardDiff.jlAutoForwardDiff()✔✖len ≤ 100
FiniteDiff.jlAutoFiniteDiff()✔✖len ≤ 100
Enzyme.jlAutoEnzyme()✔✖Only Reverse Mode

Arguments

Keyword Arguments

Example

julia
julia> f(x, y, z) = x .+ sum(abs2, y.t) + sum(y.x.z)
+  Expression: #= REPL[21]:1 =# JET.@test_opt target_modules = (Base, Core) sum(1, 1)

source

LuxTestUtils.jet_target_modules! Function
julia
jet_target_modules!(list::Vector{String}; force::Bool=false)

This sets target_modules for all JET tests when using @jet.

source

Gradient Correctness ​

LuxTestUtils.test_gradients Function
julia
test_gradients(f, args...; skip_backends=[], broken_backends=[], kwargs...)

Test the gradients of f with respect to args using the specified backends.

BackendADTypeCPUGPUNotes
Zygote.jlAutoZygote()✔✔
Tracker.jlAutoTracker()✔✔
ReverseDiff.jlAutoReverseDiff()✔✖
ForwardDiff.jlAutoForwardDiff()✔✖len ≤ 100
FiniteDiff.jlAutoFiniteDiff()✔✖len ≤ 100
Enzyme.jlAutoEnzyme()✔✖Only Reverse Mode

Arguments

Keyword Arguments

Example

julia
julia> f(x, y, z) = x .+ sum(abs2, y.t) + sum(y.x.z)
 
 julia> x = (; t=rand(10), x=(z=[2.0],))
 
-julia> test_gradients(f, 1.0, x, nothing)

source

LuxTestUtils.@test_gradients Macro
julia
@test_gradients(f, args...; kwargs...)

See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

source

Extensions to @test ​

LuxTestUtils.@test_softfail Macro
julia
@test_softfail expr

Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

source

- +julia> test_gradients(f, 1.0, x, nothing)

source

LuxTestUtils.@test_gradients Macro
julia
@test_gradients(f, args...; kwargs...)

See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

source

Extensions to @test ​

LuxTestUtils.@test_softfail Macro
julia
@test_softfail expr

Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

source

+ \ No newline at end of file diff --git a/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.BWXp7_hs.js b/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.BWXp7_hs.js new file mode 100644 index 0000000000..c85be5088a --- /dev/null +++ b/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.BWXp7_hs.js @@ -0,0 +1,25 @@ +import{_ as l,c as p,j as i,a as e,G as t,a2 as n,B as d,o as h}from"./chunks/framework.DjZDIZsN.js";const T=JSON.parse('{"title":"MLDataDevices","description":"","frontmatter":{},"headers":[],"relativePath":"api/Accelerator_Support/MLDataDevices.md","filePath":"api/Accelerator_Support/MLDataDevices.md","lastUpdated":null}'),o={name:"api/Accelerator_Support/MLDataDevices.md"},r={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"};function L(j,s,A,x,M,B){const a=d("Badge");return h(),p("div",null,[s[42]||(s[42]=i("h1",{id:"MLDataDevices-API",tabindex:"-1"},[e("MLDataDevices "),i("a",{class:"header-anchor",href:"#MLDataDevices-API","aria-label":'Permalink to "MLDataDevices {#MLDataDevices-API}"'},"​")],-1)),s[43]||(s[43]=i("p",null,[i("code",null,"MLDataDevices.jl"),e(" is a lightweight package defining rules for transferring data across devices.")],-1)),s[44]||(s[44]=i("h2",{id:"preferences",tabindex:"-1"},[e("Preferences "),i("a",{class:"header-anchor",href:"#preferences","aria-label":'Permalink to "Preferences"'},"​")],-1)),i("details",r,[i("summary",null,[s[0]||(s[0]=i("a",{id:"MLDataDevices.gpu_backend!",href:"#MLDataDevices.gpu_backend!"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_backend!")],-1)),s[1]||(s[1]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[2]||(s[2]=n(`
julia
gpu_backend!() = gpu_backend!("")
+gpu_backend!(backend) = gpu_backend!(string(backend))
+gpu_backend!(backend::AbstractGPUDevice)
+gpu_backend!(backend::String)

Creates a LocalPreferences.toml file with the desired GPU backend.

If backend == "", then the gpu_backend preference is deleted. Otherwise, backend is validated to be one of the possible backends and the preference is set to backend.

If a new backend is successfully set, then the Julia session must be restarted for the change to take effect.

source

`,5))]),s[45]||(s[45]=i("h2",{id:"Data-Transfer",tabindex:"-1"},[e("Data Transfer "),i("a",{class:"header-anchor",href:"#Data-Transfer","aria-label":'Permalink to "Data Transfer {#Data-Transfer}"'},"​")],-1)),i("details",c,[i("summary",null,[s[3]||(s[3]=i("a",{id:"MLDataDevices.cpu_device",href:"#MLDataDevices.cpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.cpu_device")],-1)),s[4]||(s[4]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=n('
julia
cpu_device() -> CPUDevice()

Return a CPUDevice object which can be used to transfer data to CPU.

source

',3))]),i("details",k,[i("summary",null,[s[6]||(s[6]=i("a",{id:"MLDataDevices.gpu_device",href:"#MLDataDevices.gpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_device")],-1)),s[7]||(s[7]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=n(`
julia
gpu_device(device_id::Union{Nothing, Integer}=nothing;
+    force::Bool=false) -> AbstractDevice

Selects GPU device based on the following criteria:

  1. If gpu_backend preference is set and the backend is functional on the system, then that device is selected.

  2. Otherwise, an automatic selection algorithm is used. We go over possible device backends in the order specified by supported_gpu_backends() and select the first functional backend.

  3. If no GPU device is functional and force is false, then cpu_device() is invoked.

  4. If nothing works, an error is thrown.

Arguments

Warning

device_id is only applicable for CUDA and AMDGPU backends. For Metal, oneAPI and CPU backends, device_id is ignored and a warning is printed.

Warning

gpu_device won't select a CUDA device unless both CUDA.jl and cuDNN.jl are loaded. This is to ensure that deep learning operations work correctly. Nonetheless, if cuDNN is not loaded you can still manually create a CUDADevice object and use it (e.g. dev = CUDADevice()).

Keyword Arguments

source

`,10))]),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"MLDataDevices.xla_device",href:"#MLDataDevices.xla_device"},[i("span",{class:"jlbinding"},"MLDataDevices.xla_device")],-1)),s[10]||(s[10]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=n('
julia
xla_device(; force::Bool=false) -> Union{XLADevice, CPUDevice}

Return a XLADevice object if functional. Otherwise, throw an error if force is true. Falls back to CPUDevice if force is false.

Danger

This is an experimental feature and might change without deprecations

source

',4))]),s[46]||(s[46]=i("h2",{id:"miscellaneous",tabindex:"-1"},[e("Miscellaneous "),i("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),i("details",u,[i("summary",null,[s[12]||(s[12]=i("a",{id:"MLDataDevices.reset_gpu_device!",href:"#MLDataDevices.reset_gpu_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.reset_gpu_device!")],-1)),s[13]||(s[13]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[14]||(s[14]=n('
julia
reset_gpu_device!()

Resets the selected GPU device. This is useful when automatic GPU selection needs to be run again.

source

',3))]),i("details",E,[i("summary",null,[s[15]||(s[15]=i("a",{id:"MLDataDevices.supported_gpu_backends",href:"#MLDataDevices.supported_gpu_backends"},[i("span",{class:"jlbinding"},"MLDataDevices.supported_gpu_backends")],-1)),s[16]||(s[16]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[17]||(s[17]=n('
julia
supported_gpu_backends() -> Tuple{String, ...}

Return a tuple of supported GPU backends.

Warning

This is not the list of functional backends on the system, but rather backends which MLDataDevices.jl supports.

source

',4))]),i("details",y,[i("summary",null,[s[18]||(s[18]=i("a",{id:"MLDataDevices.default_device_rng",href:"#MLDataDevices.default_device_rng"},[i("span",{class:"jlbinding"},"MLDataDevices.default_device_rng")],-1)),s[19]||(s[19]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=n('
julia
default_device_rng(::AbstractDevice)

Returns the default RNG for the device. This can be used to directly generate parameters and states on the device using WeightInitializers.jl.

source

',3))]),i("details",b,[i("summary",null,[s[21]||(s[21]=i("a",{id:"MLDataDevices.get_device",href:"#MLDataDevices.get_device"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device")],-1)),s[22]||(s[22]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[23]||(s[23]=n('
julia
get_device(x) -> dev::AbstractDevice | Exception | Nothing

If all arrays (on the leaves of the structure) are on the same device, we return that device. Otherwise, we throw an error. If the object is device agnostic, we return nothing.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

See also get_device_type for a faster alternative that can be used for dispatch based on device type.

source

',7))]),i("details",v,[i("summary",null,[s[24]||(s[24]=i("a",{id:"MLDataDevices.get_device_type",href:"#MLDataDevices.get_device_type"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device_type")],-1)),s[25]||(s[25]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[26]||(s[26]=n('
julia
get_device_type(x) -> Type{<:AbstractDevice} | Exception | Type{Nothing}

Similar to get_device but returns the type of the device instead of the device itself. This value is often a compile time constant and is recommended to be used instead of get_device where ever defining dispatches based on the device type.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

source

',6))]),i("details",D,[i("summary",null,[s[27]||(s[27]=i("a",{id:"MLDataDevices.loaded",href:"#MLDataDevices.loaded"},[i("span",{class:"jlbinding"},"MLDataDevices.loaded")],-1)),s[28]||(s[28]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[29]||(s[29]=n(`
julia
loaded(x::AbstractDevice) -> Bool
+loaded(::Type{<:AbstractDevice}) -> Bool

Checks if the trigger package for the device is loaded. Trigger packages are as follows:

source

`,4))]),i("details",f,[i("summary",null,[s[30]||(s[30]=i("a",{id:"MLDataDevices.functional",href:"#MLDataDevices.functional"},[i("span",{class:"jlbinding"},"MLDataDevices.functional")],-1)),s[31]||(s[31]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[32]||(s[32]=n(`
julia
functional(x::AbstractDevice) -> Bool
+functional(::Type{<:AbstractDevice}) -> Bool

Checks if the device is functional. This is used to determine if the device can be used for computation. Note that even if the backend is loaded (as checked via MLDataDevices.loaded), the device may not be functional.

Note that while this function is not exported, it is considered part of the public API.

source

`,4))]),i("details",F,[i("summary",null,[s[33]||(s[33]=i("a",{id:"MLDataDevices.isleaf",href:"#MLDataDevices.isleaf"},[i("span",{class:"jlbinding"},"MLDataDevices.isleaf")],-1)),s[34]||(s[34]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[35]||(s[35]=n('
julia
isleaf(x) -> Bool

Returns true if x is a leaf node in the data structure.

Defining MLDataDevices.isleaf(x::T) = true for custom types can be used to customize the behavior the data movement behavior when an object with nested structure containing the type is transferred to a device.

Adapt.adapt_structure(::AbstractDevice, x::T) or Adapt.adapt_structure(::AbstractDevice, x::T) will be called during data movement if isleaf(x::T) == true.

If MLDataDevices.isleaf(x::T) is not defined, then it will fall back to Functors.isleaf(x).

source

',6))]),s[47]||(s[47]=i("h2",{id:"Multi-GPU-Support",tabindex:"-1"},[e("Multi-GPU Support "),i("a",{class:"header-anchor",href:"#Multi-GPU-Support","aria-label":'Permalink to "Multi-GPU Support {#Multi-GPU-Support}"'},"​")],-1)),i("details",C,[i("summary",null,[s[36]||(s[36]=i("a",{id:"MLDataDevices.set_device!",href:"#MLDataDevices.set_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.set_device!")],-1)),s[37]||(s[37]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[38]||(s[38]=n('
julia
set_device!(T::Type{<:AbstractDevice}, dev_or_id)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

julia
set_device!(T::Type{<:AbstractDevice}, ::Nothing, rank::Integer)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

',14))]),s[48]||(s[48]=i("h2",{id:"iteration",tabindex:"-1"},[e("Iteration "),i("a",{class:"header-anchor",href:"#iteration","aria-label":'Permalink to "Iteration"'},"​")],-1)),i("details",m,[i("summary",null,[s[39]||(s[39]=i("a",{id:"MLDataDevices.DeviceIterator",href:"#MLDataDevices.DeviceIterator"},[i("span",{class:"jlbinding"},"MLDataDevices.DeviceIterator")],-1)),s[40]||(s[40]=e()),t(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[41]||(s[41]=n(`
julia
DeviceIterator(dev::AbstractDevice, iterator)

Create a DeviceIterator that iterates through the provided iterator via iterate. Upon each iteration, the current batch is copied to the device dev, and the previous iteration is marked as freeable from GPU memory (via unsafe_free!) (no-op for a CPU device).

The conversion follows the same semantics as dev(<item from iterator>).

Similarity to CUDA.CuIterator

The design inspiration was taken from CUDA.CuIterator and was generalized to work with other backends and more complex iterators (using Functors).

MLUtils.DataLoader

Calling dev(::MLUtils.DataLoader) will automatically convert the dataloader to use the same semantics as DeviceIterator. This is generally preferred over looping over the dataloader directly and transferring the data to the device.

Examples

The following was run on a computer with an NVIDIA GPU.

julia
julia> using MLDataDevices, MLUtils
+
+julia> X = rand(Float64, 3, 33);
+
+julia> dataloader = DataLoader(X; batchsize=13, shuffle=false);
+
+julia> for (i, x) in enumerate(dataloader)
+           @show i, summary(x)
+       end
+(i, summary(x)) = (1, "3×13 Matrix{Float64}")
+(i, summary(x)) = (2, "3×13 Matrix{Float64}")
+(i, summary(x)) = (3, "3×7 Matrix{Float64}")
+
+julia> for (i, x) in enumerate(CUDADevice()(dataloader))
+           @show i, summary(x)
+       end
+(i, summary(x)) = (1, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
+(i, summary(x)) = (2, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
+(i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

`,9))])])}const w=l(o,[["render",L]]);export{T as __pageData,w as default}; diff --git a/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.BWXp7_hs.lean.js b/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.BWXp7_hs.lean.js new file mode 100644 index 0000000000..c85be5088a --- /dev/null +++ b/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.BWXp7_hs.lean.js @@ -0,0 +1,25 @@ +import{_ as l,c as p,j as i,a as e,G as t,a2 as n,B as d,o as h}from"./chunks/framework.DjZDIZsN.js";const T=JSON.parse('{"title":"MLDataDevices","description":"","frontmatter":{},"headers":[],"relativePath":"api/Accelerator_Support/MLDataDevices.md","filePath":"api/Accelerator_Support/MLDataDevices.md","lastUpdated":null}'),o={name:"api/Accelerator_Support/MLDataDevices.md"},r={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"};function L(j,s,A,x,M,B){const a=d("Badge");return h(),p("div",null,[s[42]||(s[42]=i("h1",{id:"MLDataDevices-API",tabindex:"-1"},[e("MLDataDevices "),i("a",{class:"header-anchor",href:"#MLDataDevices-API","aria-label":'Permalink to "MLDataDevices {#MLDataDevices-API}"'},"​")],-1)),s[43]||(s[43]=i("p",null,[i("code",null,"MLDataDevices.jl"),e(" is a lightweight package defining rules for transferring data across devices.")],-1)),s[44]||(s[44]=i("h2",{id:"preferences",tabindex:"-1"},[e("Preferences "),i("a",{class:"header-anchor",href:"#preferences","aria-label":'Permalink to "Preferences"'},"​")],-1)),i("details",r,[i("summary",null,[s[0]||(s[0]=i("a",{id:"MLDataDevices.gpu_backend!",href:"#MLDataDevices.gpu_backend!"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_backend!")],-1)),s[1]||(s[1]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[2]||(s[2]=n(`
julia
gpu_backend!() = gpu_backend!("")
+gpu_backend!(backend) = gpu_backend!(string(backend))
+gpu_backend!(backend::AbstractGPUDevice)
+gpu_backend!(backend::String)

Creates a LocalPreferences.toml file with the desired GPU backend.

If backend == "", then the gpu_backend preference is deleted. Otherwise, backend is validated to be one of the possible backends and the preference is set to backend.

If a new backend is successfully set, then the Julia session must be restarted for the change to take effect.

source

`,5))]),s[45]||(s[45]=i("h2",{id:"Data-Transfer",tabindex:"-1"},[e("Data Transfer "),i("a",{class:"header-anchor",href:"#Data-Transfer","aria-label":'Permalink to "Data Transfer {#Data-Transfer}"'},"​")],-1)),i("details",c,[i("summary",null,[s[3]||(s[3]=i("a",{id:"MLDataDevices.cpu_device",href:"#MLDataDevices.cpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.cpu_device")],-1)),s[4]||(s[4]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=n('
julia
cpu_device() -> CPUDevice()

Return a CPUDevice object which can be used to transfer data to CPU.

source

',3))]),i("details",k,[i("summary",null,[s[6]||(s[6]=i("a",{id:"MLDataDevices.gpu_device",href:"#MLDataDevices.gpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_device")],-1)),s[7]||(s[7]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=n(`
julia
gpu_device(device_id::Union{Nothing, Integer}=nothing;
+    force::Bool=false) -> AbstractDevice

Selects GPU device based on the following criteria:

  1. If gpu_backend preference is set and the backend is functional on the system, then that device is selected.

  2. Otherwise, an automatic selection algorithm is used. We go over possible device backends in the order specified by supported_gpu_backends() and select the first functional backend.

  3. If no GPU device is functional and force is false, then cpu_device() is invoked.

  4. If nothing works, an error is thrown.

Arguments

Warning

device_id is only applicable for CUDA and AMDGPU backends. For Metal, oneAPI and CPU backends, device_id is ignored and a warning is printed.

Warning

gpu_device won't select a CUDA device unless both CUDA.jl and cuDNN.jl are loaded. This is to ensure that deep learning operations work correctly. Nonetheless, if cuDNN is not loaded you can still manually create a CUDADevice object and use it (e.g. dev = CUDADevice()).

Keyword Arguments

source

`,10))]),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"MLDataDevices.xla_device",href:"#MLDataDevices.xla_device"},[i("span",{class:"jlbinding"},"MLDataDevices.xla_device")],-1)),s[10]||(s[10]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=n('
julia
xla_device(; force::Bool=false) -> Union{XLADevice, CPUDevice}

Return a XLADevice object if functional. Otherwise, throw an error if force is true. Falls back to CPUDevice if force is false.

Danger

This is an experimental feature and might change without deprecations

source

',4))]),s[46]||(s[46]=i("h2",{id:"miscellaneous",tabindex:"-1"},[e("Miscellaneous "),i("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),i("details",u,[i("summary",null,[s[12]||(s[12]=i("a",{id:"MLDataDevices.reset_gpu_device!",href:"#MLDataDevices.reset_gpu_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.reset_gpu_device!")],-1)),s[13]||(s[13]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[14]||(s[14]=n('
julia
reset_gpu_device!()

Resets the selected GPU device. This is useful when automatic GPU selection needs to be run again.

source

',3))]),i("details",E,[i("summary",null,[s[15]||(s[15]=i("a",{id:"MLDataDevices.supported_gpu_backends",href:"#MLDataDevices.supported_gpu_backends"},[i("span",{class:"jlbinding"},"MLDataDevices.supported_gpu_backends")],-1)),s[16]||(s[16]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[17]||(s[17]=n('
julia
supported_gpu_backends() -> Tuple{String, ...}

Return a tuple of supported GPU backends.

Warning

This is not the list of functional backends on the system, but rather backends which MLDataDevices.jl supports.

source

',4))]),i("details",y,[i("summary",null,[s[18]||(s[18]=i("a",{id:"MLDataDevices.default_device_rng",href:"#MLDataDevices.default_device_rng"},[i("span",{class:"jlbinding"},"MLDataDevices.default_device_rng")],-1)),s[19]||(s[19]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=n('
julia
default_device_rng(::AbstractDevice)

Returns the default RNG for the device. This can be used to directly generate parameters and states on the device using WeightInitializers.jl.

source

',3))]),i("details",b,[i("summary",null,[s[21]||(s[21]=i("a",{id:"MLDataDevices.get_device",href:"#MLDataDevices.get_device"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device")],-1)),s[22]||(s[22]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[23]||(s[23]=n('
julia
get_device(x) -> dev::AbstractDevice | Exception | Nothing

If all arrays (on the leaves of the structure) are on the same device, we return that device. Otherwise, we throw an error. If the object is device agnostic, we return nothing.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

See also get_device_type for a faster alternative that can be used for dispatch based on device type.

source

',7))]),i("details",v,[i("summary",null,[s[24]||(s[24]=i("a",{id:"MLDataDevices.get_device_type",href:"#MLDataDevices.get_device_type"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device_type")],-1)),s[25]||(s[25]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[26]||(s[26]=n('
julia
get_device_type(x) -> Type{<:AbstractDevice} | Exception | Type{Nothing}

Similar to get_device but returns the type of the device instead of the device itself. This value is often a compile time constant and is recommended to be used instead of get_device where ever defining dispatches based on the device type.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

source

',6))]),i("details",D,[i("summary",null,[s[27]||(s[27]=i("a",{id:"MLDataDevices.loaded",href:"#MLDataDevices.loaded"},[i("span",{class:"jlbinding"},"MLDataDevices.loaded")],-1)),s[28]||(s[28]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[29]||(s[29]=n(`
julia
loaded(x::AbstractDevice) -> Bool
+loaded(::Type{<:AbstractDevice}) -> Bool

Checks if the trigger package for the device is loaded. Trigger packages are as follows:

source

`,4))]),i("details",f,[i("summary",null,[s[30]||(s[30]=i("a",{id:"MLDataDevices.functional",href:"#MLDataDevices.functional"},[i("span",{class:"jlbinding"},"MLDataDevices.functional")],-1)),s[31]||(s[31]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[32]||(s[32]=n(`
julia
functional(x::AbstractDevice) -> Bool
+functional(::Type{<:AbstractDevice}) -> Bool

Checks if the device is functional. This is used to determine if the device can be used for computation. Note that even if the backend is loaded (as checked via MLDataDevices.loaded), the device may not be functional.

Note that while this function is not exported, it is considered part of the public API.

source

`,4))]),i("details",F,[i("summary",null,[s[33]||(s[33]=i("a",{id:"MLDataDevices.isleaf",href:"#MLDataDevices.isleaf"},[i("span",{class:"jlbinding"},"MLDataDevices.isleaf")],-1)),s[34]||(s[34]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[35]||(s[35]=n('
julia
isleaf(x) -> Bool

Returns true if x is a leaf node in the data structure.

Defining MLDataDevices.isleaf(x::T) = true for custom types can be used to customize the behavior the data movement behavior when an object with nested structure containing the type is transferred to a device.

Adapt.adapt_structure(::AbstractDevice, x::T) or Adapt.adapt_structure(::AbstractDevice, x::T) will be called during data movement if isleaf(x::T) == true.

If MLDataDevices.isleaf(x::T) is not defined, then it will fall back to Functors.isleaf(x).

source

',6))]),s[47]||(s[47]=i("h2",{id:"Multi-GPU-Support",tabindex:"-1"},[e("Multi-GPU Support "),i("a",{class:"header-anchor",href:"#Multi-GPU-Support","aria-label":'Permalink to "Multi-GPU Support {#Multi-GPU-Support}"'},"​")],-1)),i("details",C,[i("summary",null,[s[36]||(s[36]=i("a",{id:"MLDataDevices.set_device!",href:"#MLDataDevices.set_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.set_device!")],-1)),s[37]||(s[37]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[38]||(s[38]=n('
julia
set_device!(T::Type{<:AbstractDevice}, dev_or_id)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

julia
set_device!(T::Type{<:AbstractDevice}, ::Nothing, rank::Integer)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

',14))]),s[48]||(s[48]=i("h2",{id:"iteration",tabindex:"-1"},[e("Iteration "),i("a",{class:"header-anchor",href:"#iteration","aria-label":'Permalink to "Iteration"'},"​")],-1)),i("details",m,[i("summary",null,[s[39]||(s[39]=i("a",{id:"MLDataDevices.DeviceIterator",href:"#MLDataDevices.DeviceIterator"},[i("span",{class:"jlbinding"},"MLDataDevices.DeviceIterator")],-1)),s[40]||(s[40]=e()),t(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[41]||(s[41]=n(`
julia
DeviceIterator(dev::AbstractDevice, iterator)

Create a DeviceIterator that iterates through the provided iterator via iterate. Upon each iteration, the current batch is copied to the device dev, and the previous iteration is marked as freeable from GPU memory (via unsafe_free!) (no-op for a CPU device).

The conversion follows the same semantics as dev(<item from iterator>).

Similarity to CUDA.CuIterator

The design inspiration was taken from CUDA.CuIterator and was generalized to work with other backends and more complex iterators (using Functors).

MLUtils.DataLoader

Calling dev(::MLUtils.DataLoader) will automatically convert the dataloader to use the same semantics as DeviceIterator. This is generally preferred over looping over the dataloader directly and transferring the data to the device.

Examples

The following was run on a computer with an NVIDIA GPU.

julia
julia> using MLDataDevices, MLUtils
+
+julia> X = rand(Float64, 3, 33);
+
+julia> dataloader = DataLoader(X; batchsize=13, shuffle=false);
+
+julia> for (i, x) in enumerate(dataloader)
+           @show i, summary(x)
+       end
+(i, summary(x)) = (1, "3×13 Matrix{Float64}")
+(i, summary(x)) = (2, "3×13 Matrix{Float64}")
+(i, summary(x)) = (3, "3×7 Matrix{Float64}")
+
+julia> for (i, x) in enumerate(CUDADevice()(dataloader))
+           @show i, summary(x)
+       end
+(i, summary(x)) = (1, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
+(i, summary(x)) = (2, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
+(i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

`,9))])])}const w=l(o,[["render",L]]);export{T as __pageData,w as default}; diff --git a/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.CGqLoJfj.js b/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.CGqLoJfj.js deleted file mode 100644 index 3ae89c777e..0000000000 --- a/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.CGqLoJfj.js +++ /dev/null @@ -1,25 +0,0 @@ -import{_ as l,c as p,j as i,a as e,G as t,a2 as n,B as d,o as h}from"./chunks/framework.DjZDIZsN.js";const T=JSON.parse('{"title":"MLDataDevices","description":"","frontmatter":{},"headers":[],"relativePath":"api/Accelerator_Support/MLDataDevices.md","filePath":"api/Accelerator_Support/MLDataDevices.md","lastUpdated":null}'),o={name:"api/Accelerator_Support/MLDataDevices.md"},r={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"};function j(L,s,A,M,x,B){const a=d("Badge");return h(),p("div",null,[s[42]||(s[42]=i("h1",{id:"MLDataDevices-API",tabindex:"-1"},[e("MLDataDevices "),i("a",{class:"header-anchor",href:"#MLDataDevices-API","aria-label":'Permalink to "MLDataDevices {#MLDataDevices-API}"'},"​")],-1)),s[43]||(s[43]=i("p",null,[i("code",null,"MLDataDevices.jl"),e(" is a lightweight package defining rules for transferring data across devices.")],-1)),s[44]||(s[44]=i("h2",{id:"preferences",tabindex:"-1"},[e("Preferences "),i("a",{class:"header-anchor",href:"#preferences","aria-label":'Permalink to "Preferences"'},"​")],-1)),i("details",r,[i("summary",null,[s[0]||(s[0]=i("a",{id:"MLDataDevices.gpu_backend!",href:"#MLDataDevices.gpu_backend!"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_backend!")],-1)),s[1]||(s[1]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[2]||(s[2]=n(`
julia
gpu_backend!() = gpu_backend!("")
-gpu_backend!(backend) = gpu_backend!(string(backend))
-gpu_backend!(backend::AbstractGPUDevice)
-gpu_backend!(backend::String)

Creates a LocalPreferences.toml file with the desired GPU backend.

If backend == "", then the gpu_backend preference is deleted. Otherwise, backend is validated to be one of the possible backends and the preference is set to backend.

If a new backend is successfully set, then the Julia session must be restarted for the change to take effect.

source

`,5))]),s[45]||(s[45]=i("h2",{id:"Data-Transfer",tabindex:"-1"},[e("Data Transfer "),i("a",{class:"header-anchor",href:"#Data-Transfer","aria-label":'Permalink to "Data Transfer {#Data-Transfer}"'},"​")],-1)),i("details",c,[i("summary",null,[s[3]||(s[3]=i("a",{id:"MLDataDevices.cpu_device",href:"#MLDataDevices.cpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.cpu_device")],-1)),s[4]||(s[4]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=n('
julia
cpu_device() -> CPUDevice()

Return a CPUDevice object which can be used to transfer data to CPU.

source

',3))]),i("details",k,[i("summary",null,[s[6]||(s[6]=i("a",{id:"MLDataDevices.gpu_device",href:"#MLDataDevices.gpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_device")],-1)),s[7]||(s[7]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=n(`
julia
gpu_device(device_id::Union{Nothing, Integer}=nothing;
-    force::Bool=false) -> AbstractDevice

Selects GPU device based on the following criteria:

  1. If gpu_backend preference is set and the backend is functional on the system, then that device is selected.

  2. Otherwise, an automatic selection algorithm is used. We go over possible device backends in the order specified by supported_gpu_backends() and select the first functional backend.

  3. If no GPU device is functional and force is false, then cpu_device() is invoked.

  4. If nothing works, an error is thrown.

Arguments

Warning

device_id is only applicable for CUDA and AMDGPU backends. For Metal, oneAPI and CPU backends, device_id is ignored and a warning is printed.

Warning

gpu_device won't select a CUDA device unless both CUDA.jl and cuDNN.jl are loaded. This is to ensure that deep learning operations work correctly. Nonetheless, if cuDNN is not loaded you can still manually create a CUDADevice object and use it (e.g. dev = CUDADevice()).

Keyword Arguments

source

`,10))]),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"MLDataDevices.xla_device",href:"#MLDataDevices.xla_device"},[i("span",{class:"jlbinding"},"MLDataDevices.xla_device")],-1)),s[10]||(s[10]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=n('
julia
xla_device(; force::Bool=false) -> Union{XLADevice, CPUDevice}

Return a XLADevice object if functional. Otherwise, throw an error if force is true. Falls back to CPUDevice if force is false.

Danger

This is an experimental feature and might change without deprecations

source

',4))]),s[46]||(s[46]=i("h2",{id:"miscellaneous",tabindex:"-1"},[e("Miscellaneous "),i("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),i("details",u,[i("summary",null,[s[12]||(s[12]=i("a",{id:"MLDataDevices.reset_gpu_device!",href:"#MLDataDevices.reset_gpu_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.reset_gpu_device!")],-1)),s[13]||(s[13]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[14]||(s[14]=n('
julia
reset_gpu_device!()

Resets the selected GPU device. This is useful when automatic GPU selection needs to be run again.

source

',3))]),i("details",E,[i("summary",null,[s[15]||(s[15]=i("a",{id:"MLDataDevices.supported_gpu_backends",href:"#MLDataDevices.supported_gpu_backends"},[i("span",{class:"jlbinding"},"MLDataDevices.supported_gpu_backends")],-1)),s[16]||(s[16]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[17]||(s[17]=n('
julia
supported_gpu_backends() -> Tuple{String, ...}

Return a tuple of supported GPU backends.

Warning

This is not the list of functional backends on the system, but rather backends which MLDataDevices.jl supports.

source

',4))]),i("details",y,[i("summary",null,[s[18]||(s[18]=i("a",{id:"MLDataDevices.default_device_rng",href:"#MLDataDevices.default_device_rng"},[i("span",{class:"jlbinding"},"MLDataDevices.default_device_rng")],-1)),s[19]||(s[19]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=n('
julia
default_device_rng(::AbstractDevice)

Returns the default RNG for the device. This can be used to directly generate parameters and states on the device using WeightInitializers.jl.

source

',3))]),i("details",v,[i("summary",null,[s[21]||(s[21]=i("a",{id:"MLDataDevices.get_device",href:"#MLDataDevices.get_device"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device")],-1)),s[22]||(s[22]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[23]||(s[23]=n('
julia
get_device(x) -> dev::AbstractDevice | Exception | Nothing

If all arrays (on the leaves of the structure) are on the same device, we return that device. Otherwise, we throw an error. If the object is device agnostic, we return nothing.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

See also get_device_type for a faster alternative that can be used for dispatch based on device type.

source

',7))]),i("details",b,[i("summary",null,[s[24]||(s[24]=i("a",{id:"MLDataDevices.get_device_type",href:"#MLDataDevices.get_device_type"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device_type")],-1)),s[25]||(s[25]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[26]||(s[26]=n('
julia
get_device_type(x) -> Type{<:AbstractDevice} | Exception | Type{Nothing}

Similar to get_device but returns the type of the device instead of the device itself. This value is often a compile time constant and is recommended to be used instead of get_device where ever defining dispatches based on the device type.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

source

',6))]),i("details",D,[i("summary",null,[s[27]||(s[27]=i("a",{id:"MLDataDevices.loaded",href:"#MLDataDevices.loaded"},[i("span",{class:"jlbinding"},"MLDataDevices.loaded")],-1)),s[28]||(s[28]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[29]||(s[29]=n(`
julia
loaded(x::AbstractDevice) -> Bool
-loaded(::Type{<:AbstractDevice}) -> Bool

Checks if the trigger package for the device is loaded. Trigger packages are as follows:

source

`,4))]),i("details",F,[i("summary",null,[s[30]||(s[30]=i("a",{id:"MLDataDevices.functional",href:"#MLDataDevices.functional"},[i("span",{class:"jlbinding"},"MLDataDevices.functional")],-1)),s[31]||(s[31]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[32]||(s[32]=n(`
julia
functional(x::AbstractDevice) -> Bool
-functional(::Type{<:AbstractDevice}) -> Bool

Checks if the device is functional. This is used to determine if the device can be used for computation. Note that even if the backend is loaded (as checked via MLDataDevices.loaded), the device may not be functional.

Note that while this function is not exported, it is considered part of the public API.

source

`,4))]),i("details",f,[i("summary",null,[s[33]||(s[33]=i("a",{id:"MLDataDevices.isleaf",href:"#MLDataDevices.isleaf"},[i("span",{class:"jlbinding"},"MLDataDevices.isleaf")],-1)),s[34]||(s[34]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[35]||(s[35]=n('
julia
isleaf(x) -> Bool

Returns true if x is a leaf node in the data structure.

Defining MLDataDevices.isleaf(x::T) = true for custom types can be used to customize the behavior the data movement behavior when an object with nested structure containing the type is transferred to a device.

Adapt.adapt_structure(::AbstractDevice, x::T) or Adapt.adapt_structure(::AbstractDevice, x::T) will be called during data movement if isleaf(x::T) == true.

If MLDataDevices.isleaf(x::T) is not defined, then it will fall back to Functors.isleaf(x).

source

',6))]),s[47]||(s[47]=i("h2",{id:"Multi-GPU-Support",tabindex:"-1"},[e("Multi-GPU Support "),i("a",{class:"header-anchor",href:"#Multi-GPU-Support","aria-label":'Permalink to "Multi-GPU Support {#Multi-GPU-Support}"'},"​")],-1)),i("details",C,[i("summary",null,[s[36]||(s[36]=i("a",{id:"MLDataDevices.set_device!",href:"#MLDataDevices.set_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.set_device!")],-1)),s[37]||(s[37]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[38]||(s[38]=n('
julia
set_device!(T::Type{<:AbstractDevice}, dev_or_id)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

julia
set_device!(T::Type{<:AbstractDevice}, ::Nothing, rank::Integer)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

',14))]),s[48]||(s[48]=i("h2",{id:"iteration",tabindex:"-1"},[e("Iteration "),i("a",{class:"header-anchor",href:"#iteration","aria-label":'Permalink to "Iteration"'},"​")],-1)),i("details",m,[i("summary",null,[s[39]||(s[39]=i("a",{id:"MLDataDevices.DeviceIterator",href:"#MLDataDevices.DeviceIterator"},[i("span",{class:"jlbinding"},"MLDataDevices.DeviceIterator")],-1)),s[40]||(s[40]=e()),t(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[41]||(s[41]=n(`
julia
DeviceIterator(dev::AbstractDevice, iterator)

Create a DeviceIterator that iterates through the provided iterator via iterate. Upon each iteration, the current batch is copied to the device dev, and the previous iteration is marked as freeable from GPU memory (via unsafe_free!) (no-op for a CPU device).

The conversion follows the same semantics as dev(<item from iterator>).

Similarity to CUDA.CuIterator

The design inspiration was taken from CUDA.CuIterator and was generalized to work with other backends and more complex iterators (using Functors).

MLUtils.DataLoader

Calling dev(::MLUtils.DataLoader) will automatically convert the dataloader to use the same semantics as DeviceIterator. This is generally preferred over looping over the dataloader directly and transferring the data to the device.

Examples

The following was run on a computer with an NVIDIA GPU.

julia
julia> using MLDataDevices, MLUtils
-
-julia> X = rand(Float64, 3, 33);
-
-julia> dataloader = DataLoader(X; batchsize=13, shuffle=false);
-
-julia> for (i, x) in enumerate(dataloader)
-           @show i, summary(x)
-       end
-(i, summary(x)) = (1, "3×13 Matrix{Float64}")
-(i, summary(x)) = (2, "3×13 Matrix{Float64}")
-(i, summary(x)) = (3, "3×7 Matrix{Float64}")
-
-julia> for (i, x) in enumerate(CUDADevice()(dataloader))
-           @show i, summary(x)
-       end
-(i, summary(x)) = (1, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
-(i, summary(x)) = (2, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
-(i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

`,9))])])}const w=l(o,[["render",j]]);export{T as __pageData,w as default}; diff --git a/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.CGqLoJfj.lean.js b/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.CGqLoJfj.lean.js deleted file mode 100644 index 3ae89c777e..0000000000 --- a/previews/PR1000/assets/api_Accelerator_Support_MLDataDevices.md.CGqLoJfj.lean.js +++ /dev/null @@ -1,25 +0,0 @@ -import{_ as l,c as p,j as i,a as e,G as t,a2 as n,B as d,o as h}from"./chunks/framework.DjZDIZsN.js";const T=JSON.parse('{"title":"MLDataDevices","description":"","frontmatter":{},"headers":[],"relativePath":"api/Accelerator_Support/MLDataDevices.md","filePath":"api/Accelerator_Support/MLDataDevices.md","lastUpdated":null}'),o={name:"api/Accelerator_Support/MLDataDevices.md"},r={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"};function j(L,s,A,M,x,B){const a=d("Badge");return h(),p("div",null,[s[42]||(s[42]=i("h1",{id:"MLDataDevices-API",tabindex:"-1"},[e("MLDataDevices "),i("a",{class:"header-anchor",href:"#MLDataDevices-API","aria-label":'Permalink to "MLDataDevices {#MLDataDevices-API}"'},"​")],-1)),s[43]||(s[43]=i("p",null,[i("code",null,"MLDataDevices.jl"),e(" is a lightweight package defining rules for transferring data across devices.")],-1)),s[44]||(s[44]=i("h2",{id:"preferences",tabindex:"-1"},[e("Preferences "),i("a",{class:"header-anchor",href:"#preferences","aria-label":'Permalink to "Preferences"'},"​")],-1)),i("details",r,[i("summary",null,[s[0]||(s[0]=i("a",{id:"MLDataDevices.gpu_backend!",href:"#MLDataDevices.gpu_backend!"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_backend!")],-1)),s[1]||(s[1]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[2]||(s[2]=n(`
julia
gpu_backend!() = gpu_backend!("")
-gpu_backend!(backend) = gpu_backend!(string(backend))
-gpu_backend!(backend::AbstractGPUDevice)
-gpu_backend!(backend::String)

Creates a LocalPreferences.toml file with the desired GPU backend.

If backend == "", then the gpu_backend preference is deleted. Otherwise, backend is validated to be one of the possible backends and the preference is set to backend.

If a new backend is successfully set, then the Julia session must be restarted for the change to take effect.

source

`,5))]),s[45]||(s[45]=i("h2",{id:"Data-Transfer",tabindex:"-1"},[e("Data Transfer "),i("a",{class:"header-anchor",href:"#Data-Transfer","aria-label":'Permalink to "Data Transfer {#Data-Transfer}"'},"​")],-1)),i("details",c,[i("summary",null,[s[3]||(s[3]=i("a",{id:"MLDataDevices.cpu_device",href:"#MLDataDevices.cpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.cpu_device")],-1)),s[4]||(s[4]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=n('
julia
cpu_device() -> CPUDevice()

Return a CPUDevice object which can be used to transfer data to CPU.

source

',3))]),i("details",k,[i("summary",null,[s[6]||(s[6]=i("a",{id:"MLDataDevices.gpu_device",href:"#MLDataDevices.gpu_device"},[i("span",{class:"jlbinding"},"MLDataDevices.gpu_device")],-1)),s[7]||(s[7]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=n(`
julia
gpu_device(device_id::Union{Nothing, Integer}=nothing;
-    force::Bool=false) -> AbstractDevice

Selects GPU device based on the following criteria:

  1. If gpu_backend preference is set and the backend is functional on the system, then that device is selected.

  2. Otherwise, an automatic selection algorithm is used. We go over possible device backends in the order specified by supported_gpu_backends() and select the first functional backend.

  3. If no GPU device is functional and force is false, then cpu_device() is invoked.

  4. If nothing works, an error is thrown.

Arguments

Warning

device_id is only applicable for CUDA and AMDGPU backends. For Metal, oneAPI and CPU backends, device_id is ignored and a warning is printed.

Warning

gpu_device won't select a CUDA device unless both CUDA.jl and cuDNN.jl are loaded. This is to ensure that deep learning operations work correctly. Nonetheless, if cuDNN is not loaded you can still manually create a CUDADevice object and use it (e.g. dev = CUDADevice()).

Keyword Arguments

source

`,10))]),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"MLDataDevices.xla_device",href:"#MLDataDevices.xla_device"},[i("span",{class:"jlbinding"},"MLDataDevices.xla_device")],-1)),s[10]||(s[10]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=n('
julia
xla_device(; force::Bool=false) -> Union{XLADevice, CPUDevice}

Return a XLADevice object if functional. Otherwise, throw an error if force is true. Falls back to CPUDevice if force is false.

Danger

This is an experimental feature and might change without deprecations

source

',4))]),s[46]||(s[46]=i("h2",{id:"miscellaneous",tabindex:"-1"},[e("Miscellaneous "),i("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),i("details",u,[i("summary",null,[s[12]||(s[12]=i("a",{id:"MLDataDevices.reset_gpu_device!",href:"#MLDataDevices.reset_gpu_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.reset_gpu_device!")],-1)),s[13]||(s[13]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[14]||(s[14]=n('
julia
reset_gpu_device!()

Resets the selected GPU device. This is useful when automatic GPU selection needs to be run again.

source

',3))]),i("details",E,[i("summary",null,[s[15]||(s[15]=i("a",{id:"MLDataDevices.supported_gpu_backends",href:"#MLDataDevices.supported_gpu_backends"},[i("span",{class:"jlbinding"},"MLDataDevices.supported_gpu_backends")],-1)),s[16]||(s[16]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[17]||(s[17]=n('
julia
supported_gpu_backends() -> Tuple{String, ...}

Return a tuple of supported GPU backends.

Warning

This is not the list of functional backends on the system, but rather backends which MLDataDevices.jl supports.

source

',4))]),i("details",y,[i("summary",null,[s[18]||(s[18]=i("a",{id:"MLDataDevices.default_device_rng",href:"#MLDataDevices.default_device_rng"},[i("span",{class:"jlbinding"},"MLDataDevices.default_device_rng")],-1)),s[19]||(s[19]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=n('
julia
default_device_rng(::AbstractDevice)

Returns the default RNG for the device. This can be used to directly generate parameters and states on the device using WeightInitializers.jl.

source

',3))]),i("details",v,[i("summary",null,[s[21]||(s[21]=i("a",{id:"MLDataDevices.get_device",href:"#MLDataDevices.get_device"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device")],-1)),s[22]||(s[22]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[23]||(s[23]=n('
julia
get_device(x) -> dev::AbstractDevice | Exception | Nothing

If all arrays (on the leaves of the structure) are on the same device, we return that device. Otherwise, we throw an error. If the object is device agnostic, we return nothing.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

See also get_device_type for a faster alternative that can be used for dispatch based on device type.

source

',7))]),i("details",b,[i("summary",null,[s[24]||(s[24]=i("a",{id:"MLDataDevices.get_device_type",href:"#MLDataDevices.get_device_type"},[i("span",{class:"jlbinding"},"MLDataDevices.get_device_type")],-1)),s[25]||(s[25]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[26]||(s[26]=n('
julia
get_device_type(x) -> Type{<:AbstractDevice} | Exception | Type{Nothing}

Similar to get_device but returns the type of the device instead of the device itself. This value is often a compile time constant and is recommended to be used instead of get_device where ever defining dispatches based on the device type.

Note

Trigger Packages must be loaded for this to return the correct device.

Special Retuened Values

source

',6))]),i("details",D,[i("summary",null,[s[27]||(s[27]=i("a",{id:"MLDataDevices.loaded",href:"#MLDataDevices.loaded"},[i("span",{class:"jlbinding"},"MLDataDevices.loaded")],-1)),s[28]||(s[28]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[29]||(s[29]=n(`
julia
loaded(x::AbstractDevice) -> Bool
-loaded(::Type{<:AbstractDevice}) -> Bool

Checks if the trigger package for the device is loaded. Trigger packages are as follows:

source

`,4))]),i("details",F,[i("summary",null,[s[30]||(s[30]=i("a",{id:"MLDataDevices.functional",href:"#MLDataDevices.functional"},[i("span",{class:"jlbinding"},"MLDataDevices.functional")],-1)),s[31]||(s[31]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[32]||(s[32]=n(`
julia
functional(x::AbstractDevice) -> Bool
-functional(::Type{<:AbstractDevice}) -> Bool

Checks if the device is functional. This is used to determine if the device can be used for computation. Note that even if the backend is loaded (as checked via MLDataDevices.loaded), the device may not be functional.

Note that while this function is not exported, it is considered part of the public API.

source

`,4))]),i("details",f,[i("summary",null,[s[33]||(s[33]=i("a",{id:"MLDataDevices.isleaf",href:"#MLDataDevices.isleaf"},[i("span",{class:"jlbinding"},"MLDataDevices.isleaf")],-1)),s[34]||(s[34]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[35]||(s[35]=n('
julia
isleaf(x) -> Bool

Returns true if x is a leaf node in the data structure.

Defining MLDataDevices.isleaf(x::T) = true for custom types can be used to customize the behavior the data movement behavior when an object with nested structure containing the type is transferred to a device.

Adapt.adapt_structure(::AbstractDevice, x::T) or Adapt.adapt_structure(::AbstractDevice, x::T) will be called during data movement if isleaf(x::T) == true.

If MLDataDevices.isleaf(x::T) is not defined, then it will fall back to Functors.isleaf(x).

source

',6))]),s[47]||(s[47]=i("h2",{id:"Multi-GPU-Support",tabindex:"-1"},[e("Multi-GPU Support "),i("a",{class:"header-anchor",href:"#Multi-GPU-Support","aria-label":'Permalink to "Multi-GPU Support {#Multi-GPU-Support}"'},"​")],-1)),i("details",C,[i("summary",null,[s[36]||(s[36]=i("a",{id:"MLDataDevices.set_device!",href:"#MLDataDevices.set_device!"},[i("span",{class:"jlbinding"},"MLDataDevices.set_device!")],-1)),s[37]||(s[37]=e()),t(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[38]||(s[38]=n('
julia
set_device!(T::Type{<:AbstractDevice}, dev_or_id)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

julia
set_device!(T::Type{<:AbstractDevice}, ::Nothing, rank::Integer)

Set the device for the given type. This is a no-op for CPUDevice. For CUDADevice and AMDGPUDevice, it prints a warning if the corresponding trigger package is not loaded.

Currently, MetalDevice and oneAPIDevice don't support setting the device.

Arguments

Danger

This specific function should be considered experimental at this point and is currently provided to support distributed training in Lux. As such please use Lux.DistributedUtils instead of using this function.

source

',14))]),s[48]||(s[48]=i("h2",{id:"iteration",tabindex:"-1"},[e("Iteration "),i("a",{class:"header-anchor",href:"#iteration","aria-label":'Permalink to "Iteration"'},"​")],-1)),i("details",m,[i("summary",null,[s[39]||(s[39]=i("a",{id:"MLDataDevices.DeviceIterator",href:"#MLDataDevices.DeviceIterator"},[i("span",{class:"jlbinding"},"MLDataDevices.DeviceIterator")],-1)),s[40]||(s[40]=e()),t(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[41]||(s[41]=n(`
julia
DeviceIterator(dev::AbstractDevice, iterator)

Create a DeviceIterator that iterates through the provided iterator via iterate. Upon each iteration, the current batch is copied to the device dev, and the previous iteration is marked as freeable from GPU memory (via unsafe_free!) (no-op for a CPU device).

The conversion follows the same semantics as dev(<item from iterator>).

Similarity to CUDA.CuIterator

The design inspiration was taken from CUDA.CuIterator and was generalized to work with other backends and more complex iterators (using Functors).

MLUtils.DataLoader

Calling dev(::MLUtils.DataLoader) will automatically convert the dataloader to use the same semantics as DeviceIterator. This is generally preferred over looping over the dataloader directly and transferring the data to the device.

Examples

The following was run on a computer with an NVIDIA GPU.

julia
julia> using MLDataDevices, MLUtils
-
-julia> X = rand(Float64, 3, 33);
-
-julia> dataloader = DataLoader(X; batchsize=13, shuffle=false);
-
-julia> for (i, x) in enumerate(dataloader)
-           @show i, summary(x)
-       end
-(i, summary(x)) = (1, "3×13 Matrix{Float64}")
-(i, summary(x)) = (2, "3×13 Matrix{Float64}")
-(i, summary(x)) = (3, "3×7 Matrix{Float64}")
-
-julia> for (i, x) in enumerate(CUDADevice()(dataloader))
-           @show i, summary(x)
-       end
-(i, summary(x)) = (1, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
-(i, summary(x)) = (2, "3×13 CuArray{Float32, 2, CUDA.DeviceMemory}")
-(i, summary(x)) = (3, "3×7 CuArray{Float32, 2, CUDA.DeviceMemory}")

source

`,9))])])}const w=l(o,[["render",j]]);export{T as __pageData,w as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.D2jtdUfi.js b/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.D2jtdUfi.js new file mode 100644 index 0000000000..72baf0dc1d --- /dev/null +++ b/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.D2jtdUfi.js @@ -0,0 +1 @@ +import{_ as o,c as n,a2 as a,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const S=JSON.parse('{"title":"LuxCore","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxCore.md","filePath":"api/Building_Blocks/LuxCore.md","lastUpdated":null}'),d={name:"api/Building_Blocks/LuxCore.md"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"};function T(w,e,D,P,R,N){const i=r("Badge");return p(),n("div",null,[e[54]||(e[54]=a('

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

',3)),s("details",c,[s("summary",null,[e[0]||(e[0]=s("a",{id:"LuxCore.AbstractLuxLayer",href:"#LuxCore.AbstractLuxLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxLayer")],-1)),e[1]||(e[1]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[2]||(e[2]=a('
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

Optionally:

See also AbstractLuxContainerLayer

source

',8))]),s("details",u,[s("summary",null,[e[3]||(e[3]=s("a",{id:"LuxCore.AbstractLuxWrapperLayer",href:"#LuxCore.AbstractLuxWrapperLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxWrapperLayer")],-1)),e[4]||(e[4]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[5]||(e[5]=a('
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

',5))]),s("details",h,[s("summary",null,[e[6]||(e[6]=s("a",{id:"LuxCore.AbstractLuxContainerLayer",href:"#LuxCore.AbstractLuxContainerLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxContainerLayer")],-1)),e[7]||(e[7]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[8]||(e[8]=a('
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

',7))]),e[55]||(e[55]=s("h2",{id:"general",tabindex:"-1"},[t("General "),s("a",{class:"header-anchor",href:"#general","aria-label":'Permalink to "General"'},"​")],-1)),s("details",k,[s("summary",null,[e[9]||(e[9]=s("a",{id:"LuxCore.apply",href:"#LuxCore.apply"},[s("span",{class:"jlbinding"},"LuxCore.apply")],-1)),e[10]||(e[10]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[11]||(e[11]=a('
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

',5))]),s("details",b,[s("summary",null,[e[12]||(e[12]=s("a",{id:"LuxCore.stateless_apply",href:"#LuxCore.stateless_apply"},[s("span",{class:"jlbinding"},"LuxCore.stateless_apply")],-1)),e[13]||(e[13]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=a('
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

',3))]),s("details",g,[s("summary",null,[e[15]||(e[15]=s("a",{id:"LuxCore.check_fmap_condition",href:"#LuxCore.check_fmap_condition"},[s("span",{class:"jlbinding"},"LuxCore.check_fmap_condition")],-1)),e[16]||(e[16]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[17]||(e[17]=a('
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

Returns

A Boolean Value

source

',7))]),s("details",y,[s("summary",null,[e[18]||(e[18]=s("a",{id:"LuxCore.contains_lux_layer",href:"#LuxCore.contains_lux_layer"},[s("span",{class:"jlbinding"},"LuxCore.contains_lux_layer")],-1)),e[19]||(e[19]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[20]||(e[20]=a('
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

',3))]),s("details",m,[s("summary",null,[e[21]||(e[21]=s("a",{id:"LuxCore.display_name",href:"#LuxCore.display_name"},[s("span",{class:"jlbinding"},"LuxCore.display_name")],-1)),e[22]||(e[22]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[23]||(e[23]=a('
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

',3))]),s("details",L,[s("summary",null,[e[24]||(e[24]=s("a",{id:"LuxCore.replicate",href:"#LuxCore.replicate"},[s("span",{class:"jlbinding"},"LuxCore.replicate")],-1)),e[25]||(e[25]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[26]||(e[26]=a('
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

',3))]),s("details",x,[s("summary",null,[e[27]||(e[27]=s("a",{id:"LuxCore.setup",href:"#LuxCore.setup"},[s("span",{class:"jlbinding"},"LuxCore.setup")],-1)),e[28]||(e[28]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[29]||(e[29]=a('
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

',4))]),e[56]||(e[56]=s("h2",{id:"parameters",tabindex:"-1"},[t("Parameters "),s("a",{class:"header-anchor",href:"#parameters","aria-label":'Permalink to "Parameters"'},"​")],-1)),s("details",f,[s("summary",null,[e[30]||(e[30]=s("a",{id:"LuxCore.initialparameters",href:"#LuxCore.initialparameters"},[s("span",{class:"jlbinding"},"LuxCore.initialparameters")],-1)),e[31]||(e[31]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[32]||(e[32]=a('
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

',3))]),s("details",C,[s("summary",null,[e[33]||(e[33]=s("a",{id:"LuxCore.parameterlength",href:"#LuxCore.parameterlength"},[s("span",{class:"jlbinding"},"LuxCore.parameterlength")],-1)),e[34]||(e[34]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[35]||(e[35]=a('
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

',3))]),e[57]||(e[57]=s("h2",{id:"states",tabindex:"-1"},[t("States "),s("a",{class:"header-anchor",href:"#states","aria-label":'Permalink to "States"'},"​")],-1)),s("details",j,[s("summary",null,[e[36]||(e[36]=s("a",{id:"LuxCore.initialstates",href:"#LuxCore.initialstates"},[s("span",{class:"jlbinding"},"LuxCore.initialstates")],-1)),e[37]||(e[37]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[38]||(e[38]=a('
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

',3))]),s("details",v,[s("summary",null,[e[39]||(e[39]=s("a",{id:"LuxCore.statelength",href:"#LuxCore.statelength"},[s("span",{class:"jlbinding"},"LuxCore.statelength")],-1)),e[40]||(e[40]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[41]||(e[41]=a('
julia
statelength(layer)

Return the total number of states of the layer l.

source

',3))]),s("details",E,[s("summary",null,[e[42]||(e[42]=s("a",{id:"LuxCore.testmode",href:"#LuxCore.testmode"},[s("span",{class:"jlbinding"},"LuxCore.testmode")],-1)),e[43]||(e[43]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[44]||(e[44]=a('
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

',3))]),s("details",F,[s("summary",null,[e[45]||(e[45]=s("a",{id:"LuxCore.trainmode",href:"#LuxCore.trainmode"},[s("span",{class:"jlbinding"},"LuxCore.trainmode")],-1)),e[46]||(e[46]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[47]||(e[47]=a('
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

',3))]),s("details",A,[s("summary",null,[e[48]||(e[48]=s("a",{id:"LuxCore.update_state",href:"#LuxCore.update_state"},[s("span",{class:"jlbinding"},"LuxCore.update_state")],-1)),e[49]||(e[49]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[50]||(e[50]=a('
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

',4))]),e[58]||(e[58]=s("h2",{id:"Layer-size",tabindex:"-1"},[t("Layer size "),s("a",{class:"header-anchor",href:"#Layer-size","aria-label":'Permalink to "Layer size {#Layer-size}"'},"​")],-1)),s("details",B,[s("summary",null,[e[51]||(e[51]=s("a",{id:"LuxCore.outputsize",href:"#LuxCore.outputsize"},[s("span",{class:"jlbinding"},"LuxCore.outputsize")],-1)),e[52]||(e[52]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[53]||(e[53]=a('
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

',6))])])}const z=o(d,[["render",T]]);export{S as __pageData,z as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.D2jtdUfi.lean.js b/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.D2jtdUfi.lean.js new file mode 100644 index 0000000000..72baf0dc1d --- /dev/null +++ b/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.D2jtdUfi.lean.js @@ -0,0 +1 @@ +import{_ as o,c as n,a2 as a,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const S=JSON.parse('{"title":"LuxCore","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxCore.md","filePath":"api/Building_Blocks/LuxCore.md","lastUpdated":null}'),d={name:"api/Building_Blocks/LuxCore.md"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"};function T(w,e,D,P,R,N){const i=r("Badge");return p(),n("div",null,[e[54]||(e[54]=a('

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

',3)),s("details",c,[s("summary",null,[e[0]||(e[0]=s("a",{id:"LuxCore.AbstractLuxLayer",href:"#LuxCore.AbstractLuxLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxLayer")],-1)),e[1]||(e[1]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[2]||(e[2]=a('
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

Optionally:

See also AbstractLuxContainerLayer

source

',8))]),s("details",u,[s("summary",null,[e[3]||(e[3]=s("a",{id:"LuxCore.AbstractLuxWrapperLayer",href:"#LuxCore.AbstractLuxWrapperLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxWrapperLayer")],-1)),e[4]||(e[4]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[5]||(e[5]=a('
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

',5))]),s("details",h,[s("summary",null,[e[6]||(e[6]=s("a",{id:"LuxCore.AbstractLuxContainerLayer",href:"#LuxCore.AbstractLuxContainerLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxContainerLayer")],-1)),e[7]||(e[7]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[8]||(e[8]=a('
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

',7))]),e[55]||(e[55]=s("h2",{id:"general",tabindex:"-1"},[t("General "),s("a",{class:"header-anchor",href:"#general","aria-label":'Permalink to "General"'},"​")],-1)),s("details",k,[s("summary",null,[e[9]||(e[9]=s("a",{id:"LuxCore.apply",href:"#LuxCore.apply"},[s("span",{class:"jlbinding"},"LuxCore.apply")],-1)),e[10]||(e[10]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[11]||(e[11]=a('
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

',5))]),s("details",b,[s("summary",null,[e[12]||(e[12]=s("a",{id:"LuxCore.stateless_apply",href:"#LuxCore.stateless_apply"},[s("span",{class:"jlbinding"},"LuxCore.stateless_apply")],-1)),e[13]||(e[13]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=a('
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

',3))]),s("details",g,[s("summary",null,[e[15]||(e[15]=s("a",{id:"LuxCore.check_fmap_condition",href:"#LuxCore.check_fmap_condition"},[s("span",{class:"jlbinding"},"LuxCore.check_fmap_condition")],-1)),e[16]||(e[16]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[17]||(e[17]=a('
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

Returns

A Boolean Value

source

',7))]),s("details",y,[s("summary",null,[e[18]||(e[18]=s("a",{id:"LuxCore.contains_lux_layer",href:"#LuxCore.contains_lux_layer"},[s("span",{class:"jlbinding"},"LuxCore.contains_lux_layer")],-1)),e[19]||(e[19]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[20]||(e[20]=a('
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

',3))]),s("details",m,[s("summary",null,[e[21]||(e[21]=s("a",{id:"LuxCore.display_name",href:"#LuxCore.display_name"},[s("span",{class:"jlbinding"},"LuxCore.display_name")],-1)),e[22]||(e[22]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[23]||(e[23]=a('
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

',3))]),s("details",L,[s("summary",null,[e[24]||(e[24]=s("a",{id:"LuxCore.replicate",href:"#LuxCore.replicate"},[s("span",{class:"jlbinding"},"LuxCore.replicate")],-1)),e[25]||(e[25]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[26]||(e[26]=a('
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

',3))]),s("details",x,[s("summary",null,[e[27]||(e[27]=s("a",{id:"LuxCore.setup",href:"#LuxCore.setup"},[s("span",{class:"jlbinding"},"LuxCore.setup")],-1)),e[28]||(e[28]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[29]||(e[29]=a('
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

',4))]),e[56]||(e[56]=s("h2",{id:"parameters",tabindex:"-1"},[t("Parameters "),s("a",{class:"header-anchor",href:"#parameters","aria-label":'Permalink to "Parameters"'},"​")],-1)),s("details",f,[s("summary",null,[e[30]||(e[30]=s("a",{id:"LuxCore.initialparameters",href:"#LuxCore.initialparameters"},[s("span",{class:"jlbinding"},"LuxCore.initialparameters")],-1)),e[31]||(e[31]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[32]||(e[32]=a('
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

',3))]),s("details",C,[s("summary",null,[e[33]||(e[33]=s("a",{id:"LuxCore.parameterlength",href:"#LuxCore.parameterlength"},[s("span",{class:"jlbinding"},"LuxCore.parameterlength")],-1)),e[34]||(e[34]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[35]||(e[35]=a('
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

',3))]),e[57]||(e[57]=s("h2",{id:"states",tabindex:"-1"},[t("States "),s("a",{class:"header-anchor",href:"#states","aria-label":'Permalink to "States"'},"​")],-1)),s("details",j,[s("summary",null,[e[36]||(e[36]=s("a",{id:"LuxCore.initialstates",href:"#LuxCore.initialstates"},[s("span",{class:"jlbinding"},"LuxCore.initialstates")],-1)),e[37]||(e[37]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[38]||(e[38]=a('
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

',3))]),s("details",v,[s("summary",null,[e[39]||(e[39]=s("a",{id:"LuxCore.statelength",href:"#LuxCore.statelength"},[s("span",{class:"jlbinding"},"LuxCore.statelength")],-1)),e[40]||(e[40]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[41]||(e[41]=a('
julia
statelength(layer)

Return the total number of states of the layer l.

source

',3))]),s("details",E,[s("summary",null,[e[42]||(e[42]=s("a",{id:"LuxCore.testmode",href:"#LuxCore.testmode"},[s("span",{class:"jlbinding"},"LuxCore.testmode")],-1)),e[43]||(e[43]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[44]||(e[44]=a('
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

',3))]),s("details",F,[s("summary",null,[e[45]||(e[45]=s("a",{id:"LuxCore.trainmode",href:"#LuxCore.trainmode"},[s("span",{class:"jlbinding"},"LuxCore.trainmode")],-1)),e[46]||(e[46]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[47]||(e[47]=a('
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

',3))]),s("details",A,[s("summary",null,[e[48]||(e[48]=s("a",{id:"LuxCore.update_state",href:"#LuxCore.update_state"},[s("span",{class:"jlbinding"},"LuxCore.update_state")],-1)),e[49]||(e[49]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[50]||(e[50]=a('
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

',4))]),e[58]||(e[58]=s("h2",{id:"Layer-size",tabindex:"-1"},[t("Layer size "),s("a",{class:"header-anchor",href:"#Layer-size","aria-label":'Permalink to "Layer size {#Layer-size}"'},"​")],-1)),s("details",B,[s("summary",null,[e[51]||(e[51]=s("a",{id:"LuxCore.outputsize",href:"#LuxCore.outputsize"},[s("span",{class:"jlbinding"},"LuxCore.outputsize")],-1)),e[52]||(e[52]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[53]||(e[53]=a('
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

',6))])])}const z=o(d,[["render",T]]);export{S as __pageData,z as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.Tk3NWXme.js b/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.Tk3NWXme.js deleted file mode 100644 index 34ba247d80..0000000000 --- a/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.Tk3NWXme.js +++ /dev/null @@ -1 +0,0 @@ -import{_ as o,c as n,a2 as a,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const S=JSON.parse('{"title":"LuxCore","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxCore.md","filePath":"api/Building_Blocks/LuxCore.md","lastUpdated":null}'),d={name:"api/Building_Blocks/LuxCore.md"},u={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"};function T(w,e,D,P,R,N){const i=r("Badge");return p(),n("div",null,[e[54]||(e[54]=a('

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

',3)),s("details",u,[s("summary",null,[e[0]||(e[0]=s("a",{id:"LuxCore.AbstractLuxLayer",href:"#LuxCore.AbstractLuxLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxLayer")],-1)),e[1]||(e[1]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[2]||(e[2]=a('
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

Optionally:

See also AbstractLuxContainerLayer

source

',8))]),s("details",c,[s("summary",null,[e[3]||(e[3]=s("a",{id:"LuxCore.AbstractLuxWrapperLayer",href:"#LuxCore.AbstractLuxWrapperLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxWrapperLayer")],-1)),e[4]||(e[4]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[5]||(e[5]=a('
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

',5))]),s("details",h,[s("summary",null,[e[6]||(e[6]=s("a",{id:"LuxCore.AbstractLuxContainerLayer",href:"#LuxCore.AbstractLuxContainerLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxContainerLayer")],-1)),e[7]||(e[7]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[8]||(e[8]=a('
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

',7))]),e[55]||(e[55]=s("h2",{id:"general",tabindex:"-1"},[t("General "),s("a",{class:"header-anchor",href:"#general","aria-label":'Permalink to "General"'},"​")],-1)),s("details",k,[s("summary",null,[e[9]||(e[9]=s("a",{id:"LuxCore.apply",href:"#LuxCore.apply"},[s("span",{class:"jlbinding"},"LuxCore.apply")],-1)),e[10]||(e[10]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[11]||(e[11]=a('
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

',5))]),s("details",g,[s("summary",null,[e[12]||(e[12]=s("a",{id:"LuxCore.stateless_apply",href:"#LuxCore.stateless_apply"},[s("span",{class:"jlbinding"},"LuxCore.stateless_apply")],-1)),e[13]||(e[13]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=a('
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

',3))]),s("details",y,[s("summary",null,[e[15]||(e[15]=s("a",{id:"LuxCore.check_fmap_condition",href:"#LuxCore.check_fmap_condition"},[s("span",{class:"jlbinding"},"LuxCore.check_fmap_condition")],-1)),e[16]||(e[16]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[17]||(e[17]=a('
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

Returns

A Boolean Value

source

',7))]),s("details",b,[s("summary",null,[e[18]||(e[18]=s("a",{id:"LuxCore.contains_lux_layer",href:"#LuxCore.contains_lux_layer"},[s("span",{class:"jlbinding"},"LuxCore.contains_lux_layer")],-1)),e[19]||(e[19]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[20]||(e[20]=a('
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

',3))]),s("details",m,[s("summary",null,[e[21]||(e[21]=s("a",{id:"LuxCore.display_name",href:"#LuxCore.display_name"},[s("span",{class:"jlbinding"},"LuxCore.display_name")],-1)),e[22]||(e[22]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[23]||(e[23]=a('
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

',3))]),s("details",L,[s("summary",null,[e[24]||(e[24]=s("a",{id:"LuxCore.replicate",href:"#LuxCore.replicate"},[s("span",{class:"jlbinding"},"LuxCore.replicate")],-1)),e[25]||(e[25]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[26]||(e[26]=a('
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

',3))]),s("details",x,[s("summary",null,[e[27]||(e[27]=s("a",{id:"LuxCore.setup",href:"#LuxCore.setup"},[s("span",{class:"jlbinding"},"LuxCore.setup")],-1)),e[28]||(e[28]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[29]||(e[29]=a('
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

',4))]),e[56]||(e[56]=s("h2",{id:"parameters",tabindex:"-1"},[t("Parameters "),s("a",{class:"header-anchor",href:"#parameters","aria-label":'Permalink to "Parameters"'},"​")],-1)),s("details",C,[s("summary",null,[e[30]||(e[30]=s("a",{id:"LuxCore.initialparameters",href:"#LuxCore.initialparameters"},[s("span",{class:"jlbinding"},"LuxCore.initialparameters")],-1)),e[31]||(e[31]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[32]||(e[32]=a('
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

',3))]),s("details",f,[s("summary",null,[e[33]||(e[33]=s("a",{id:"LuxCore.parameterlength",href:"#LuxCore.parameterlength"},[s("span",{class:"jlbinding"},"LuxCore.parameterlength")],-1)),e[34]||(e[34]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[35]||(e[35]=a('
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

',3))]),e[57]||(e[57]=s("h2",{id:"states",tabindex:"-1"},[t("States "),s("a",{class:"header-anchor",href:"#states","aria-label":'Permalink to "States"'},"​")],-1)),s("details",j,[s("summary",null,[e[36]||(e[36]=s("a",{id:"LuxCore.initialstates",href:"#LuxCore.initialstates"},[s("span",{class:"jlbinding"},"LuxCore.initialstates")],-1)),e[37]||(e[37]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[38]||(e[38]=a('
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

',3))]),s("details",v,[s("summary",null,[e[39]||(e[39]=s("a",{id:"LuxCore.statelength",href:"#LuxCore.statelength"},[s("span",{class:"jlbinding"},"LuxCore.statelength")],-1)),e[40]||(e[40]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[41]||(e[41]=a('
julia
statelength(layer)

Return the total number of states of the layer l.

source

',3))]),s("details",E,[s("summary",null,[e[42]||(e[42]=s("a",{id:"LuxCore.testmode",href:"#LuxCore.testmode"},[s("span",{class:"jlbinding"},"LuxCore.testmode")],-1)),e[43]||(e[43]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[44]||(e[44]=a('
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

',3))]),s("details",F,[s("summary",null,[e[45]||(e[45]=s("a",{id:"LuxCore.trainmode",href:"#LuxCore.trainmode"},[s("span",{class:"jlbinding"},"LuxCore.trainmode")],-1)),e[46]||(e[46]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[47]||(e[47]=a('
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

',3))]),s("details",A,[s("summary",null,[e[48]||(e[48]=s("a",{id:"LuxCore.update_state",href:"#LuxCore.update_state"},[s("span",{class:"jlbinding"},"LuxCore.update_state")],-1)),e[49]||(e[49]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[50]||(e[50]=a('
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

',4))]),e[58]||(e[58]=s("h2",{id:"Layer-size",tabindex:"-1"},[t("Layer size "),s("a",{class:"header-anchor",href:"#Layer-size","aria-label":'Permalink to "Layer size {#Layer-size}"'},"​")],-1)),s("details",B,[s("summary",null,[e[51]||(e[51]=s("a",{id:"LuxCore.outputsize",href:"#LuxCore.outputsize"},[s("span",{class:"jlbinding"},"LuxCore.outputsize")],-1)),e[52]||(e[52]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[53]||(e[53]=a('
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

',6))])])}const z=o(d,[["render",T]]);export{S as __pageData,z as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.Tk3NWXme.lean.js b/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.Tk3NWXme.lean.js deleted file mode 100644 index 34ba247d80..0000000000 --- a/previews/PR1000/assets/api_Building_Blocks_LuxCore.md.Tk3NWXme.lean.js +++ /dev/null @@ -1 +0,0 @@ -import{_ as o,c as n,a2 as a,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const S=JSON.parse('{"title":"LuxCore","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxCore.md","filePath":"api/Building_Blocks/LuxCore.md","lastUpdated":null}'),d={name:"api/Building_Blocks/LuxCore.md"},u={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"};function T(w,e,D,P,R,N){const i=r("Badge");return p(),n("div",null,[e[54]||(e[54]=a('

LuxCore ​

LuxCore.jl defines the abstract layers for Lux. Allows users to be compatible with the entirely of Lux.jl without having such a heavy dependency. If you are depending on Lux.jl directly, you do not need to depend on LuxCore.jl (all the functionality is exported via Lux.jl).

Abstract Types ​

',3)),s("details",u,[s("summary",null,[e[0]||(e[0]=s("a",{id:"LuxCore.AbstractLuxLayer",href:"#LuxCore.AbstractLuxLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxLayer")],-1)),e[1]||(e[1]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[2]||(e[2]=a('
julia
abstract type AbstractLuxLayer

Abstract Type for all Lux Layers

Users implementing their custom layer, must implement

Optionally:

See also AbstractLuxContainerLayer

source

',8))]),s("details",c,[s("summary",null,[e[3]||(e[3]=s("a",{id:"LuxCore.AbstractLuxWrapperLayer",href:"#LuxCore.AbstractLuxWrapperLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxWrapperLayer")],-1)),e[4]||(e[4]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[5]||(e[5]=a('
julia
abstract type AbstractLuxWrapperLayer{layer} <: AbstractLuxLayer

See AbstractLuxContainerLayer for detailed documentation. This abstract type is very similar to AbstractLuxContainerLayer except that it allows for a single layer to be wrapped in a container.

Additionally, on calling initialparameters and initialstates, the parameters and states are not wrapped in a NamedTuple with the same name as the field.

As a convenience, we define the fallback call (::AbstractLuxWrapperLayer)(x, ps, st), which calls getfield(x, layer)(x, ps, st).

source

',5))]),s("details",h,[s("summary",null,[e[6]||(e[6]=s("a",{id:"LuxCore.AbstractLuxContainerLayer",href:"#LuxCore.AbstractLuxContainerLayer"},[s("span",{class:"jlbinding"},"LuxCore.AbstractLuxContainerLayer")],-1)),e[7]||(e[7]=t()),l(i,{type:"info",class:"jlObjectType jlType",text:"Type"})]),e[8]||(e[8]=a('
julia
abstract type AbstractLuxContainerLayer{layers} <: AbstractLuxLayer

Abstract Container Type for certain Lux Layers. layers is a tuple containing fieldnames for the layer, and constructs the parameters and states using those.

Users implementing their custom layer can extend the same functions as in AbstractLuxLayer.

Advanced Structure Manipulation

Advanced structure manipulation of these layers post construction is possible via Functors.fmap. For a more flexible interface, we recommend using Lux.Experimental.@layer_map.

fmap Support

fmap support needs to be explicitly enabled by loading Functors.jl and Setfield.jl.

Changes from Pre-1.0 Behavior

Previously if layers was a singleton tuple, initialparameters and initialstates would return the parameters and states for the single field layers. From v1.0.0 onwards, even for singleton tuples, the parameters/states are wrapped in a NamedTuple with the same name as the field. See AbstractLuxWrapperLayer to replicate the previous behavior of singleton tuples.

source

',7))]),e[55]||(e[55]=s("h2",{id:"general",tabindex:"-1"},[t("General "),s("a",{class:"header-anchor",href:"#general","aria-label":'Permalink to "General"'},"​")],-1)),s("details",k,[s("summary",null,[e[9]||(e[9]=s("a",{id:"LuxCore.apply",href:"#LuxCore.apply"},[s("span",{class:"jlbinding"},"LuxCore.apply")],-1)),e[10]||(e[10]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[11]||(e[11]=a('
julia
apply(model, x, ps, st)

In most cases this function simply calls model(x, ps, st). However, it is still recommended to call apply instead of model(x, ps, st) directly. Some of the reasons for this include:

  1. For certain types of inputs x, we might want to perform preprocessing before calling model. For eg, if x is an Array of ReverseDiff.TrackedReals this can cause significant regressions in model(x, ps, st) (since it won't hit any of the BLAS dispatches). In those cases, we would automatically convert x to a ReverseDiff.TrackedArray.

  2. Certain user defined inputs need to be applied to specific layers but we want the datatype of propagate through all the layers (even unsupported ones). In these cases, we can unpack the input in apply and pass it to the appropriate layer and then repack it before returning. See the Lux manual on Custom Input Types for a motivating example.

Tip

apply is integrated with DispatchDoctor.jl that allows automatic verification of type stability. By default this is "disable"d. For more information, see the documentation.

source

',5))]),s("details",g,[s("summary",null,[e[12]||(e[12]=s("a",{id:"LuxCore.stateless_apply",href:"#LuxCore.stateless_apply"},[s("span",{class:"jlbinding"},"LuxCore.stateless_apply")],-1)),e[13]||(e[13]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=a('
julia
stateless_apply(model, x, ps)

Calls apply and only returns the first argument. This function requires that model has an empty state of NamedTuple(). Behavior of other kinds of models are undefined and it is the responsibility of the user to ensure that the model has an empty state.

source

',3))]),s("details",y,[s("summary",null,[e[15]||(e[15]=s("a",{id:"LuxCore.check_fmap_condition",href:"#LuxCore.check_fmap_condition"},[s("span",{class:"jlbinding"},"LuxCore.check_fmap_condition")],-1)),e[16]||(e[16]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[17]||(e[17]=a('
julia
check_fmap_condition(cond, tmatch::Union{Type, Nothing}, x) -> Bool

fmaps into the structure x and see if cond is satisfied for any of the leaf elements.

Arguments

Returns

A Boolean Value

source

',7))]),s("details",b,[s("summary",null,[e[18]||(e[18]=s("a",{id:"LuxCore.contains_lux_layer",href:"#LuxCore.contains_lux_layer"},[s("span",{class:"jlbinding"},"LuxCore.contains_lux_layer")],-1)),e[19]||(e[19]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[20]||(e[20]=a('
julia
contains_lux_layer(l) -> Bool

Check if the structure l is a Lux AbstractLuxLayer or a container of such a layer.

source

',3))]),s("details",m,[s("summary",null,[e[21]||(e[21]=s("a",{id:"LuxCore.display_name",href:"#LuxCore.display_name"},[s("span",{class:"jlbinding"},"LuxCore.display_name")],-1)),e[22]||(e[22]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[23]||(e[23]=a('
julia
display_name(layer::AbstractLuxLayer)

Printed Name of the layer. If the layer has a field name that is used, else the type name is used.

source

',3))]),s("details",L,[s("summary",null,[e[24]||(e[24]=s("a",{id:"LuxCore.replicate",href:"#LuxCore.replicate"},[s("span",{class:"jlbinding"},"LuxCore.replicate")],-1)),e[25]||(e[25]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[26]||(e[26]=a('
julia
replicate(rng::AbstractRNG)

Creates a copy of the rng state depending on its type.

source

',3))]),s("details",x,[s("summary",null,[e[27]||(e[27]=s("a",{id:"LuxCore.setup",href:"#LuxCore.setup"},[s("span",{class:"jlbinding"},"LuxCore.setup")],-1)),e[28]||(e[28]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[29]||(e[29]=a('
julia
setup(rng::AbstractRNG, layer)

Shorthand for getting the parameters and states of the layer l. Is equivalent to (initialparameters(rng, l), initialstates(rng, l)).

Warning

This function is not pure, it mutates rng.

source

',4))]),e[56]||(e[56]=s("h2",{id:"parameters",tabindex:"-1"},[t("Parameters "),s("a",{class:"header-anchor",href:"#parameters","aria-label":'Permalink to "Parameters"'},"​")],-1)),s("details",C,[s("summary",null,[e[30]||(e[30]=s("a",{id:"LuxCore.initialparameters",href:"#LuxCore.initialparameters"},[s("span",{class:"jlbinding"},"LuxCore.initialparameters")],-1)),e[31]||(e[31]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[32]||(e[32]=a('
julia
initialparameters(rng::AbstractRNG, layer)

Generate the initial parameters of the layer l.

source

',3))]),s("details",f,[s("summary",null,[e[33]||(e[33]=s("a",{id:"LuxCore.parameterlength",href:"#LuxCore.parameterlength"},[s("span",{class:"jlbinding"},"LuxCore.parameterlength")],-1)),e[34]||(e[34]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[35]||(e[35]=a('
julia
parameterlength(layer)

Return the total number of parameters of the layer l.

source

',3))]),e[57]||(e[57]=s("h2",{id:"states",tabindex:"-1"},[t("States "),s("a",{class:"header-anchor",href:"#states","aria-label":'Permalink to "States"'},"​")],-1)),s("details",j,[s("summary",null,[e[36]||(e[36]=s("a",{id:"LuxCore.initialstates",href:"#LuxCore.initialstates"},[s("span",{class:"jlbinding"},"LuxCore.initialstates")],-1)),e[37]||(e[37]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[38]||(e[38]=a('
julia
initialstates(rng::AbstractRNG, layer)

Generate the initial states of the layer l.

source

',3))]),s("details",v,[s("summary",null,[e[39]||(e[39]=s("a",{id:"LuxCore.statelength",href:"#LuxCore.statelength"},[s("span",{class:"jlbinding"},"LuxCore.statelength")],-1)),e[40]||(e[40]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[41]||(e[41]=a('
julia
statelength(layer)

Return the total number of states of the layer l.

source

',3))]),s("details",E,[s("summary",null,[e[42]||(e[42]=s("a",{id:"LuxCore.testmode",href:"#LuxCore.testmode"},[s("span",{class:"jlbinding"},"LuxCore.testmode")],-1)),e[43]||(e[43]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[44]||(e[44]=a('
julia
testmode(st::NamedTuple)

Make all occurrences of training in state st – Val(false).

source

',3))]),s("details",F,[s("summary",null,[e[45]||(e[45]=s("a",{id:"LuxCore.trainmode",href:"#LuxCore.trainmode"},[s("span",{class:"jlbinding"},"LuxCore.trainmode")],-1)),e[46]||(e[46]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[47]||(e[47]=a('
julia
trainmode(st::NamedTuple)

Make all occurrences of training in state st – Val(true).

source

',3))]),s("details",A,[s("summary",null,[e[48]||(e[48]=s("a",{id:"LuxCore.update_state",href:"#LuxCore.update_state"},[s("span",{class:"jlbinding"},"LuxCore.update_state")],-1)),e[49]||(e[49]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[50]||(e[50]=a('
julia
update_state(st::NamedTuple, key::Symbol, value; exclude=Internal.isleaf)

Recursively update all occurrences of the key in the state st with the value. exclude is a function that is passed to Functors.fmap_with_path's exclude keyword.

Needs Functors.jl

This function requires Functors.jl to be loaded.

source

',4))]),e[58]||(e[58]=s("h2",{id:"Layer-size",tabindex:"-1"},[t("Layer size "),s("a",{class:"header-anchor",href:"#Layer-size","aria-label":'Permalink to "Layer size {#Layer-size}"'},"​")],-1)),s("details",B,[s("summary",null,[e[51]||(e[51]=s("a",{id:"LuxCore.outputsize",href:"#LuxCore.outputsize"},[s("span",{class:"jlbinding"},"LuxCore.outputsize")],-1)),e[52]||(e[52]=t()),l(i,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[53]||(e[53]=a('
julia
outputsize(layer, x, rng)

Return the output size of the layer.

The fallback implementation of this function assumes the inputs were batched, i.e., if any of the outputs are Arrays, with ndims(A) > 1, it will return size(A)[1:(end - 1)]. If this behavior is undesirable, provide a custom outputsize(layer, x, rng) implementation).

Fallback Implementation

The fallback implementation of this function is defined once Lux.jl is loaded.

Changes from Pre-1.0 Behavior

Previously it was possible to override this function by defining outputsize(layer). However, this can potentially introduce a bug that is hard to bypass. See this PR for more information.

source

',6))])])}const z=o(d,[["render",T]]);export{S as __pageData,z as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.DM8CkpLX.js b/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.VqNljgWV.js similarity index 72% rename from previews/PR1000/assets/api_Building_Blocks_LuxLib.md.DM8CkpLX.js rename to previews/PR1000/assets/api_Building_Blocks_LuxLib.md.VqNljgWV.js index 8d3487fe92..dc691c0fb6 100644 --- a/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.DM8CkpLX.js +++ b/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.VqNljgWV.js @@ -1,10 +1,10 @@ -import{_ as r,c as l,j as t,a as s,G as n,a2 as e,B as d,o}from"./chunks/framework.DjZDIZsN.js";const st=JSON.parse('{"title":"LuxLib","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxLib.md","filePath":"api/Building_Blocks/LuxLib.md","lastUpdated":null}'),p={name:"api/Building_Blocks/LuxLib.md"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},x={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.07ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 11080.9 891","aria-hidden":"true"},L={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},w={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},F={class:"jldocstring custom-block"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},j={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},D={class:"jldocstring custom-block"},M={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},z={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},O={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},S={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},X={class:"jldocstring custom-block"};function K(W,i,$,Y,_,tt){const a=d("Badge");return o(),l("div",null,[i[144]||(i[144]=t("h1",{id:"LuxLib-API",tabindex:"-1"},[s("LuxLib "),t("a",{class:"header-anchor",href:"#LuxLib-API","aria-label":'Permalink to "LuxLib {#LuxLib-API}"'},"​")],-1)),i[145]||(i[145]=t("p",null,"Backend for Lux.jl",-1)),i[146]||(i[146]=t("h2",{id:"Apply-Activation",tabindex:"-1"},[s("Apply Activation "),t("a",{class:"header-anchor",href:"#Apply-Activation","aria-label":'Permalink to "Apply Activation {#Apply-Activation}"'},"​")],-1)),t("details",h,[t("summary",null,[i[0]||(i[0]=t("a",{id:"LuxLib.API.fast_activation",href:"#LuxLib.API.fast_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation")],-1)),i[1]||(i[1]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=e('
julia
fast_activation(σ::F, x::AbstractArray) where {F}

Compute σ.(x) with the best possible implementation available. On CPUs we unroll the loop and use LoopVectorization.jl to vectorize the computation. On GPUs we use simply use broadcasting.

Note

This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.

Arguments

Returns

source

',8))]),t("details",u,[t("summary",null,[i[3]||(i[3]=t("a",{id:"LuxLib.API.fast_activation!!",href:"#LuxLib.API.fast_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation!!")],-1)),i[4]||(i[4]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=e('
julia
fast_activation!!(σ::F, x::AbstractArray) where {F}

Compute σ.(x) with the best possible implementation available. If it is possible to rewrite x in-place, it does so. If x is an immutable array, it falls back to the generic implementation.

Note

This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.

Load SLEEFPirates.jl to get faster activations

Certain activation functions are replaced with specialized implementations from SLEEFPirates.jl for FP32. This might lead to faster performance but can cause slight decrease in accuracy (in the floating point limit).

Arguments

Returns

source

',9))]),i[147]||(i[147]=t("h2",{id:"Batched-Operations",tabindex:"-1"},[s("Batched Operations "),t("a",{class:"header-anchor",href:"#Batched-Operations","aria-label":'Permalink to "Batched Operations {#Batched-Operations}"'},"​")],-1)),t("details",Q,[t("summary",null,[i[6]||(i[6]=t("a",{id:"LuxLib.API.batched_matmul",href:"#LuxLib.API.batched_matmul"},[t("span",{class:"jlbinding"},"LuxLib.API.batched_matmul")],-1)),i[7]||(i[7]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
julia
batched_matmul(x, y)

Computes the batched matrix multiplication of x and y. For more details see the NNlib documentation on NNlib.batched_mul. This function is mostly a wrapper around batched_mul but attempts to be faster on CPUs.

Load LoopVectorization.jl to get faster batched matrix multiplication

On CPUs loading LoopVectorization adds faster implementations of batched matrix multiplication.

source

',4))]),i[148]||(i[148]=t("h2",{id:"Bias-Activation",tabindex:"-1"},[s("Bias Activation "),t("a",{class:"header-anchor",href:"#Bias-Activation","aria-label":'Permalink to "Bias Activation {#Bias-Activation}"'},"​")],-1)),t("details",m,[t("summary",null,[i[9]||(i[9]=t("a",{id:"LuxLib.API.bias_activation",href:"#LuxLib.API.bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation")],-1)),i[10]||(i[10]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
julia
bias_activation(σ, x, bias)

Applies the activation function σ elementwise to the result of broadcasted addition of x and bias along the penultimate dimension. A vector x is treated as a matrix with a single last dimension.

Arguments

See also bias_activation!!, fast_activation.

source

',6))]),t("details",k,[t("summary",null,[i[12]||(i[12]=t("a",{id:"LuxLib.API.bias_activation!!",href:"#LuxLib.API.bias_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation!!")],-1)),i[13]||(i[13]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
julia
bias_activation!!(σ, x, bias)

Same as bias_activation but might update x in-place if possible. Users should not rely on x being mutated, it is recommended to use it like y = bias_activation!!(σ, x, bias). If x is updated in-place, y aliases x.

See also bias_activation, fast_activation!!.

source

',4))]),i[149]||(i[149]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",g,[t("summary",null,[i[15]||(i[15]=t("a",{id:"LuxLib.API.fused_conv_bias_activation",href:"#LuxLib.API.fused_conv_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_conv_bias_activation")],-1)),i[16]||(i[16]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e(`
julia
fused_conv_bias_activation(σ::F, weight::AbstractArray, x::AbstractArray,
-    b::Optional{<:AbstractVector}, cdims::ConvDims) where {F}

Computes σ.(conv(x, weight, cdims) .+ b) (b is not exactly broadcasted like this, rather it is reshaped and broadcasted to the penultimate dimension) with the best possible implementation available. This operation fuses operations into a single kernel if possible, and minimizes reallocations by reusing the output buffer for multiple operations.

Arguments

Notes on implementation

source

`,7))]),i[150]||(i[150]=t("h2",{id:"dropout",tabindex:"-1"},[s("Dropout "),t("a",{class:"header-anchor",href:"#dropout","aria-label":'Permalink to "Dropout"'},"​")],-1)),t("details",c,[t("summary",null,[i[18]||(i[18]=t("a",{id:"LuxLib.API.alpha_dropout",href:"#LuxLib.API.alpha_dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.alpha_dropout")],-1)),i[19]||(i[19]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
julia
alpha_dropout(rng::AbstractRNG, x, p, training)\nalpha_dropout(rng::AbstractRNG, x, p, training, α, A, B)

Alpha Dropout: Dropout ensuring that the mean and variance of the output remains same as the input. For details see [1]. Use the second call signature to avoid recomputing the constants for a fixed dropout probability.

Arguments

Returns

References

[1] Klambauer, Günter, et al. "Self-normalizing neural networks." Advances in neural information processing systems 30 (2017).

source

',9))]),t("details",T,[t("summary",null,[i[21]||(i[21]=t("a",{id:"LuxLib.API.dropout",href:"#LuxLib.API.dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.dropout")],-1)),i[22]||(i[22]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
julia
dropout(rng::AbstractRNG, x, p, training, invp, dims)
+import{_ as r,c as l,j as t,a as s,G as n,a2 as e,B as d,o}from"./chunks/framework.DjZDIZsN.js";const st=JSON.parse('{"title":"LuxLib","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxLib.md","filePath":"api/Building_Blocks/LuxLib.md","lastUpdated":null}'),p={name:"api/Building_Blocks/LuxLib.md"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},x={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.07ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 11080.9 891","aria-hidden":"true"},L={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},w={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},F={class:"jldocstring custom-block"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},j={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},D={class:"jldocstring custom-block"},M={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},z={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},O={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},S={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},X={class:"jldocstring custom-block"};function K(W,i,$,Y,_,tt){const a=d("Badge");return o(),l("div",null,[i[144]||(i[144]=t("h1",{id:"LuxLib-API",tabindex:"-1"},[s("LuxLib "),t("a",{class:"header-anchor",href:"#LuxLib-API","aria-label":'Permalink to "LuxLib {#LuxLib-API}"'},"​")],-1)),i[145]||(i[145]=t("p",null,"Backend for Lux.jl",-1)),i[146]||(i[146]=t("h2",{id:"Apply-Activation",tabindex:"-1"},[s("Apply Activation "),t("a",{class:"header-anchor",href:"#Apply-Activation","aria-label":'Permalink to "Apply Activation {#Apply-Activation}"'},"​")],-1)),t("details",h,[t("summary",null,[i[0]||(i[0]=t("a",{id:"LuxLib.API.fast_activation",href:"#LuxLib.API.fast_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation")],-1)),i[1]||(i[1]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=e('
julia
fast_activation(σ::F, x::AbstractArray) where {F}
Compute σ.(x) with the best possible implementation available. On CPUs we unroll the loop and use LoopVectorization.jl to vectorize the computation. On GPUs we use simply use broadcasting.
Note
This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.
Arguments
  • σ: Activation function
  • x: Input array
Returns
  • Output Array with the same size as x
source
',8))]),t("details",u,[t("summary",null,[i[3]||(i[3]=t("a",{id:"LuxLib.API.fast_activation!!",href:"#LuxLib.API.fast_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation!!")],-1)),i[4]||(i[4]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=e('
julia
fast_activation!!(σ::F, x::AbstractArray) where {F}
Compute σ.(x) with the best possible implementation available. If it is possible to rewrite x in-place, it does so. If x is an immutable array, it falls back to the generic implementation.
Note
This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.
Load SLEEFPirates.jl to get faster activations
Certain activation functions are replaced with specialized implementations from SLEEFPirates.jl for FP32. This might lead to faster performance but can cause slight decrease in accuracy (in the floating point limit).
Arguments
  • σ: Activation function
  • x: Input array
Returns
  • Output Array with the same size as x
source
',9))]),i[147]||(i[147]=t("h2",{id:"Batched-Operations",tabindex:"-1"},[s("Batched Operations "),t("a",{class:"header-anchor",href:"#Batched-Operations","aria-label":'Permalink to "Batched Operations {#Batched-Operations}"'},"​")],-1)),t("details",Q,[t("summary",null,[i[6]||(i[6]=t("a",{id:"LuxLib.API.batched_matmul",href:"#LuxLib.API.batched_matmul"},[t("span",{class:"jlbinding"},"LuxLib.API.batched_matmul")],-1)),i[7]||(i[7]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
julia
batched_matmul(x, y)
Computes the batched matrix multiplication of x and y. For more details see the NNlib documentation on NNlib.batched_mul. This function is mostly a wrapper around batched_mul but attempts to be faster on CPUs.
Load LoopVectorization.jl to get faster batched matrix multiplication
On CPUs loading LoopVectorization adds faster implementations of batched matrix multiplication.
source
',4))]),i[148]||(i[148]=t("h2",{id:"Bias-Activation",tabindex:"-1"},[s("Bias Activation "),t("a",{class:"header-anchor",href:"#Bias-Activation","aria-label":'Permalink to "Bias Activation {#Bias-Activation}"'},"​")],-1)),t("details",m,[t("summary",null,[i[9]||(i[9]=t("a",{id:"LuxLib.API.bias_activation",href:"#LuxLib.API.bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation")],-1)),i[10]||(i[10]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
julia
bias_activation(σ, x, bias)
Applies the activation function σ elementwise to the result of broadcasted addition of x and bias along the penultimate dimension. A vector x is treated as a matrix with a single last dimension.
Arguments
  • σ: Activation function
  • x: Input to be transformed
  • bias: Bias to be added. Can be nothing.
See also bias_activation!!, fast_activation.
source
',6))]),t("details",k,[t("summary",null,[i[12]||(i[12]=t("a",{id:"LuxLib.API.bias_activation!!",href:"#LuxLib.API.bias_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation!!")],-1)),i[13]||(i[13]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
julia
bias_activation!!(σ, x, bias)
Same as bias_activation but might update x in-place if possible. Users should not rely on x being mutated, it is recommended to use it like y = bias_activation!!(σ, x, bias). If x is updated in-place, y aliases x.
See also bias_activation, fast_activation!!.
source
',4))]),i[149]||(i[149]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[i[15]||(i[15]=t("a",{id:"LuxLib.API.fused_conv_bias_activation",href:"#LuxLib.API.fused_conv_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_conv_bias_activation")],-1)),i[16]||(i[16]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e(`
julia
fused_conv_bias_activation(σ::F, weight::AbstractArray, x::AbstractArray,
+    b::Optional{<:AbstractVector}, cdims::ConvDims) where {F}
Computes σ.(conv(x, weight, cdims) .+ b) (b is not exactly broadcasted like this, rather it is reshaped and broadcasted to the penultimate dimension) with the best possible implementation available. This operation fuses operations into a single kernel if possible, and minimizes reallocations by reusing the output buffer for multiple operations.
Arguments
  • σ: Activation function
  • weight: Weight tensor
  • x: Input tensor
  • b: Bias tensor (can be nothing)
  • cdims: ConvDims object
Notes on implementation
  • For CUDA Arrays, this uses fused CUDNN kernels when the activation is identity or relu. For other activations, it tries to fuse the operations on the Julia side.
  • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
  • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
  • For Mixed-Precision Inputs on GPU, we type promote the inputs to the highest precision, with a warning.
source
`,7))]),i[150]||(i[150]=t("h2",{id:"dropout",tabindex:"-1"},[s("Dropout "),t("a",{class:"header-anchor",href:"#dropout","aria-label":'Permalink to "Dropout"'},"​")],-1)),t("details",g,[t("summary",null,[i[18]||(i[18]=t("a",{id:"LuxLib.API.alpha_dropout",href:"#LuxLib.API.alpha_dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.alpha_dropout")],-1)),i[19]||(i[19]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
julia
alpha_dropout(rng::AbstractRNG, x, p, training)\nalpha_dropout(rng::AbstractRNG, x, p, training, α, A, B)
Alpha Dropout: Dropout ensuring that the mean and variance of the output remains same as the input. For details see [1]. Use the second call signature to avoid recomputing the constants for a fixed dropout probability.
Arguments
  • rng: Random number generator
  • x: Input Array
  • p: Probability of an element to be dropped out
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context`
  • α: -1.7580993408473766. Computed at limit x tends to infinity, selu(x) = -λβ = α
  • A: Scaling factor for the mean
  • B: Scaling factor for the variance
Returns
  • Output Array after applying alpha dropout
  • Updated state for the random number generator
References
[1] Klambauer, Günter, et al. "Self-normalizing neural networks." Advances in neural information processing systems 30 (2017).
source
',9))]),t("details",T,[t("summary",null,[i[21]||(i[21]=t("a",{id:"LuxLib.API.dropout",href:"#LuxLib.API.dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.dropout")],-1)),i[22]||(i[22]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
julia
dropout(rng::AbstractRNG, x, p, training, invp, dims)
 dropout(rng::AbstractRNG, x, mask, p, training, update_mask::Union{Val, StaticBool},
-    invp, dims)
Dropout: Simple Way to prevent Neural Networks for Overfitting. For details see [1].
Arguments
  • rng: Random number generator
  • x: Input Array
  • mask: Dropout Mask. If not used then it is constructed automatically
  • p: Probability of an element to be dropped out
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • update_mask: If Val(true) or True() then the mask is generated and used. Else, the mask provided is directly used
  • invp: Inverse multiplied to the mask. Calculated as invp = 1 / (1 - p).
Returns
  • Output Array after applying dropout
  • Dropout Mask (if training == false, the returned value is meaningless)
  • Updated state for the random number generator
References
[1] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The journal of machine learning research 15.1 (2014): 1929-1958.
source
`,9))]),i[151]||(i[151]=t("h2",{id:"Fully-Connected-Layers",tabindex:"-1"},[s("Fully Connected Layers "),t("a",{class:"header-anchor",href:"#Fully-Connected-Layers","aria-label":'Permalink to "Fully Connected Layers {#Fully-Connected-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[i[24]||(i[24]=t("a",{id:"LuxLib.API.fused_dense_bias_activation",href:"#LuxLib.API.fused_dense_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_dense_bias_activation")],-1)),i[25]||(i[25]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e('
julia
fused_dense_bias_activation(σ::F, weight::AbstractMatrix, x::AbstractMatrix,\n    b::Optional{<:AbstractVector}) where {F}
Compute σ.(weight * x .+ b) with the best possible implementation available. Currently this implementation attempts to minimize reallocations by reusing the output buffer for multiple operations.
Arguments
  • σ: Activation function
  • weight: Weight matrix
  • x: Input matrix
  • b: Bias vector (can be nothing)
Notes on implementation
  • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
  • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
  • For CUDA Arrays, this uses a special fused implementation via cuBLASLt.
  • For small CPU Arrays, we use LoopVectorization.jl. On x86_64 we use Octavian for medium sized matrices. This is overridden if special BLAS implementations are loaded (currently MKL, AppleAccelerate, and BLISBLAS).
!!! tip "Load Octavian.jl
Loading `Octavian.jl` enables a polyalgorithm that uses different backends based on the\ninput sizes.
source
',9))]),i[152]||(i[152]=t("h2",{id:"normalization",tabindex:"-1"},[s("Normalization "),t("a",{class:"header-anchor",href:"#normalization","aria-label":'Permalink to "Normalization"'},"​")],-1)),t("details",y,[t("summary",null,[i[27]||(i[27]=t("a",{id:"LuxLib.API.batchnorm",href:"#LuxLib.API.batchnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.batchnorm")],-1)),i[28]||(i[28]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[49]||(i[49]=e(`
julia
batchnorm(x, scale, bias, running_mean, running_var, training,
-    σ=identity, momentum = 0.1f0, epsilon = eps(eltype(x)) ^ (5 // 7))
Batch Normalization. For details see [1].
`,2)),t("p",null,[i[31]||(i[31]=s("Batch Normalization computes the mean and variance for each ")),t("mjx-container",x,[(o(),l("svg",f,i[29]||(i[29]=[e('',1)]))),i[30]||(i[30]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),i[32]||(i[32]=s(" input slice and normalises the input accordingly."))]),i[50]||(i[50]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[47]||(i[47]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[35]||(i[35]=t("code",null,"scale",-1)),i[36]||(i[36]=s(": Scale factor (")),t("mjx-container",L,[(o(),l("svg",v,i[33]||(i[33]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[34]||(i[34]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[37]||(i[37]=s(") (can be ")),i[38]||(i[38]=t("code",null,"nothing",-1)),i[39]||(i[39]=s(")"))])]),t("li",null,[t("p",null,[i[42]||(i[42]=t("code",null,"bias",-1)),i[43]||(i[43]=s(": Bias factor (")),t("mjx-container",E,[(o(),l("svg",w,i[40]||(i[40]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[41]||(i[41]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[44]||(i[44]=s(") (can be ")),i[45]||(i[45]=t("code",null,"nothing",-1)),i[46]||(i[46]=s(")"))])]),i[48]||(i[48]=e("
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",6))]),i[51]||(i[51]=t("p",null,[t("strong",null,"Returns")],-1)),i[52]||(i[52]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[53]||(i[53]=t("p",null,[t("strong",null,"References")],-1)),i[54]||(i[54]=t("p",null,'[1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International conference on machine learning. PMLR, 2015.',-1)),i[55]||(i[55]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/batchnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",F,[t("summary",null,[i[56]||(i[56]=t("a",{id:"LuxLib.API.groupnorm",href:"#LuxLib.API.groupnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.groupnorm")],-1)),i[57]||(i[57]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[74]||(i[74]=e(`
    julia
    groupnorm(x, scale, bias, groups::Int, σ::F=identity,
-    epsilon::Real=eps(eltype(x)) ^ (5 // 7))
    Group Normalization. For details see [1].
    This op is similar to batch normalization, but statistics are shared across equally-sized groups of channels and not shared across batch dimension. Thus, group normalization does not depend on the batch composition and does not require maintaining internal state for storing statistics.
    Arguments
    `,4)),t("ul",null,[i[72]||(i[72]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[60]||(i[60]=t("code",null,"scale",-1)),i[61]||(i[61]=s(": Scale factor (")),t("mjx-container",C,[(o(),l("svg",A,i[58]||(i[58]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[59]||(i[59]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[62]||(i[62]=s(") (can be ")),i[63]||(i[63]=t("code",null,"nothing",-1)),i[64]||(i[64]=s(")"))])]),t("li",null,[t("p",null,[i[67]||(i[67]=t("code",null,"bias",-1)),i[68]||(i[68]=s(": Bias factor (")),t("mjx-container",j,[(o(),l("svg",H,i[65]||(i[65]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[66]||(i[66]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[69]||(i[69]=s(") (can be ")),i[70]||(i[70]=t("code",null,"nothing",-1)),i[71]||(i[71]=s(")"))])]),i[73]||(i[73]=e("
  • groups: Number of groups
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",3))]),i[75]||(i[75]=t("p",null,[t("strong",null,"Returns")],-1)),i[76]||(i[76]=t("p",null,"The normalized array is returned.",-1)),i[77]||(i[77]=t("p",null,[t("strong",null,"References")],-1)),i[78]||(i[78]=t("p",null,'[1] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European conference on computer vision (ECCV). 2018.',-1)),i[79]||(i[79]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/groupnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",D,[t("summary",null,[i[80]||(i[80]=t("a",{id:"LuxLib.API.instancenorm",href:"#LuxLib.API.instancenorm"},[t("span",{class:"jlbinding"},"LuxLib.API.instancenorm")],-1)),i[81]||(i[81]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=e(`
    julia
    instancenorm(x, scale, bias, training, act, epsilon = eps(eltype(x)) ^ (5 // 7))
+    invp, dims)
    Dropout: Simple Way to prevent Neural Networks for Overfitting. For details see [1].
    Arguments
    • rng: Random number generator
    • x: Input Array
    • mask: Dropout Mask. If not used then it is constructed automatically
    • p: Probability of an element to be dropped out
    • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
    • update_mask: If Val(true) or True() then the mask is generated and used. Else, the mask provided is directly used
    • invp: Inverse multiplied to the mask. Calculated as invp = 1 / (1 - p).
    Returns
    • Output Array after applying dropout
    • Dropout Mask (if training == false, the returned value is meaningless)
    • Updated state for the random number generator
    References
    [1] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The journal of machine learning research 15.1 (2014): 1929-1958.
    source
    `,9))]),i[151]||(i[151]=t("h2",{id:"Fully-Connected-Layers",tabindex:"-1"},[s("Fully Connected Layers "),t("a",{class:"header-anchor",href:"#Fully-Connected-Layers","aria-label":'Permalink to "Fully Connected Layers {#Fully-Connected-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[i[24]||(i[24]=t("a",{id:"LuxLib.API.fused_dense_bias_activation",href:"#LuxLib.API.fused_dense_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_dense_bias_activation")],-1)),i[25]||(i[25]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e('
    julia
    fused_dense_bias_activation(σ::F, weight::AbstractMatrix, x::AbstractMatrix,\n    b::Optional{<:AbstractVector}) where {F}
    Compute σ.(weight * x .+ b) with the best possible implementation available. Currently this implementation attempts to minimize reallocations by reusing the output buffer for multiple operations.
    Arguments
    • σ: Activation function
    • weight: Weight matrix
    • x: Input matrix
    • b: Bias vector (can be nothing)
    Notes on implementation
    • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
    • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
    • For CUDA Arrays, this uses a special fused implementation via cuBLASLt.
    • For small CPU Arrays, we use LoopVectorization.jl. On x86_64 we use Octavian for medium sized matrices. This is overridden if special BLAS implementations are loaded (currently MKL, AppleAccelerate, and BLISBLAS).
    !!! tip "Load Octavian.jl
    Loading `Octavian.jl` enables a polyalgorithm that uses different backends based on the\ninput sizes.
    source
    ',9))]),i[152]||(i[152]=t("h2",{id:"normalization",tabindex:"-1"},[s("Normalization "),t("a",{class:"header-anchor",href:"#normalization","aria-label":'Permalink to "Normalization"'},"​")],-1)),t("details",y,[t("summary",null,[i[27]||(i[27]=t("a",{id:"LuxLib.API.batchnorm",href:"#LuxLib.API.batchnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.batchnorm")],-1)),i[28]||(i[28]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[49]||(i[49]=e(`
    julia
    batchnorm(x, scale, bias, running_mean, running_var, training,
+    σ=identity, momentum = 0.1f0, epsilon = eps(eltype(x)) ^ (5 // 7))
    Batch Normalization. For details see [1].
    `,2)),t("p",null,[i[31]||(i[31]=s("Batch Normalization computes the mean and variance for each ")),t("mjx-container",x,[(o(),l("svg",f,i[29]||(i[29]=[e('',1)]))),i[30]||(i[30]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),i[32]||(i[32]=s(" input slice and normalises the input accordingly."))]),i[50]||(i[50]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[47]||(i[47]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[35]||(i[35]=t("code",null,"scale",-1)),i[36]||(i[36]=s(": Scale factor (")),t("mjx-container",L,[(o(),l("svg",E,i[33]||(i[33]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[34]||(i[34]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[37]||(i[37]=s(") (can be ")),i[38]||(i[38]=t("code",null,"nothing",-1)),i[39]||(i[39]=s(")"))])]),t("li",null,[t("p",null,[i[42]||(i[42]=t("code",null,"bias",-1)),i[43]||(i[43]=s(": Bias factor (")),t("mjx-container",v,[(o(),l("svg",w,i[40]||(i[40]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[41]||(i[41]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[44]||(i[44]=s(") (can be ")),i[45]||(i[45]=t("code",null,"nothing",-1)),i[46]||(i[46]=s(")"))])]),i[48]||(i[48]=e("
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",6))]),i[51]||(i[51]=t("p",null,[t("strong",null,"Returns")],-1)),i[52]||(i[52]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[53]||(i[53]=t("p",null,[t("strong",null,"References")],-1)),i[54]||(i[54]=t("p",null,'[1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International conference on machine learning. PMLR, 2015.',-1)),i[55]||(i[55]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/batchnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",F,[t("summary",null,[i[56]||(i[56]=t("a",{id:"LuxLib.API.groupnorm",href:"#LuxLib.API.groupnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.groupnorm")],-1)),i[57]||(i[57]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[74]||(i[74]=e(`
    julia
    groupnorm(x, scale, bias, groups::Int, σ::F=identity,
+    epsilon::Real=eps(eltype(x)) ^ (5 // 7))
    Group Normalization. For details see [1].
    This op is similar to batch normalization, but statistics are shared across equally-sized groups of channels and not shared across batch dimension. Thus, group normalization does not depend on the batch composition and does not require maintaining internal state for storing statistics.
    Arguments
    `,4)),t("ul",null,[i[72]||(i[72]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[60]||(i[60]=t("code",null,"scale",-1)),i[61]||(i[61]=s(": Scale factor (")),t("mjx-container",C,[(o(),l("svg",A,i[58]||(i[58]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[59]||(i[59]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[62]||(i[62]=s(") (can be ")),i[63]||(i[63]=t("code",null,"nothing",-1)),i[64]||(i[64]=s(")"))])]),t("li",null,[t("p",null,[i[67]||(i[67]=t("code",null,"bias",-1)),i[68]||(i[68]=s(": Bias factor (")),t("mjx-container",j,[(o(),l("svg",H,i[65]||(i[65]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[66]||(i[66]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[69]||(i[69]=s(") (can be ")),i[70]||(i[70]=t("code",null,"nothing",-1)),i[71]||(i[71]=s(")"))])]),i[73]||(i[73]=e("
  • groups: Number of groups
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",3))]),i[75]||(i[75]=t("p",null,[t("strong",null,"Returns")],-1)),i[76]||(i[76]=t("p",null,"The normalized array is returned.",-1)),i[77]||(i[77]=t("p",null,[t("strong",null,"References")],-1)),i[78]||(i[78]=t("p",null,'[1] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European conference on computer vision (ECCV). 2018.',-1)),i[79]||(i[79]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/groupnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",D,[t("summary",null,[i[80]||(i[80]=t("a",{id:"LuxLib.API.instancenorm",href:"#LuxLib.API.instancenorm"},[t("span",{class:"jlbinding"},"LuxLib.API.instancenorm")],-1)),i[81]||(i[81]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=e(`
    julia
    instancenorm(x, scale, bias, training, act, epsilon = eps(eltype(x)) ^ (5 // 7))
 instancenorm(x, scale, bias, running_mean, running_var, training, act, momentum,
-    epsilon = eps(eltype(x)) ^ (5 // 7))
    Instance Normalization. For details see [1].
    `,2)),t("p",null,[i[84]||(i[84]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",M,[(o(),l("svg",B,i[82]||(i[82]=[e('',1)]))),i[83]||(i[83]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),i[85]||(i[85]=s(" input slice and normalises the input accordingly."))]),i[103]||(i[103]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[100]||(i[100]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized (must be atleast 3D)")])],-1)),t("li",null,[t("p",null,[i[88]||(i[88]=t("code",null,"scale",-1)),i[89]||(i[89]=s(": Scale factor (")),t("mjx-container",V,[(o(),l("svg",P,i[86]||(i[86]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[87]||(i[87]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[90]||(i[90]=s(") (can be ")),i[91]||(i[91]=t("code",null,"nothing",-1)),i[92]||(i[92]=s(")"))])]),t("li",null,[t("p",null,[i[95]||(i[95]=t("code",null,"bias",-1)),i[96]||(i[96]=s(": Bias factor (")),t("mjx-container",I,[(o(),l("svg",Z,i[93]||(i[93]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[94]||(i[94]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[97]||(i[97]=s(") (can be ")),i[98]||(i[98]=t("code",null,"nothing",-1)),i[99]||(i[99]=s(")"))])]),i[101]||(i[101]=e("
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
    • ",6))]),i[104]||(i[104]=t("p",null,[t("strong",null,"Returns")],-1)),i[105]||(i[105]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[106]||(i[106]=t("p",null,[t("strong",null,"References")],-1)),i[107]||(i[107]=t("p",null,'[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).',-1)),i[108]||(i[108]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/instancenorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",z,[t("summary",null,[i[109]||(i[109]=t("a",{id:"LuxLib.API.layernorm",href:"#LuxLib.API.layernorm"},[t("span",{class:"jlbinding"},"LuxLib.API.layernorm")],-1)),i[110]||(i[110]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[133]||(i[133]=e(`
    julia
    layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
-    epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}
    Layer Normalization. For details see [1].
    `,2)),t("p",null,[i[113]||(i[113]=s("Given an input array ")),t("mjx-container",N,[(o(),l("svg",R,i[111]||(i[111]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),i[112]||(i[112]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),i[114]||(i[114]=s(", this layer computes"))]),t("mjx-container",O,[(o(),l("svg",S,i[115]||(i[115]=[e('',1)]))),i[116]||(i[116]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),i[134]||(i[134]=t("p",null,[s("and applies the activation function "),t("code",null,"σ"),s(" elementwise to "),t("code",null,"y"),s(".")],-1)),i[135]||(i[135]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[131]||(i[131]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[119]||(i[119]=t("code",null,"scale",-1)),i[120]||(i[120]=s(": Scale factor (")),t("mjx-container",G,[(o(),l("svg",U,i[117]||(i[117]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[118]||(i[118]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[121]||(i[121]=s(") (can be ")),i[122]||(i[122]=t("code",null,"nothing",-1)),i[123]||(i[123]=s(")"))])]),t("li",null,[t("p",null,[i[126]||(i[126]=t("code",null,"bias",-1)),i[127]||(i[127]=s(": Bias factor (")),t("mjx-container",J,[(o(),l("svg",q,i[124]||(i[124]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[125]||(i[125]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[128]||(i[128]=s(") (can be ")),i[129]||(i[129]=t("code",null,"nothing",-1)),i[130]||(i[130]=s(")"))])]),i[132]||(i[132]=e("
  • σ: Activation function (default: identity)
  • dims: Dimensions along which the mean and std of x is computed. If nothing is passed, the dims are inferred based on the dimensions of scale and bias. For example, if x is N dimensional and scale and bias are M dimensional, then the dims will be 1:(N - M).
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",3))]),i[136]||(i[136]=t("p",null,[t("strong",null,"Returns")],-1)),i[137]||(i[137]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(".")],-1)),i[138]||(i[138]=t("p",null,[t("strong",null,"References")],-1)),i[139]||(i[139]=t("p",null,'[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).',-1)),i[140]||(i[140]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/layernorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),i[153]||(i[153]=t("h2",{id:"Helper-Functions",tabindex:"-1"},[s("Helper Functions "),t("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),t("details",X,[t("summary",null,[i[141]||(i[141]=t("a",{id:"LuxLib.internal_operation_mode",href:"#LuxLib.internal_operation_mode"},[t("span",{class:"jlbinding"},"LuxLib.internal_operation_mode")],-1)),i[142]||(i[142]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[143]||(i[143]=e(`
    julia
    internal_operation_mode(xs::Tuple)
-internal_operation_mode(x::AbstractArray)
    Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.
    Currently supported modes are:
    • GenericBroadcastOp: This is the fallback for most types. For the following types this is the preferred mode:
      • Arrays with fast_scalar_indexing set to False.
      • Static Arrays
      • ReverseDiff Arrays
      • Tracker Arrays
      • ForwardDiff.Dual Arrays
    • GPUBroadcastOp{dev}: GPU Arrays where dev is obtained from get_device_type(xs). This option dispatches should preferably use KernelAbstractions or specialized vendor dispatches.
    • LoopedArrayOp: CPU arrays that can be optimized using SIMD Loops, ideally using LoopVectorization.jl or Polyester.jl.
    source
    `,5))])])}const et=r(p,[["render",K]]);export{st as __pageData,et as default};
+    epsilon = eps(eltype(x)) ^ (5 // 7))

    Instance Normalization. For details see [1].

    `,2)),t("p",null,[i[84]||(i[84]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",M,[(o(),l("svg",B,i[82]||(i[82]=[e('',1)]))),i[83]||(i[83]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),i[85]||(i[85]=s(" input slice and normalises the input accordingly."))]),i[103]||(i[103]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[100]||(i[100]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized (must be atleast 3D)")])],-1)),t("li",null,[t("p",null,[i[88]||(i[88]=t("code",null,"scale",-1)),i[89]||(i[89]=s(": Scale factor (")),t("mjx-container",V,[(o(),l("svg",P,i[86]||(i[86]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[87]||(i[87]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[90]||(i[90]=s(") (can be ")),i[91]||(i[91]=t("code",null,"nothing",-1)),i[92]||(i[92]=s(")"))])]),t("li",null,[t("p",null,[i[95]||(i[95]=t("code",null,"bias",-1)),i[96]||(i[96]=s(": Bias factor (")),t("mjx-container",I,[(o(),l("svg",Z,i[93]||(i[93]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[94]||(i[94]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[97]||(i[97]=s(") (can be ")),i[98]||(i[98]=t("code",null,"nothing",-1)),i[99]||(i[99]=s(")"))])]),i[101]||(i[101]=e("

  • running_mean: Running mean (can be nothing)

  • running_var: Running variance (can be nothing)

  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context

  • σ: Activation function (default: identity)

  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))

  • momentum: Momentum for updating running mean and variance (default: 0.1f0)

    • ",6))]),i[104]||(i[104]=t("p",null,[t("strong",null,"Returns")],-1)),i[105]||(i[105]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[106]||(i[106]=t("p",null,[t("strong",null,"References")],-1)),i[107]||(i[107]=t("p",null,'[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).',-1)),i[108]||(i[108]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/instancenorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",z,[t("summary",null,[i[109]||(i[109]=t("a",{id:"LuxLib.API.layernorm",href:"#LuxLib.API.layernorm"},[t("span",{class:"jlbinding"},"LuxLib.API.layernorm")],-1)),i[110]||(i[110]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[133]||(i[133]=e(`
    julia
    layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
+    epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}

    Layer Normalization. For details see [1].

    `,2)),t("p",null,[i[113]||(i[113]=s("Given an input array ")),t("mjx-container",N,[(o(),l("svg",R,i[111]||(i[111]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),i[112]||(i[112]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),i[114]||(i[114]=s(", this layer computes"))]),t("mjx-container",O,[(o(),l("svg",S,i[115]||(i[115]=[e('',1)]))),i[116]||(i[116]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),i[134]||(i[134]=t("p",null,[s("and applies the activation function "),t("code",null,"σ"),s(" elementwise to "),t("code",null,"y"),s(".")],-1)),i[135]||(i[135]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[131]||(i[131]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[119]||(i[119]=t("code",null,"scale",-1)),i[120]||(i[120]=s(": Scale factor (")),t("mjx-container",G,[(o(),l("svg",U,i[117]||(i[117]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[118]||(i[118]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[121]||(i[121]=s(") (can be ")),i[122]||(i[122]=t("code",null,"nothing",-1)),i[123]||(i[123]=s(")"))])]),t("li",null,[t("p",null,[i[126]||(i[126]=t("code",null,"bias",-1)),i[127]||(i[127]=s(": Bias factor (")),t("mjx-container",J,[(o(),l("svg",q,i[124]||(i[124]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[125]||(i[125]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[128]||(i[128]=s(") (can be ")),i[129]||(i[129]=t("code",null,"nothing",-1)),i[130]||(i[130]=s(")"))])]),i[132]||(i[132]=e("

  • σ: Activation function (default: identity)

  • dims: Dimensions along which the mean and std of x is computed. If nothing is passed, the dims are inferred based on the dimensions of scale and bias. For example, if x is N dimensional and scale and bias are M dimensional, then the dims will be 1:(N - M).

  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))

    • ",3))]),i[136]||(i[136]=t("p",null,[t("strong",null,"Returns")],-1)),i[137]||(i[137]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(".")],-1)),i[138]||(i[138]=t("p",null,[t("strong",null,"References")],-1)),i[139]||(i[139]=t("p",null,'[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).',-1)),i[140]||(i[140]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/layernorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),i[153]||(i[153]=t("h2",{id:"Helper-Functions",tabindex:"-1"},[s("Helper Functions "),t("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),t("details",X,[t("summary",null,[i[141]||(i[141]=t("a",{id:"LuxLib.internal_operation_mode",href:"#LuxLib.internal_operation_mode"},[t("span",{class:"jlbinding"},"LuxLib.internal_operation_mode")],-1)),i[142]||(i[142]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[143]||(i[143]=e(`
    julia
    internal_operation_mode(xs::Tuple)
+internal_operation_mode(x::AbstractArray)

    Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.

    Currently supported modes are:

    source

    `,5))])])}const et=r(p,[["render",K]]);export{st as __pageData,et as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.DM8CkpLX.lean.js b/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.VqNljgWV.lean.js similarity index 72% rename from previews/PR1000/assets/api_Building_Blocks_LuxLib.md.DM8CkpLX.lean.js rename to previews/PR1000/assets/api_Building_Blocks_LuxLib.md.VqNljgWV.lean.js index 8d3487fe92..dc691c0fb6 100644 --- a/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.DM8CkpLX.lean.js +++ b/previews/PR1000/assets/api_Building_Blocks_LuxLib.md.VqNljgWV.lean.js @@ -1,10 +1,10 @@ -import{_ as r,c as l,j as t,a as s,G as n,a2 as e,B as d,o}from"./chunks/framework.DjZDIZsN.js";const st=JSON.parse('{"title":"LuxLib","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxLib.md","filePath":"api/Building_Blocks/LuxLib.md","lastUpdated":null}'),p={name:"api/Building_Blocks/LuxLib.md"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},x={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.07ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 11080.9 891","aria-hidden":"true"},L={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},w={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},F={class:"jldocstring custom-block"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},j={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},D={class:"jldocstring custom-block"},M={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},z={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},O={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},S={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},X={class:"jldocstring custom-block"};function K(W,i,$,Y,_,tt){const a=d("Badge");return o(),l("div",null,[i[144]||(i[144]=t("h1",{id:"LuxLib-API",tabindex:"-1"},[s("LuxLib "),t("a",{class:"header-anchor",href:"#LuxLib-API","aria-label":'Permalink to "LuxLib {#LuxLib-API}"'},"​")],-1)),i[145]||(i[145]=t("p",null,"Backend for Lux.jl",-1)),i[146]||(i[146]=t("h2",{id:"Apply-Activation",tabindex:"-1"},[s("Apply Activation "),t("a",{class:"header-anchor",href:"#Apply-Activation","aria-label":'Permalink to "Apply Activation {#Apply-Activation}"'},"​")],-1)),t("details",h,[t("summary",null,[i[0]||(i[0]=t("a",{id:"LuxLib.API.fast_activation",href:"#LuxLib.API.fast_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation")],-1)),i[1]||(i[1]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=e('
    julia
    fast_activation(σ::F, x::AbstractArray) where {F}

    Compute σ.(x) with the best possible implementation available. On CPUs we unroll the loop and use LoopVectorization.jl to vectorize the computation. On GPUs we use simply use broadcasting.

    Note

    This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.

    Arguments

    Returns

    source

    ',8))]),t("details",u,[t("summary",null,[i[3]||(i[3]=t("a",{id:"LuxLib.API.fast_activation!!",href:"#LuxLib.API.fast_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation!!")],-1)),i[4]||(i[4]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=e('
    julia
    fast_activation!!(σ::F, x::AbstractArray) where {F}

    Compute σ.(x) with the best possible implementation available. If it is possible to rewrite x in-place, it does so. If x is an immutable array, it falls back to the generic implementation.

    Note

    This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.

    Load SLEEFPirates.jl to get faster activations

    Certain activation functions are replaced with specialized implementations from SLEEFPirates.jl for FP32. This might lead to faster performance but can cause slight decrease in accuracy (in the floating point limit).

    Arguments

    Returns

    source

    ',9))]),i[147]||(i[147]=t("h2",{id:"Batched-Operations",tabindex:"-1"},[s("Batched Operations "),t("a",{class:"header-anchor",href:"#Batched-Operations","aria-label":'Permalink to "Batched Operations {#Batched-Operations}"'},"​")],-1)),t("details",Q,[t("summary",null,[i[6]||(i[6]=t("a",{id:"LuxLib.API.batched_matmul",href:"#LuxLib.API.batched_matmul"},[t("span",{class:"jlbinding"},"LuxLib.API.batched_matmul")],-1)),i[7]||(i[7]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
    julia
    batched_matmul(x, y)

    Computes the batched matrix multiplication of x and y. For more details see the NNlib documentation on NNlib.batched_mul. This function is mostly a wrapper around batched_mul but attempts to be faster on CPUs.

    Load LoopVectorization.jl to get faster batched matrix multiplication

    On CPUs loading LoopVectorization adds faster implementations of batched matrix multiplication.

    source

    ',4))]),i[148]||(i[148]=t("h2",{id:"Bias-Activation",tabindex:"-1"},[s("Bias Activation "),t("a",{class:"header-anchor",href:"#Bias-Activation","aria-label":'Permalink to "Bias Activation {#Bias-Activation}"'},"​")],-1)),t("details",m,[t("summary",null,[i[9]||(i[9]=t("a",{id:"LuxLib.API.bias_activation",href:"#LuxLib.API.bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation")],-1)),i[10]||(i[10]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
    julia
    bias_activation(σ, x, bias)

    Applies the activation function σ elementwise to the result of broadcasted addition of x and bias along the penultimate dimension. A vector x is treated as a matrix with a single last dimension.

    Arguments

    See also bias_activation!!, fast_activation.

    source

    ',6))]),t("details",k,[t("summary",null,[i[12]||(i[12]=t("a",{id:"LuxLib.API.bias_activation!!",href:"#LuxLib.API.bias_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation!!")],-1)),i[13]||(i[13]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
    julia
    bias_activation!!(σ, x, bias)

    Same as bias_activation but might update x in-place if possible. Users should not rely on x being mutated, it is recommended to use it like y = bias_activation!!(σ, x, bias). If x is updated in-place, y aliases x.

    See also bias_activation, fast_activation!!.

    source

    ',4))]),i[149]||(i[149]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",g,[t("summary",null,[i[15]||(i[15]=t("a",{id:"LuxLib.API.fused_conv_bias_activation",href:"#LuxLib.API.fused_conv_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_conv_bias_activation")],-1)),i[16]||(i[16]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e(`
    julia
    fused_conv_bias_activation(σ::F, weight::AbstractArray, x::AbstractArray,
-    b::Optional{<:AbstractVector}, cdims::ConvDims) where {F}

    Computes σ.(conv(x, weight, cdims) .+ b) (b is not exactly broadcasted like this, rather it is reshaped and broadcasted to the penultimate dimension) with the best possible implementation available. This operation fuses operations into a single kernel if possible, and minimizes reallocations by reusing the output buffer for multiple operations.

    Arguments

    Notes on implementation

    source

    `,7))]),i[150]||(i[150]=t("h2",{id:"dropout",tabindex:"-1"},[s("Dropout "),t("a",{class:"header-anchor",href:"#dropout","aria-label":'Permalink to "Dropout"'},"​")],-1)),t("details",c,[t("summary",null,[i[18]||(i[18]=t("a",{id:"LuxLib.API.alpha_dropout",href:"#LuxLib.API.alpha_dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.alpha_dropout")],-1)),i[19]||(i[19]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
    julia
    alpha_dropout(rng::AbstractRNG, x, p, training)\nalpha_dropout(rng::AbstractRNG, x, p, training, α, A, B)

    Alpha Dropout: Dropout ensuring that the mean and variance of the output remains same as the input. For details see [1]. Use the second call signature to avoid recomputing the constants for a fixed dropout probability.

    Arguments

    Returns

    References

    [1] Klambauer, Günter, et al. "Self-normalizing neural networks." Advances in neural information processing systems 30 (2017).

    source

    ',9))]),t("details",T,[t("summary",null,[i[21]||(i[21]=t("a",{id:"LuxLib.API.dropout",href:"#LuxLib.API.dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.dropout")],-1)),i[22]||(i[22]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
    julia
    dropout(rng::AbstractRNG, x, p, training, invp, dims)
+import{_ as r,c as l,j as t,a as s,G as n,a2 as e,B as d,o}from"./chunks/framework.DjZDIZsN.js";const st=JSON.parse('{"title":"LuxLib","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/LuxLib.md","filePath":"api/Building_Blocks/LuxLib.md","lastUpdated":null}'),p={name:"api/Building_Blocks/LuxLib.md"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},x={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.07ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 11080.9 891","aria-hidden":"true"},L={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},w={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},F={class:"jldocstring custom-block"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},j={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},D={class:"jldocstring custom-block"},M={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.471ex"},xmlns:"http://www.w3.org/2000/svg",width:"22.72ex",height:"2.016ex",role:"img",focusable:"false",viewBox:"0 -683 10042 891","aria-hidden":"true"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},P={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},Z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},z={class:"jldocstring custom-block"},N={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},R={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.294ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 572 453","aria-hidden":"true"},O={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},S={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.76ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.034ex",height:"6.063ex",role:"img",focusable:"false",viewBox:"0 -1460 11064.9 2680","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.281ex",height:"2.034ex",role:"img",focusable:"false",viewBox:"0 -705 566 899","aria-hidden":"true"},X={class:"jldocstring custom-block"};function K(W,i,$,Y,_,tt){const a=d("Badge");return o(),l("div",null,[i[144]||(i[144]=t("h1",{id:"LuxLib-API",tabindex:"-1"},[s("LuxLib "),t("a",{class:"header-anchor",href:"#LuxLib-API","aria-label":'Permalink to "LuxLib {#LuxLib-API}"'},"​")],-1)),i[145]||(i[145]=t("p",null,"Backend for Lux.jl",-1)),i[146]||(i[146]=t("h2",{id:"Apply-Activation",tabindex:"-1"},[s("Apply Activation "),t("a",{class:"header-anchor",href:"#Apply-Activation","aria-label":'Permalink to "Apply Activation {#Apply-Activation}"'},"​")],-1)),t("details",h,[t("summary",null,[i[0]||(i[0]=t("a",{id:"LuxLib.API.fast_activation",href:"#LuxLib.API.fast_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation")],-1)),i[1]||(i[1]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=e('
    julia
    fast_activation(σ::F, x::AbstractArray) where {F}
    Compute σ.(x) with the best possible implementation available. On CPUs we unroll the loop and use LoopVectorization.jl to vectorize the computation. On GPUs we use simply use broadcasting.
    Note
    This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.
    Arguments
    • σ: Activation function
    • x: Input array
    Returns
    • Output Array with the same size as x
    source
    ',8))]),t("details",u,[t("summary",null,[i[3]||(i[3]=t("a",{id:"LuxLib.API.fast_activation!!",href:"#LuxLib.API.fast_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.fast_activation!!")],-1)),i[4]||(i[4]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=e('
    julia
    fast_activation!!(σ::F, x::AbstractArray) where {F}
    Compute σ.(x) with the best possible implementation available. If it is possible to rewrite x in-place, it does so. If x is an immutable array, it falls back to the generic implementation.
    Note
    This function doesn't replace σ with NNlib.fast_act(σ, ...), that needs to be done by the user if needed.
    Load SLEEFPirates.jl to get faster activations
    Certain activation functions are replaced with specialized implementations from SLEEFPirates.jl for FP32. This might lead to faster performance but can cause slight decrease in accuracy (in the floating point limit).
    Arguments
    • σ: Activation function
    • x: Input array
    Returns
    • Output Array with the same size as x
    source
    ',9))]),i[147]||(i[147]=t("h2",{id:"Batched-Operations",tabindex:"-1"},[s("Batched Operations "),t("a",{class:"header-anchor",href:"#Batched-Operations","aria-label":'Permalink to "Batched Operations {#Batched-Operations}"'},"​")],-1)),t("details",Q,[t("summary",null,[i[6]||(i[6]=t("a",{id:"LuxLib.API.batched_matmul",href:"#LuxLib.API.batched_matmul"},[t("span",{class:"jlbinding"},"LuxLib.API.batched_matmul")],-1)),i[7]||(i[7]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
    julia
    batched_matmul(x, y)
    Computes the batched matrix multiplication of x and y. For more details see the NNlib documentation on NNlib.batched_mul. This function is mostly a wrapper around batched_mul but attempts to be faster on CPUs.
    Load LoopVectorization.jl to get faster batched matrix multiplication
    On CPUs loading LoopVectorization adds faster implementations of batched matrix multiplication.
    source
    ',4))]),i[148]||(i[148]=t("h2",{id:"Bias-Activation",tabindex:"-1"},[s("Bias Activation "),t("a",{class:"header-anchor",href:"#Bias-Activation","aria-label":'Permalink to "Bias Activation {#Bias-Activation}"'},"​")],-1)),t("details",m,[t("summary",null,[i[9]||(i[9]=t("a",{id:"LuxLib.API.bias_activation",href:"#LuxLib.API.bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation")],-1)),i[10]||(i[10]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
    julia
    bias_activation(σ, x, bias)
    Applies the activation function σ elementwise to the result of broadcasted addition of x and bias along the penultimate dimension. A vector x is treated as a matrix with a single last dimension.
    Arguments
    • σ: Activation function
    • x: Input to be transformed
    • bias: Bias to be added. Can be nothing.
    See also bias_activation!!, fast_activation.
    source
    ',6))]),t("details",k,[t("summary",null,[i[12]||(i[12]=t("a",{id:"LuxLib.API.bias_activation!!",href:"#LuxLib.API.bias_activation!!"},[t("span",{class:"jlbinding"},"LuxLib.API.bias_activation!!")],-1)),i[13]||(i[13]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
    julia
    bias_activation!!(σ, x, bias)
    Same as bias_activation but might update x in-place if possible. Users should not rely on x being mutated, it is recommended to use it like y = bias_activation!!(σ, x, bias). If x is updated in-place, y aliases x.
    See also bias_activation, fast_activation!!.
    source
    ',4))]),i[149]||(i[149]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[i[15]||(i[15]=t("a",{id:"LuxLib.API.fused_conv_bias_activation",href:"#LuxLib.API.fused_conv_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_conv_bias_activation")],-1)),i[16]||(i[16]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e(`
    julia
    fused_conv_bias_activation(σ::F, weight::AbstractArray, x::AbstractArray,
+    b::Optional{<:AbstractVector}, cdims::ConvDims) where {F}
    Computes σ.(conv(x, weight, cdims) .+ b) (b is not exactly broadcasted like this, rather it is reshaped and broadcasted to the penultimate dimension) with the best possible implementation available. This operation fuses operations into a single kernel if possible, and minimizes reallocations by reusing the output buffer for multiple operations.
    Arguments
    • σ: Activation function
    • weight: Weight tensor
    • x: Input tensor
    • b: Bias tensor (can be nothing)
    • cdims: ConvDims object
    Notes on implementation
    • For CUDA Arrays, this uses fused CUDNN kernels when the activation is identity or relu. For other activations, it tries to fuse the operations on the Julia side.
    • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
    • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
    • For Mixed-Precision Inputs on GPU, we type promote the inputs to the highest precision, with a warning.
    source
    `,7))]),i[150]||(i[150]=t("h2",{id:"dropout",tabindex:"-1"},[s("Dropout "),t("a",{class:"header-anchor",href:"#dropout","aria-label":'Permalink to "Dropout"'},"​")],-1)),t("details",g,[t("summary",null,[i[18]||(i[18]=t("a",{id:"LuxLib.API.alpha_dropout",href:"#LuxLib.API.alpha_dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.alpha_dropout")],-1)),i[19]||(i[19]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
    julia
    alpha_dropout(rng::AbstractRNG, x, p, training)\nalpha_dropout(rng::AbstractRNG, x, p, training, α, A, B)
    Alpha Dropout: Dropout ensuring that the mean and variance of the output remains same as the input. For details see [1]. Use the second call signature to avoid recomputing the constants for a fixed dropout probability.
    Arguments
    • rng: Random number generator
    • x: Input Array
    • p: Probability of an element to be dropped out
    • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context`
    • α: -1.7580993408473766. Computed at limit x tends to infinity, selu(x) = -λβ = α
    • A: Scaling factor for the mean
    • B: Scaling factor for the variance
    Returns
    • Output Array after applying alpha dropout
    • Updated state for the random number generator
    References
    [1] Klambauer, Günter, et al. "Self-normalizing neural networks." Advances in neural information processing systems 30 (2017).
    source
    ',9))]),t("details",T,[t("summary",null,[i[21]||(i[21]=t("a",{id:"LuxLib.API.dropout",href:"#LuxLib.API.dropout"},[t("span",{class:"jlbinding"},"LuxLib.API.dropout")],-1)),i[22]||(i[22]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
    julia
    dropout(rng::AbstractRNG, x, p, training, invp, dims)
 dropout(rng::AbstractRNG, x, mask, p, training, update_mask::Union{Val, StaticBool},
-    invp, dims)
    Dropout: Simple Way to prevent Neural Networks for Overfitting. For details see [1].
    Arguments
    • rng: Random number generator
    • x: Input Array
    • mask: Dropout Mask. If not used then it is constructed automatically
    • p: Probability of an element to be dropped out
    • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
    • update_mask: If Val(true) or True() then the mask is generated and used. Else, the mask provided is directly used
    • invp: Inverse multiplied to the mask. Calculated as invp = 1 / (1 - p).
    Returns
    • Output Array after applying dropout
    • Dropout Mask (if training == false, the returned value is meaningless)
    • Updated state for the random number generator
    References
    [1] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The journal of machine learning research 15.1 (2014): 1929-1958.
    source
    `,9))]),i[151]||(i[151]=t("h2",{id:"Fully-Connected-Layers",tabindex:"-1"},[s("Fully Connected Layers "),t("a",{class:"header-anchor",href:"#Fully-Connected-Layers","aria-label":'Permalink to "Fully Connected Layers {#Fully-Connected-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[i[24]||(i[24]=t("a",{id:"LuxLib.API.fused_dense_bias_activation",href:"#LuxLib.API.fused_dense_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_dense_bias_activation")],-1)),i[25]||(i[25]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e('
    julia
    fused_dense_bias_activation(σ::F, weight::AbstractMatrix, x::AbstractMatrix,\n    b::Optional{<:AbstractVector}) where {F}
    Compute σ.(weight * x .+ b) with the best possible implementation available. Currently this implementation attempts to minimize reallocations by reusing the output buffer for multiple operations.
    Arguments
    • σ: Activation function
    • weight: Weight matrix
    • x: Input matrix
    • b: Bias vector (can be nothing)
    Notes on implementation
    • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
    • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
    • For CUDA Arrays, this uses a special fused implementation via cuBLASLt.
    • For small CPU Arrays, we use LoopVectorization.jl. On x86_64 we use Octavian for medium sized matrices. This is overridden if special BLAS implementations are loaded (currently MKL, AppleAccelerate, and BLISBLAS).
    !!! tip "Load Octavian.jl
    Loading `Octavian.jl` enables a polyalgorithm that uses different backends based on the\ninput sizes.
    source
    ',9))]),i[152]||(i[152]=t("h2",{id:"normalization",tabindex:"-1"},[s("Normalization "),t("a",{class:"header-anchor",href:"#normalization","aria-label":'Permalink to "Normalization"'},"​")],-1)),t("details",y,[t("summary",null,[i[27]||(i[27]=t("a",{id:"LuxLib.API.batchnorm",href:"#LuxLib.API.batchnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.batchnorm")],-1)),i[28]||(i[28]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[49]||(i[49]=e(`
    julia
    batchnorm(x, scale, bias, running_mean, running_var, training,
-    σ=identity, momentum = 0.1f0, epsilon = eps(eltype(x)) ^ (5 // 7))
    Batch Normalization. For details see [1].
    `,2)),t("p",null,[i[31]||(i[31]=s("Batch Normalization computes the mean and variance for each ")),t("mjx-container",x,[(o(),l("svg",f,i[29]||(i[29]=[e('',1)]))),i[30]||(i[30]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),i[32]||(i[32]=s(" input slice and normalises the input accordingly."))]),i[50]||(i[50]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[47]||(i[47]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[35]||(i[35]=t("code",null,"scale",-1)),i[36]||(i[36]=s(": Scale factor (")),t("mjx-container",L,[(o(),l("svg",v,i[33]||(i[33]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[34]||(i[34]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[37]||(i[37]=s(") (can be ")),i[38]||(i[38]=t("code",null,"nothing",-1)),i[39]||(i[39]=s(")"))])]),t("li",null,[t("p",null,[i[42]||(i[42]=t("code",null,"bias",-1)),i[43]||(i[43]=s(": Bias factor (")),t("mjx-container",E,[(o(),l("svg",w,i[40]||(i[40]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[41]||(i[41]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[44]||(i[44]=s(") (can be ")),i[45]||(i[45]=t("code",null,"nothing",-1)),i[46]||(i[46]=s(")"))])]),i[48]||(i[48]=e("
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",6))]),i[51]||(i[51]=t("p",null,[t("strong",null,"Returns")],-1)),i[52]||(i[52]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[53]||(i[53]=t("p",null,[t("strong",null,"References")],-1)),i[54]||(i[54]=t("p",null,'[1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International conference on machine learning. PMLR, 2015.',-1)),i[55]||(i[55]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/batchnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",F,[t("summary",null,[i[56]||(i[56]=t("a",{id:"LuxLib.API.groupnorm",href:"#LuxLib.API.groupnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.groupnorm")],-1)),i[57]||(i[57]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[74]||(i[74]=e(`
    julia
    groupnorm(x, scale, bias, groups::Int, σ::F=identity,
-    epsilon::Real=eps(eltype(x)) ^ (5 // 7))
    Group Normalization. For details see [1].
    This op is similar to batch normalization, but statistics are shared across equally-sized groups of channels and not shared across batch dimension. Thus, group normalization does not depend on the batch composition and does not require maintaining internal state for storing statistics.
    Arguments
    `,4)),t("ul",null,[i[72]||(i[72]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[60]||(i[60]=t("code",null,"scale",-1)),i[61]||(i[61]=s(": Scale factor (")),t("mjx-container",C,[(o(),l("svg",A,i[58]||(i[58]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[59]||(i[59]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[62]||(i[62]=s(") (can be ")),i[63]||(i[63]=t("code",null,"nothing",-1)),i[64]||(i[64]=s(")"))])]),t("li",null,[t("p",null,[i[67]||(i[67]=t("code",null,"bias",-1)),i[68]||(i[68]=s(": Bias factor (")),t("mjx-container",j,[(o(),l("svg",H,i[65]||(i[65]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[66]||(i[66]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[69]||(i[69]=s(") (can be ")),i[70]||(i[70]=t("code",null,"nothing",-1)),i[71]||(i[71]=s(")"))])]),i[73]||(i[73]=e("
  • groups: Number of groups
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",3))]),i[75]||(i[75]=t("p",null,[t("strong",null,"Returns")],-1)),i[76]||(i[76]=t("p",null,"The normalized array is returned.",-1)),i[77]||(i[77]=t("p",null,[t("strong",null,"References")],-1)),i[78]||(i[78]=t("p",null,'[1] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European conference on computer vision (ECCV). 2018.',-1)),i[79]||(i[79]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/groupnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",D,[t("summary",null,[i[80]||(i[80]=t("a",{id:"LuxLib.API.instancenorm",href:"#LuxLib.API.instancenorm"},[t("span",{class:"jlbinding"},"LuxLib.API.instancenorm")],-1)),i[81]||(i[81]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=e(`
    julia
    instancenorm(x, scale, bias, training, act, epsilon = eps(eltype(x)) ^ (5 // 7))
+    invp, dims)
    Dropout: Simple Way to prevent Neural Networks for Overfitting. For details see [1].
    Arguments
    • rng: Random number generator
    • x: Input Array
    • mask: Dropout Mask. If not used then it is constructed automatically
    • p: Probability of an element to be dropped out
    • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
    • update_mask: If Val(true) or True() then the mask is generated and used. Else, the mask provided is directly used
    • invp: Inverse multiplied to the mask. Calculated as invp = 1 / (1 - p).
    Returns
    • Output Array after applying dropout
    • Dropout Mask (if training == false, the returned value is meaningless)
    • Updated state for the random number generator
    References
    [1] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." The journal of machine learning research 15.1 (2014): 1929-1958.
    source
    `,9))]),i[151]||(i[151]=t("h2",{id:"Fully-Connected-Layers",tabindex:"-1"},[s("Fully Connected Layers "),t("a",{class:"header-anchor",href:"#Fully-Connected-Layers","aria-label":'Permalink to "Fully Connected Layers {#Fully-Connected-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[i[24]||(i[24]=t("a",{id:"LuxLib.API.fused_dense_bias_activation",href:"#LuxLib.API.fused_dense_bias_activation"},[t("span",{class:"jlbinding"},"LuxLib.API.fused_dense_bias_activation")],-1)),i[25]||(i[25]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e('
    julia
    fused_dense_bias_activation(σ::F, weight::AbstractMatrix, x::AbstractMatrix,\n    b::Optional{<:AbstractVector}) where {F}
    Compute σ.(weight * x .+ b) with the best possible implementation available. Currently this implementation attempts to minimize reallocations by reusing the output buffer for multiple operations.
    Arguments
    • σ: Activation function
    • weight: Weight matrix
    • x: Input matrix
    • b: Bias vector (can be nothing)
    Notes on implementation
    • If any of the inputs, don't support setindexing (aka immutable arrays) we fallback to the generic non-mutating implementation.
    • Maximum memory reuse and operation fusion is guaranteed for ChainRules compatible AD backends or backends that support mutation. Backends like Tracker and ReverseDiff fallback to the generic implementation.
    • For CUDA Arrays, this uses a special fused implementation via cuBLASLt.
    • For small CPU Arrays, we use LoopVectorization.jl. On x86_64 we use Octavian for medium sized matrices. This is overridden if special BLAS implementations are loaded (currently MKL, AppleAccelerate, and BLISBLAS).
    !!! tip "Load Octavian.jl
    Loading `Octavian.jl` enables a polyalgorithm that uses different backends based on the\ninput sizes.
    source
    ',9))]),i[152]||(i[152]=t("h2",{id:"normalization",tabindex:"-1"},[s("Normalization "),t("a",{class:"header-anchor",href:"#normalization","aria-label":'Permalink to "Normalization"'},"​")],-1)),t("details",y,[t("summary",null,[i[27]||(i[27]=t("a",{id:"LuxLib.API.batchnorm",href:"#LuxLib.API.batchnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.batchnorm")],-1)),i[28]||(i[28]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[49]||(i[49]=e(`
    julia
    batchnorm(x, scale, bias, running_mean, running_var, training,
+    σ=identity, momentum = 0.1f0, epsilon = eps(eltype(x)) ^ (5 // 7))
    Batch Normalization. For details see [1].
    `,2)),t("p",null,[i[31]||(i[31]=s("Batch Normalization computes the mean and variance for each ")),t("mjx-container",x,[(o(),l("svg",f,i[29]||(i[29]=[e('',1)]))),i[30]||(i[30]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),i[32]||(i[32]=s(" input slice and normalises the input accordingly."))]),i[50]||(i[50]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[47]||(i[47]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[35]||(i[35]=t("code",null,"scale",-1)),i[36]||(i[36]=s(": Scale factor (")),t("mjx-container",L,[(o(),l("svg",E,i[33]||(i[33]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[34]||(i[34]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[37]||(i[37]=s(") (can be ")),i[38]||(i[38]=t("code",null,"nothing",-1)),i[39]||(i[39]=s(")"))])]),t("li",null,[t("p",null,[i[42]||(i[42]=t("code",null,"bias",-1)),i[43]||(i[43]=s(": Bias factor (")),t("mjx-container",v,[(o(),l("svg",w,i[40]||(i[40]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[41]||(i[41]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[44]||(i[44]=s(") (can be ")),i[45]||(i[45]=t("code",null,"nothing",-1)),i[46]||(i[46]=s(")"))])]),i[48]||(i[48]=e("
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",6))]),i[51]||(i[51]=t("p",null,[t("strong",null,"Returns")],-1)),i[52]||(i[52]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[53]||(i[53]=t("p",null,[t("strong",null,"References")],-1)),i[54]||(i[54]=t("p",null,'[1] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International conference on machine learning. PMLR, 2015.',-1)),i[55]||(i[55]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/batchnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",F,[t("summary",null,[i[56]||(i[56]=t("a",{id:"LuxLib.API.groupnorm",href:"#LuxLib.API.groupnorm"},[t("span",{class:"jlbinding"},"LuxLib.API.groupnorm")],-1)),i[57]||(i[57]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[74]||(i[74]=e(`
    julia
    groupnorm(x, scale, bias, groups::Int, σ::F=identity,
+    epsilon::Real=eps(eltype(x)) ^ (5 // 7))
    Group Normalization. For details see [1].
    This op is similar to batch normalization, but statistics are shared across equally-sized groups of channels and not shared across batch dimension. Thus, group normalization does not depend on the batch composition and does not require maintaining internal state for storing statistics.
    Arguments
    `,4)),t("ul",null,[i[72]||(i[72]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[60]||(i[60]=t("code",null,"scale",-1)),i[61]||(i[61]=s(": Scale factor (")),t("mjx-container",C,[(o(),l("svg",A,i[58]||(i[58]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[59]||(i[59]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[62]||(i[62]=s(") (can be ")),i[63]||(i[63]=t("code",null,"nothing",-1)),i[64]||(i[64]=s(")"))])]),t("li",null,[t("p",null,[i[67]||(i[67]=t("code",null,"bias",-1)),i[68]||(i[68]=s(": Bias factor (")),t("mjx-container",j,[(o(),l("svg",H,i[65]||(i[65]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[66]||(i[66]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[69]||(i[69]=s(") (can be ")),i[70]||(i[70]=t("code",null,"nothing",-1)),i[71]||(i[71]=s(")"))])]),i[73]||(i[73]=e("
  • groups: Number of groups
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",3))]),i[75]||(i[75]=t("p",null,[t("strong",null,"Returns")],-1)),i[76]||(i[76]=t("p",null,"The normalized array is returned.",-1)),i[77]||(i[77]=t("p",null,[t("strong",null,"References")],-1)),i[78]||(i[78]=t("p",null,'[1] Wu, Yuxin, and Kaiming He. "Group normalization." Proceedings of the European conference on computer vision (ECCV). 2018.',-1)),i[79]||(i[79]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/groupnorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",D,[t("summary",null,[i[80]||(i[80]=t("a",{id:"LuxLib.API.instancenorm",href:"#LuxLib.API.instancenorm"},[t("span",{class:"jlbinding"},"LuxLib.API.instancenorm")],-1)),i[81]||(i[81]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=e(`
    julia
    instancenorm(x, scale, bias, training, act, epsilon = eps(eltype(x)) ^ (5 // 7))
 instancenorm(x, scale, bias, running_mean, running_var, training, act, momentum,
-    epsilon = eps(eltype(x)) ^ (5 // 7))
    Instance Normalization. For details see [1].
    `,2)),t("p",null,[i[84]||(i[84]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",M,[(o(),l("svg",B,i[82]||(i[82]=[e('',1)]))),i[83]||(i[83]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),i[85]||(i[85]=s(" input slice and normalises the input accordingly."))]),i[103]||(i[103]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[100]||(i[100]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized (must be atleast 3D)")])],-1)),t("li",null,[t("p",null,[i[88]||(i[88]=t("code",null,"scale",-1)),i[89]||(i[89]=s(": Scale factor (")),t("mjx-container",V,[(o(),l("svg",P,i[86]||(i[86]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[87]||(i[87]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[90]||(i[90]=s(") (can be ")),i[91]||(i[91]=t("code",null,"nothing",-1)),i[92]||(i[92]=s(")"))])]),t("li",null,[t("p",null,[i[95]||(i[95]=t("code",null,"bias",-1)),i[96]||(i[96]=s(": Bias factor (")),t("mjx-container",I,[(o(),l("svg",Z,i[93]||(i[93]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[94]||(i[94]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[97]||(i[97]=s(") (can be ")),i[98]||(i[98]=t("code",null,"nothing",-1)),i[99]||(i[99]=s(")"))])]),i[101]||(i[101]=e("
  • running_mean: Running mean (can be nothing)
  • running_var: Running variance (can be nothing)
  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context
  • σ: Activation function (default: identity)
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
  • momentum: Momentum for updating running mean and variance (default: 0.1f0)
    • ",6))]),i[104]||(i[104]=t("p",null,[t("strong",null,"Returns")],-1)),i[105]||(i[105]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[106]||(i[106]=t("p",null,[t("strong",null,"References")],-1)),i[107]||(i[107]=t("p",null,'[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).',-1)),i[108]||(i[108]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/instancenorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",z,[t("summary",null,[i[109]||(i[109]=t("a",{id:"LuxLib.API.layernorm",href:"#LuxLib.API.layernorm"},[t("span",{class:"jlbinding"},"LuxLib.API.layernorm")],-1)),i[110]||(i[110]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[133]||(i[133]=e(`
    julia
    layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
-    epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}
    Layer Normalization. For details see [1].
    `,2)),t("p",null,[i[113]||(i[113]=s("Given an input array ")),t("mjx-container",N,[(o(),l("svg",R,i[111]||(i[111]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),i[112]||(i[112]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),i[114]||(i[114]=s(", this layer computes"))]),t("mjx-container",O,[(o(),l("svg",S,i[115]||(i[115]=[e('',1)]))),i[116]||(i[116]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),i[134]||(i[134]=t("p",null,[s("and applies the activation function "),t("code",null,"σ"),s(" elementwise to "),t("code",null,"y"),s(".")],-1)),i[135]||(i[135]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[131]||(i[131]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[119]||(i[119]=t("code",null,"scale",-1)),i[120]||(i[120]=s(": Scale factor (")),t("mjx-container",G,[(o(),l("svg",U,i[117]||(i[117]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[118]||(i[118]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[121]||(i[121]=s(") (can be ")),i[122]||(i[122]=t("code",null,"nothing",-1)),i[123]||(i[123]=s(")"))])]),t("li",null,[t("p",null,[i[126]||(i[126]=t("code",null,"bias",-1)),i[127]||(i[127]=s(": Bias factor (")),t("mjx-container",J,[(o(),l("svg",q,i[124]||(i[124]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[125]||(i[125]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[128]||(i[128]=s(") (can be ")),i[129]||(i[129]=t("code",null,"nothing",-1)),i[130]||(i[130]=s(")"))])]),i[132]||(i[132]=e("
  • σ: Activation function (default: identity)
  • dims: Dimensions along which the mean and std of x is computed. If nothing is passed, the dims are inferred based on the dimensions of scale and bias. For example, if x is N dimensional and scale and bias are M dimensional, then the dims will be 1:(N - M).
  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))
    • ",3))]),i[136]||(i[136]=t("p",null,[t("strong",null,"Returns")],-1)),i[137]||(i[137]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(".")],-1)),i[138]||(i[138]=t("p",null,[t("strong",null,"References")],-1)),i[139]||(i[139]=t("p",null,'[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).',-1)),i[140]||(i[140]=t("p",null,[t("a",{href:"https://github.com/LuxDL/LuxLib.jl/blob/v1.3.6/src/api/layernorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),i[153]||(i[153]=t("h2",{id:"Helper-Functions",tabindex:"-1"},[s("Helper Functions "),t("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),t("details",X,[t("summary",null,[i[141]||(i[141]=t("a",{id:"LuxLib.internal_operation_mode",href:"#LuxLib.internal_operation_mode"},[t("span",{class:"jlbinding"},"LuxLib.internal_operation_mode")],-1)),i[142]||(i[142]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[143]||(i[143]=e(`
    julia
    internal_operation_mode(xs::Tuple)
-internal_operation_mode(x::AbstractArray)
    Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.
    Currently supported modes are:
    • GenericBroadcastOp: This is the fallback for most types. For the following types this is the preferred mode:
      • Arrays with fast_scalar_indexing set to False.
      • Static Arrays
      • ReverseDiff Arrays
      • Tracker Arrays
      • ForwardDiff.Dual Arrays
    • GPUBroadcastOp{dev}: GPU Arrays where dev is obtained from get_device_type(xs). This option dispatches should preferably use KernelAbstractions or specialized vendor dispatches.
    • LoopedArrayOp: CPU arrays that can be optimized using SIMD Loops, ideally using LoopVectorization.jl or Polyester.jl.
    source
    `,5))])])}const et=r(p,[["render",K]]);export{st as __pageData,et as default};
+    epsilon = eps(eltype(x)) ^ (5 // 7))

    Instance Normalization. For details see [1].

    `,2)),t("p",null,[i[84]||(i[84]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",M,[(o(),l("svg",B,i[82]||(i[82]=[e('',1)]))),i[83]||(i[83]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),i[85]||(i[85]=s(" input slice and normalises the input accordingly."))]),i[103]||(i[103]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[100]||(i[100]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized (must be atleast 3D)")])],-1)),t("li",null,[t("p",null,[i[88]||(i[88]=t("code",null,"scale",-1)),i[89]||(i[89]=s(": Scale factor (")),t("mjx-container",V,[(o(),l("svg",P,i[86]||(i[86]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[87]||(i[87]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[90]||(i[90]=s(") (can be ")),i[91]||(i[91]=t("code",null,"nothing",-1)),i[92]||(i[92]=s(")"))])]),t("li",null,[t("p",null,[i[95]||(i[95]=t("code",null,"bias",-1)),i[96]||(i[96]=s(": Bias factor (")),t("mjx-container",I,[(o(),l("svg",Z,i[93]||(i[93]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[94]||(i[94]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[97]||(i[97]=s(") (can be ")),i[98]||(i[98]=t("code",null,"nothing",-1)),i[99]||(i[99]=s(")"))])]),i[101]||(i[101]=e("

  • running_mean: Running mean (can be nothing)

  • running_var: Running variance (can be nothing)

  • training: Set to Val(true) or True() if running in training mode. Can be set to nothing to automatically determine if the function is being called within an autodiff context

  • σ: Activation function (default: identity)

  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))

  • momentum: Momentum for updating running mean and variance (default: 0.1f0)

    • ",6))]),i[104]||(i[104]=t("p",null,[t("strong",null,"Returns")],-1)),i[105]||(i[105]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(". And a Named Tuple containing the updated running mean and variance.")],-1)),i[106]||(i[106]=t("p",null,[t("strong",null,"References")],-1)),i[107]||(i[107]=t("p",null,'[1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).',-1)),i[108]||(i[108]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/instancenorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",z,[t("summary",null,[i[109]||(i[109]=t("a",{id:"LuxLib.API.layernorm",href:"#LuxLib.API.layernorm"},[t("span",{class:"jlbinding"},"LuxLib.API.layernorm")],-1)),i[110]||(i[110]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[133]||(i[133]=e(`
    julia
    layernorm(x::AbstractArray{xT, N}, scale, bias, σ = identity, dims=1:(N - 1),
+    epsilon = eps(eltype(x)) ^ (5 / 7)) where {xT, N}

    Layer Normalization. For details see [1].

    `,2)),t("p",null,[i[113]||(i[113]=s("Given an input array ")),t("mjx-container",N,[(o(),l("svg",R,i[111]||(i[111]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),i[112]||(i[112]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),i[114]||(i[114]=s(", this layer computes"))]),t("mjx-container",O,[(o(),l("svg",S,i[115]||(i[115]=[e('',1)]))),i[116]||(i[116]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),i[134]||(i[134]=t("p",null,[s("and applies the activation function "),t("code",null,"σ"),s(" elementwise to "),t("code",null,"y"),s(".")],-1)),i[135]||(i[135]=t("p",null,[t("strong",null,"Arguments")],-1)),t("ul",null,[i[131]||(i[131]=t("li",null,[t("p",null,[t("code",null,"x"),s(": Input to be Normalized")])],-1)),t("li",null,[t("p",null,[i[119]||(i[119]=t("code",null,"scale",-1)),i[120]||(i[120]=s(": Scale factor (")),t("mjx-container",G,[(o(),l("svg",U,i[117]||(i[117]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[118]||(i[118]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),i[121]||(i[121]=s(") (can be ")),i[122]||(i[122]=t("code",null,"nothing",-1)),i[123]||(i[123]=s(")"))])]),t("li",null,[t("p",null,[i[126]||(i[126]=t("code",null,"bias",-1)),i[127]||(i[127]=s(": Bias factor (")),t("mjx-container",J,[(o(),l("svg",q,i[124]||(i[124]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),i[125]||(i[125]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),i[128]||(i[128]=s(") (can be ")),i[129]||(i[129]=t("code",null,"nothing",-1)),i[130]||(i[130]=s(")"))])]),i[132]||(i[132]=e("

  • σ: Activation function (default: identity)

  • dims: Dimensions along which the mean and std of x is computed. If nothing is passed, the dims are inferred based on the dimensions of scale and bias. For example, if x is N dimensional and scale and bias are M dimensional, then the dims will be 1:(N - M).

  • epsilon: Value added to the denominator for numerical stability (default: eps(eltype(x)) ^ (5 / 7))

    • ",3))]),i[136]||(i[136]=t("p",null,[t("strong",null,"Returns")],-1)),i[137]||(i[137]=t("p",null,[s("Normalized Array of same size as "),t("code",null,"x"),s(".")],-1)),i[138]||(i[138]=t("p",null,[t("strong",null,"References")],-1)),i[139]||(i[139]=t("p",null,'[1] Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer normalization." arXiv preprint arXiv:1607.06450 (2016).',-1)),i[140]||(i[140]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/LuxLib/src/api/layernorm.jl#L1",target:"_blank",rel:"noreferrer"},"source")],-1))]),i[153]||(i[153]=t("h2",{id:"Helper-Functions",tabindex:"-1"},[s("Helper Functions "),t("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),t("details",X,[t("summary",null,[i[141]||(i[141]=t("a",{id:"LuxLib.internal_operation_mode",href:"#LuxLib.internal_operation_mode"},[t("span",{class:"jlbinding"},"LuxLib.internal_operation_mode")],-1)),i[142]||(i[142]=s()),n(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[143]||(i[143]=e(`
    julia
    internal_operation_mode(xs::Tuple)
+internal_operation_mode(x::AbstractArray)

    Returns the internal operation mode for the given array(s). This is useful to define custom implementations using different backends like simple Julia broadcasting, Kernel Abstractions, Loop Vectorization, etc.

    Currently supported modes are:

    source

    `,5))])])}const et=r(p,[["render",K]]);export{st as __pageData,et as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.jFCbAP4G.js b/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.C7zduFBi.js similarity index 54% rename from previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.jFCbAP4G.js rename to previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.C7zduFBi.js index acb1fd2d84..0a2a73daa6 100644 --- a/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.jFCbAP4G.js +++ b/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.C7zduFBi.js @@ -1,6 +1,6 @@ -import{_ as r,c as l,a2 as t,j as s,a,G as e,B as p,o as h}from"./chunks/framework.DjZDIZsN.js";const Y=JSON.parse('{"title":"WeightInitializers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/WeightInitializers.md","filePath":"api/Building_Blocks/WeightInitializers.md","lastUpdated":null}'),k={name:"api/Building_Blocks/WeightInitializers.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.612ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2922.7 1000","aria-hidden":"true"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},z={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},I={class:"jldocstring custom-block"},W={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},R={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},N={class:"jldocstring custom-block"},G={class:"jldocstring custom-block"},U={class:"jldocstring custom-block"},O={class:"jldocstring custom-block"},P={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},q={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"};function V(S,i,X,Z,$,J){const n=p("Badge");return h(),l("div",null,[i[109]||(i[109]=t('

    WeightInitializers ​

    This package is a light dependency providing common weight initialization schemes for deep learning models.

    Supported RNG Types ​

    RNG Type / PackageReturned Array TypeUnsupported Functions
    Random.jlArray
    StableRNGs.jlArray
    CUDA.default_rng()CuArray
    GPUArrays.default_rng(CuArray)CuArray
    AMDGPU.rocrand_rng()ROCArray
    AMDGPU.gpuarrays_rng()ROCArray
    GPUArrays.default_rng(ROCArray)ROCArray
    Metal.gpuarrays_rng()MtlArrayorthogonal, truncated_normal
    GPUArrays.default_rng(MtlArray)MtlArrayorthogonal, truncated_normal
    oneAPI.gpuarrays_rng()oneArrayorthogonal, truncated_normal
    GPUArrays.default_rng(oneArray)oneArrayorthogonal, truncated_normal

    API Reference ​

    Main Functions ​

    ',6)),s("details",d,[s("summary",null,[i[0]||(i[0]=s("a",{id:"WeightInitializers.glorot_normal",href:"#WeightInitializers.glorot_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_normal")],-1)),i[1]||(i[1]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=t(`
    julia
    glorot_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = 1) -> AbstractArray{T, length(size)}

    Return an AbstractArray{T} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.

    References

    [1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.

    source

    `,5))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"WeightInitializers.glorot_uniform",href:"#WeightInitializers.glorot_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_uniform")],-1)),i[4]||(i[4]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[15]||(i[15]=t(`
    julia
    glorot_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = 1) -> AbstractArray{T, length(size)}
    `,1)),s("p",null,[i[7]||(i[7]=a("Return an ")),i[8]||(i[8]=s("code",null,"AbstractArray{T}",-1)),i[9]||(i[9]=a(" of the given ")),i[10]||(i[10]=s("code",null,"size",-1)),i[11]||(i[11]=a(" containing random numbers drawn from a uniform distribution on the interval ")),s("mjx-container",g,[(h(),l("svg",E,i[5]||(i[5]=[t('',1)]))),i[6]||(i[6]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",{stretchy:"false"},"["),s("mo",null,"−"),s("mi",null,"x"),s("mo",null,","),s("mi",null,"x"),s("mo",{stretchy:"false"},"]")])],-1))]),i[12]||(i[12]=a(", where ")),i[13]||(i[13]=s("code",null,"x = gain * sqrt(6 / (fan_in + fan_out))",-1)),i[14]||(i[14]=a(". This method is described in [1] and also known as Xavier initialization."))]),i[16]||(i[16]=s("p",null,[s("strong",null,"References")],-1)),i[17]||(i[17]=s("p",null,[a('[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." '),s("em",null,"Proceedings of the thirteenth international conference on artificial intelligence and statistics"),a(". 2010.")],-1)),i[18]||(i[18]=s("p",null,[s("a",{href:"https://github.com/LuxDL/WeightInitializers.jl/blob/v1.0.4/src/initializers.jl#L13-L27",target:"_blank",rel:"noreferrer"},"source")],-1))]),s("details",y,[s("summary",null,[i[19]||(i[19]=s("a",{id:"WeightInitializers.identity_init",href:"#WeightInitializers.identity_init"},[s("span",{class:"jlbinding"},"WeightInitializers.identity_init")],-1)),i[20]||(i[20]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[21]||(i[21]=t(`
    julia
    identity_init([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; gain::Number=1,
+import{_ as r,c as e,a2 as t,j as s,a,G as l,B as p,o as h}from"./chunks/framework.DjZDIZsN.js";const Y=JSON.parse('{"title":"WeightInitializers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/WeightInitializers.md","filePath":"api/Building_Blocks/WeightInitializers.md","lastUpdated":null}'),k={name:"api/Building_Blocks/WeightInitializers.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.612ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2922.7 1000","aria-hidden":"true"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},z={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"},I={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},W={class:"jldocstring custom-block"},R={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},N={class:"jldocstring custom-block"},G={class:"jldocstring custom-block"},U={class:"jldocstring custom-block"},O={class:"jldocstring custom-block"},P={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},q={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"};function V(S,i,X,Z,$,J){const n=p("Badge");return h(),e("div",null,[i[109]||(i[109]=t('
    WeightInitializers ​
    This package is a light dependency providing common weight initialization schemes for deep learning models.
    Supported RNG Types ​
    RNG Type / PackageReturned Array TypeUnsupported Functions
    Random.jlArray
    StableRNGs.jlArray
    CUDA.default_rng()CuArray
    GPUArrays.default_rng(CuArray)CuArray
    AMDGPU.rocrand_rng()ROCArray
    AMDGPU.gpuarrays_rng()ROCArray
    GPUArrays.default_rng(ROCArray)ROCArray
    Metal.gpuarrays_rng()MtlArrayorthogonal
    GPUArrays.default_rng(MtlArray)MtlArrayorthogonal
    oneAPI.gpuarrays_rng()oneArrayorthogonal, truncated_normal
    GPUArrays.default_rng(oneArray)oneArrayorthogonal, truncated_normal
    API Reference ​
    Main Functions ​
    ',6)),s("details",d,[s("summary",null,[i[0]||(i[0]=s("a",{id:"WeightInitializers.glorot_normal",href:"#WeightInitializers.glorot_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_normal")],-1)),i[1]||(i[1]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=t(`
    julia
    glorot_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = 1) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.
    References
    [1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.
    source
    `,5))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"WeightInitializers.glorot_uniform",href:"#WeightInitializers.glorot_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_uniform")],-1)),i[4]||(i[4]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[15]||(i[15]=t(`
    julia
    glorot_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = 1) -> AbstractArray{T, length(size)}
    `,1)),s("p",null,[i[7]||(i[7]=a("Return an ")),i[8]||(i[8]=s("code",null,"AbstractArray{T}",-1)),i[9]||(i[9]=a(" of the given ")),i[10]||(i[10]=s("code",null,"size",-1)),i[11]||(i[11]=a(" containing random numbers drawn from a uniform distribution on the interval ")),s("mjx-container",g,[(h(),e("svg",E,i[5]||(i[5]=[t('',1)]))),i[6]||(i[6]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",{stretchy:"false"},"["),s("mo",null,"−"),s("mi",null,"x"),s("mo",null,","),s("mi",null,"x"),s("mo",{stretchy:"false"},"]")])],-1))]),i[12]||(i[12]=a(", where ")),i[13]||(i[13]=s("code",null,"x = gain * sqrt(6 / (fan_in + fan_out))",-1)),i[14]||(i[14]=a(". This method is described in [1] and also known as Xavier initialization."))]),i[16]||(i[16]=s("p",null,[s("strong",null,"References")],-1)),i[17]||(i[17]=s("p",null,[a('[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." '),s("em",null,"Proceedings of the thirteenth international conference on artificial intelligence and statistics"),a(". 2010.")],-1)),i[18]||(i[18]=s("p",null,[s("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/WeightInitializers/src/initializers.jl#L13-L27",target:"_blank",rel:"noreferrer"},"source")],-1))]),s("details",y,[s("summary",null,[i[19]||(i[19]=s("a",{id:"WeightInitializers.identity_init",href:"#WeightInitializers.identity_init"},[s("span",{class:"jlbinding"},"WeightInitializers.identity_init")],-1)),i[20]||(i[20]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[21]||(i[21]=t(`
    julia
    identity_init([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; gain::Number=1,
     shift::Union{Integer, Tuple{Integer, Integer}}=0) -> AbstractArray{T}
    Constructs an array that aims to provide an identity mapping when used as parameters in most layers of a neural network. The identity mapping is scaled by the gain parameter.
    Behavior
    • 1D: Returns a Vector of zeros (useful for biases in layers where input_size == output_size).
    • 2D: Returns an identity matrix (useful for fully connected layers with equal input and output sizes).
    • More than 2D: Returns a tensor where the central slice along the last two dimensions is an identity matrix, and the rest are zeros (useful for convolutional layers, simulating an identity convolution).
    Caveats
    • Not all layers will result in an identity mapping when using this initializer. Exceptions include recurrent and normalization layers.
    • Layers must have input_size == output_size for a perfect identity mapping. In cases where this condition is not met, the function pads extra dimensions with zeros.
    • For convolutional layers to achieve an identity mapping, kernel sizes must be odd, and appropriate padding must be applied to ensure the output feature maps are the same size as the input feature maps.
    Arguments
    • rng::AbstractRNG: An optional random number generator, included for consistency with other initializers but ignored since the output is deterministic.
    • T::Type{<:Number}: The numeric type of the array elements.
    • size...: The dimensions of the array to be initialized.
    • gain::Number=1: A scaling factor applied to the identity mapping.
    • shift::Union{Integer, Tuple{Integer, Integer}}=0: An integer or a tuple specifying the circular shift applied to the output array.
    Returns
    • AbstractArray{T}: An array initialized to represent an identity mapping, scaled by gain and optionally shifted by shift.
    Examples
    julia
    julia> identity_init(Xoshiro(123), Float32, 5, 5)
 5×5 Matrix{Float32}:
  1.0  1.0  1.0  1.0  1.0
@@ -14,39 +14,39 @@ import{_ as r,c as l,a2 as t,j as s,a,G as e,B as p,o as h}from"./chunks/framewo
 [:, :, 1, 1] =
  0.0  0.0  0.0
  0.0  1.5  0.0
- 0.0  0.0  0.0
    source
    `,13))]),s("details",c,[s("summary",null,[i[22]||(i[22]=s("a",{id:"WeightInitializers.kaiming_normal",href:"#WeightInitializers.kaiming_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_normal")],-1)),i[23]||(i[23]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[24]||(i[24]=t(`
    julia
    kaiming_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers taken from a normal distribution standard deviation gain / sqrt(fan_in)
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",u,[s("summary",null,[i[25]||(i[25]=s("a",{id:"WeightInitializers.kaiming_uniform",href:"#WeightInitializers.kaiming_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_uniform")],-1)),i[26]||(i[26]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[27]||(i[27]=t(`
    julia
    kaiming_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [-x, x], where x = gain * sqrt(3/fan_in).
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",F,[s("summary",null,[i[28]||(i[28]=s("a",{id:"WeightInitializers.sparse_init",href:"#WeightInitializers.sparse_init"},[s("span",{class:"jlbinding"},"WeightInitializers.sparse_init")],-1)),i[29]||(i[29]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[30]||(i[30]=t(`
    julia
    sparse_init([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
+ 0.0  0.0  0.0
    source
    `,13))]),s("details",c,[s("summary",null,[i[22]||(i[22]=s("a",{id:"WeightInitializers.kaiming_normal",href:"#WeightInitializers.kaiming_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_normal")],-1)),i[23]||(i[23]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[24]||(i[24]=t(`
    julia
    kaiming_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers taken from a normal distribution standard deviation gain / sqrt(fan_in)
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",u,[s("summary",null,[i[25]||(i[25]=s("a",{id:"WeightInitializers.kaiming_uniform",href:"#WeightInitializers.kaiming_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_uniform")],-1)),i[26]||(i[26]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[27]||(i[27]=t(`
    julia
    kaiming_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [-x, x], where x = gain * sqrt(3/fan_in).
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",F,[s("summary",null,[i[28]||(i[28]=s("a",{id:"WeightInitializers.sparse_init",href:"#WeightInitializers.sparse_init"},[s("span",{class:"jlbinding"},"WeightInitializers.sparse_init")],-1)),i[29]||(i[29]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[30]||(i[30]=t(`
    julia
    sparse_init([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
     sparsity::Number, std::Number=0.01) -> AbstractArray{T}
    Creates a sparsely initialized weight matrix with a specified proportion of zeroed elements, using random numbers drawn from a normal distribution for the non-zero elements. This method was introduced in [1].
    Note
    The sparsity parameter controls the proportion of the matrix that will be zeroed. For example, a sparsity of 0.3 means that approximately 30% of the elements will be set to zero. The non-zero elements are distributed according to a normal distribution, scaled by the std parameter.
    Arguments
    • rng::AbstractRNG: The random number generator to use.
    • T::Type{<:Number}: The numeric type of the elements in the returned array.
    • dims::Integer...: The dimensions of the weight matrix to be generated.
    • sparsity::Number: The proportion of elements to be zeroed. Must be between 0 and 1.
    • std::Number=0.01: The standard deviation of the normal distribution before applying gain.
    Returns
    • AbstractArray{T}: A sparsely initialized weight matrix of dimensions dims and type T.
    Examples
    julia
    julia> y = sparse_init(Xoshiro(123), Float32, 5, 5; sparsity=0.3, std=0.01);
 
 julia> y isa Matrix{Float32}
 true
 
 julia> size(y) == (5, 5)
-true
    References
    [1] Martens, J, "Deep learning via Hessian-free optimization" Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.
    source
    `,12))]),s("details",b,[s("summary",null,[i[31]||(i[31]=s("a",{id:"WeightInitializers.truncated_normal",href:"#WeightInitializers.truncated_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.truncated_normal")],-1)),i[32]||(i[32]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[33]||(i[33]=t(`
    julia
    truncated_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; mean = 0,
-    std = 1, lo = -2, hi = 2) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size where each element is drawn from a truncated normal distribution. The numbers are distributed like filter(x -> lo ≤ x ≤ hi, mean .+ std .* randn(100)).
    source
    `,3))]),s("details",C,[s("summary",null,[i[34]||(i[34]=s("a",{id:"WeightInitializers.orthogonal",href:"#WeightInitializers.orthogonal"},[s("span",{class:"jlbinding"},"WeightInitializers.orthogonal")],-1)),i[35]||(i[35]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[36]||(i[36]=t(`
    julia
    orthogonal([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
-    gain = 1)  -> AbstractArray{T, length(dims)}
    Return an AbstractArray{T} of the given dimensions (dims) which is a (semi) orthogonal matrix, as described in [1].
    The function constructs an orthogonal or semi-orthogonal matrix depending on the specified dimensions. For two dimensions, it returns a matrix where dims = (rows, cols). For more than two dimensions, it computes an orthogonal matrix of size prod(dims[1:(end - 1)]) by dims[end] before reshaping it to the original dimensions.
    Cannot construct a vector, i.e., length(dims) == 1 is forbidden.
    Arguments
    • rng::AbstractRNG: Random number generator.
    • T::Type{<:Real}: The type of the elements in the array.
    • dims::Integer...: The dimensions of the array.
    • gain::Number: Scaling factor for the elements of the orthogonal matrix.
    References
    [1] Saxe, McClelland, Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", ICLR 2014, https://arxiv.org/abs/1312.6120
    source
    `,9))]),i[110]||(i[110]=s("h3",{id:"Other-Convenience-Functions",tabindex:"-1"},[a("Other Convenience Functions "),s("a",{class:"header-anchor",href:"#Other-Convenience-Functions","aria-label":'Permalink to "Other Convenience Functions {#Other-Convenience-Functions}"'},"​")],-1)),i[111]||(i[111]=s("div",{class:"warning custom-block"},[s("p",{class:"custom-block-title"},"Beware"),s("p",null,"Unlike the other functions these ones don't take a type argument.")],-1)),s("details",m,[s("summary",null,[i[37]||(i[37]=s("a",{id:"WeightInitializers.zeros16",href:"#WeightInitializers.zeros16"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros16")],-1)),i[38]||(i[38]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[39]||(i[39]=t(`
    julia
    zeros16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",A,[s("summary",null,[i[40]||(i[40]=s("a",{id:"WeightInitializers.ones16",href:"#WeightInitializers.ones16"},[s("span",{class:"jlbinding"},"WeightInitializers.ones16")],-1)),i[41]||(i[41]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[42]||(i[42]=t(`
    julia
    ones16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",f,[s("summary",null,[i[43]||(i[43]=s("a",{id:"WeightInitializers.rand16",href:"#WeightInitializers.rand16"},[s("span",{class:"jlbinding"},"WeightInitializers.rand16")],-1)),i[44]||(i[44]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[45]||(i[45]=t(`
    julia
    rand16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",z,[s("summary",null,[i[46]||(i[46]=s("a",{id:"WeightInitializers.randn16",href:"#WeightInitializers.randn16"},[s("span",{class:"jlbinding"},"WeightInitializers.randn16")],-1)),i[47]||(i[47]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[48]||(i[48]=t(`
    julia
    randn16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",j,[s("summary",null,[i[49]||(i[49]=s("a",{id:"WeightInitializers.zeros32",href:"#WeightInitializers.zeros32"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros32")],-1)),i[50]||(i[50]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[51]||(i[51]=t(`
    julia
    zeros32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",v,[s("summary",null,[i[52]||(i[52]=s("a",{id:"WeightInitializers.ones32",href:"#WeightInitializers.ones32"},[s("span",{class:"jlbinding"},"WeightInitializers.ones32")],-1)),i[53]||(i[53]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[54]||(i[54]=t(`
    julia
    ones32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",D,[s("summary",null,[i[55]||(i[55]=s("a",{id:"WeightInitializers.rand32",href:"#WeightInitializers.rand32"},[s("span",{class:"jlbinding"},"WeightInitializers.rand32")],-1)),i[56]||(i[56]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[57]||(i[57]=t(`
    julia
    rand32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",B,[s("summary",null,[i[58]||(i[58]=s("a",{id:"WeightInitializers.randn32",href:"#WeightInitializers.randn32"},[s("span",{class:"jlbinding"},"WeightInitializers.randn32")],-1)),i[59]||(i[59]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[60]||(i[60]=t(`
    julia
    randn32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",x,[s("summary",null,[i[61]||(i[61]=s("a",{id:"WeightInitializers.zeros64",href:"#WeightInitializers.zeros64"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros64")],-1)),i[62]||(i[62]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[63]||(i[63]=t(`
    julia
    zeros64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",I,[s("summary",null,[i[64]||(i[64]=s("a",{id:"WeightInitializers.ones64",href:"#WeightInitializers.ones64"},[s("span",{class:"jlbinding"},"WeightInitializers.ones64")],-1)),i[65]||(i[65]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[66]||(i[66]=t(`
    julia
    ones64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",W,[s("summary",null,[i[67]||(i[67]=s("a",{id:"WeightInitializers.rand64",href:"#WeightInitializers.rand64"},[s("span",{class:"jlbinding"},"WeightInitializers.rand64")],-1)),i[68]||(i[68]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[69]||(i[69]=t(`
    julia
    rand64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",T,[s("summary",null,[i[70]||(i[70]=s("a",{id:"WeightInitializers.randn64",href:"#WeightInitializers.randn64"},[s("span",{class:"jlbinding"},"WeightInitializers.randn64")],-1)),i[71]||(i[71]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[72]||(i[72]=t(`
    julia
    randn64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",L,[s("summary",null,[i[73]||(i[73]=s("a",{id:"WeightInitializers.zerosC16",href:"#WeightInitializers.zerosC16"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC16")],-1)),i[74]||(i[74]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[75]||(i[75]=t(`
    julia
    zerosC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",R,[s("summary",null,[i[76]||(i[76]=s("a",{id:"WeightInitializers.onesC16",href:"#WeightInitializers.onesC16"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC16")],-1)),i[77]||(i[77]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[78]||(i[78]=t(`
    julia
    onesC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",w,[s("summary",null,[i[79]||(i[79]=s("a",{id:"WeightInitializers.randC16",href:"#WeightInitializers.randC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randC16")],-1)),i[80]||(i[80]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[81]||(i[81]=t(`
    julia
    randC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",Q,[s("summary",null,[i[82]||(i[82]=s("a",{id:"WeightInitializers.randnC16",href:"#WeightInitializers.randnC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC16")],-1)),i[83]||(i[83]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[84]||(i[84]=t(`
    julia
    randnC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",N,[s("summary",null,[i[85]||(i[85]=s("a",{id:"WeightInitializers.zerosC32",href:"#WeightInitializers.zerosC32"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC32")],-1)),i[86]||(i[86]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[87]||(i[87]=t(`
    julia
    zerosC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",G,[s("summary",null,[i[88]||(i[88]=s("a",{id:"WeightInitializers.onesC32",href:"#WeightInitializers.onesC32"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC32")],-1)),i[89]||(i[89]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[90]||(i[90]=t(`
    julia
    onesC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",U,[s("summary",null,[i[91]||(i[91]=s("a",{id:"WeightInitializers.randC32",href:"#WeightInitializers.randC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randC32")],-1)),i[92]||(i[92]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[93]||(i[93]=t(`
    julia
    randC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",O,[s("summary",null,[i[94]||(i[94]=s("a",{id:"WeightInitializers.randnC32",href:"#WeightInitializers.randnC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC32")],-1)),i[95]||(i[95]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[96]||(i[96]=t(`
    julia
    randnC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",P,[s("summary",null,[i[97]||(i[97]=s("a",{id:"WeightInitializers.zerosC64",href:"#WeightInitializers.zerosC64"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC64")],-1)),i[98]||(i[98]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[99]||(i[99]=t(`
    julia
    zerosC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",M,[s("summary",null,[i[100]||(i[100]=s("a",{id:"WeightInitializers.onesC64",href:"#WeightInitializers.onesC64"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC64")],-1)),i[101]||(i[101]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=t(`
    julia
    onesC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",q,[s("summary",null,[i[103]||(i[103]=s("a",{id:"WeightInitializers.randC64",href:"#WeightInitializers.randC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randC64")],-1)),i[104]||(i[104]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[105]||(i[105]=t(`
    julia
    randC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",H,[s("summary",null,[i[106]||(i[106]=s("a",{id:"WeightInitializers.randnC64",href:"#WeightInitializers.randnC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC64")],-1)),i[107]||(i[107]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[108]||(i[108]=t(`
    julia
    randnC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))])])}const _=r(k,[["render",V]]);export{Y as __pageData,_ as default};
+true

    References

    [1] Martens, J, "Deep learning via Hessian-free optimization" Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.

    source

    `,12))]),s("details",b,[s("summary",null,[i[31]||(i[31]=s("a",{id:"WeightInitializers.truncated_normal",href:"#WeightInitializers.truncated_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.truncated_normal")],-1)),i[32]||(i[32]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[33]||(i[33]=t(`
    julia
    truncated_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; mean = 0,
+    std = 1, lo = -2, hi = 2) -> AbstractArray{T, length(size)}

    Return an AbstractArray{T} of the given size where each element is drawn from a truncated normal distribution. The numbers are distributed like filter(x -> lo ≤ x ≤ hi, mean .+ std .* randn(100)).

    source

    `,3))]),s("details",C,[s("summary",null,[i[34]||(i[34]=s("a",{id:"WeightInitializers.orthogonal",href:"#WeightInitializers.orthogonal"},[s("span",{class:"jlbinding"},"WeightInitializers.orthogonal")],-1)),i[35]||(i[35]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[36]||(i[36]=t(`
    julia
    orthogonal([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
+    gain = 1)  -> AbstractArray{T, length(dims)}

    Return an AbstractArray{T} of the given dimensions (dims) which is a (semi) orthogonal matrix, as described in [1].

    The function constructs an orthogonal or semi-orthogonal matrix depending on the specified dimensions. For two dimensions, it returns a matrix where dims = (rows, cols). For more than two dimensions, it computes an orthogonal matrix of size prod(dims[1:(end - 1)]) by dims[end] before reshaping it to the original dimensions.

    Cannot construct a vector, i.e., length(dims) == 1 is forbidden.

    Arguments

    References

    [1] Saxe, McClelland, Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", ICLR 2014, https://arxiv.org/abs/1312.6120

    source

    `,9))]),i[110]||(i[110]=s("h3",{id:"Other-Convenience-Functions",tabindex:"-1"},[a("Other Convenience Functions "),s("a",{class:"header-anchor",href:"#Other-Convenience-Functions","aria-label":'Permalink to "Other Convenience Functions {#Other-Convenience-Functions}"'},"​")],-1)),i[111]||(i[111]=s("div",{class:"warning custom-block"},[s("p",{class:"custom-block-title"},"Beware"),s("p",null,"Unlike the other functions these ones don't take a type argument.")],-1)),s("details",m,[s("summary",null,[i[37]||(i[37]=s("a",{id:"WeightInitializers.zeros16",href:"#WeightInitializers.zeros16"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros16")],-1)),i[38]||(i[38]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[39]||(i[39]=t(`
    julia
    zeros16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",A,[s("summary",null,[i[40]||(i[40]=s("a",{id:"WeightInitializers.ones16",href:"#WeightInitializers.ones16"},[s("span",{class:"jlbinding"},"WeightInitializers.ones16")],-1)),i[41]||(i[41]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[42]||(i[42]=t(`
    julia
    ones16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",f,[s("summary",null,[i[43]||(i[43]=s("a",{id:"WeightInitializers.rand16",href:"#WeightInitializers.rand16"},[s("span",{class:"jlbinding"},"WeightInitializers.rand16")],-1)),i[44]||(i[44]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[45]||(i[45]=t(`
    julia
    rand16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",z,[s("summary",null,[i[46]||(i[46]=s("a",{id:"WeightInitializers.randn16",href:"#WeightInitializers.randn16"},[s("span",{class:"jlbinding"},"WeightInitializers.randn16")],-1)),i[47]||(i[47]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[48]||(i[48]=t(`
    julia
    randn16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",j,[s("summary",null,[i[49]||(i[49]=s("a",{id:"WeightInitializers.zeros32",href:"#WeightInitializers.zeros32"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros32")],-1)),i[50]||(i[50]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[51]||(i[51]=t(`
    julia
    zeros32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",D,[s("summary",null,[i[52]||(i[52]=s("a",{id:"WeightInitializers.ones32",href:"#WeightInitializers.ones32"},[s("span",{class:"jlbinding"},"WeightInitializers.ones32")],-1)),i[53]||(i[53]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[54]||(i[54]=t(`
    julia
    ones32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",v,[s("summary",null,[i[55]||(i[55]=s("a",{id:"WeightInitializers.rand32",href:"#WeightInitializers.rand32"},[s("span",{class:"jlbinding"},"WeightInitializers.rand32")],-1)),i[56]||(i[56]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[57]||(i[57]=t(`
    julia
    rand32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",x,[s("summary",null,[i[58]||(i[58]=s("a",{id:"WeightInitializers.randn32",href:"#WeightInitializers.randn32"},[s("span",{class:"jlbinding"},"WeightInitializers.randn32")],-1)),i[59]||(i[59]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[60]||(i[60]=t(`
    julia
    randn32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",B,[s("summary",null,[i[61]||(i[61]=s("a",{id:"WeightInitializers.zeros64",href:"#WeightInitializers.zeros64"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros64")],-1)),i[62]||(i[62]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[63]||(i[63]=t(`
    julia
    zeros64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",I,[s("summary",null,[i[64]||(i[64]=s("a",{id:"WeightInitializers.ones64",href:"#WeightInitializers.ones64"},[s("span",{class:"jlbinding"},"WeightInitializers.ones64")],-1)),i[65]||(i[65]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[66]||(i[66]=t(`
    julia
    ones64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",L,[s("summary",null,[i[67]||(i[67]=s("a",{id:"WeightInitializers.rand64",href:"#WeightInitializers.rand64"},[s("span",{class:"jlbinding"},"WeightInitializers.rand64")],-1)),i[68]||(i[68]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[69]||(i[69]=t(`
    julia
    rand64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",T,[s("summary",null,[i[70]||(i[70]=s("a",{id:"WeightInitializers.randn64",href:"#WeightInitializers.randn64"},[s("span",{class:"jlbinding"},"WeightInitializers.randn64")],-1)),i[71]||(i[71]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[72]||(i[72]=t(`
    julia
    randn64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",W,[s("summary",null,[i[73]||(i[73]=s("a",{id:"WeightInitializers.zerosC16",href:"#WeightInitializers.zerosC16"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC16")],-1)),i[74]||(i[74]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[75]||(i[75]=t(`
    julia
    zerosC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",R,[s("summary",null,[i[76]||(i[76]=s("a",{id:"WeightInitializers.onesC16",href:"#WeightInitializers.onesC16"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC16")],-1)),i[77]||(i[77]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[78]||(i[78]=t(`
    julia
    onesC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",w,[s("summary",null,[i[79]||(i[79]=s("a",{id:"WeightInitializers.randC16",href:"#WeightInitializers.randC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randC16")],-1)),i[80]||(i[80]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[81]||(i[81]=t(`
    julia
    randC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",Q,[s("summary",null,[i[82]||(i[82]=s("a",{id:"WeightInitializers.randnC16",href:"#WeightInitializers.randnC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC16")],-1)),i[83]||(i[83]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[84]||(i[84]=t(`
    julia
    randnC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",N,[s("summary",null,[i[85]||(i[85]=s("a",{id:"WeightInitializers.zerosC32",href:"#WeightInitializers.zerosC32"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC32")],-1)),i[86]||(i[86]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[87]||(i[87]=t(`
    julia
    zerosC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",G,[s("summary",null,[i[88]||(i[88]=s("a",{id:"WeightInitializers.onesC32",href:"#WeightInitializers.onesC32"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC32")],-1)),i[89]||(i[89]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[90]||(i[90]=t(`
    julia
    onesC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",U,[s("summary",null,[i[91]||(i[91]=s("a",{id:"WeightInitializers.randC32",href:"#WeightInitializers.randC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randC32")],-1)),i[92]||(i[92]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[93]||(i[93]=t(`
    julia
    randC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",O,[s("summary",null,[i[94]||(i[94]=s("a",{id:"WeightInitializers.randnC32",href:"#WeightInitializers.randnC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC32")],-1)),i[95]||(i[95]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[96]||(i[96]=t(`
    julia
    randnC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",P,[s("summary",null,[i[97]||(i[97]=s("a",{id:"WeightInitializers.zerosC64",href:"#WeightInitializers.zerosC64"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC64")],-1)),i[98]||(i[98]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[99]||(i[99]=t(`
    julia
    zerosC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",M,[s("summary",null,[i[100]||(i[100]=s("a",{id:"WeightInitializers.onesC64",href:"#WeightInitializers.onesC64"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC64")],-1)),i[101]||(i[101]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=t(`
    julia
    onesC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",q,[s("summary",null,[i[103]||(i[103]=s("a",{id:"WeightInitializers.randC64",href:"#WeightInitializers.randC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randC64")],-1)),i[104]||(i[104]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[105]||(i[105]=t(`
    julia
    randC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",H,[s("summary",null,[i[106]||(i[106]=s("a",{id:"WeightInitializers.randnC64",href:"#WeightInitializers.randnC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC64")],-1)),i[107]||(i[107]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[108]||(i[108]=t(`
    julia
    randnC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))])])}const _=r(k,[["render",V]]);export{Y as __pageData,_ as default}; diff --git a/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.jFCbAP4G.lean.js b/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.C7zduFBi.lean.js similarity index 54% rename from previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.jFCbAP4G.lean.js rename to previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.C7zduFBi.lean.js index acb1fd2d84..0a2a73daa6 100644 --- a/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.jFCbAP4G.lean.js +++ b/previews/PR1000/assets/api_Building_Blocks_WeightInitializers.md.C7zduFBi.lean.js @@ -1,6 +1,6 @@ -import{_ as r,c as l,a2 as t,j as s,a,G as e,B as p,o as h}from"./chunks/framework.DjZDIZsN.js";const Y=JSON.parse('{"title":"WeightInitializers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/WeightInitializers.md","filePath":"api/Building_Blocks/WeightInitializers.md","lastUpdated":null}'),k={name:"api/Building_Blocks/WeightInitializers.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.612ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2922.7 1000","aria-hidden":"true"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},z={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},I={class:"jldocstring custom-block"},W={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},R={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},N={class:"jldocstring custom-block"},G={class:"jldocstring custom-block"},U={class:"jldocstring custom-block"},O={class:"jldocstring custom-block"},P={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},q={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"};function V(S,i,X,Z,$,J){const n=p("Badge");return h(),l("div",null,[i[109]||(i[109]=t('

    WeightInitializers ​

    This package is a light dependency providing common weight initialization schemes for deep learning models.

    Supported RNG Types ​

    RNG Type / PackageReturned Array TypeUnsupported Functions
    Random.jlArray
    StableRNGs.jlArray
    CUDA.default_rng()CuArray
    GPUArrays.default_rng(CuArray)CuArray
    AMDGPU.rocrand_rng()ROCArray
    AMDGPU.gpuarrays_rng()ROCArray
    GPUArrays.default_rng(ROCArray)ROCArray
    Metal.gpuarrays_rng()MtlArrayorthogonal, truncated_normal
    GPUArrays.default_rng(MtlArray)MtlArrayorthogonal, truncated_normal
    oneAPI.gpuarrays_rng()oneArrayorthogonal, truncated_normal
    GPUArrays.default_rng(oneArray)oneArrayorthogonal, truncated_normal

    API Reference ​

    Main Functions ​

    ',6)),s("details",d,[s("summary",null,[i[0]||(i[0]=s("a",{id:"WeightInitializers.glorot_normal",href:"#WeightInitializers.glorot_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_normal")],-1)),i[1]||(i[1]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=t(`
    julia
    glorot_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = 1) -> AbstractArray{T, length(size)}

    Return an AbstractArray{T} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.

    References

    [1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.

    source

    `,5))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"WeightInitializers.glorot_uniform",href:"#WeightInitializers.glorot_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_uniform")],-1)),i[4]||(i[4]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[15]||(i[15]=t(`
    julia
    glorot_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = 1) -> AbstractArray{T, length(size)}
    `,1)),s("p",null,[i[7]||(i[7]=a("Return an ")),i[8]||(i[8]=s("code",null,"AbstractArray{T}",-1)),i[9]||(i[9]=a(" of the given ")),i[10]||(i[10]=s("code",null,"size",-1)),i[11]||(i[11]=a(" containing random numbers drawn from a uniform distribution on the interval ")),s("mjx-container",g,[(h(),l("svg",E,i[5]||(i[5]=[t('',1)]))),i[6]||(i[6]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",{stretchy:"false"},"["),s("mo",null,"−"),s("mi",null,"x"),s("mo",null,","),s("mi",null,"x"),s("mo",{stretchy:"false"},"]")])],-1))]),i[12]||(i[12]=a(", where ")),i[13]||(i[13]=s("code",null,"x = gain * sqrt(6 / (fan_in + fan_out))",-1)),i[14]||(i[14]=a(". This method is described in [1] and also known as Xavier initialization."))]),i[16]||(i[16]=s("p",null,[s("strong",null,"References")],-1)),i[17]||(i[17]=s("p",null,[a('[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." '),s("em",null,"Proceedings of the thirteenth international conference on artificial intelligence and statistics"),a(". 2010.")],-1)),i[18]||(i[18]=s("p",null,[s("a",{href:"https://github.com/LuxDL/WeightInitializers.jl/blob/v1.0.4/src/initializers.jl#L13-L27",target:"_blank",rel:"noreferrer"},"source")],-1))]),s("details",y,[s("summary",null,[i[19]||(i[19]=s("a",{id:"WeightInitializers.identity_init",href:"#WeightInitializers.identity_init"},[s("span",{class:"jlbinding"},"WeightInitializers.identity_init")],-1)),i[20]||(i[20]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[21]||(i[21]=t(`
    julia
    identity_init([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; gain::Number=1,
+import{_ as r,c as e,a2 as t,j as s,a,G as l,B as p,o as h}from"./chunks/framework.DjZDIZsN.js";const Y=JSON.parse('{"title":"WeightInitializers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Building_Blocks/WeightInitializers.md","filePath":"api/Building_Blocks/WeightInitializers.md","lastUpdated":null}'),k={name:"api/Building_Blocks/WeightInitializers.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.612ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2922.7 1000","aria-hidden":"true"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},F={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},A={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},z={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"},D={class:"jldocstring custom-block"},v={class:"jldocstring custom-block"},x={class:"jldocstring custom-block"},B={class:"jldocstring custom-block"},I={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},W={class:"jldocstring custom-block"},R={class:"jldocstring custom-block"},w={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},N={class:"jldocstring custom-block"},G={class:"jldocstring custom-block"},U={class:"jldocstring custom-block"},O={class:"jldocstring custom-block"},P={class:"jldocstring custom-block"},M={class:"jldocstring custom-block"},q={class:"jldocstring custom-block"},H={class:"jldocstring custom-block"};function V(S,i,X,Z,$,J){const n=p("Badge");return h(),e("div",null,[i[109]||(i[109]=t('
    WeightInitializers ​
    This package is a light dependency providing common weight initialization schemes for deep learning models.
    Supported RNG Types ​
    RNG Type / PackageReturned Array TypeUnsupported Functions
    Random.jlArray
    StableRNGs.jlArray
    CUDA.default_rng()CuArray
    GPUArrays.default_rng(CuArray)CuArray
    AMDGPU.rocrand_rng()ROCArray
    AMDGPU.gpuarrays_rng()ROCArray
    GPUArrays.default_rng(ROCArray)ROCArray
    Metal.gpuarrays_rng()MtlArrayorthogonal
    GPUArrays.default_rng(MtlArray)MtlArrayorthogonal
    oneAPI.gpuarrays_rng()oneArrayorthogonal, truncated_normal
    GPUArrays.default_rng(oneArray)oneArrayorthogonal, truncated_normal
    API Reference ​
    Main Functions ​
    ',6)),s("details",d,[s("summary",null,[i[0]||(i[0]=s("a",{id:"WeightInitializers.glorot_normal",href:"#WeightInitializers.glorot_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_normal")],-1)),i[1]||(i[1]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[2]||(i[2]=t(`
    julia
    glorot_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = 1) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers drawn from a normal distribution with standard deviation gain * sqrt(2 / (fan_in + fan_out)). This method is described in [1] and also known as Xavier initialization.
    References
    [1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.
    source
    `,5))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"WeightInitializers.glorot_uniform",href:"#WeightInitializers.glorot_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.glorot_uniform")],-1)),i[4]||(i[4]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[15]||(i[15]=t(`
    julia
    glorot_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = 1) -> AbstractArray{T, length(size)}
    `,1)),s("p",null,[i[7]||(i[7]=a("Return an ")),i[8]||(i[8]=s("code",null,"AbstractArray{T}",-1)),i[9]||(i[9]=a(" of the given ")),i[10]||(i[10]=s("code",null,"size",-1)),i[11]||(i[11]=a(" containing random numbers drawn from a uniform distribution on the interval ")),s("mjx-container",g,[(h(),e("svg",E,i[5]||(i[5]=[t('',1)]))),i[6]||(i[6]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",{stretchy:"false"},"["),s("mo",null,"−"),s("mi",null,"x"),s("mo",null,","),s("mi",null,"x"),s("mo",{stretchy:"false"},"]")])],-1))]),i[12]||(i[12]=a(", where ")),i[13]||(i[13]=s("code",null,"x = gain * sqrt(6 / (fan_in + fan_out))",-1)),i[14]||(i[14]=a(". This method is described in [1] and also known as Xavier initialization."))]),i[16]||(i[16]=s("p",null,[s("strong",null,"References")],-1)),i[17]||(i[17]=s("p",null,[a('[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." '),s("em",null,"Proceedings of the thirteenth international conference on artificial intelligence and statistics"),a(". 2010.")],-1)),i[18]||(i[18]=s("p",null,[s("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/lib/WeightInitializers/src/initializers.jl#L13-L27",target:"_blank",rel:"noreferrer"},"source")],-1))]),s("details",y,[s("summary",null,[i[19]||(i[19]=s("a",{id:"WeightInitializers.identity_init",href:"#WeightInitializers.identity_init"},[s("span",{class:"jlbinding"},"WeightInitializers.identity_init")],-1)),i[20]||(i[20]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[21]||(i[21]=t(`
    julia
    identity_init([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; gain::Number=1,
     shift::Union{Integer, Tuple{Integer, Integer}}=0) -> AbstractArray{T}
    Constructs an array that aims to provide an identity mapping when used as parameters in most layers of a neural network. The identity mapping is scaled by the gain parameter.
    Behavior
    • 1D: Returns a Vector of zeros (useful for biases in layers where input_size == output_size).
    • 2D: Returns an identity matrix (useful for fully connected layers with equal input and output sizes).
    • More than 2D: Returns a tensor where the central slice along the last two dimensions is an identity matrix, and the rest are zeros (useful for convolutional layers, simulating an identity convolution).
    Caveats
    • Not all layers will result in an identity mapping when using this initializer. Exceptions include recurrent and normalization layers.
    • Layers must have input_size == output_size for a perfect identity mapping. In cases where this condition is not met, the function pads extra dimensions with zeros.
    • For convolutional layers to achieve an identity mapping, kernel sizes must be odd, and appropriate padding must be applied to ensure the output feature maps are the same size as the input feature maps.
    Arguments
    • rng::AbstractRNG: An optional random number generator, included for consistency with other initializers but ignored since the output is deterministic.
    • T::Type{<:Number}: The numeric type of the array elements.
    • size...: The dimensions of the array to be initialized.
    • gain::Number=1: A scaling factor applied to the identity mapping.
    • shift::Union{Integer, Tuple{Integer, Integer}}=0: An integer or a tuple specifying the circular shift applied to the output array.
    Returns
    • AbstractArray{T}: An array initialized to represent an identity mapping, scaled by gain and optionally shifted by shift.
    Examples
    julia
    julia> identity_init(Xoshiro(123), Float32, 5, 5)
 5×5 Matrix{Float32}:
  1.0  1.0  1.0  1.0  1.0
@@ -14,39 +14,39 @@ import{_ as r,c as l,a2 as t,j as s,a,G as e,B as p,o as h}from"./chunks/framewo
 [:, :, 1, 1] =
  0.0  0.0  0.0
  0.0  1.5  0.0
- 0.0  0.0  0.0
    source
    `,13))]),s("details",c,[s("summary",null,[i[22]||(i[22]=s("a",{id:"WeightInitializers.kaiming_normal",href:"#WeightInitializers.kaiming_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_normal")],-1)),i[23]||(i[23]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[24]||(i[24]=t(`
    julia
    kaiming_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers taken from a normal distribution standard deviation gain / sqrt(fan_in)
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",u,[s("summary",null,[i[25]||(i[25]=s("a",{id:"WeightInitializers.kaiming_uniform",href:"#WeightInitializers.kaiming_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_uniform")],-1)),i[26]||(i[26]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[27]||(i[27]=t(`
    julia
    kaiming_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
-    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [-x, x], where x = gain * sqrt(3/fan_in).
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",F,[s("summary",null,[i[28]||(i[28]=s("a",{id:"WeightInitializers.sparse_init",href:"#WeightInitializers.sparse_init"},[s("span",{class:"jlbinding"},"WeightInitializers.sparse_init")],-1)),i[29]||(i[29]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[30]||(i[30]=t(`
    julia
    sparse_init([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
+ 0.0  0.0  0.0
    source
    `,13))]),s("details",c,[s("summary",null,[i[22]||(i[22]=s("a",{id:"WeightInitializers.kaiming_normal",href:"#WeightInitializers.kaiming_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_normal")],-1)),i[23]||(i[23]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[24]||(i[24]=t(`
    julia
    kaiming_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers taken from a normal distribution standard deviation gain / sqrt(fan_in)
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",u,[s("summary",null,[i[25]||(i[25]=s("a",{id:"WeightInitializers.kaiming_uniform",href:"#WeightInitializers.kaiming_uniform"},[s("span",{class:"jlbinding"},"WeightInitializers.kaiming_uniform")],-1)),i[26]||(i[26]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[27]||(i[27]=t(`
    julia
    kaiming_uniform([::AbstractRNG=Utils.default_rng()], [T=Float32], size...;
+    gain = √T(2)) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size containing random numbers drawn from a uniform distribution on the interval [-x, x], where x = gain * sqrt(3/fan_in).
    References
    [1] He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." Proceedings of the IEEE international conference on computer vision. 2015.
    source
    `,5))]),s("details",F,[s("summary",null,[i[28]||(i[28]=s("a",{id:"WeightInitializers.sparse_init",href:"#WeightInitializers.sparse_init"},[s("span",{class:"jlbinding"},"WeightInitializers.sparse_init")],-1)),i[29]||(i[29]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[30]||(i[30]=t(`
    julia
    sparse_init([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
     sparsity::Number, std::Number=0.01) -> AbstractArray{T}
    Creates a sparsely initialized weight matrix with a specified proportion of zeroed elements, using random numbers drawn from a normal distribution for the non-zero elements. This method was introduced in [1].
    Note
    The sparsity parameter controls the proportion of the matrix that will be zeroed. For example, a sparsity of 0.3 means that approximately 30% of the elements will be set to zero. The non-zero elements are distributed according to a normal distribution, scaled by the std parameter.
    Arguments
    • rng::AbstractRNG: The random number generator to use.
    • T::Type{<:Number}: The numeric type of the elements in the returned array.
    • dims::Integer...: The dimensions of the weight matrix to be generated.
    • sparsity::Number: The proportion of elements to be zeroed. Must be between 0 and 1.
    • std::Number=0.01: The standard deviation of the normal distribution before applying gain.
    Returns
    • AbstractArray{T}: A sparsely initialized weight matrix of dimensions dims and type T.
    Examples
    julia
    julia> y = sparse_init(Xoshiro(123), Float32, 5, 5; sparsity=0.3, std=0.01);
 
 julia> y isa Matrix{Float32}
 true
 
 julia> size(y) == (5, 5)
-true
    References
    [1] Martens, J, "Deep learning via Hessian-free optimization" Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.
    source
    `,12))]),s("details",b,[s("summary",null,[i[31]||(i[31]=s("a",{id:"WeightInitializers.truncated_normal",href:"#WeightInitializers.truncated_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.truncated_normal")],-1)),i[32]||(i[32]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[33]||(i[33]=t(`
    julia
    truncated_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; mean = 0,
-    std = 1, lo = -2, hi = 2) -> AbstractArray{T, length(size)}
    Return an AbstractArray{T} of the given size where each element is drawn from a truncated normal distribution. The numbers are distributed like filter(x -> lo ≤ x ≤ hi, mean .+ std .* randn(100)).
    source
    `,3))]),s("details",C,[s("summary",null,[i[34]||(i[34]=s("a",{id:"WeightInitializers.orthogonal",href:"#WeightInitializers.orthogonal"},[s("span",{class:"jlbinding"},"WeightInitializers.orthogonal")],-1)),i[35]||(i[35]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[36]||(i[36]=t(`
    julia
    orthogonal([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
-    gain = 1)  -> AbstractArray{T, length(dims)}
    Return an AbstractArray{T} of the given dimensions (dims) which is a (semi) orthogonal matrix, as described in [1].
    The function constructs an orthogonal or semi-orthogonal matrix depending on the specified dimensions. For two dimensions, it returns a matrix where dims = (rows, cols). For more than two dimensions, it computes an orthogonal matrix of size prod(dims[1:(end - 1)]) by dims[end] before reshaping it to the original dimensions.
    Cannot construct a vector, i.e., length(dims) == 1 is forbidden.
    Arguments
    • rng::AbstractRNG: Random number generator.
    • T::Type{<:Real}: The type of the elements in the array.
    • dims::Integer...: The dimensions of the array.
    • gain::Number: Scaling factor for the elements of the orthogonal matrix.
    References
    [1] Saxe, McClelland, Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", ICLR 2014, https://arxiv.org/abs/1312.6120
    source
    `,9))]),i[110]||(i[110]=s("h3",{id:"Other-Convenience-Functions",tabindex:"-1"},[a("Other Convenience Functions "),s("a",{class:"header-anchor",href:"#Other-Convenience-Functions","aria-label":'Permalink to "Other Convenience Functions {#Other-Convenience-Functions}"'},"​")],-1)),i[111]||(i[111]=s("div",{class:"warning custom-block"},[s("p",{class:"custom-block-title"},"Beware"),s("p",null,"Unlike the other functions these ones don't take a type argument.")],-1)),s("details",m,[s("summary",null,[i[37]||(i[37]=s("a",{id:"WeightInitializers.zeros16",href:"#WeightInitializers.zeros16"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros16")],-1)),i[38]||(i[38]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[39]||(i[39]=t(`
    julia
    zeros16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",A,[s("summary",null,[i[40]||(i[40]=s("a",{id:"WeightInitializers.ones16",href:"#WeightInitializers.ones16"},[s("span",{class:"jlbinding"},"WeightInitializers.ones16")],-1)),i[41]||(i[41]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[42]||(i[42]=t(`
    julia
    ones16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",f,[s("summary",null,[i[43]||(i[43]=s("a",{id:"WeightInitializers.rand16",href:"#WeightInitializers.rand16"},[s("span",{class:"jlbinding"},"WeightInitializers.rand16")],-1)),i[44]||(i[44]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[45]||(i[45]=t(`
    julia
    rand16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",z,[s("summary",null,[i[46]||(i[46]=s("a",{id:"WeightInitializers.randn16",href:"#WeightInitializers.randn16"},[s("span",{class:"jlbinding"},"WeightInitializers.randn16")],-1)),i[47]||(i[47]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[48]||(i[48]=t(`
    julia
    randn16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float16, length(size)}
    Return an AbstractArray{Float16} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",j,[s("summary",null,[i[49]||(i[49]=s("a",{id:"WeightInitializers.zeros32",href:"#WeightInitializers.zeros32"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros32")],-1)),i[50]||(i[50]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[51]||(i[51]=t(`
    julia
    zeros32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",v,[s("summary",null,[i[52]||(i[52]=s("a",{id:"WeightInitializers.ones32",href:"#WeightInitializers.ones32"},[s("span",{class:"jlbinding"},"WeightInitializers.ones32")],-1)),i[53]||(i[53]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[54]||(i[54]=t(`
    julia
    ones32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",D,[s("summary",null,[i[55]||(i[55]=s("a",{id:"WeightInitializers.rand32",href:"#WeightInitializers.rand32"},[s("span",{class:"jlbinding"},"WeightInitializers.rand32")],-1)),i[56]||(i[56]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[57]||(i[57]=t(`
    julia
    rand32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",B,[s("summary",null,[i[58]||(i[58]=s("a",{id:"WeightInitializers.randn32",href:"#WeightInitializers.randn32"},[s("span",{class:"jlbinding"},"WeightInitializers.randn32")],-1)),i[59]||(i[59]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[60]||(i[60]=t(`
    julia
    randn32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float32, length(size)}
    Return an AbstractArray{Float32} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",x,[s("summary",null,[i[61]||(i[61]=s("a",{id:"WeightInitializers.zeros64",href:"#WeightInitializers.zeros64"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros64")],-1)),i[62]||(i[62]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[63]||(i[63]=t(`
    julia
    zeros64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",I,[s("summary",null,[i[64]||(i[64]=s("a",{id:"WeightInitializers.ones64",href:"#WeightInitializers.ones64"},[s("span",{class:"jlbinding"},"WeightInitializers.ones64")],-1)),i[65]||(i[65]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[66]||(i[66]=t(`
    julia
    ones64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",W,[s("summary",null,[i[67]||(i[67]=s("a",{id:"WeightInitializers.rand64",href:"#WeightInitializers.rand64"},[s("span",{class:"jlbinding"},"WeightInitializers.rand64")],-1)),i[68]||(i[68]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[69]||(i[69]=t(`
    julia
    rand64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",T,[s("summary",null,[i[70]||(i[70]=s("a",{id:"WeightInitializers.randn64",href:"#WeightInitializers.randn64"},[s("span",{class:"jlbinding"},"WeightInitializers.randn64")],-1)),i[71]||(i[71]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[72]||(i[72]=t(`
    julia
    randn64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{Float64, length(size)}
    Return an AbstractArray{Float64} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",L,[s("summary",null,[i[73]||(i[73]=s("a",{id:"WeightInitializers.zerosC16",href:"#WeightInitializers.zerosC16"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC16")],-1)),i[74]||(i[74]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[75]||(i[75]=t(`
    julia
    zerosC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",R,[s("summary",null,[i[76]||(i[76]=s("a",{id:"WeightInitializers.onesC16",href:"#WeightInitializers.onesC16"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC16")],-1)),i[77]||(i[77]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[78]||(i[78]=t(`
    julia
    onesC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",w,[s("summary",null,[i[79]||(i[79]=s("a",{id:"WeightInitializers.randC16",href:"#WeightInitializers.randC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randC16")],-1)),i[80]||(i[80]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[81]||(i[81]=t(`
    julia
    randC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",Q,[s("summary",null,[i[82]||(i[82]=s("a",{id:"WeightInitializers.randnC16",href:"#WeightInitializers.randnC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC16")],-1)),i[83]||(i[83]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[84]||(i[84]=t(`
    julia
    randnC16([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF16, length(size)}
    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",N,[s("summary",null,[i[85]||(i[85]=s("a",{id:"WeightInitializers.zerosC32",href:"#WeightInitializers.zerosC32"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC32")],-1)),i[86]||(i[86]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[87]||(i[87]=t(`
    julia
    zerosC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",G,[s("summary",null,[i[88]||(i[88]=s("a",{id:"WeightInitializers.onesC32",href:"#WeightInitializers.onesC32"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC32")],-1)),i[89]||(i[89]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[90]||(i[90]=t(`
    julia
    onesC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",U,[s("summary",null,[i[91]||(i[91]=s("a",{id:"WeightInitializers.randC32",href:"#WeightInitializers.randC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randC32")],-1)),i[92]||(i[92]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[93]||(i[93]=t(`
    julia
    randC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",O,[s("summary",null,[i[94]||(i[94]=s("a",{id:"WeightInitializers.randnC32",href:"#WeightInitializers.randnC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC32")],-1)),i[95]||(i[95]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[96]||(i[96]=t(`
    julia
    randnC32([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF32, length(size)}
    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))]),s("details",P,[s("summary",null,[i[97]||(i[97]=s("a",{id:"WeightInitializers.zerosC64",href:"#WeightInitializers.zerosC64"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC64")],-1)),i[98]||(i[98]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[99]||(i[99]=t(`
    julia
    zerosC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of zeros.
    source
    `,3))]),s("details",M,[s("summary",null,[i[100]||(i[100]=s("a",{id:"WeightInitializers.onesC64",href:"#WeightInitializers.onesC64"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC64")],-1)),i[101]||(i[101]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=t(`
    julia
    onesC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.
    source
    `,3))]),s("details",q,[s("summary",null,[i[103]||(i[103]=s("a",{id:"WeightInitializers.randC64",href:"#WeightInitializers.randC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randC64")],-1)),i[104]||(i[104]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[105]||(i[105]=t(`
    julia
    randC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.
    source
    `,3))]),s("details",H,[s("summary",null,[i[106]||(i[106]=s("a",{id:"WeightInitializers.randnC64",href:"#WeightInitializers.randnC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC64")],-1)),i[107]||(i[107]=a()),e(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[108]||(i[108]=t(`
    julia
    randnC64([::AbstractRNG=Utils.default_rng()], size...;
-    kwargs...) -> AbstractArray{ComplexF64, length(size)}
    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.
    source
    `,3))])])}const _=r(k,[["render",V]]);export{Y as __pageData,_ as default};
+true

    References

    [1] Martens, J, "Deep learning via Hessian-free optimization" Proceedings of the 27th International Conference on International Conference on Machine Learning. 2010.

    source

    `,12))]),s("details",b,[s("summary",null,[i[31]||(i[31]=s("a",{id:"WeightInitializers.truncated_normal",href:"#WeightInitializers.truncated_normal"},[s("span",{class:"jlbinding"},"WeightInitializers.truncated_normal")],-1)),i[32]||(i[32]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[33]||(i[33]=t(`
    julia
    truncated_normal([::AbstractRNG=Utils.default_rng()], [T=Float32], size...; mean = 0,
+    std = 1, lo = -2, hi = 2) -> AbstractArray{T, length(size)}

    Return an AbstractArray{T} of the given size where each element is drawn from a truncated normal distribution. The numbers are distributed like filter(x -> lo ≤ x ≤ hi, mean .+ std .* randn(100)).

    source

    `,3))]),s("details",C,[s("summary",null,[i[34]||(i[34]=s("a",{id:"WeightInitializers.orthogonal",href:"#WeightInitializers.orthogonal"},[s("span",{class:"jlbinding"},"WeightInitializers.orthogonal")],-1)),i[35]||(i[35]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[36]||(i[36]=t(`
    julia
    orthogonal([::AbstractRNG=Utils.default_rng()], [T=Float32], dims::Integer...;
+    gain = 1)  -> AbstractArray{T, length(dims)}

    Return an AbstractArray{T} of the given dimensions (dims) which is a (semi) orthogonal matrix, as described in [1].

    The function constructs an orthogonal or semi-orthogonal matrix depending on the specified dimensions. For two dimensions, it returns a matrix where dims = (rows, cols). For more than two dimensions, it computes an orthogonal matrix of size prod(dims[1:(end - 1)]) by dims[end] before reshaping it to the original dimensions.

    Cannot construct a vector, i.e., length(dims) == 1 is forbidden.

    Arguments

    References

    [1] Saxe, McClelland, Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks", ICLR 2014, https://arxiv.org/abs/1312.6120

    source

    `,9))]),i[110]||(i[110]=s("h3",{id:"Other-Convenience-Functions",tabindex:"-1"},[a("Other Convenience Functions "),s("a",{class:"header-anchor",href:"#Other-Convenience-Functions","aria-label":'Permalink to "Other Convenience Functions {#Other-Convenience-Functions}"'},"​")],-1)),i[111]||(i[111]=s("div",{class:"warning custom-block"},[s("p",{class:"custom-block-title"},"Beware"),s("p",null,"Unlike the other functions these ones don't take a type argument.")],-1)),s("details",m,[s("summary",null,[i[37]||(i[37]=s("a",{id:"WeightInitializers.zeros16",href:"#WeightInitializers.zeros16"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros16")],-1)),i[38]||(i[38]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[39]||(i[39]=t(`
    julia
    zeros16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",A,[s("summary",null,[i[40]||(i[40]=s("a",{id:"WeightInitializers.ones16",href:"#WeightInitializers.ones16"},[s("span",{class:"jlbinding"},"WeightInitializers.ones16")],-1)),i[41]||(i[41]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[42]||(i[42]=t(`
    julia
    ones16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",f,[s("summary",null,[i[43]||(i[43]=s("a",{id:"WeightInitializers.rand16",href:"#WeightInitializers.rand16"},[s("span",{class:"jlbinding"},"WeightInitializers.rand16")],-1)),i[44]||(i[44]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[45]||(i[45]=t(`
    julia
    rand16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",z,[s("summary",null,[i[46]||(i[46]=s("a",{id:"WeightInitializers.randn16",href:"#WeightInitializers.randn16"},[s("span",{class:"jlbinding"},"WeightInitializers.randn16")],-1)),i[47]||(i[47]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[48]||(i[48]=t(`
    julia
    randn16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float16, length(size)}

    Return an AbstractArray{Float16} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",j,[s("summary",null,[i[49]||(i[49]=s("a",{id:"WeightInitializers.zeros32",href:"#WeightInitializers.zeros32"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros32")],-1)),i[50]||(i[50]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[51]||(i[51]=t(`
    julia
    zeros32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",D,[s("summary",null,[i[52]||(i[52]=s("a",{id:"WeightInitializers.ones32",href:"#WeightInitializers.ones32"},[s("span",{class:"jlbinding"},"WeightInitializers.ones32")],-1)),i[53]||(i[53]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[54]||(i[54]=t(`
    julia
    ones32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",v,[s("summary",null,[i[55]||(i[55]=s("a",{id:"WeightInitializers.rand32",href:"#WeightInitializers.rand32"},[s("span",{class:"jlbinding"},"WeightInitializers.rand32")],-1)),i[56]||(i[56]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[57]||(i[57]=t(`
    julia
    rand32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",x,[s("summary",null,[i[58]||(i[58]=s("a",{id:"WeightInitializers.randn32",href:"#WeightInitializers.randn32"},[s("span",{class:"jlbinding"},"WeightInitializers.randn32")],-1)),i[59]||(i[59]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[60]||(i[60]=t(`
    julia
    randn32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float32, length(size)}

    Return an AbstractArray{Float32} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",B,[s("summary",null,[i[61]||(i[61]=s("a",{id:"WeightInitializers.zeros64",href:"#WeightInitializers.zeros64"},[s("span",{class:"jlbinding"},"WeightInitializers.zeros64")],-1)),i[62]||(i[62]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[63]||(i[63]=t(`
    julia
    zeros64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",I,[s("summary",null,[i[64]||(i[64]=s("a",{id:"WeightInitializers.ones64",href:"#WeightInitializers.ones64"},[s("span",{class:"jlbinding"},"WeightInitializers.ones64")],-1)),i[65]||(i[65]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[66]||(i[66]=t(`
    julia
    ones64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",L,[s("summary",null,[i[67]||(i[67]=s("a",{id:"WeightInitializers.rand64",href:"#WeightInitializers.rand64"},[s("span",{class:"jlbinding"},"WeightInitializers.rand64")],-1)),i[68]||(i[68]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[69]||(i[69]=t(`
    julia
    rand64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",T,[s("summary",null,[i[70]||(i[70]=s("a",{id:"WeightInitializers.randn64",href:"#WeightInitializers.randn64"},[s("span",{class:"jlbinding"},"WeightInitializers.randn64")],-1)),i[71]||(i[71]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[72]||(i[72]=t(`
    julia
    randn64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{Float64, length(size)}

    Return an AbstractArray{Float64} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",W,[s("summary",null,[i[73]||(i[73]=s("a",{id:"WeightInitializers.zerosC16",href:"#WeightInitializers.zerosC16"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC16")],-1)),i[74]||(i[74]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[75]||(i[75]=t(`
    julia
    zerosC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",R,[s("summary",null,[i[76]||(i[76]=s("a",{id:"WeightInitializers.onesC16",href:"#WeightInitializers.onesC16"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC16")],-1)),i[77]||(i[77]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[78]||(i[78]=t(`
    julia
    onesC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",w,[s("summary",null,[i[79]||(i[79]=s("a",{id:"WeightInitializers.randC16",href:"#WeightInitializers.randC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randC16")],-1)),i[80]||(i[80]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[81]||(i[81]=t(`
    julia
    randC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",Q,[s("summary",null,[i[82]||(i[82]=s("a",{id:"WeightInitializers.randnC16",href:"#WeightInitializers.randnC16"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC16")],-1)),i[83]||(i[83]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[84]||(i[84]=t(`
    julia
    randnC16([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF16, length(size)}

    Return an AbstractArray{ComplexF16} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",N,[s("summary",null,[i[85]||(i[85]=s("a",{id:"WeightInitializers.zerosC32",href:"#WeightInitializers.zerosC32"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC32")],-1)),i[86]||(i[86]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[87]||(i[87]=t(`
    julia
    zerosC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",G,[s("summary",null,[i[88]||(i[88]=s("a",{id:"WeightInitializers.onesC32",href:"#WeightInitializers.onesC32"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC32")],-1)),i[89]||(i[89]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[90]||(i[90]=t(`
    julia
    onesC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",U,[s("summary",null,[i[91]||(i[91]=s("a",{id:"WeightInitializers.randC32",href:"#WeightInitializers.randC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randC32")],-1)),i[92]||(i[92]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[93]||(i[93]=t(`
    julia
    randC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",O,[s("summary",null,[i[94]||(i[94]=s("a",{id:"WeightInitializers.randnC32",href:"#WeightInitializers.randnC32"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC32")],-1)),i[95]||(i[95]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[96]||(i[96]=t(`
    julia
    randnC32([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF32, length(size)}

    Return an AbstractArray{ComplexF32} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))]),s("details",P,[s("summary",null,[i[97]||(i[97]=s("a",{id:"WeightInitializers.zerosC64",href:"#WeightInitializers.zerosC64"},[s("span",{class:"jlbinding"},"WeightInitializers.zerosC64")],-1)),i[98]||(i[98]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[99]||(i[99]=t(`
    julia
    zerosC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of zeros.

    source

    `,3))]),s("details",M,[s("summary",null,[i[100]||(i[100]=s("a",{id:"WeightInitializers.onesC64",href:"#WeightInitializers.onesC64"},[s("span",{class:"jlbinding"},"WeightInitializers.onesC64")],-1)),i[101]||(i[101]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[102]||(i[102]=t(`
    julia
    onesC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing an AbstractArray of ones.

    source

    `,3))]),s("details",q,[s("summary",null,[i[103]||(i[103]=s("a",{id:"WeightInitializers.randC64",href:"#WeightInitializers.randC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randC64")],-1)),i[104]||(i[104]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[105]||(i[105]=t(`
    julia
    randC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a uniform distribution.

    source

    `,3))]),s("details",H,[s("summary",null,[i[106]||(i[106]=s("a",{id:"WeightInitializers.randnC64",href:"#WeightInitializers.randnC64"},[s("span",{class:"jlbinding"},"WeightInitializers.randnC64")],-1)),i[107]||(i[107]=a()),l(n,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[108]||(i[108]=t(`
    julia
    randnC64([::AbstractRNG=Utils.default_rng()], size...;
+    kwargs...) -> AbstractArray{ComplexF64, length(size)}

    Return an AbstractArray{ComplexF64} of the given size containing random numbers from a standard normal distribution.

    source

    `,3))])])}const _=r(k,[["render",V]]);export{Y as __pageData,_ as default}; diff --git a/previews/PR1000/assets/api_Lux_autodiff.md.65bMeWBg.js b/previews/PR1000/assets/api_Lux_autodiff.md.kxYwKcE-.js similarity index 98% rename from previews/PR1000/assets/api_Lux_autodiff.md.65bMeWBg.js rename to previews/PR1000/assets/api_Lux_autodiff.md.kxYwKcE-.js index 966ab4e4fb..e7504c7d6b 100644 --- a/previews/PR1000/assets/api_Lux_autodiff.md.65bMeWBg.js +++ b/previews/PR1000/assets/api_Lux_autodiff.md.kxYwKcE-.js @@ -1 +1 @@ -import{_ as d,c as i,j as t,a,G as l,a2 as s,B as r,o as n}from"./chunks/framework.DjZDIZsN.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),Q={name:"api/Lux/autodiff.md"},p={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return n(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",p,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
    julia
    jacobian_vector_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",c,[(n(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
    julia
    vector_jacobian_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(n(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
    julia
    batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

    Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

    1. ndims(y) ≥ 2

    2. size(y, ndims(y)) == size(x, ndims(x))

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff
    AutoZygoteZygote.jl

    Arguments

    Returns

    Danger

    f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

    source

    ',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/previews/PR1000/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=d(Q,[["render",f]]);export{v as __pageData,L as default}; +import{_ as d,c as i,j as t,a,G as l,a2 as s,B as r,o as n}from"./chunks/framework.DjZDIZsN.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),Q={name:"api/Lux/autodiff.md"},p={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return n(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",p,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
    julia
    jacobian_vector_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",c,[(n(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
    julia
    vector_jacobian_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(n(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
    julia
    batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

    Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

    1. ndims(y) ≥ 2

    2. size(y, ndims(y)) == size(x, ndims(x))

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff
    AutoZygoteZygote.jl

    Arguments

    Returns

    Danger

    f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

    source

    ',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/previews/PR1000/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=d(Q,[["render",f]]);export{v as __pageData,L as default}; diff --git a/previews/PR1000/assets/api_Lux_autodiff.md.65bMeWBg.lean.js b/previews/PR1000/assets/api_Lux_autodiff.md.kxYwKcE-.lean.js similarity index 98% rename from previews/PR1000/assets/api_Lux_autodiff.md.65bMeWBg.lean.js rename to previews/PR1000/assets/api_Lux_autodiff.md.kxYwKcE-.lean.js index 966ab4e4fb..e7504c7d6b 100644 --- a/previews/PR1000/assets/api_Lux_autodiff.md.65bMeWBg.lean.js +++ b/previews/PR1000/assets/api_Lux_autodiff.md.kxYwKcE-.lean.js @@ -1 +1 @@ -import{_ as d,c as i,j as t,a,G as l,a2 as s,B as r,o as n}from"./chunks/framework.DjZDIZsN.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),Q={name:"api/Lux/autodiff.md"},p={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return n(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",p,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
    julia
    jacobian_vector_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",c,[(n(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
    julia
    vector_jacobian_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(n(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
    julia
    batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

    Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

    1. ndims(y) ≥ 2

    2. size(y, ndims(y)) == size(x, ndims(x))

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff
    AutoZygoteZygote.jl

    Arguments

    Returns

    Danger

    f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

    source

    ',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/previews/PR1000/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=d(Q,[["render",f]]);export{v as __pageData,L as default}; +import{_ as d,c as i,j as t,a,G as l,a2 as s,B as r,o as n}from"./chunks/framework.DjZDIZsN.js";const v=JSON.parse('{"title":"Automatic Differentiation Helpers","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/autodiff.md","filePath":"api/Lux/autodiff.md","lastUpdated":null}'),Q={name:"api/Lux/autodiff.md"},p={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},T={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.812ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 3010.7 1799","aria-hidden":"true"},m={class:"jldocstring custom-block"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},h={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.126ex",height:"4.536ex",role:"img",focusable:"false",viewBox:"0 -1355.3 3591.5 2004.8","aria-hidden":"true"},g={class:"jldocstring custom-block"};function f(b,e,x,k,y,w){const o=r("Badge");return n(),i("div",null,[e[19]||(e[19]=t("h1",{id:"autodiff-lux-helpers",tabindex:"-1"},[a("Automatic Differentiation Helpers "),t("a",{class:"header-anchor",href:"#autodiff-lux-helpers","aria-label":'Permalink to "Automatic Differentiation Helpers {#autodiff-lux-helpers}"'},"​")],-1)),e[20]||(e[20]=t("h2",{id:"JVP-and-VJP-Wrappers",tabindex:"-1"},[a("JVP & VJP Wrappers "),t("a",{class:"header-anchor",href:"#JVP-and-VJP-Wrappers","aria-label":'Permalink to "JVP & VJP Wrappers {#JVP-and-VJP-Wrappers}"'},"​")],-1)),t("details",p,[t("summary",null,[e[0]||(e[0]=t("a",{id:"Lux.jacobian_vector_product",href:"#Lux.jacobian_vector_product"},[t("span",{class:"jlbinding"},"Lux.jacobian_vector_product")],-1)),e[1]||(e[1]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[6]||(e[6]=s('
    julia
    jacobian_vector_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[4]||(e[4]=a("Compute the Jacobian-Vector Product ")),t("mjx-container",c,[(n(),i("svg",T,e[2]||(e[2]=[s('',1)]))),e[3]||(e[3]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"u")])],-1))]),e[5]||(e[5]=a(". This is a wrapper around AD backends but allows us to compute gradients of jacobian-vector products efficiently using mixed-mode AD."))]),e[7]||(e[7]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),t("details",m,[t("summary",null,[e[8]||(e[8]=t("a",{id:"Lux.vector_jacobian_product",href:"#Lux.vector_jacobian_product"},[t("span",{class:"jlbinding"},"Lux.vector_jacobian_product")],-1)),e[9]||(e[9]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[14]||(e[14]=s('
    julia
    vector_jacobian_product(f, backend::AbstractADType, x, u)
    ',1)),t("p",null,[e[12]||(e[12]=a("Compute the Vector-Jacobian Product ")),t("mjx-container",u,[(n(),i("svg",h,e[10]||(e[10]=[s('',1)]))),e[11]||(e[11]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msup",null,[t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"("),t("mfrac",null,[t("mrow",null,[t("mi",null,"∂"),t("mi",null,"f")]),t("mrow",null,[t("mi",null,"∂"),t("mi",null,"x")])]),t("mo",{"data-mjx-texclass":"CLOSE"},")")]),t("mi",null,"T")]),t("mi",null,"u")])],-1))]),e[13]||(e[13]=a(". This is a wrapper around AD backends but allows us to compute gradients of vector-jacobian products efficiently using mixed-mode AD."))]),e[15]||(e[15]=s('

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl

    Warning

    Gradient wrt u in the reverse pass is always dropped.

    Arguments

    Returns

    source

    ',8))]),e[21]||(e[21]=t("h2",{id:"Batched-AD",tabindex:"-1"},[a("Batched AD "),t("a",{class:"header-anchor",href:"#Batched-AD","aria-label":'Permalink to "Batched AD {#Batched-AD}"'},"​")],-1)),t("details",g,[t("summary",null,[e[16]||(e[16]=t("a",{id:"Lux.batched_jacobian",href:"#Lux.batched_jacobian"},[t("span",{class:"jlbinding"},"Lux.batched_jacobian")],-1)),e[17]||(e[17]=a()),l(o,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),e[18]||(e[18]=s('
    julia
    batched_jacobian(f, backend::AbstractADType, x::AbstractArray)

    Computes the Jacobian of a function f with respect to a batch of inputs x. This expects the following properties for y = f(x):

    1. ndims(y) ≥ 2

    2. size(y, ndims(y)) == size(x, ndims(x))

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoForwardDiff
    AutoZygoteZygote.jl

    Arguments

    Returns

    Danger

    f(x) must not be inter-mixing the batch dimensions, else the result will be incorrect. For example, if f contains operations like batch normalization, then the result will be incorrect.

    source

    ',11))]),e[22]||(e[22]=t("h2",{id:"Nested-2nd-Order-AD",tabindex:"-1"},[a("Nested 2nd Order AD "),t("a",{class:"header-anchor",href:"#Nested-2nd-Order-AD","aria-label":'Permalink to "Nested 2nd Order AD {#Nested-2nd-Order-AD}"'},"​")],-1)),e[23]||(e[23]=t("p",null,[a("Consult the "),t("a",{href:"/previews/PR1000/manual/nested_autodiff#nested_autodiff"},"manual page on Nested AD"),a(" for information on nested automatic differentiation.")],-1))])}const L=d(Q,[["render",f]]);export{v as __pageData,L as default}; diff --git a/previews/PR1000/assets/api_Lux_contrib.md.Cn9lWBAP.js b/previews/PR1000/assets/api_Lux_contrib.md.DGQu-ivG.js similarity index 98% rename from previews/PR1000/assets/api_Lux_contrib.md.Cn9lWBAP.js rename to previews/PR1000/assets/api_Lux_contrib.md.DGQu-ivG.js index 90beb49fd1..a3ea83976b 100644 --- a/previews/PR1000/assets/api_Lux_contrib.md.Cn9lWBAP.js +++ b/previews/PR1000/assets/api_Lux_contrib.md.DGQu-ivG.js @@ -1,6 +1,6 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framework.DjZDIZsN.js";const A=JSON.parse('{"title":"Experimental Features","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/contrib.md","filePath":"api/Lux/contrib.md","lastUpdated":null}'),r={name:"api/Lux/contrib.md"},d={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"};function F(m,s,C,b,D,f){const t=p("Badge");return k(),l("div",null,[s[21]||(s[21]=e('

    Experimental Features ​

    All features listed on this page are experimental which means:

    1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.

    2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.

    3. None of the features are exported.

    Parameter Freezing ​

    ',4)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Lux.Experimental.FrozenLayer",href:"#Lux.Experimental.FrozenLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.FrozenLayer")],-1)),s[1]||(s[1]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[2]||(s[2]=e(`
    julia
    FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})

    Freeze the parameters with name which_params of the layer l.

    Use Lux.Experimental.freeze instead

    It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.

    No checks for which_params

    There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.

    Arguments

    Extended Help

    Parameters

    States

    Note on Internal Layer Implementation

    The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.

    Example

    julia
    julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
-FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

    See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

    source

    `,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
    julia
    freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

    Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

    source

    julia
    freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
-    which_params::Optional{Tuple} = nothing)

    Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

    source

    `,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
    julia
    unfreeze(l::FrozenLayer)

    Unfreezes the layer l.

    source

    julia
    unfreeze(l::FrozenLayer, ps, st::NamedTuple)

    Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

    source

    ',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/previews/PR1000/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
    julia
    layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

    Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

    KeyPath provided to the function

    The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

    Call Signature for f

    Extended Help

    Example

    julia
    julia> using Lux, Random
+FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

    See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

    source

    `,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
    julia
    freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

    Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

    source

    julia
    freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
+    which_params::Optional{Tuple} = nothing)

    Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

    source

    `,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
    julia
    unfreeze(l::FrozenLayer)

    Unfreezes the layer l.

    source

    julia
    unfreeze(l::FrozenLayer, ps, st::NamedTuple)

    Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

    source

    ',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/previews/PR1000/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
    julia
    layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

    Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

    KeyPath provided to the function

    The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

    Call Signature for f

    Extended Help

    Example

    julia
    julia> using Lux, Random
 
 julia> c = Parallel(
            +; chain=Chain(; dense_1=Dense(2 => 3), bn=BatchNorm(3), dense_2=Dense(3 => 5)),
@@ -27,10 +27,10 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
 julia> all(iszero, (ps_new.chain.dense_1.weight, ps_new.chain.dense_1.bias,
                     ps_new.chain.dense_2.weight, ps_new.chain.dense_2.bias,
                     ps_new.dense_3.weight, ps_new.dense_3.bias))
-true

    source

    `,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @debug_mode layer kwargs...

    Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

    See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

    source

    ',4))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
    julia
    DebugLayer(layer::AbstractLuxLayer;
+true

    source

    `,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @debug_mode layer kwargs...

    Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

    See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

    source

    ',4))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
    julia
    DebugLayer(layer::AbstractLuxLayer;
     nan_check::Union{Symbol, StaticSymbol, Val}=static(:both),
     error_check::Union{StaticBool, Bool, Val{true}, Val{false}}=True(),
-    location::KeyPath=KeyPath())

    A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

    Arguments

    Extended Help

    Keyword Arguments

    Input / Output

    Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

    If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

    ChainRules Compatible Reverse Mode AD Tools

    nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

    Disable After Debugging

    This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

    See Lux.Experimental.@debug_mode to construct this layer.

    source

    `,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
    julia
    share_parameters(ps, sharing)
+    location::KeyPath=KeyPath())

    A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

    Arguments

    Extended Help

    Keyword Arguments

    Input / Output

    Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

    If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

    ChainRules Compatible Reverse Mode AD Tools

    nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

    Disable After Debugging

    This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

    See Lux.Experimental.@debug_mode to construct this layer.

    source

    `,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
    julia
    share_parameters(ps, sharing)
 share_parameters(ps, sharing, new_parameters)

    Updates the parameters in ps with a common set of parameters new_parameters that are shared between each list in the nested list sharing. (That was kind of a mouthful, the example should make it clear).

    Arguments

    Returns

    Updated Parameters having the same structure as ps.

    Example

    julia
    julia> model = Chain(; d1=Dense(2 => 4, tanh),
            d3=Chain(; l1=Dense(4 => 2), l2=Dense(2 => 4)), d2=Dense(4 => 2))
 Chain(
@@ -52,4 +52,4 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
            ps.d3.l2.bias === ps.d1.bias &&
            ps.d2.weight === ps.d3.l1.weight &&
            ps.d2.bias === ps.d3.l1.bias
-true

    ComponentArrays

    ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

    source

    `,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default}; +true

    ComponentArrays

    ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

    source

    `,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default}; diff --git a/previews/PR1000/assets/api_Lux_contrib.md.Cn9lWBAP.lean.js b/previews/PR1000/assets/api_Lux_contrib.md.DGQu-ivG.lean.js similarity index 98% rename from previews/PR1000/assets/api_Lux_contrib.md.Cn9lWBAP.lean.js rename to previews/PR1000/assets/api_Lux_contrib.md.DGQu-ivG.lean.js index 90beb49fd1..a3ea83976b 100644 --- a/previews/PR1000/assets/api_Lux_contrib.md.Cn9lWBAP.lean.js +++ b/previews/PR1000/assets/api_Lux_contrib.md.DGQu-ivG.lean.js @@ -1,6 +1,6 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framework.DjZDIZsN.js";const A=JSON.parse('{"title":"Experimental Features","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/contrib.md","filePath":"api/Lux/contrib.md","lastUpdated":null}'),r={name:"api/Lux/contrib.md"},d={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"};function F(m,s,C,b,D,f){const t=p("Badge");return k(),l("div",null,[s[21]||(s[21]=e('

    Experimental Features ​

    All features listed on this page are experimental which means:

    1. No SemVer Guarantees. We use code here to iterate fast. That said, historically we have never broken any code in this module and have always provided a deprecation period.

    2. Expect edge-cases and report them. It will help us move these features out of experimental sooner.

    3. None of the features are exported.

    Parameter Freezing ​

    ',4)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Lux.Experimental.FrozenLayer",href:"#Lux.Experimental.FrozenLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.FrozenLayer")],-1)),s[1]||(s[1]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[2]||(s[2]=e(`
    julia
    FrozenLayer(l::AbstractLuxLayer, which_params::Optional{Tuple})

    Freeze the parameters with name which_params of the layer l.

    Use Lux.Experimental.freeze instead

    It is always recommended to use the Lux.Experimental.freeze function instead of directly using the FrozenLayer constructor.

    No checks for which_params

    There are no checks for which_params. For example, if the original layer has parameters named (:weight, :bias), and which_params is set to (:myweight,) then none of the parameters are frozen and no error is thrown.

    Arguments

    Extended Help

    Parameters

    States

    Note on Internal Layer Implementation

    The inner layer should work with NamedTuple parameters. In order to support custom parameter types, users need to implement Lux.Utils.merge(::CustomParamType, ::NamedTuple) or extend Lux.Utils.named_tuple(::CustomParamType) to return a NamedTuple.

    Example

    julia
    julia> Lux.Experimental.FrozenLayer(Dense(2 => 2), (:weight,))
-FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

    See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

    source

    `,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
    julia
    freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

    Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

    source

    julia
    freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
-    which_params::Optional{Tuple} = nothing)

    Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

    source

    `,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
    julia
    unfreeze(l::FrozenLayer)

    Unfreezes the layer l.

    source

    julia
    unfreeze(l::FrozenLayer, ps, st::NamedTuple)

    Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

    source

    ',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/previews/PR1000/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
    julia
    layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

    Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

    KeyPath provided to the function

    The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

    Call Signature for f

    Extended Help

    Example

    julia
    julia> using Lux, Random
+FrozenLayer(Dense(2 => 2), (:weight,))  # 2 parameters, plus 4 non-trainable

    See also Lux.Experimental.freeze, Lux.Experimental.unfreeze.

    source

    `,17))]),i("details",E,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.Experimental.freeze",href:"#Lux.Experimental.freeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.freeze")],-1)),s[4]||(s[4]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e(`
    julia
    freeze(l::AbstractLuxLayer, which_params::Optional{Tuple} = nothing)

    Constructs a version of l with which_params frozen. If which_params is nothing, then all parameters are frozen.

    source

    julia
    freeze(l::AbstractLuxLayer, ps, st::NamedTuple,
+    which_params::Optional{Tuple} = nothing)

    Construct a Lux.Experimental.FrozenLayer for l with the current parameters and states. If which_params is nothing, then all parameters are frozen.

    source

    `,6))]),i("details",o,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.Experimental.unfreeze",href:"#Lux.Experimental.unfreeze"},[i("span",{class:"jlbinding"},"Lux.Experimental.unfreeze")],-1)),s[7]||(s[7]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e('
    julia
    unfreeze(l::FrozenLayer)

    Unfreezes the layer l.

    source

    julia
    unfreeze(l::FrozenLayer, ps, st::NamedTuple)

    Unwraps a Lux.Experimental.FrozenLayer l with the current parameters and states.

    source

    ',6))]),s[22]||(s[22]=i("p",null,[a("For detailed usage example look at the "),i("a",{href:"/previews/PR1000/manual/freezing_model_parameters#freezing-model-parameters"},"manual page"),a(".")],-1)),s[23]||(s[23]=i("h2",{id:"Map-over-Layer",tabindex:"-1"},[a("Map over Layer "),i("a",{class:"header-anchor",href:"#Map-over-Layer","aria-label":'Permalink to "Map over Layer {#Map-over-Layer}"'},"​")],-1)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Lux.Experimental.layer_map",href:"#Lux.Experimental.layer_map"},[i("span",{class:"jlbinding"},"Lux.Experimental.layer_map")],-1)),s[10]||(s[10]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[11]||(s[11]=e(`
    julia
    layer_map(f, l::AbstractLuxLayer, ps, st::NamedTuple)

    Map the function f over the model l, with the parameters ps and states st. This is different from Functors.fmap since it zips the layers, parameters, and states and invokes the function on all of them together.

    KeyPath provided to the function

    The KeyPath depths on the structure of the parameters and states. This is of consequence exclusively for AbstractLuxWrapperLayer where the structure of the layer doesn't match the structure of the parameters and states. In the example, provided below, the KeyPath is (:chain, :dense_1) for the first layer (following the structure in ps) while accessing the same layer in the chain is done with ( :chain, :layers, :dense_1).

    Call Signature for f

    Extended Help

    Example

    julia
    julia> using Lux, Random
 
 julia> c = Parallel(
            +; chain=Chain(; dense_1=Dense(2 => 3), bn=BatchNorm(3), dense_2=Dense(3 => 5)),
@@ -27,10 +27,10 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
 julia> all(iszero, (ps_new.chain.dense_1.weight, ps_new.chain.dense_1.bias,
                     ps_new.chain.dense_2.weight, ps_new.chain.dense_2.bias,
                     ps_new.dense_3.weight, ps_new.dense_3.bias))
-true

    source

    `,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @debug_mode layer kwargs...

    Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

    See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

    source

    ',4))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
    julia
    DebugLayer(layer::AbstractLuxLayer;
+true

    source

    `,9))]),s[24]||(s[24]=i("h2",{id:"Debugging-Functionality",tabindex:"-1"},[a("Debugging Functionality "),i("a",{class:"header-anchor",href:"#Debugging-Functionality","aria-label":'Permalink to "Debugging Functionality {#Debugging-Functionality}"'},"​")],-1)),s[25]||(s[25]=i("p",null,"Model not working properly! Here are some functionalities to help you debug you Lux model.",-1)),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.Experimental.@debug_mode",href:"#Lux.Experimental.@debug_mode"},[i("span",{class:"jlbinding"},"Lux.Experimental.@debug_mode")],-1)),s[13]||(s[13]=a()),n(t,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @debug_mode layer kwargs...

    Recurses into the layer and replaces the inner most non Container Layers with a Lux.Experimental.DebugLayer.

    See Lux.Experimental.DebugLayer for details about the Keyword Arguments.

    source

    ',4))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.Experimental.DebugLayer",href:"#Lux.Experimental.DebugLayer"},[i("span",{class:"jlbinding"},"Lux.Experimental.DebugLayer")],-1)),s[16]||(s[16]=a()),n(t,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=e(`
    julia
    DebugLayer(layer::AbstractLuxLayer;
     nan_check::Union{Symbol, StaticSymbol, Val}=static(:both),
     error_check::Union{StaticBool, Bool, Val{true}, Val{false}}=True(),
-    location::KeyPath=KeyPath())

    A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

    Arguments

    Extended Help

    Keyword Arguments

    Input / Output

    Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

    If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

    ChainRules Compatible Reverse Mode AD Tools

    nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

    Disable After Debugging

    This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

    See Lux.Experimental.@debug_mode to construct this layer.

    source

    `,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
    julia
    share_parameters(ps, sharing)
+    location::KeyPath=KeyPath())

    A wrapper over Lux layers that adds checks for NaNs and errors. This is useful for debugging.

    Arguments

    Extended Help

    Keyword Arguments

    Input / Output

    Inputs and outputs are the same as the layer unless one of the nan_check or error_check criteria is met.

    If nan_check is enabled and NaNs are detected then a DomainError is thrown. If error_check is enabled, then any errors in the layer are thrown with useful information to track where the error originates.

    ChainRules Compatible Reverse Mode AD Tools

    nan_check for the backward mode only works with ChainRules Compatible Reverse Mode AD Tools currently.

    Disable After Debugging

    This layer is only meant to be used for debugging. If used for actual training or inference, will lead to extremely bad performance.

    See Lux.Experimental.@debug_mode to construct this layer.

    source

    `,14))]),s[26]||(s[26]=i("h2",{id:"Tied-Parameters",tabindex:"-1"},[a("Tied Parameters "),i("a",{class:"header-anchor",href:"#Tied-Parameters","aria-label":'Permalink to "Tied Parameters {#Tied-Parameters}"'},"​")],-1)),i("details",u,[i("summary",null,[s[18]||(s[18]=i("a",{id:"Lux.Experimental.share_parameters",href:"#Lux.Experimental.share_parameters"},[i("span",{class:"jlbinding"},"Lux.Experimental.share_parameters")],-1)),s[19]||(s[19]=a()),n(t,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[20]||(s[20]=e(`
    julia
    share_parameters(ps, sharing)
 share_parameters(ps, sharing, new_parameters)

    Updates the parameters in ps with a common set of parameters new_parameters that are shared between each list in the nested list sharing. (That was kind of a mouthful, the example should make it clear).

    Arguments

    Returns

    Updated Parameters having the same structure as ps.

    Example

    julia
    julia> model = Chain(; d1=Dense(2 => 4, tanh),
            d3=Chain(; l1=Dense(4 => 2), l2=Dense(2 => 4)), d2=Dense(4 => 2))
 Chain(
@@ -52,4 +52,4 @@ import{_ as h,c as l,a2 as e,j as i,a,G as n,B as p,o as k}from"./chunks/framewo
            ps.d3.l2.bias === ps.d1.bias &&
            ps.d2.weight === ps.d3.l1.weight &&
            ps.d2.bias === ps.d3.l1.bias
-true

    ComponentArrays

    ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

    source

    `,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default}; +true

    ComponentArrays

    ComponentArrays doesn't allow sharing parameters. Converting the returned parameters to a ComponentArray will silently cause the parameter sharing to be undone.

    source

    `,10))])])}const L=h(r,[["render",F]]);export{A as __pageData,L as default}; diff --git a/previews/PR1000/assets/api_Lux_distributed_utils.md.DrmUj9oe.js b/previews/PR1000/assets/api_Lux_distributed_utils.md.DUxpyBki.js similarity index 93% rename from previews/PR1000/assets/api_Lux_distributed_utils.md.DrmUj9oe.js rename to previews/PR1000/assets/api_Lux_distributed_utils.md.DUxpyBki.js index 5b85acbb16..7b0fdf8137 100644 --- a/previews/PR1000/assets/api_Lux_distributed_utils.md.DrmUj9oe.js +++ b/previews/PR1000/assets/api_Lux_distributed_utils.md.DUxpyBki.js @@ -1,4 +1,4 @@ -import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

    Distributed Utils ​

    Note

    These functionalities are available via the Lux.DistributedUtils module.

    Backends ​

    ',3)),s("details",c,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
    julia
    MPIBackend(comm = nothing)

    Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

    source

    ',3))]),s("details",k,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
    julia
    NCCLBackend(comm = nothing, mpi_backend = nothing)

    Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

    source

    ',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
    julia
    initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

    Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

    Possible values for backend are:

    source

    ',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
    julia
    initialized(backend::Type{<:AbstractLuxDistributedBackend})

    Check if the given backend is initialized.

    source

    ',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
    julia
    get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

    Get the distributed backend for the given backend type. Possible values are:

    Danger

    initialize(backend; kwargs...) must be called before calling this function.

    source

    ',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
    julia
    local_rank(backend::AbstractLuxDistributedBackend)

    Get the local rank for the given backend.

    source

    ',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
    julia
    total_workers(backend::AbstractLuxDistributedBackend)

    Get the total number of workers for the given backend.

    source

    ',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",m,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
    julia
    allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
-allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

    Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",E,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
    julia
    bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
-bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

    Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

    source

    `,3))]),s("details",C,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
    julia
    reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
-reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

    Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",f,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
    julia
    synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

    Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

    source

    ',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
    julia
    DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

    Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

    Arguments

    source

    ',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
    julia
    DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

    data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

    Load MLUtils.jl

    MLUtils.jl must be installed and loaded before using this.

    source

    ',4))])])}const P=n(o,[["render",F]]);export{I as __pageData,P as default}; +import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

    Distributed Utils ​

    Note

    These functionalities are available via the Lux.DistributedUtils module.

    Backends ​

    ',3)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
    julia
    MPIBackend(comm = nothing)

    Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

    source

    ',3))]),s("details",c,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
    julia
    NCCLBackend(comm = nothing, mpi_backend = nothing)

    Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

    source

    ',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
    julia
    initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

    Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

    Possible values for backend are:

    source

    ',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
    julia
    initialized(backend::Type{<:AbstractLuxDistributedBackend})

    Check if the given backend is initialized.

    source

    ',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
    julia
    get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

    Get the distributed backend for the given backend type. Possible values are:

    Danger

    initialize(backend; kwargs...) must be called before calling this function.

    source

    ',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
    julia
    local_rank(backend::AbstractLuxDistributedBackend)

    Get the local rank for the given backend.

    source

    ',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
    julia
    total_workers(backend::AbstractLuxDistributedBackend)

    Get the total number of workers for the given backend.

    source

    ',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",m,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
    julia
    allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
+allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

    Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",E,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
    julia
    bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
+bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

    Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

    source

    `,3))]),s("details",C,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
    julia
    reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
+reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

    Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",f,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
    julia
    synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

    Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

    source

    ',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
    julia
    DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

    Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

    Arguments

    source

    ',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
    julia
    DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

    data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

    Load MLUtils.jl

    MLUtils.jl must be installed and loaded before using this.

    source

    ',4))])])}const P=n(o,[["render",F]]);export{I as __pageData,P as default}; diff --git a/previews/PR1000/assets/api_Lux_distributed_utils.md.DrmUj9oe.lean.js b/previews/PR1000/assets/api_Lux_distributed_utils.md.DUxpyBki.lean.js similarity index 93% rename from previews/PR1000/assets/api_Lux_distributed_utils.md.DrmUj9oe.lean.js rename to previews/PR1000/assets/api_Lux_distributed_utils.md.DUxpyBki.lean.js index 5b85acbb16..7b0fdf8137 100644 --- a/previews/PR1000/assets/api_Lux_distributed_utils.md.DrmUj9oe.lean.js +++ b/previews/PR1000/assets/api_Lux_distributed_utils.md.DUxpyBki.lean.js @@ -1,4 +1,4 @@ -import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},c={class:"jldocstring custom-block"},k={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

    Distributed Utils ​

    Note

    These functionalities are available via the Lux.DistributedUtils module.

    Backends ​

    ',3)),s("details",c,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
    julia
    MPIBackend(comm = nothing)

    Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

    source

    ',3))]),s("details",k,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
    julia
    NCCLBackend(comm = nothing, mpi_backend = nothing)

    Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

    source

    ',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
    julia
    initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

    Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

    Possible values for backend are:

    source

    ',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
    julia
    initialized(backend::Type{<:AbstractLuxDistributedBackend})

    Check if the given backend is initialized.

    source

    ',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
    julia
    get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

    Get the distributed backend for the given backend type. Possible values are:

    Danger

    initialize(backend; kwargs...) must be called before calling this function.

    source

    ',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
    julia
    local_rank(backend::AbstractLuxDistributedBackend)

    Get the local rank for the given backend.

    source

    ',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
    julia
    total_workers(backend::AbstractLuxDistributedBackend)

    Get the total number of workers for the given backend.

    source

    ',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",m,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
    julia
    allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
-allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

    Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",E,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
    julia
    bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
-bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

    Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

    source

    `,3))]),s("details",C,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
    julia
    reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
-reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

    Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",f,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
    julia
    synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

    Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

    source

    ',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
    julia
    DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

    Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

    Arguments

    source

    ',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
    julia
    DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

    data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

    Load MLUtils.jl

    MLUtils.jl must be installed and loaded before using this.

    source

    ',4))])])}const P=n(o,[["render",F]]);export{I as __pageData,P as default}; +import{_ as n,c as d,a2 as e,j as s,a as t,G as l,B as r,o as p}from"./chunks/framework.DjZDIZsN.js";const I=JSON.parse('{"title":"Distributed Utils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/distributed_utils.md","filePath":"api/Lux/distributed_utils.md","lastUpdated":null}'),o={name:"api/Lux/distributed_utils.md"},k={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},h={class:"jldocstring custom-block"},u={class:"jldocstring custom-block"},b={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},C={class:"jldocstring custom-block"},f={class:"jldocstring custom-block"},L={class:"jldocstring custom-block"},j={class:"jldocstring custom-block"};function F(v,i,x,D,B,U){const a=r("Badge");return p(),d("div",null,[i[39]||(i[39]=e('

    Distributed Utils ​

    Note

    These functionalities are available via the Lux.DistributedUtils module.

    Backends ​

    ',3)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.MPIBackend",href:"#Lux.MPIBackend"},[s("span",{class:"jlbinding"},"Lux.MPIBackend")],-1)),i[1]||(i[1]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=e('
    julia
    MPIBackend(comm = nothing)

    Create an MPI backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(MPIBackend).

    source

    ',3))]),s("details",c,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.NCCLBackend",href:"#Lux.NCCLBackend"},[s("span",{class:"jlbinding"},"Lux.NCCLBackend")],-1)),i[4]||(i[4]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[5]||(i[5]=e('
    julia
    NCCLBackend(comm = nothing, mpi_backend = nothing)

    Create an NCCL backend for distributed training. Users should not use this function directly. Instead use DistributedUtils.get_distributed_backend(NCCLBackend).

    source

    ',3))]),i[40]||(i[40]=s("h2",{id:"initialization",tabindex:"-1"},[t("Initialization "),s("a",{class:"header-anchor",href:"#initialization","aria-label":'Permalink to "Initialization"'},"​")],-1)),s("details",h,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.DistributedUtils.initialize",href:"#Lux.DistributedUtils.initialize"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialize")],-1)),i[7]||(i[7]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=e('
    julia
    initialize(backend::Type{<:AbstractLuxDistributedBackend}; kwargs...)

    Initialize the given backend. Users can supply cuda_devices and amdgpu_devices to initialize the backend with the given devices. These can be set to missing to prevent initialization of the given device type. If set to nothing, and the backend is functional we assign GPUs in a round-robin fashion. Finally, a list of integers can be supplied to initialize the backend with the given devices.

    Possible values for backend are:

    source

    ',5))]),s("details",u,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.DistributedUtils.initialized",href:"#Lux.DistributedUtils.initialized"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.initialized")],-1)),i[10]||(i[10]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=e('
    julia
    initialized(backend::Type{<:AbstractLuxDistributedBackend})

    Check if the given backend is initialized.

    source

    ',3))]),s("details",b,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.DistributedUtils.get_distributed_backend",href:"#Lux.DistributedUtils.get_distributed_backend"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.get_distributed_backend")],-1)),i[13]||(i[13]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=e('
    julia
    get_distributed_backend(backend::Type{<:AbstractLuxDistributedBackend})

    Get the distributed backend for the given backend type. Possible values are:

    Danger

    initialize(backend; kwargs...) must be called before calling this function.

    source

    ',5))]),i[41]||(i[41]=s("h2",{id:"Helper-Functions",tabindex:"-1"},[t("Helper Functions "),s("a",{class:"header-anchor",href:"#Helper-Functions","aria-label":'Permalink to "Helper Functions {#Helper-Functions}"'},"​")],-1)),s("details",g,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.DistributedUtils.local_rank",href:"#Lux.DistributedUtils.local_rank"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.local_rank")],-1)),i[16]||(i[16]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=e('
    julia
    local_rank(backend::AbstractLuxDistributedBackend)

    Get the local rank for the given backend.

    source

    ',3))]),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.DistributedUtils.total_workers",href:"#Lux.DistributedUtils.total_workers"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.total_workers")],-1)),i[19]||(i[19]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[20]||(i[20]=e('
    julia
    total_workers(backend::AbstractLuxDistributedBackend)

    Get the total number of workers for the given backend.

    source

    ',3))]),i[42]||(i[42]=s("h2",{id:"Communication-Primitives",tabindex:"-1"},[t("Communication Primitives "),s("a",{class:"header-anchor",href:"#Communication-Primitives","aria-label":'Permalink to "Communication Primitives {#Communication-Primitives}"'},"​")],-1)),s("details",m,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.DistributedUtils.allreduce!",href:"#Lux.DistributedUtils.allreduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.allreduce!")],-1)),i[22]||(i[22]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[23]||(i[23]=e(`
    julia
    allreduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op)
+allreduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op)

    Backend Agnostic API to perform an allreduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",E,[s("summary",null,[i[24]||(i[24]=s("a",{id:"Lux.DistributedUtils.bcast!",href:"#Lux.DistributedUtils.bcast!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.bcast!")],-1)),i[25]||(i[25]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[26]||(i[26]=e(`
    julia
    bcast!(backend::AbstractLuxDistributedBackend, sendrecvbuf; root::Int=0)
+bcast!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf; root::Int=0)

    Backend Agnostic API to broadcast the given buffer sendrecvbuf or sendbuf to all workers into recvbuf. The value at root will be broadcasted to all other workers.

    source

    `,3))]),s("details",C,[s("summary",null,[i[27]||(i[27]=s("a",{id:"Lux.DistributedUtils.reduce!",href:"#Lux.DistributedUtils.reduce!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.reduce!")],-1)),i[28]||(i[28]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[29]||(i[29]=e(`
    julia
    reduce!(backend::AbstractLuxDistributedBackend, sendrecvbuf, op; root::Int=0)
+reduce!(backend::AbstractLuxDistributedBackend, sendbuf, recvbuf, op; root::Int=0)

    Backend Agnostic API to perform a reduce operation on the given buffer sendrecvbuf or sendbuf and store the result in recvbuf.

    op allows a special DistributedUtils.avg operation that averages the result across all workers.

    source

    `,4))]),s("details",f,[s("summary",null,[i[30]||(i[30]=s("a",{id:"Lux.DistributedUtils.synchronize!!",href:"#Lux.DistributedUtils.synchronize!!"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.synchronize!!")],-1)),i[31]||(i[31]=t()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[32]||(i[32]=e('
    julia
    synchronize!!(backend::AbstractLuxDistributedBackend, ps; root::Int=0)

    Synchronize the given structure ps using the given backend. The value at root will be broadcasted to all other workers.

    source

    ',3))]),i[43]||(i[43]=s("h2",{id:"Optimizers.jl-Integration",tabindex:"-1"},[t("Optimizers.jl Integration "),s("a",{class:"header-anchor",href:"#Optimizers.jl-Integration","aria-label":'Permalink to "Optimizers.jl Integration {#Optimizers.jl-Integration}"'},"​")],-1)),s("details",L,[s("summary",null,[i[33]||(i[33]=s("a",{id:"Lux.DistributedUtils.DistributedOptimizer",href:"#Lux.DistributedUtils.DistributedOptimizer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedOptimizer")],-1)),i[34]||(i[34]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[35]||(i[35]=e('
    julia
    DistributedOptimizer(backend::AbstractLuxDistributedBacked, optimizer)

    Wrap the optimizer in a DistributedOptimizer. Before updating the parameters, this averages the gradients across the processes using Allreduce.

    Arguments

    source

    ',5))]),i[44]||(i[44]=s("h2",{id:"MLUtils.jl-Integration",tabindex:"-1"},[t("MLUtils.jl Integration "),s("a",{class:"header-anchor",href:"#MLUtils.jl-Integration","aria-label":'Permalink to "MLUtils.jl Integration {#MLUtils.jl-Integration}"'},"​")],-1)),s("details",j,[s("summary",null,[i[36]||(i[36]=s("a",{id:"Lux.DistributedUtils.DistributedDataContainer",href:"#Lux.DistributedUtils.DistributedDataContainer"},[s("span",{class:"jlbinding"},"Lux.DistributedUtils.DistributedDataContainer")],-1)),i[37]||(i[37]=t()),l(a,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[38]||(i[38]=e('
    julia
    DistributedDataContainer(backend::AbstractLuxDistributedBackend, data)

    data must be compatible with MLUtils interface. The returned container is compatible with MLUtils interface and is used to partition the dataset across the available processes.

    Load MLUtils.jl

    MLUtils.jl must be installed and loaded before using this.

    source

    ',4))])])}const P=n(o,[["render",F]]);export{I as __pageData,P as default}; diff --git a/previews/PR1000/assets/api_Lux_interop.md.xen3KeOd.js b/previews/PR1000/assets/api_Lux_interop.md.CfzA7PpN.js similarity index 97% rename from previews/PR1000/assets/api_Lux_interop.md.xen3KeOd.js rename to previews/PR1000/assets/api_Lux_interop.md.CfzA7PpN.js index 5abcf6f11c..42bb649dea 100644 --- a/previews/PR1000/assets/api_Lux_interop.md.xen3KeOd.js +++ b/previews/PR1000/assets/api_Lux_interop.md.CfzA7PpN.js @@ -1,4 +1,4 @@ -import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.DjZDIZsN.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('

    Interoperability between Lux and other packages ​

    Switching from older frameworks ​

    Flux Models to Lux Models ​

    Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.

    Tip

    Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.

    ',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
    julia
    Adapt.adapt(from::FromFluxAdaptor, L)

    Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.

    source

    ',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
    julia
    FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)

    Convert a Flux Model to Lux Model.

    active field

    This always ignores the active field of some of the Flux layers. This is almost never going to be supported.

    Keyword Arguments

    Example

    julia
    julia> import Flux
+import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.DjZDIZsN.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('
    Interoperability between Lux and other packages ​
    Switching from older frameworks ​
    Flux Models to Lux Models ​
    Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.
    Tip
    Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.
    ',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
    julia
    Adapt.adapt(from::FromFluxAdaptor, L)
    Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.
    source
    ',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
    julia
    FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)
    Convert a Flux Model to Lux Model.
    active field
    This always ignores the active field of some of the Flux layers. This is almost never going to be supported.
    Keyword Arguments
    • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.
    • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.
    Example
    julia
    julia> import Flux
 
 julia> using Adapt, Lux, Random
 
@@ -11,7 +11,7 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> ps, st = Lux.setup(Random.default_rng(), m2);
 
 julia> size(first(m2(x, ps, st)))
-(2, 32)
    source
    `,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
    julia
    FluxLayer(layer)
    Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
    Warning
    Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
    Warning
    Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
    Arguments
    • layer: Flux layer
    Parameters
    • p: Flattened parameters of the layer
    source
    ',9))]),s[19]||(s[19]=a('
    Using a different backend for Lux ​
    Lux Models to Simple Chains ​
    SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
    Tip
    Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
    ',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
    julia
    Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
    Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
    source
    ',3))]),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
    julia
    ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
    Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
    Warning
    There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
    Warning
    Any kind of initialization function is not preserved when converting to SimpleChains.jl.
    Arguments
    • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
    • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
    Example
    julia
    julia> import SimpleChains
+(2, 32)
    source
    `,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
    julia
    FluxLayer(layer)
    Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
    Warning
    Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
    Warning
    Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
    Arguments
    • layer: Flux layer
    Parameters
    • p: Flattened parameters of the layer
    source
    ',9))]),s[19]||(s[19]=a('
    Using a different backend for Lux ​
    Lux Models to Simple Chains ​
    SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
    Tip
    Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
    ',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
    julia
    Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
    Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
    source
    ',3))]),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
    julia
    ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
    Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
    Warning
    There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
    Warning
    Any kind of initialization function is not preserved when converting to SimpleChains.jl.
    Arguments
    • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
    • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
    Example
    julia
    julia> import SimpleChains
 
 julia> using Adapt, Lux, Random
 
@@ -28,5 +28,5 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> x = randn(Float32, 28, 28, 1, 1);
 
 julia> size(first(simple_chains_model(x, ps, st)))
-(10, 1)
    source
    `,9))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
    julia
    SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
-SimpleChainsLayer(layer, lux_layer, to_array)
    Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.
    ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.
    Arguments
    • layer: SimpleChains layer
    • lux_layer: Potentially equivalent Lux layer that is used for printing
    source
    `,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default};
+(10, 1)

    source

    `,9))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
    julia
    SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
+SimpleChainsLayer(layer, lux_layer, to_array)

    Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.

    ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.

    Arguments

    source

    `,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default}; diff --git a/previews/PR1000/assets/api_Lux_interop.md.xen3KeOd.lean.js b/previews/PR1000/assets/api_Lux_interop.md.CfzA7PpN.lean.js similarity index 97% rename from previews/PR1000/assets/api_Lux_interop.md.xen3KeOd.lean.js rename to previews/PR1000/assets/api_Lux_interop.md.CfzA7PpN.lean.js index 5abcf6f11c..42bb649dea 100644 --- a/previews/PR1000/assets/api_Lux_interop.md.xen3KeOd.lean.js +++ b/previews/PR1000/assets/api_Lux_interop.md.CfzA7PpN.lean.js @@ -1,4 +1,4 @@ -import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.DjZDIZsN.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('

    Interoperability between Lux and other packages ​

    Switching from older frameworks ​

    Flux Models to Lux Models ​

    Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.

    Tip

    Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.

    ',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
    julia
    Adapt.adapt(from::FromFluxAdaptor, L)

    Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.

    source

    ',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
    julia
    FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)

    Convert a Flux Model to Lux Model.

    active field

    This always ignores the active field of some of the Flux layers. This is almost never going to be supported.

    Keyword Arguments

    Example

    julia
    julia> import Flux
+import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/framework.DjZDIZsN.js";const A=JSON.parse('{"title":"Interoperability between Lux and other packages","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/interop.md","filePath":"api/Lux/interop.md","lastUpdated":null}'),r={name:"api/Lux/interop.md"},d={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"};function u(F,s,C,m,b,x){const e=h("Badge");return k(),p("div",null,[s[18]||(s[18]=a('
    Interoperability between Lux and other packages ​
    Switching from older frameworks ​
    Flux Models to Lux Models ​
    Flux.jl has been around in the Julia ecosystem for a long time and has a large userbase, hence we provide a way to convert Flux models to Lux models.
    Tip
    Accessing these functions require manually loading Flux, i.e., using Flux must be present somewhere in the code for these to be used.
    ',5)),i("details",d,[i("summary",null,[s[0]||(s[0]=i("a",{id:"Adapt.adapt-Tuple{FromFluxAdaptor, Any}",href:"#Adapt.adapt-Tuple{FromFluxAdaptor, Any}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[1]||(s[1]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[2]||(s[2]=a('
    julia
    Adapt.adapt(from::FromFluxAdaptor, L)
    Adapt a Flux model L to Lux model. See FromFluxAdaptor for more details.
    source
    ',3))]),i("details",o,[i("summary",null,[s[3]||(s[3]=i("a",{id:"Lux.FromFluxAdaptor",href:"#Lux.FromFluxAdaptor"},[i("span",{class:"jlbinding"},"Lux.FromFluxAdaptor")],-1)),s[4]||(s[4]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[5]||(s[5]=a(`
    julia
    FromFluxAdaptor(preserve_ps_st::Bool=false, force_preserve::Bool=false)
    Convert a Flux Model to Lux Model.
    active field
    This always ignores the active field of some of the Flux layers. This is almost never going to be supported.
    Keyword Arguments
    • preserve_ps_st: Set to true to preserve the states and parameters of the layer. This attempts the best possible way to preserve the original model. But it might fail. If you need to override possible failures, set force_preserve to true.
    • force_preserve: Some of the transformations with state and parameters preservation haven't been implemented yet, in these cases, if force_transform is false a warning will be printed and a core Lux layer will be returned. Else, it will create a FluxLayer.
    Example
    julia
    julia> import Flux
 
 julia> using Adapt, Lux, Random
 
@@ -11,7 +11,7 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> ps, st = Lux.setup(Random.default_rng(), m2);
 
 julia> size(first(m2(x, ps, st)))
-(2, 32)
    source
    `,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
    julia
    FluxLayer(layer)
    Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
    Warning
    Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
    Warning
    Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
    Arguments
    • layer: Flux layer
    Parameters
    • p: Flattened parameters of the layer
    source
    ',9))]),s[19]||(s[19]=a('
    Using a different backend for Lux ​
    Lux Models to Simple Chains ​
    SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
    Tip
    Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
    ',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
    julia
    Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
    Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
    source
    ',3))]),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
    julia
    ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
    Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
    Warning
    There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
    Warning
    Any kind of initialization function is not preserved when converting to SimpleChains.jl.
    Arguments
    • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
    • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
    Example
    julia
    julia> import SimpleChains
+(2, 32)
    source
    `,8))]),i("details",E,[i("summary",null,[s[6]||(s[6]=i("a",{id:"Lux.FluxLayer",href:"#Lux.FluxLayer"},[i("span",{class:"jlbinding"},"Lux.FluxLayer")],-1)),s[7]||(s[7]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[8]||(s[8]=a('
    julia
    FluxLayer(layer)
    Serves as a compatibility layer between Flux and Lux. This uses Optimisers.destructure API internally.
    Warning
    Lux was written to overcome the limitations of destructure + Flux. It is recommended to rewrite your layer in Lux instead of using this layer.
    Warning
    Introducing this Layer in your model will lead to type instabilities, given the way Optimisers.destructure works.
    Arguments
    • layer: Flux layer
    Parameters
    • p: Flattened parameters of the layer
    source
    ',9))]),s[19]||(s[19]=a('
    Using a different backend for Lux ​
    Lux Models to Simple Chains ​
    SimpleChains.jl provides a way to train Small Neural Networks really fast on CPUs. See this blog post for more details. This section describes how to convert Lux models to SimpleChains models while preserving the layer interface.
    Tip
    Accessing these functions require manually loading SimpleChains, i.e., using SimpleChains must be present somewhere in the code for these to be used.
    ',4)),i("details",g,[i("summary",null,[s[9]||(s[9]=i("a",{id:"Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}",href:"#Adapt.adapt-Tuple{ToSimpleChainsAdaptor, AbstractLuxLayer}"},[i("span",{class:"jlbinding"},"Adapt.adapt")],-1)),s[10]||(s[10]=t()),l(e,{type:"info",class:"jlObjectType jlMethod",text:"Method"})]),s[11]||(s[11]=a('
    julia
    Adapt.adapt(from::ToSimpleChainsAdaptor, L::AbstractLuxLayer)
    Adapt a Simple Chains model to Lux model. See ToSimpleChainsAdaptor for more details.
    source
    ',3))]),i("details",y,[i("summary",null,[s[12]||(s[12]=i("a",{id:"Lux.ToSimpleChainsAdaptor",href:"#Lux.ToSimpleChainsAdaptor"},[i("span",{class:"jlbinding"},"Lux.ToSimpleChainsAdaptor")],-1)),s[13]||(s[13]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[14]||(s[14]=a(`
    julia
    ToSimpleChainsAdaptor(input_dims, convert_to_array::Bool=false)
    Adaptor for converting a Lux Model to SimpleChains. The returned model is still a Lux model, and satisfies the AbstractLuxLayer interfacem but all internal calculations are performed using SimpleChains.
    Warning
    There is no way to preserve trained parameters and states when converting to SimpleChains.jl.
    Warning
    Any kind of initialization function is not preserved when converting to SimpleChains.jl.
    Arguments
    • input_dims: Tuple of input dimensions excluding the batch dimension. These must be of static type as SimpleChains expects.
    • convert_to_array: SimpleChains.jl by default outputs StrideArraysCore.StrideArray, but this might not compose well with other packages. If convert_to_array is set to true, the output will be converted to a regular Array.
    Example
    julia
    julia> import SimpleChains
 
 julia> using Adapt, Lux, Random
 
@@ -28,5 +28,5 @@ import{_ as n,c as p,a2 as a,j as i,a as t,G as l,B as h,o as k}from"./chunks/fr
 julia> x = randn(Float32, 28, 28, 1, 1);
 
 julia> size(first(simple_chains_model(x, ps, st)))
-(10, 1)
    source
    `,9))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
    julia
    SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
-SimpleChainsLayer(layer, lux_layer, to_array)
    Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.
    ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.
    Arguments
    • layer: SimpleChains layer
    • lux_layer: Potentially equivalent Lux layer that is used for printing
    source
    `,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default};
+(10, 1)

    source

    `,9))]),i("details",c,[i("summary",null,[s[15]||(s[15]=i("a",{id:"Lux.SimpleChainsLayer",href:"#Lux.SimpleChainsLayer"},[i("span",{class:"jlbinding"},"Lux.SimpleChainsLayer")],-1)),s[16]||(s[16]=t()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),s[17]||(s[17]=a(`
    julia
    SimpleChainsLayer(layer, to_array::Union{Bool, Val}=Val(false))
+SimpleChainsLayer(layer, lux_layer, to_array)

    Wraps a SimpleChains layer into a Lux layer. All operations are performed using SimpleChains but the layer satisfies the AbstractLuxLayer interface.

    ToArray is a boolean flag that determines whether the output should be converted to a regular Array or not. Default is false.

    Arguments

    source

    `,6))])])}const L=n(r,[["render",u]]);export{A as __pageData,L as default}; diff --git a/previews/PR1000/assets/api_Lux_layers.md.sNH-X5ys.js b/previews/PR1000/assets/api_Lux_layers.md.mQAmD5HD.js similarity index 99% rename from previews/PR1000/assets/api_Lux_layers.md.sNH-X5ys.js rename to previews/PR1000/assets/api_Lux_layers.md.mQAmD5HD.js index 76ff91c0b6..3c319491bb 100644 --- a/previews/PR1000/assets/api_Lux_layers.md.sNH-X5ys.js +++ b/previews/PR1000/assets/api_Lux_layers.md.mQAmD5HD.js @@ -5,7 +5,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr layer_2 = NoOpLayer(), layer_3 = NoOpLayer(), ) # Total: 0 parameters, - # plus 0 states.

    source

    `,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
    julia
    Chain(layers...; name=nothing)
+          #        plus 0 states.

    source

    `,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
    julia
    Chain(layers...; name=nothing)
 Chain(; layers..., name=nothing)

    Collects multiple layers / functions to be called in sequence on a given input.

    Arguments

    Extended Help

    Inputs

    Input x is passed sequentially to each layer, and must conform to the input requirements of the internal layers.

    Returns

    Parameters

    States

    Miscellaneous Properties

    Example

    julia
    julia> Chain(Dense(2, 3, relu), BatchNorm(3), Dense(3, 2))
 Chain(
     layer_1 = Dense(2 => 3, relu),      # 9 parameters
@@ -20,7 +20,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_2 = BatchNorm(3, affine=true, track_stats=true),  # 6 parameters, plus 7
     layer_3 = Dense(3 => 2),            # 8 parameters
 )         # Total: 23 parameters,
-          #        plus 7 states.

    source

    `,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
    julia
    PairwiseFusion(connection, layers...; name=nothing)
+          #        plus 7 states.

    source

    `,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
    julia
    PairwiseFusion(connection, layers...; name=nothing)
 PairwiseFusion(connection; name=nothing, layers...)
 PairwiseFusion(; connection, layers..., name=nothing)
    x1 → layer1 → y1 ↘
                   connection → layer2 → y2 ↘
@@ -31,7 +31,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 end
    1. Any other kind of input
    julia
    y = x
 for i in 1:N
     y = connection(x, layers[i](y))
-end

    Returns

    Parameters

    States

    source

    `,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
    julia
    Parallel(connection, layers...; name=nothing)
+end

    Returns

    Parameters

    States

    source

    `,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
    julia
    Parallel(connection, layers...; name=nothing)
 Parallel(connection; name=nothing, layers...)
 Parallel(; connection, layers..., name=nothing)

    Create a layer which passes an input to each path in layers, before reducing the output with connection.

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    See also SkipConnection which is Parallel with one identity.

    Example

    julia
    julia> model = Parallel(nothing, Dense(2, 1), Dense(2, 1))
 Parallel(
@@ -47,37 +47,37 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x2 = randn(rng, Float32, 2);
 
 julia> size.(first(model((x1, x2), ps, st)))
-((1,), (1,))

    source

    `,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
    julia
    SkipConnection(layers, connection; name=nothing)
-SkipConnection(; layers, connection, name=nothing)

    Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

    The simplest "ResNet"-type connection is just SkipConnection(layer, +).

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    See Parallel for a more general implementation.

    source

    `,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
    julia
    RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

    Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

    julia
    res = x
+((1,), (1,))

    source

    `,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
    julia
    SkipConnection(layers, connection; name=nothing)
+SkipConnection(; layers, connection, name=nothing)

    Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

    The simplest "ResNet"-type connection is just SkipConnection(layer, +).

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    See Parallel for a more general implementation.

    source

    `,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
    julia
    RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

    Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

    julia
    res = x
 for i in 1:repeats
     res, st = model(res, ps, st)
 end
    julia
    res = x
 for i in 1:repeats
     res, st = model((res, x), ps, st)
-end

    It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
    julia
    Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+end

    It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
    julia
    Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
      activation=identity; init_weight=nothing, init_bias=nothing, stride=1,
-     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

    Standard convolutional layer.

    Conv 2D

    Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

    Warning

    Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

    Arguments

    Extended Help

    Keyword Arguments

    Inputs

    Returns

    `,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('

    Parameters

    source

    ',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
    julia
    ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

    Standard convolutional layer.

    Conv 2D

    Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

    Warning

    Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

    Arguments

    Extended Help

    Keyword Arguments

    Inputs

    Returns

    `,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('

    Parameters

    source

    ',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
    julia
    ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
               activation=identity; init_weight=glorot_uniform, init_bias=zeros32,
               stride=1, pad=0, outpad=0, dilation=1, groups=1, use_bias=True(),
-              cross_correlation=False())

    Standard convolutional transpose layer.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    `,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
    julia
    AlphaDropout(p::Real)

    AlphaDropout layer.

    Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also Dropout, VariationalHiddenDropout

    source

    ',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
    julia
    Dropout(p; dims=:)

    Dropout layer.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, VariationalHiddenDropout

    source

    ',15))]),t("details",x,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
    julia
    VariationalHiddenDropout(p; dims=:)

    VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, Dropout

    source

    ',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
    julia
    AdaptiveLPPool(output_size; p=2)

    Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
    julia
    AdaptiveMaxPool(output_size)

    Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
    julia
    AdaptiveMeanPool(output_size)

    Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
    julia
    GlobalLPPool(; p=2)

    Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
    julia
    GlobalMaxPool()

    Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
    julia
    GlobalMeanPool()

    Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
    julia
    LPPool(window; stride=window, pad=0, dilation=1, p=2)

    LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

    Arguments

    Keyword Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Extended Help

    Inputs

    Returns

    ',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
    julia
    MaxPool(window; stride=window, pad=0, dilation=1)

    Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
    julia
    MeanPool(window; stride=window, pad=0, dilation=1)

    Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",R,[(Q(),n("svg",N,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
    julia
    GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
-        init_weight=nothing, init_bias=nothing, init_state=zeros32)

    Gated Recurrent Unit (GRU) Cell

    `,2)),t("mjx-container",z,[(Q(),n("svg",P,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",I,[(Q(),n("svg",S,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
    julia
    LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
+              cross_correlation=False())

    Standard convolutional transpose layer.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    `,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
    julia
    AlphaDropout(p::Real)

    AlphaDropout layer.

    Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also Dropout, VariationalHiddenDropout

    source

    ',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
    julia
    Dropout(p; dims=:)

    Dropout layer.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, VariationalHiddenDropout

    source

    ',15))]),t("details",x,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
    julia
    VariationalHiddenDropout(p; dims=:)

    VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, Dropout

    source

    ',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
    julia
    AdaptiveLPPool(output_size; p=2)

    Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
    julia
    AdaptiveMaxPool(output_size)

    Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
    julia
    AdaptiveMeanPool(output_size)

    Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
    julia
    GlobalLPPool(; p=2)

    Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
    julia
    GlobalMaxPool()

    Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
    julia
    GlobalMeanPool()

    Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
    julia
    LPPool(window; stride=window, pad=0, dilation=1, p=2)

    LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

    Arguments

    Keyword Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Extended Help

    Inputs

    Returns

    ',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
    julia
    MaxPool(window; stride=window, pad=0, dilation=1)

    Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
    julia
    MeanPool(window; stride=window, pad=0, dilation=1)

    Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",R,[(Q(),n("svg",N,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
    julia
    GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
+        init_weight=nothing, init_bias=nothing, init_state=zeros32)

    Gated Recurrent Unit (GRU) Cell

    `,2)),t("mjx-container",z,[(Q(),n("svg",P,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",I,[(Q(),n("svg",S,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
    julia
    LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
          train_memory::Bool=false, init_weight=nothing, init_bias=nothing,
-         init_state=zeros32, init_memory=zeros32)

    Long Short-Term (LSTM) Cell

    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("

  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)

  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)

  • hidden_state: Initial hidden state vector (not present if train_state=false)

  • memory: Initial memory vector (not present if train_memory=false)

    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
-    init_bias=nothing, init_weight=nothing, init_state=zeros32)

    An Elman RNNCell cell with activation (typically set to tanh or relu).

    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('

    Parameters

    States

    source

    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
+         init_state=zeros32, init_memory=zeros32)

    Long Short-Term (LSTM) Cell

    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("

  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)

  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)

  • hidden_state: Initial hidden state vector (not present if train_state=false)

  • memory: Initial memory vector (not present if train_memory=false)

    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
+    init_bias=nothing, init_weight=nothing, init_state=zeros32)

    An Elman RNNCell cell with activation (typically set to tanh or relu).

    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('

    Parameters

    States

    source

    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
     ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex(),
     return_sequence::Bool=false)

    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.

    Relation to Flux.Recur

    This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Tip

    Frameworks like Tensorflow have special implementation of StackedRNNCells to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.

    Chain(
     Recurrence(RNNCell(inputsize => latentsize); return_sequence=true),
     Recurrence(RNNCell(latentsize => latentsize); return_sequence=true),
     :
     x -> stack(x; dims=2)
-)

    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

    source

    `,14))]),t("details",x1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)

    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

    Arguments

    Inputs

    Returns

    States

    source

    ',12))]),t("details",b1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
+)

    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

    source

    `,14))]),t("details",x1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)

    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

    Arguments

    Inputs

    Returns

    States

    source

    ',12))]),t("details",b1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
     backward_cell::Union{AbstractRecurrentCell, Nothing}=nothing;
     merge_mode::Union{Function, Nothing}=vcat,
-    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

    Bidirectional RNN wrapper.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
+    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

    Bidirectional RNN wrapper.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
          init_bias=nothing, use_bias=True())
 Bilinear(in12_dims => out, activation=identity; init_weight=nothing,
-         init_bias=nothing, use_bias=True())

    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

    z[i] = activation(x' * W[i, :, :] * y + bias[i])

    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
-      init_bias=nothing, use_bias=True())

    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)

    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

    This layer is often used to store word embeddings and retrieve them using indices.

    Arguments

    Keyword Arguments

    Input

    Returns

    source

    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)

    Flattens the passed array into a matrix.

    Keyword Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = FlattenLayer()
+         init_bias=nothing, use_bias=True())

    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

    z[i] = activation(x' * W[i, :, :] * y + bias[i])

    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
+      init_bias=nothing, use_bias=True())

    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)

    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

    This layer is often used to store word embeddings and retrieve them using indices.

    Arguments

    Keyword Arguments

    Input

    Returns

    source

    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)

    Flattens the passed array into a matrix.

    Keyword Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = FlattenLayer()
 FlattenLayer{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -87,9 +87,9 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(8, 2)

    source

    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
+(8, 2)

    source

    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
 Maxout(; layers...)
-Maxout(f::Function, n_alts::Int)

    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

    See also Parallel to reduce with other operators.

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    References

    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

    source

    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()

    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

    Example

    julia
    julia> model = NoOpLayer()
+Maxout(f::Function, n_alts::Int)

    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

    See also Parallel to reduce with other operators.

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    References

    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

    source

    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()

    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

    Example

    julia
    julia> model = NoOpLayer()
 NoOpLayer()
 
 julia> rng = Random.default_rng();
@@ -99,7 +99,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 1
 
 julia> y, st_new = model(x, ps, st)
-(1, NamedTuple())

    source

    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)

    Reshapes the passed array to have a size of (dims..., :)

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReshapeLayer((2, 2))
+(1, NamedTuple())

    source

    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)

    Reshapes the passed array to have a size of (dims..., :)

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReshapeLayer((2, 2))
 ReshapeLayer(output_dims = (2, 2, :))
 
 julia> rng = Random.default_rng();
@@ -109,7 +109,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(2, 2, 3)

    source

    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)

    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)

    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)

    Reverse the specified dimension dims of the passed array

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReverseSequence()
+(2, 2, 3)

    source

    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)

    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)

    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)

    Reverse the specified dimension dims of the passed array

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReverseSequence()
 ReverseSequence{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -118,7 +118,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x = [1.0, 2.0, 3.0];
 
 julia> y, st_new = model(x, ps, st)
-([3.0, 2.0, 1.0], NamedTuple())

    source

    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+([3.0, 2.0, 1.0], NamedTuple())

    source

    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
           affine=True(), track_stats=True(), epsilon=1f-5, momentum=0.1f0)

    Batch Normalization layer.

    `,2)),t("p",null,[a[222]||(a[222]=t("code",null,"BatchNorm",-1)),a[223]||(a[223]=s(" computes the mean and variance for each ")),t("mjx-container",R1,[(Q(),n("svg",N1,a[220]||(a[220]=[i('',1)]))),a[221]||(a[221]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mi",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mi",null,"×"),t("mn",null,"1"),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),a[224]||(a[224]=s(" input slice and normalises the input accordingly."))]),a[226]||(a[226]=i(`

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    Use Lux.testmode during inference.

    Example

    julia
    julia> Chain(Dense(784 => 64), BatchNorm(64, relu), Dense(64 => 10), BatchNorm(10))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -126,7 +126,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = BatchNorm(10, affine=true, track_stats=true),  # 20 parameters, plus 21
 )         # Total: 51_038 parameters,
-          #        plus 150 states.

    Warning

    Passing a batch size of 1, during training will result in an error.

    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
+          #        plus 150 states.

    Warning

    Passing a batch size of 1, during training will result in an error.

    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
           init_scale=ones32, affine=true, epsilon=1f-5)

    Group Normalization layer.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    Use Lux.testmode during inference.

    Example

    julia
    julia> Chain(Dense(784 => 64), GroupNorm(64, 4, relu), Dense(64 => 10), GroupNorm(10, 5))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -134,7 +134,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = GroupNorm(10, 5, affine=true),  # 20 parameters
 )         # Total: 51_038 parameters,
-          #        plus 0 states.

    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+          #        plus 0 states.

    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
              affine=False(), track_stats=False(), epsilon=1f-5, momentum=0.1f0)

    Instance Normalization. For details see [1].

    `,2)),t("p",null,[a[234]||(a[234]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",P1,[(Q(),n("svg",I1,a[232]||(a[232]=[i('',1)]))),a[233]||(a[233]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),a[235]||(a[235]=s("` input slice and normalises the input accordingly."))]),a[237]||(a[237]=i(`

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    Use Lux.testmode during inference.

    Example

    julia
    julia> Chain(Dense(784 => 64), InstanceNorm(64, relu; affine=true), Dense(64 => 10),
            InstanceNorm(10, relu; affine=true))
 Chain(
@@ -143,7 +143,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = InstanceNorm(10, relu, affine=true, track_stats=false),  # 20 parameters, plus 1
 )         # Total: 51_038 parameters,
-          #        plus 2 states.

    References

    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",S1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
-          affine=true, init_bias=zeros32, init_scale=ones32)

    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
-           dims::Union{Tuple, Nothing}=nothing)

    Applies weight normalization to a parameter in the given layer.

    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('

    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

    Arguments

    Inputs

    Returns

    Parameters

    States

    source

    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)

    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

    See NNlib.pixel_shuffle for more details.

    Arguments

    Inputs

    Returns

    source

    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
-Upsample(scale, mode = :nearest)

    Upsampling Layer.

    Layer Construction

    Option 1

    Exactly one of two keywords must be specified:

    Option 2

    Currently supported upsampling modes and corresponding NNlib's methods are:

    Extended Help

    Other Keyword Arguments

    Inputs

    Returns

    source

    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default}; + # plus 2 states.

    References

    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",S1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
+          affine=true, init_bias=zeros32, init_scale=ones32)

    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
+           dims::Union{Tuple, Nothing}=nothing)

    Applies weight normalization to a parameter in the given layer.

    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('

    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

    Arguments

    Inputs

    Returns

    Parameters

    States

    source

    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)

    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

    See NNlib.pixel_shuffle for more details.

    Arguments

    Inputs

    Returns

    source

    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
+Upsample(scale, mode = :nearest)

    Upsampling Layer.

    Layer Construction

    Option 1

    Exactly one of two keywords must be specified:

    Option 2

    Currently supported upsampling modes and corresponding NNlib's methods are:

    Extended Help

    Other Keyword Arguments

    Inputs

    Returns

    source

    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default}; diff --git a/previews/PR1000/assets/api_Lux_layers.md.sNH-X5ys.lean.js b/previews/PR1000/assets/api_Lux_layers.md.mQAmD5HD.lean.js similarity index 99% rename from previews/PR1000/assets/api_Lux_layers.md.sNH-X5ys.lean.js rename to previews/PR1000/assets/api_Lux_layers.md.mQAmD5HD.lean.js index 76ff91c0b6..3c319491bb 100644 --- a/previews/PR1000/assets/api_Lux_layers.md.sNH-X5ys.lean.js +++ b/previews/PR1000/assets/api_Lux_layers.md.mQAmD5HD.lean.js @@ -5,7 +5,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr layer_2 = NoOpLayer(), layer_3 = NoOpLayer(), ) # Total: 0 parameters, - # plus 0 states.

    source

    `,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
    julia
    Chain(layers...; name=nothing)
+          #        plus 0 states.

    source

    `,18))]),t("details",p,[t("summary",null,[a[3]||(a[3]=t("a",{id:"Lux.Chain",href:"#Lux.Chain"},[t("span",{class:"jlbinding"},"Lux.Chain")],-1)),a[4]||(a[4]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[5]||(a[5]=i(`
    julia
    Chain(layers...; name=nothing)
 Chain(; layers..., name=nothing)

    Collects multiple layers / functions to be called in sequence on a given input.

    Arguments

    Extended Help

    Inputs

    Input x is passed sequentially to each layer, and must conform to the input requirements of the internal layers.

    Returns

    Parameters

    States

    Miscellaneous Properties

    Example

    julia
    julia> Chain(Dense(2, 3, relu), BatchNorm(3), Dense(3, 2))
 Chain(
     layer_1 = Dense(2 => 3, relu),      # 9 parameters
@@ -20,7 +20,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_2 = BatchNorm(3, affine=true, track_stats=true),  # 6 parameters, plus 7
     layer_3 = Dense(3 => 2),            # 8 parameters
 )         # Total: 23 parameters,
-          #        plus 7 states.

    source

    `,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
    julia
    PairwiseFusion(connection, layers...; name=nothing)
+          #        plus 7 states.

    source

    `,18))]),t("details",h,[t("summary",null,[a[6]||(a[6]=t("a",{id:"Lux.PairwiseFusion",href:"#Lux.PairwiseFusion"},[t("span",{class:"jlbinding"},"Lux.PairwiseFusion")],-1)),a[7]||(a[7]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[8]||(a[8]=i(`
    julia
    PairwiseFusion(connection, layers...; name=nothing)
 PairwiseFusion(connection; name=nothing, layers...)
 PairwiseFusion(; connection, layers..., name=nothing)
    x1 → layer1 → y1 ↘
                   connection → layer2 → y2 ↘
@@ -31,7 +31,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 end
    1. Any other kind of input
    julia
    y = x
 for i in 1:N
     y = connection(x, layers[i](y))
-end

    Returns

    Parameters

    States

    source

    `,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
    julia
    Parallel(connection, layers...; name=nothing)
+end

    Returns

    Parameters

    States

    source

    `,18))]),t("details",m,[t("summary",null,[a[9]||(a[9]=t("a",{id:"Lux.Parallel",href:"#Lux.Parallel"},[t("span",{class:"jlbinding"},"Lux.Parallel")],-1)),a[10]||(a[10]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[11]||(a[11]=i(`
    julia
    Parallel(connection, layers...; name=nothing)
 Parallel(connection; name=nothing, layers...)
 Parallel(; connection, layers..., name=nothing)

    Create a layer which passes an input to each path in layers, before reducing the output with connection.

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    See also SkipConnection which is Parallel with one identity.

    Example

    julia
    julia> model = Parallel(nothing, Dense(2, 1), Dense(2, 1))
 Parallel(
@@ -47,37 +47,37 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x2 = randn(rng, Float32, 2);
 
 julia> size.(first(model((x1, x2), ps, st)))
-((1,), (1,))

    source

    `,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
    julia
    SkipConnection(layers, connection; name=nothing)
-SkipConnection(; layers, connection, name=nothing)

    Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

    The simplest "ResNet"-type connection is just SkipConnection(layer, +).

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    See Parallel for a more general implementation.

    source

    `,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
    julia
    RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

    Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

    julia
    res = x
+((1,), (1,))

    source

    `,17))]),t("details",g,[t("summary",null,[a[12]||(a[12]=t("a",{id:"Lux.SkipConnection",href:"#Lux.SkipConnection"},[t("span",{class:"jlbinding"},"Lux.SkipConnection")],-1)),a[13]||(a[13]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[14]||(a[14]=i(`
    julia
    SkipConnection(layers, connection; name=nothing)
+SkipConnection(; layers, connection, name=nothing)

    Create a skip connection which consists of a layer or Chain of consecutive layers and a shortcut connection linking the block's input to the output through a user-supplied 2-argument callable. The first argument to the callable will be propagated through the given layer while the second is the unchanged, "skipped" input.

    The simplest "ResNet"-type connection is just SkipConnection(layer, +).

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    See Parallel for a more general implementation.

    source

    `,16))]),t("details",k,[t("summary",null,[a[15]||(a[15]=t("a",{id:"Lux.RepeatedLayer",href:"#Lux.RepeatedLayer"},[t("span",{class:"jlbinding"},"Lux.RepeatedLayer")],-1)),a[16]||(a[16]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[17]||(a[17]=i(`
    julia
    RepeatedLayer(model; repeats::Val = Val(10), input_injection::Val = Val(false))

    Iteratively applies model for repeats number of times. The initial input is passed into the model repeatedly if input_injection = Val(true). This layer unrolls the computation, however, semantically this is same as:

    julia
    res = x
 for i in 1:repeats
     res, st = model(res, ps, st)
 end
    julia
    res = x
 for i in 1:repeats
     res, st = model((res, x), ps, st)
-end

    It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
    julia
    Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+end

    It is expected that repeats will be a reasonable number below 20, beyond that compile times for gradients might be unreasonably high.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,21))]),a[271]||(a[271]=t("h2",{id:"Convolutional-Layers",tabindex:"-1"},[s("Convolutional Layers "),t("a",{class:"header-anchor",href:"#Convolutional-Layers","aria-label":'Permalink to "Convolutional Layers {#Convolutional-Layers}"'},"​")],-1)),t("details",c,[t("summary",null,[a[18]||(a[18]=t("a",{id:"Lux.Conv",href:"#Lux.Conv"},[t("span",{class:"jlbinding"},"Lux.Conv")],-1)),a[19]||(a[19]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[22]||(a[22]=i(`
    julia
    Conv(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
      activation=identity; init_weight=nothing, init_bias=nothing, stride=1,
-     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

    Standard convolutional layer.

    Conv 2D

    Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

    Warning

    Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

    Arguments

    Extended Help

    Keyword Arguments

    Inputs

    Returns

    `,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('

    Parameters

    source

    ',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
    julia
    ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
+     pad=0, dilation=1, groups=1, use_bias=True(), cross_correlation=False())

    Standard convolutional layer.

    Conv 2D

    Image data should be stored in WHCN order (width, height, channels, batch). In other words, a 100 x 100 RGB image would be a 100 x 100 x 3 x 1 array, and a batch of 50 would be a 100 x 100 x 3 x 50 array. This has N = 2 spatial dimensions, and needs a kernel size like (5, 5), a 2-tuple of integers. To take convolutions along N feature dimensions, this layer expects as input an array with ndims(x) == N + 2, where size(x, N + 1) == in_chs is the number of input channels, and size(x, ndims(x)) is the number of observations in a batch.

    Warning

    Frameworks like Pytorch perform cross-correlation in their convolution layers. Pass cross_correlation=true to use cross-correlation instead.

    Arguments

    Extended Help

    Keyword Arguments

    Inputs

    Returns

    `,13)),t("mjx-container",u,[(Q(),n("svg",y,a[20]||(a[20]=[i('',1)]))),a[21]||(a[21]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[23]||(a[23]=i('

    Parameters

    source

    ',4))]),t("details",E,[t("summary",null,[a[24]||(a[24]=t("a",{id:"Lux.ConvTranspose",href:"#Lux.ConvTranspose"},[t("span",{class:"jlbinding"},"Lux.ConvTranspose")],-1)),a[25]||(a[25]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[26]||(a[26]=i(`
    julia
    ConvTranspose(k::NTuple{N,Integer}, (in_chs => out_chs)::Pair{<:Integer,<:Integer},
               activation=identity; init_weight=glorot_uniform, init_bias=zeros32,
               stride=1, pad=0, outpad=0, dilation=1, groups=1, use_bias=True(),
-              cross_correlation=False())

    Standard convolutional transpose layer.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    `,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
    julia
    AlphaDropout(p::Real)

    AlphaDropout layer.

    Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also Dropout, VariationalHiddenDropout

    source

    ',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
    julia
    Dropout(p; dims=:)

    Dropout layer.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, VariationalHiddenDropout

    source

    ',15))]),t("details",x,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
    julia
    VariationalHiddenDropout(p; dims=:)

    VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, Dropout

    source

    ',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
    julia
    AdaptiveLPPool(output_size; p=2)

    Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
    julia
    AdaptiveMaxPool(output_size)

    Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
    julia
    AdaptiveMeanPool(output_size)

    Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
    julia
    GlobalLPPool(; p=2)

    Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
    julia
    GlobalMaxPool()

    Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
    julia
    GlobalMeanPool()

    Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
    julia
    LPPool(window; stride=window, pad=0, dilation=1, p=2)

    LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

    Arguments

    Keyword Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Extended Help

    Inputs

    Returns

    ',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
    julia
    MaxPool(window; stride=window, pad=0, dilation=1)

    Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
    julia
    MeanPool(window; stride=window, pad=0, dilation=1)

    Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",R,[(Q(),n("svg",N,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
    julia
    GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
-        init_weight=nothing, init_bias=nothing, init_state=zeros32)

    Gated Recurrent Unit (GRU) Cell

    `,2)),t("mjx-container",z,[(Q(),n("svg",P,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",I,[(Q(),n("svg",S,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
    julia
    LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
+              cross_correlation=False())

    Standard convolutional transpose layer.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    `,14))]),a[272]||(a[272]=t("h2",{id:"Dropout-Layers",tabindex:"-1"},[s("Dropout Layers "),t("a",{class:"header-anchor",href:"#Dropout-Layers","aria-label":'Permalink to "Dropout Layers {#Dropout-Layers}"'},"​")],-1)),t("details",f,[t("summary",null,[a[27]||(a[27]=t("a",{id:"Lux.AlphaDropout",href:"#Lux.AlphaDropout"},[t("span",{class:"jlbinding"},"Lux.AlphaDropout")],-1)),a[28]||(a[28]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[29]||(a[29]=i('
    julia
    AlphaDropout(p::Real)

    AlphaDropout layer.

    Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also Dropout, VariationalHiddenDropout

    source

    ',13))]),t("details",L,[t("summary",null,[a[30]||(a[30]=t("a",{id:"Lux.Dropout",href:"#Lux.Dropout"},[t("span",{class:"jlbinding"},"Lux.Dropout")],-1)),a[31]||(a[31]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[32]||(a[32]=i('
    julia
    Dropout(p; dims=:)

    Dropout layer.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, VariationalHiddenDropout

    source

    ',15))]),t("details",x,[t("summary",null,[a[33]||(a[33]=t("a",{id:"Lux.VariationalHiddenDropout",href:"#Lux.VariationalHiddenDropout"},[t("span",{class:"jlbinding"},"Lux.VariationalHiddenDropout")],-1)),a[34]||(a[34]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[35]||(a[35]=i('
    julia
    VariationalHiddenDropout(p; dims=:)

    VariationalHiddenDropout layer. The only difference from Dropout is that the mask is retained until Lux.update_state(l, :update_mask, Val(true)) is called.

    Arguments

    Keyword Arguments

    Inputs

    Returns

    States

    Call Lux.testmode to switch to test mode.

    See also AlphaDropout, Dropout

    source

    ',15))]),a[273]||(a[273]=t("h2",{id:"Pooling-Layers",tabindex:"-1"},[s("Pooling Layers "),t("a",{class:"header-anchor",href:"#Pooling-Layers","aria-label":'Permalink to "Pooling Layers {#Pooling-Layers}"'},"​")],-1)),t("details",b,[t("summary",null,[a[36]||(a[36]=t("a",{id:"Lux.AdaptiveLPPool",href:"#Lux.AdaptiveLPPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveLPPool")],-1)),a[37]||(a[37]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[38]||(a[38]=i('
    julia
    AdaptiveLPPool(output_size; p=2)

    Adaptive LP Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',10))]),t("details",w,[t("summary",null,[a[39]||(a[39]=t("a",{id:"Lux.AdaptiveMaxPool",href:"#Lux.AdaptiveMaxPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMaxPool")],-1)),a[40]||(a[40]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[41]||(a[41]=i('
    julia
    AdaptiveMaxPool(output_size)

    Adaptive Max Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",H,[t("summary",null,[a[42]||(a[42]=t("a",{id:"Lux.AdaptiveMeanPool",href:"#Lux.AdaptiveMeanPool"},[t("span",{class:"jlbinding"},"Lux.AdaptiveMeanPool")],-1)),a[43]||(a[43]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[44]||(a[44]=i('
    julia
    AdaptiveMeanPool(output_size)

    Adaptive Mean Pooling layer. Calculates the necessary window size such that its output has size(y)[1:N] == output_size.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",C,[t("summary",null,[a[45]||(a[45]=t("a",{id:"Lux.GlobalLPPool",href:"#Lux.GlobalLPPool"},[t("span",{class:"jlbinding"},"Lux.GlobalLPPool")],-1)),a[46]||(a[46]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[47]||(a[47]=i('
    julia
    GlobalLPPool(; p=2)

    Global LP Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    GPU Support

    This layer is currently only supported on CPU.

    Inputs

    Returns

    source

    ',8))]),t("details",F,[t("summary",null,[a[48]||(a[48]=t("a",{id:"Lux.GlobalMaxPool",href:"#Lux.GlobalMaxPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMaxPool")],-1)),a[49]||(a[49]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[50]||(a[50]=i('
    julia
    GlobalMaxPool()

    Global Max Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",D,[t("summary",null,[a[51]||(a[51]=t("a",{id:"Lux.GlobalMeanPool",href:"#Lux.GlobalMeanPool"},[t("span",{class:"jlbinding"},"Lux.GlobalMeanPool")],-1)),a[52]||(a[52]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[53]||(a[53]=i('
    julia
    GlobalMeanPool()

    Global Mean Pooling layer. Transforms (w, h, c, b)-shaped input into (1, 1, c, b)-shaped output, by performing mean pooling on the complete (w, h)-shaped feature maps.

    Inputs

    Returns

    source

    ',7))]),t("details",M,[t("summary",null,[a[54]||(a[54]=t("a",{id:"Lux.LPPool",href:"#Lux.LPPool"},[t("span",{class:"jlbinding"},"Lux.LPPool")],-1)),a[55]||(a[55]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[58]||(a[58]=i('
    julia
    LPPool(window; stride=window, pad=0, dilation=1, p=2)

    LP Pooling layer, which replaces all pixels in a block of size window with the reduction operation: lp.

    Arguments

    Keyword Arguments

    GPU Support

    This layer is currently only supported on CPU.

    Extended Help

    Inputs

    Returns

    ',12)),t("mjx-container",v,[(Q(),n("svg",Z,a[56]||(a[56]=[i('',1)]))),a[57]||(a[57]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[59]||(a[59]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[60]||(a[60]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/pooling.jl#L188-L236",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",j,[t("summary",null,[a[61]||(a[61]=t("a",{id:"Lux.MaxPool",href:"#Lux.MaxPool"},[t("span",{class:"jlbinding"},"Lux.MaxPool")],-1)),a[62]||(a[62]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[65]||(a[65]=i('
    julia
    MaxPool(window; stride=window, pad=0, dilation=1)

    Max Pooling layer, which replaces all pixels in a block of size window with the reduction operation: max.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",A,[(Q(),n("svg",B,a[63]||(a[63]=[i('',1)]))),a[64]||(a[64]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[66]||(a[66]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[67]||(a[67]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",V,[t("summary",null,[a[68]||(a[68]=t("a",{id:"Lux.MeanPool",href:"#Lux.MeanPool"},[t("span",{class:"jlbinding"},"Lux.MeanPool")],-1)),a[69]||(a[69]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[72]||(a[72]=i('
    julia
    MeanPool(window; stride=window, pad=0, dilation=1)

    Mean Pooling layer, which replaces all pixels in a block of size window with the reduction operation: mean.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    ',11)),t("mjx-container",R,[(Q(),n("svg",N,a[70]||(a[70]=[i('',1)]))),a[71]||(a[71]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("msub",null,[t("mi",null,"O"),t("mi",null,"i")]),t("mo",null,"="),t("mrow",{"data-mjx-texclass":"INNER"},[t("mo",{"data-mjx-texclass":"OPEN"},"⌊"),t("mfrac",null,[t("mrow",null,[t("msub",null,[t("mi",null,"I"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mi",null,"i")]),t("mo",null,"+"),t("msub",null,[t("mi",null,"p"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mo",{stretchy:"false"},"("),t("mi",null,"i"),t("mo",null,"+"),t("mi",null,"N"),t("mo",{stretchy:"false"},")"),t("mi",{mathvariant:"normal"},"%"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),t("mi",null,"p"),t("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|")])]),t("mo",null,"−"),t("msub",null,[t("mi",null,"d"),t("mi",null,"i")]),t("mo",null,"×"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"k"),t("mi",null,"i")]),t("mo",null,"−"),t("mn",null,"1"),t("mo",{stretchy:"false"},")")]),t("msub",null,[t("mi",null,"s"),t("mi",null,"i")])]),t("mo",null,"+"),t("mn",null,"1"),t("mo",{"data-mjx-texclass":"CLOSE"},"⌋")])])],-1))]),a[73]||(a[73]=t("ul",null,[t("li",null,[s("Empty "),t("code",null,"NamedTuple()")])],-1)),a[74]||(a[74]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/pooling.jl#L188-L232",target:"_blank",rel:"noreferrer"},"source")],-1))]),a[274]||(a[274]=t("h2",{id:"Recurrent-Layers",tabindex:"-1"},[s("Recurrent Layers "),t("a",{class:"header-anchor",href:"#Recurrent-Layers","aria-label":'Permalink to "Recurrent Layers {#Recurrent-Layers}"'},"​")],-1)),t("details",O,[t("summary",null,[a[75]||(a[75]=t("a",{id:"Lux.GRUCell",href:"#Lux.GRUCell"},[t("span",{class:"jlbinding"},"Lux.GRUCell")],-1)),a[76]||(a[76]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[120]||(a[120]=i(`
    julia
    GRUCell((in_dims, out_dims)::Pair{<:Int,<:Int}; use_bias=true, train_state::Bool=false,
+        init_weight=nothing, init_bias=nothing, init_state=zeros32)

    Gated Recurrent Unit (GRU) Cell

    `,2)),t("mjx-container",z,[(Q(),n("svg",P,a[77]||(a[77]=[i('',1)]))),a[78]||(a[78]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"r")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"z")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"n")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"tanh"),t("mo",{"data-mjx-texclass":"NONE"},"⁡"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",null,"+"),t("mi",null,"r"),t("mo",null,"⋅"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{stretchy:"false"},")"),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mo",{stretchy:"false"},"("),t("mn",null,"1"),t("mo",null,"−"),t("mi",null,"z"),t("mo",{stretchy:"false"},")"),t("mo",null,"⋅"),t("mi",null,"n"),t("mo",null,"+"),t("mi",null,"z"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])])])])])])],-1))]),a[121]||(a[121]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[87]||(a[87]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[81]||(a[81]=s("Output ")),t("mjx-container",I,[(Q(),n("svg",S,a[79]||(a[79]=[i('',1)]))),a[80]||(a[80]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[82]||(a[82]=s(" of shape ")),a[83]||(a[83]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[86]||(a[86]=s("Tuple containing new hidden state ")),t("mjx-container",_,[(Q(),n("svg",G,a[84]||(a[84]=[i('',1)]))),a[85]||(a[85]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[88]||(a[88]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[122]||(a[122]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[91]||(a[91]=t("code",null,"weight_ih",-1)),a[92]||(a[92]=s(": Concatenated Weights to map from input space ")),t("mjx-container",W,[(Q(),n("svg",X,a[89]||(a[89]=[i('',1)]))),a[90]||(a[90]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[93]||(a[93]=s("."))])]),t("li",null,[t("p",null,[a[96]||(a[96]=t("code",null,"weight_hh",-1)),a[97]||(a[97]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",U,[(Q(),n("svg",q,a[94]||(a[94]=[i('',1)]))),a[95]||(a[95]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[98]||(a[98]=s("."))])]),t("li",null,[t("p",null,[a[101]||(a[101]=t("code",null,"bias_ih",-1)),a[102]||(a[102]=s(": Concatenated Bias vector for the input space ")),t("mjx-container",J,[(Q(),n("svg",K,a[99]||(a[99]=[i('',1)]))),a[100]||(a[100]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[103]||(a[103]=s(" (not present if ")),a[104]||(a[104]=t("code",null,"use_bias=false",-1)),a[105]||(a[105]=s(")."))])]),t("li",null,[t("p",null,[a[108]||(a[108]=t("code",null,"bias_hh",-1)),a[109]||(a[109]=s(": Concatenated Bias vector for the hidden space ")),t("mjx-container",$,[(Q(),n("svg",Y,a[106]||(a[106]=[i('',1)]))),a[107]||(a[107]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[110]||(a[110]=s(" (not present if ")),a[111]||(a[111]=t("code",null,"use_bias=false",-1)),a[112]||(a[112]=s(")."))])]),t("li",null,[t("p",null,[a[115]||(a[115]=t("code",null,"hidden_state",-1)),a[116]||(a[116]=s(": Initial hidden state vector (not present if ")),a[117]||(a[117]=t("code",null,"train_state=false",-1)),a[118]||(a[118]=s(") ")),t("mjx-container",t1,[(Q(),n("svg",a1,a[113]||(a[113]=[i('',1)]))),a[114]||(a[114]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"r")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"z")])]),t("mo",null,","),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"n")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[119]||(a[119]=s("."))])])]),a[123]||(a[123]=t("p",null,[t("strong",null,"States")],-1)),a[124]||(a[124]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[125]||(a[125]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/recurrent.jl#L488",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",s1,[t("summary",null,[a[126]||(a[126]=t("a",{id:"Lux.LSTMCell",href:"#Lux.LSTMCell"},[t("span",{class:"jlbinding"},"Lux.LSTMCell")],-1)),a[127]||(a[127]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[153]||(a[153]=i(`
    julia
    LSTMCell(in_dims => out_dims; use_bias::Bool=true, train_state::Bool=false,
          train_memory::Bool=false, init_weight=nothing, init_bias=nothing,
-         init_state=zeros32, init_memory=zeros32)

    Long Short-Term (LSTM) Cell

    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("

  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)

  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)

  • hidden_state: Initial hidden state vector (not present if train_state=false)

  • memory: Initial memory vector (not present if train_memory=false)

    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/778479a21fc81f3ac741cb26289192deb4ec7210/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
-    init_bias=nothing, init_weight=nothing, init_state=zeros32)

    An Elman RNNCell cell with activation (typically set to tanh or relu).

    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('

    Parameters

    States

    source

    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
+         init_state=zeros32, init_memory=zeros32)

    Long Short-Term (LSTM) Cell

    `,2)),t("mjx-container",i1,[(Q(),n("svg",e1,a[128]||(a[128]=[i('',1)]))),a[129]||(a[129]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mtable",{displaystyle:"true",columnalign:"right left",columnspacing:"0em",rowspacing:"3pt"},[t("mtr",null,[t("mtd",null,[t("mi",null,"i")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"f")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"f")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"g")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"g")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("mi",null,"o")]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"σ"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("msub",null,[t("mi",null,"b"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"o")])]),t("mo",{stretchy:"false"},")")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"f"),t("mo",null,"⋅"),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"i"),t("mo",null,"⋅"),t("mi",null,"g")])]),t("mtr",null,[t("mtd",null,[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])]),t("mtd",null,[t("mi"),t("mo",null,"="),t("mi",null,"o"),t("mo",null,"⋅"),t("mi",null,"t"),t("mi",null,"a"),t("mi",null,"n"),t("mi",null,"h"),t("mo",{stretchy:"false"},"("),t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",{stretchy:"false"},")")])])])])],-1))]),a[154]||(a[154]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[141]||(a[141]=t("p",null,"Tuple Containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[132]||(a[132]=s("Output ")),t("mjx-container",l1,[(Q(),n("svg",n1,a[130]||(a[130]=[i('',1)]))),a[131]||(a[131]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[133]||(a[133]=s(" of shape ")),a[134]||(a[134]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[139]||(a[139]=s("Tuple containing new hidden state ")),t("mjx-container",Q1,[(Q(),n("svg",T1,a[135]||(a[135]=[i('',1)]))),a[136]||(a[136]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[140]||(a[140]=s(" and new memory ")),t("mjx-container",d1,[(Q(),n("svg",o1,a[137]||(a[137]=[i('',1)]))),a[138]||(a[138]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"c"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[142]||(a[142]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[155]||(a[155]=t("p",null,[t("strong",null,"Parameters")],-1)),t("ul",null,[t("li",null,[t("p",null,[a[145]||(a[145]=t("code",null,"weight_ih",-1)),a[146]||(a[146]=s(": Concatenated Weights to map from input space ")),t("mjx-container",r1,[(Q(),n("svg",p1,a[143]||(a[143]=[i('',1)]))),a[144]||(a[144]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))]),a[147]||(a[147]=s("."))])]),t("li",null,[t("p",null,[a[150]||(a[150]=t("code",null,"weight_hh",-1)),a[151]||(a[151]=s(": Concatenated Weights to map from hidden space ")),t("mjx-container",h1,[(Q(),n("svg",m1,a[148]||(a[148]=[i('',1)]))),a[149]||(a[149]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mo",{fence:"false",stretchy:"false"},"{"),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"i")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"f")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"g")])]),t("mo",null,","),t("msub",null,[t("mi",null,"W"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"o")])]),t("mo",{fence:"false",stretchy:"false"},"}")])],-1))])])]),a[152]||(a[152]=i("

  • bias_ih: Bias vector for the input-hidden connection (not present if use_bias=false)

  • bias_hh: Concatenated Bias vector for the hidden-hidden connection (not present if use_bias=false)

  • hidden_state: Initial hidden state vector (not present if train_state=false)

  • memory: Initial memory vector (not present if train_memory=false)

    • ",4))]),a[156]||(a[156]=t("p",null,[t("strong",null,"States")],-1)),a[157]||(a[157]=t("ul",null,[t("li",null,[t("code",null,"rng"),s(": Controls the randomness (if any) in the initial state generation")])],-1)),a[158]||(a[158]=t("p",null,[t("a",{href:"https://github.com/LuxDL/Lux.jl/blob/4a020327b3869c808c3f8643560e5a8d914dc27f/src/layers/recurrent.jl#L309",target:"_blank",rel:"noreferrer"},"source")],-1))]),t("details",g1,[t("summary",null,[a[159]||(a[159]=t("a",{id:"Lux.RNNCell",href:"#Lux.RNNCell"},[t("span",{class:"jlbinding"},"Lux.RNNCell")],-1)),a[160]||(a[160]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[173]||(a[173]=i(`
    julia
    RNNCell(in_dims => out_dims, activation=tanh; use_bias=True(), train_state=False(),
+    init_bias=nothing, init_weight=nothing, init_state=zeros32)

    An Elman RNNCell cell with activation (typically set to tanh or relu).

    `,2)),t("p",null,[t("mjx-container",k1,[(Q(),n("svg",c1,a[161]||(a[161]=[i('',1)]))),a[162]||(a[162]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])]),t("mo",null,"="),t("mi",null,"a"),t("mi",null,"c"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"v"),t("mi",null,"a"),t("mi",null,"t"),t("mi",null,"i"),t("mi",null,"o"),t("mi",null,"n"),t("mo",{stretchy:"false"},"("),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"×"),t("mi",null,"x"),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"i"),t("mi",null,"h")])]),t("mo",null,"+"),t("mi",null,"w"),t("mi",null,"e"),t("mi",null,"i"),t("mi",null,"g"),t("mi",null,"h"),t("msub",null,[t("mi",null,"t"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",null,"×"),t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"p"),t("mi",null,"r"),t("mi",null,"e"),t("mi",null,"v")])]),t("mo",null,"+"),t("mi",null,"b"),t("mi",null,"i"),t("mi",null,"a"),t("msub",null,[t("mi",null,"s"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"h"),t("mi",null,"h")])]),t("mo",{stretchy:"false"},")")])],-1))])]),a[174]||(a[174]=i("

    Arguments

    Inputs

    Returns

    ",5)),t("ul",null,[t("li",null,[a[171]||(a[171]=t("p",null,"Tuple containing",-1)),t("ul",null,[t("li",null,[t("p",null,[a[165]||(a[165]=s("Output ")),t("mjx-container",u1,[(Q(),n("svg",y1,a[163]||(a[163]=[i('',1)]))),a[164]||(a[164]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))]),a[166]||(a[166]=s(" of shape ")),a[167]||(a[167]=t("code",null,"(out_dims, batch_size)",-1))])]),t("li",null,[t("p",null,[a[170]||(a[170]=s("Tuple containing new hidden state ")),t("mjx-container",E1,[(Q(),n("svg",f1,a[168]||(a[168]=[i('',1)]))),a[169]||(a[169]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"h"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"n"),t("mi",null,"e"),t("mi",null,"w")])])])],-1))])])])])]),a[172]||(a[172]=t("li",null,[t("p",null,"Updated model state")],-1))]),a[175]||(a[175]=i('

    Parameters

    States

    source

    ',5))]),t("details",L1,[t("summary",null,[a[176]||(a[176]=t("a",{id:"Lux.Recurrence",href:"#Lux.Recurrence"},[t("span",{class:"jlbinding"},"Lux.Recurrence")],-1)),a[177]||(a[177]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[178]||(a[178]=i(`
    julia
    Recurrence(cell;
     ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex(),
     return_sequence::Bool=false)

    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) to automatically operate over a sequence of inputs.

    Relation to Flux.Recur

    This is completely distinct from Flux.Recur. It doesn't make the cell stateful, rather allows operating on an entire sequence of inputs at once. See StatefulRecurrentCell for functionality similar to Flux.Recur.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Tip

    Frameworks like Tensorflow have special implementation of StackedRNNCells to handle sequentially composed RNN Cells. In Lux, one can simple stack multiple Recurrence blocks in a Chain to achieve the same.

    Chain(
     Recurrence(RNNCell(inputsize => latentsize); return_sequence=true),
     Recurrence(RNNCell(latentsize => latentsize); return_sequence=true),
     :
     x -> stack(x; dims=2)
-)

    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

    source

    `,14))]),t("details",x1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)

    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

    Arguments

    Inputs

    Returns

    States

    source

    ',12))]),t("details",b1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
+)

    For some discussion on this topic, see https://github.com/LuxDL/Lux.jl/issues/472.

    source

    `,14))]),t("details",x1,[t("summary",null,[a[179]||(a[179]=t("a",{id:"Lux.StatefulRecurrentCell",href:"#Lux.StatefulRecurrentCell"},[t("span",{class:"jlbinding"},"Lux.StatefulRecurrentCell")],-1)),a[180]||(a[180]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[181]||(a[181]=i('
    julia
    StatefulRecurrentCell(cell)

    Wraps a recurrent cell (like RNNCell, LSTMCell, GRUCell) and makes it stateful.

    To avoid undefined behavior, once the processing of a single sequence of data is complete, update the state with Lux.update_state(st, :carry, nothing).

    Arguments

    Inputs

    Returns

    States

    source

    ',12))]),t("details",b1,[t("summary",null,[a[182]||(a[182]=t("a",{id:"Lux.BidirectionalRNN",href:"#Lux.BidirectionalRNN"},[t("span",{class:"jlbinding"},"Lux.BidirectionalRNN")],-1)),a[183]||(a[183]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[184]||(a[184]=i(`
    julia
    BidirectionalRNN(cell::AbstractRecurrentCell,
     backward_cell::Union{AbstractRecurrentCell, Nothing}=nothing;
     merge_mode::Union{Function, Nothing}=vcat,
-    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

    Bidirectional RNN wrapper.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
+    ordering::AbstractTimeSeriesDataBatchOrdering=BatchLastIndex())

    Bidirectional RNN wrapper.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    source

    `,16))]),a[275]||(a[275]=t("h2",{id:"Linear-Layers",tabindex:"-1"},[s("Linear Layers "),t("a",{class:"header-anchor",href:"#Linear-Layers","aria-label":'Permalink to "Linear Layers {#Linear-Layers}"'},"​")],-1)),t("details",w1,[t("summary",null,[a[185]||(a[185]=t("a",{id:"Lux.Bilinear",href:"#Lux.Bilinear"},[t("span",{class:"jlbinding"},"Lux.Bilinear")],-1)),a[186]||(a[186]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[187]||(a[187]=i(`
    julia
    Bilinear((in1_dims, in2_dims) => out, activation=identity; init_weight=nothing,
          init_bias=nothing, use_bias=True())
 Bilinear(in12_dims => out, activation=identity; init_weight=nothing,
-         init_bias=nothing, use_bias=True())

    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

    z[i] = activation(x' * W[i, :, :] * y + bias[i])

    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
-      init_bias=nothing, use_bias=True())

    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)

    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

    This layer is often used to store word embeddings and retrieve them using indices.

    Arguments

    Keyword Arguments

    Input

    Returns

    source

    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)

    Flattens the passed array into a matrix.

    Keyword Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = FlattenLayer()
+         init_bias=nothing, use_bias=True())

    Create a fully connected layer between two inputs and an output, and otherwise similar to Dense. Its output, given vectors x & y, is another vector z with, for all i in 1:out:

    z[i] = activation(x' * W[i, :, :] * y + bias[i])

    If x and y are matrices, then each column of the output z = B(x, y) is of this form, with B the Bilinear layer.

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,15))]),t("details",H1,[t("summary",null,[a[188]||(a[188]=t("a",{id:"Lux.Dense",href:"#Lux.Dense"},[t("span",{class:"jlbinding"},"Lux.Dense")],-1)),a[189]||(a[189]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[190]||(a[190]=i(`
    julia
    Dense(in_dims => out_dims, activation=identity; init_weight=nothing,
+      init_bias=nothing, use_bias=True())

    Create a traditional fully connected layer, whose forward pass is given by: y = activation.(weight * x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    `,13))]),t("details",C1,[t("summary",null,[a[191]||(a[191]=t("a",{id:"Lux.Embedding",href:"#Lux.Embedding"},[t("span",{class:"jlbinding"},"Lux.Embedding")],-1)),a[192]||(a[192]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[193]||(a[193]=i('
    julia
    Embedding(in_dims => out_dims; init_weight=rand32)

    A lookup table that stores embeddings of dimension out_dims for a vocabulary of size in_dims. When the vocabulary is multi-dimensional, the input is expected to be a tuple of Cartesian indices.

    This layer is often used to store word embeddings and retrieve them using indices.

    Arguments

    Keyword Arguments

    Input

    Returns

    source

    ',12))]),t("details",F1,[t("summary",null,[a[194]||(a[194]=t("a",{id:"Lux.Scale",href:"#Lux.Scale"},[t("span",{class:"jlbinding"},"Lux.Scale")],-1)),a[195]||(a[195]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[196]||(a[196]=i('
    julia
    Scale(dims, activation=identity; init_weight=ones32, init_bias=zeros32, use_bias=True())

    Create a Sparsely Connected Layer with a very specific structure (only Diagonal Elements are non-zero). The forward pass is given by: y = activation.(weight .* x .+ bias)

    Arguments

    Keyword Arguments

    Input

    Returns

    Parameters

    source

    ',13))]),a[276]||(a[276]=t("h2",{id:"Misc.-Helper-Layers",tabindex:"-1"},[s("Misc. Helper Layers "),t("a",{class:"header-anchor",href:"#Misc.-Helper-Layers","aria-label":'Permalink to "Misc. Helper Layers {#Misc.-Helper-Layers}"'},"​")],-1)),t("details",D1,[t("summary",null,[a[197]||(a[197]=t("a",{id:"Lux.FlattenLayer",href:"#Lux.FlattenLayer"},[t("span",{class:"jlbinding"},"Lux.FlattenLayer")],-1)),a[198]||(a[198]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[199]||(a[199]=i(`
    julia
    FlattenLayer(; N = nothing)

    Flattens the passed array into a matrix.

    Keyword Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = FlattenLayer()
 FlattenLayer{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -87,9 +87,9 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(8, 2)

    source

    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
+(8, 2)

    source

    `,11))]),t("details",M1,[t("summary",null,[a[200]||(a[200]=t("a",{id:"Lux.Maxout",href:"#Lux.Maxout"},[t("span",{class:"jlbinding"},"Lux.Maxout")],-1)),a[201]||(a[201]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[202]||(a[202]=i(`
    julia
    Maxout(layers...)
 Maxout(; layers...)
-Maxout(f::Function, n_alts::Int)

    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

    See also Parallel to reduce with other operators.

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    References

    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

    source

    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()

    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

    Example

    julia
    julia> model = NoOpLayer()
+Maxout(f::Function, n_alts::Int)

    This contains a number of internal layers, each of which receives the same input. Its output is the elementwise maximum of the the internal layers' outputs.

    Maxout over linear dense layers satisfies the universal approximation theorem. See [1].

    See also Parallel to reduce with other operators.

    Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    References

    [1] Goodfellow, Warde-Farley, Mirza, Courville & Bengio "Maxout Networks" https://arxiv.org/abs/1302.4389

    source

    `,18))]),t("details",v1,[t("summary",null,[a[203]||(a[203]=t("a",{id:"Lux.NoOpLayer",href:"#Lux.NoOpLayer"},[t("span",{class:"jlbinding"},"Lux.NoOpLayer")],-1)),a[204]||(a[204]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[205]||(a[205]=i(`
    julia
    NoOpLayer()

    As the name suggests does nothing but allows pretty printing of layers. Whatever input is passed is returned.

    Example

    julia
    julia> model = NoOpLayer()
 NoOpLayer()
 
 julia> rng = Random.default_rng();
@@ -99,7 +99,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 1
 
 julia> y, st_new = model(x, ps, st)
-(1, NamedTuple())

    source

    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)

    Reshapes the passed array to have a size of (dims..., :)

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReshapeLayer((2, 2))
+(1, NamedTuple())

    source

    `,5))]),t("details",Z1,[t("summary",null,[a[206]||(a[206]=t("a",{id:"Lux.ReshapeLayer",href:"#Lux.ReshapeLayer"},[t("span",{class:"jlbinding"},"Lux.ReshapeLayer")],-1)),a[207]||(a[207]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[208]||(a[208]=i(`
    julia
    ReshapeLayer(dims)

    Reshapes the passed array to have a size of (dims..., :)

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReshapeLayer((2, 2))
 ReshapeLayer(output_dims = (2, 2, :))
 
 julia> rng = Random.default_rng();
@@ -109,7 +109,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
 
 julia> y, st_new = model(x, ps, st);
        size(y)
-(2, 2, 3)

    source

    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)

    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)

    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)

    Reverse the specified dimension dims of the passed array

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReverseSequence()
+(2, 2, 3)

    source

    `,11))]),t("details",j1,[t("summary",null,[a[209]||(a[209]=t("a",{id:"Lux.SelectDim",href:"#Lux.SelectDim"},[t("span",{class:"jlbinding"},"Lux.SelectDim")],-1)),a[210]||(a[210]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[211]||(a[211]=i('
    julia
    SelectDim(dim, i)

    Return a view of all the data of the input x where the index for dimension dim equals i. Equivalent to view(x,:,:,...,i,:,:,...) where i is in position d.

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",A1,[t("summary",null,[a[212]||(a[212]=t("a",{id:"Lux.WrappedFunction",href:"#Lux.WrappedFunction"},[t("span",{class:"jlbinding"},"Lux.WrappedFunction")],-1)),a[213]||(a[213]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[214]||(a[214]=i('
    julia
    WrappedFunction(f)

    Wraps a stateless and parameter less function. Might be used when a function is added to Chain. For example, Chain(x -> relu.(x)) would not work and the right thing to do would be Chain((x, ps, st) -> (relu.(x), st)). An easier thing to do would be Chain(WrappedFunction(Base.Fix1(broadcast, relu)))

    Arguments

    Inputs

    Returns

    source

    ',9))]),t("details",B1,[t("summary",null,[a[215]||(a[215]=t("a",{id:"Lux.ReverseSequence",href:"#Lux.ReverseSequence"},[t("span",{class:"jlbinding"},"Lux.ReverseSequence")],-1)),a[216]||(a[216]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[217]||(a[217]=i(`
    julia
    ReverseSequence(dim = nothing)

    Reverse the specified dimension dims of the passed array

    Arguments

    Inputs

    Returns

    Example

    julia
    julia> model = ReverseSequence()
 ReverseSequence{Nothing}(nothing)
 
 julia> rng = Random.default_rng();
@@ -118,7 +118,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
        x = [1.0, 2.0, 3.0];
 
 julia> y, st_new = model(x, ps, st)
-([3.0, 2.0, 1.0], NamedTuple())

    source

    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+([3.0, 2.0, 1.0], NamedTuple())

    source

    `,11))]),a[277]||(a[277]=t("h2",{id:"Normalization-Layers",tabindex:"-1"},[s("Normalization Layers "),t("a",{class:"header-anchor",href:"#Normalization-Layers","aria-label":'Permalink to "Normalization Layers {#Normalization-Layers}"'},"​")],-1)),t("details",V1,[t("summary",null,[a[218]||(a[218]=t("a",{id:"Lux.BatchNorm",href:"#Lux.BatchNorm"},[t("span",{class:"jlbinding"},"Lux.BatchNorm")],-1)),a[219]||(a[219]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[225]||(a[225]=i(`
    julia
    BatchNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
           affine=True(), track_stats=True(), epsilon=1f-5, momentum=0.1f0)

    Batch Normalization layer.

    `,2)),t("p",null,[a[222]||(a[222]=t("code",null,"BatchNorm",-1)),a[223]||(a[223]=s(" computes the mean and variance for each ")),t("mjx-container",R1,[(Q(),n("svg",N1,a[220]||(a[220]=[i('',1)]))),a[221]||(a[221]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mi",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mi",null,"×"),t("mn",null,"1"),t("mi",null,"×"),t("msub",null,[t("mi",null,"D"),t("mi",null,"N")])])],-1))]),a[224]||(a[224]=s(" input slice and normalises the input accordingly."))]),a[226]||(a[226]=i(`

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    Use Lux.testmode during inference.

    Example

    julia
    julia> Chain(Dense(784 => 64), BatchNorm(64, relu), Dense(64 => 10), BatchNorm(10))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -126,7 +126,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = BatchNorm(10, affine=true, track_stats=true),  # 20 parameters, plus 21
 )         # Total: 51_038 parameters,
-          #        plus 150 states.

    Warning

    Passing a batch size of 1, during training will result in an error.

    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
+          #        plus 150 states.

    Warning

    Passing a batch size of 1, during training will result in an error.

    See also BatchNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,19))]),t("details",O1,[t("summary",null,[a[227]||(a[227]=t("a",{id:"Lux.GroupNorm",href:"#Lux.GroupNorm"},[t("span",{class:"jlbinding"},"Lux.GroupNorm")],-1)),a[228]||(a[228]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[229]||(a[229]=i(`
    julia
    GroupNorm(chs::Integer, groups::Integer, activation=identity; init_bias=zeros32,
           init_scale=ones32, affine=true, epsilon=1f-5)

    Group Normalization layer.

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    Use Lux.testmode during inference.

    Example

    julia
    julia> Chain(Dense(784 => 64), GroupNorm(64, 4, relu), Dense(64 => 10), GroupNorm(10, 5))
 Chain(
     layer_1 = Dense(784 => 64),         # 50_240 parameters
@@ -134,7 +134,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = GroupNorm(10, 5, affine=true),  # 20 parameters
 )         # Total: 51_038 parameters,
-          #        plus 0 states.

    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
+          #        plus 0 states.

    See also GroupNorm, InstanceNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",z1,[t("summary",null,[a[230]||(a[230]=t("a",{id:"Lux.InstanceNorm",href:"#Lux.InstanceNorm"},[t("span",{class:"jlbinding"},"Lux.InstanceNorm")],-1)),a[231]||(a[231]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[236]||(a[236]=i(`
    julia
    InstanceNorm(chs::Integer, activation=identity; init_bias=zeros32, init_scale=ones32,
              affine=False(), track_stats=False(), epsilon=1f-5, momentum=0.1f0)

    Instance Normalization. For details see [1].

    `,2)),t("p",null,[a[234]||(a[234]=s("Instance Normalization computes the mean and variance for each ")),t("mjx-container",P1,[(Q(),n("svg",I1,a[232]||(a[232]=[i('',1)]))),a[233]||(a[233]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("msub",null,[t("mi",null,"D"),t("mn",null,"1")]),t("mo",null,"×"),t("mo",null,"."),t("mo",null,"."),t("mo",null,"."),t("mo",null,"×"),t("msub",null,[t("mi",null,"D"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",null,"N"),t("mo",null,"−"),t("mn",null,"2")])]),t("mo",null,"×"),t("mn",null,"1"),t("mo",null,"×"),t("mn",null,"1")])],-1))]),a[235]||(a[235]=s("` input slice and normalises the input accordingly."))]),a[237]||(a[237]=i(`

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    States

    Use Lux.testmode during inference.

    Example

    julia
    julia> Chain(Dense(784 => 64), InstanceNorm(64, relu; affine=true), Dense(64 => 10),
            InstanceNorm(10, relu; affine=true))
 Chain(
@@ -143,7 +143,7 @@ import{_ as T,c as n,j as t,a as s,G as l,a2 as i,B as d,o as Q}from"./chunks/fr
     layer_3 = Dense(64 => 10),          # 650 parameters
     layer_4 = InstanceNorm(10, relu, affine=true, track_stats=false),  # 20 parameters, plus 1
 )         # Total: 51_038 parameters,
-          #        plus 2 states.

    References

    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",S1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
-          affine=true, init_bias=zeros32, init_scale=ones32)

    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
-           dims::Union{Tuple, Nothing}=nothing)

    Applies weight normalization to a parameter in the given layer.

    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('

    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

    Arguments

    Inputs

    Returns

    Parameters

    States

    source

    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)

    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

    See NNlib.pixel_shuffle for more details.

    Arguments

    Inputs

    Returns

    source

    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
-Upsample(scale, mode = :nearest)

    Upsampling Layer.

    Layer Construction

    Option 1

    Exactly one of two keywords must be specified:

    Option 2

    Currently supported upsampling modes and corresponding NNlib's methods are:

    Extended Help

    Other Keyword Arguments

    Inputs

    Returns

    source

    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default}; + # plus 2 states.

    References

    [1] Ulyanov, Dmitry, Andrea Vedaldi, and Victor Lempitsky. "Instance normalization: The missing ingredient for fast stylization." arXiv preprint arXiv:1607.08022 (2016).

    See also BatchNorm, GroupNorm, LayerNorm, WeightNorm

    source

    `,20))]),t("details",S1,[t("summary",null,[a[238]||(a[238]=t("a",{id:"Lux.LayerNorm",href:"#Lux.LayerNorm"},[t("span",{class:"jlbinding"},"Lux.LayerNorm")],-1)),a[239]||(a[239]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[255]||(a[255]=i(`
    julia
    LayerNorm(shape::NTuple{N, Int}, activation=identity; epsilon=1f-5, dims=Colon(),
+          affine=true, init_bias=zeros32, init_scale=ones32)

    Computes mean and standard deviation over the whole input array, and uses these to normalize the whole array. Optionally applies an elementwise affine transformation afterwards.

    `,2)),t("p",null,[a[242]||(a[242]=s("Given an input array ")),t("mjx-container",_1,[(Q(),n("svg",G1,a[240]||(a[240]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D465",d:"M52 289Q59 331 106 386T222 442Q257 442 286 424T329 379Q371 442 430 442Q467 442 494 420T522 361Q522 332 508 314T481 292T458 288Q439 288 427 299T415 328Q415 374 465 391Q454 404 425 404Q412 404 406 402Q368 386 350 336Q290 115 290 78Q290 50 306 38T341 26Q378 26 414 59T463 140Q466 150 469 151T485 153H489Q504 153 504 145Q504 144 502 134Q486 77 440 33T333 -11Q263 -11 227 52Q186 -10 133 -10H127Q78 -10 57 16T35 71Q35 103 54 123T99 143Q142 143 142 101Q142 81 130 66T107 46T94 41L91 40Q91 39 97 36T113 29T132 26Q168 26 194 71Q203 87 217 139T245 247T261 313Q266 340 266 352Q266 380 251 392T217 404Q177 404 142 372T93 290Q91 281 88 280T72 278H58Q52 284 52 289Z",style:{"stroke-width":"3"}})])])],-1)]))),a[241]||(a[241]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"x")])],-1))]),a[243]||(a[243]=s(", this layer computes"))]),t("mjx-container",W1,[(Q(),n("svg",X1,a[244]||(a[244]=[i('',1)]))),a[245]||(a[245]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"y"),t("mo",null,"="),t("mfrac",null,[t("mrow",null,[t("mi",null,"x"),t("mo",null,"−"),t("mrow",{"data-mjx-texclass":"ORD"},[t("mi",{mathvariant:"double-struck"},"E")]),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]")]),t("msqrt",null,[t("mi",null,"V"),t("mi",null,"a"),t("mi",null,"r"),t("mo",{stretchy:"false"},"["),t("mi",null,"x"),t("mo",{stretchy:"false"},"]"),t("mo",null,"+"),t("mi",null,"ϵ")])]),t("mo",null,"∗"),t("mi",null,"γ"),t("mo",null,"+"),t("mi",null,"β")])],-1))]),t("p",null,[a[250]||(a[250]=s("where ")),t("mjx-container",U1,[(Q(),n("svg",q1,a[246]||(a[246]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),a[247]||(a[247]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"γ")])],-1))]),a[251]||(a[251]=s(" & ")),t("mjx-container",J1,[(Q(),n("svg",K1,a[248]||(a[248]=[t("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[t("g",{"data-mml-node":"math"},[t("g",{"data-mml-node":"mi"},[t("path",{"data-c":"1D6FD",d:"M29 -194Q23 -188 23 -186Q23 -183 102 134T186 465Q208 533 243 584T309 658Q365 705 429 705H431Q493 705 533 667T573 570Q573 465 469 396L482 383Q533 332 533 252Q533 139 448 65T257 -10Q227 -10 203 -2T165 17T143 40T131 59T126 65L62 -188Q60 -194 42 -194H29ZM353 431Q392 431 427 419L432 422Q436 426 439 429T449 439T461 453T472 471T484 495T493 524T501 560Q503 569 503 593Q503 611 502 616Q487 667 426 667Q384 667 347 643T286 582T247 514T224 455Q219 439 186 308T152 168Q151 163 151 147Q151 99 173 68Q204 26 260 26Q302 26 349 51T425 137Q441 171 449 214T457 279Q457 337 422 372Q380 358 347 358H337Q258 358 258 389Q258 396 261 403Q275 431 353 431Z",style:{"stroke-width":"3"}})])])],-1)]))),a[249]||(a[249]=t("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[t("mi",null,"β")])],-1))]),a[252]||(a[252]=s(" are trainable parameters if ")),a[253]||(a[253]=t("code",null,"affine=true",-1)),a[254]||(a[254]=s("."))]),a[256]||(a[256]=i('

    Arguments

    Keyword Arguments

    Extended Help

    Inputs

    Returns

    Parameters

    source

    ',12))]),t("details",$1,[t("summary",null,[a[257]||(a[257]=t("a",{id:"Lux.WeightNorm",href:"#Lux.WeightNorm"},[t("span",{class:"jlbinding"},"Lux.WeightNorm")],-1)),a[258]||(a[258]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[261]||(a[261]=i(`
    julia
    WeightNorm(layer::AbstractLuxLayer, which_params::NTuple{N, Symbol},
+           dims::Union{Tuple, Nothing}=nothing)

    Applies weight normalization to a parameter in the given layer.

    `,2)),t("mjx-container",Y1,[(Q(),n("svg",t2,a[259]||(a[259]=[i('',1)]))),a[260]||(a[260]=t("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[t("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[t("mi",null,"w"),t("mo",null,"="),t("mi",null,"g"),t("mfrac",null,[t("mi",null,"v"),t("mrow",null,[t("mo",{"data-mjx-texclass":"ORD"},"∥"),t("mi",null,"v"),t("mo",{"data-mjx-texclass":"ORD"},"∥")])])])],-1))]),a[262]||(a[262]=i('

    Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This updates the parameters in which_params (e.g. weight) using two parameters: one specifying the magnitude (e.g. weight_g) and one specifying the direction (e.g. weight_v).

    Arguments

    Inputs

    Returns

    Parameters

    States

    source

    ',12))]),a[278]||(a[278]=t("h2",{id:"upsampling",tabindex:"-1"},[s("Upsampling "),t("a",{class:"header-anchor",href:"#upsampling","aria-label":'Permalink to "Upsampling"'},"​")],-1)),t("details",a2,[t("summary",null,[a[263]||(a[263]=t("a",{id:"Lux.PixelShuffle",href:"#Lux.PixelShuffle"},[t("span",{class:"jlbinding"},"Lux.PixelShuffle")],-1)),a[264]||(a[264]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[265]||(a[265]=i('
    julia
    PixelShuffle(r::Int)

    Pixel shuffling layer with upscale factor r. Usually used for generating higher resolution images while upscaling them.

    See NNlib.pixel_shuffle for more details.

    Arguments

    Inputs

    Returns

    source

    ',10))]),t("details",s2,[t("summary",null,[a[266]||(a[266]=t("a",{id:"Lux.Upsample",href:"#Lux.Upsample"},[t("span",{class:"jlbinding"},"Lux.Upsample")],-1)),a[267]||(a[267]=s()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),a[268]||(a[268]=i(`
    julia
    Upsample(mode = :nearest; [scale, size, align_corners=false])
+Upsample(scale, mode = :nearest)

    Upsampling Layer.

    Layer Construction

    Option 1

    Exactly one of two keywords must be specified:

    Option 2

    Currently supported upsampling modes and corresponding NNlib's methods are:

    Extended Help

    Other Keyword Arguments

    Inputs

    Returns

    source

    `,19))])])}const r2=T(o,[["render",i2]]);export{o2 as __pageData,r2 as default}; diff --git a/previews/PR1000/assets/api_Lux_utilities.md.DwOGkpib.js b/previews/PR1000/assets/api_Lux_utilities.md.C5yiud75.js similarity index 98% rename from previews/PR1000/assets/api_Lux_utilities.md.DwOGkpib.js rename to previews/PR1000/assets/api_Lux_utilities.md.C5yiud75.js index 69ad3919af..3b3a8f6f16 100644 --- a/previews/PR1000/assets/api_Lux_utilities.md.DwOGkpib.js +++ b/previews/PR1000/assets/api_Lux_utilities.md.C5yiud75.js @@ -1,12 +1,12 @@ -import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.DjZDIZsN.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(c2,i,E2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState

    Training State containing:

    Internal fields:

    Warning

    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.

    source

    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
-    ts::TrainState)

    Compute the gradients of the objective function wrt parameters stored in ts.

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl

    Arguments

    Return

    A 4-Tuple containing:

    Known Limitations

    Aliased Gradients

    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.

    source

    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.

    In most cases you should use single_train_step! instead of this function.

    Return

    Returned values are the same as compute_gradients.

    source

    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.

    Return

    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).

    source

    ',5))]),i[290]||(i[290]=t('

    Loss Functions ​

    Loss Functions Objects take 2 forms of inputs:

    1. ŷ and y where ŷ is the predicted output and y is the target output.

    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.

    Warning

    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.

    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)

    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.

    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
+import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.DjZDIZsN.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},E1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},c1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(E2,i,c2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState
    Training State containing:
    • model: Lux model.
    • parameters: Trainable Variables of the model.
    • states: Non-trainable Variables of the model.
    • optimizer: Optimizer from Optimisers.jl.
    • optimizer_state: Optimizer State.
    • step: Number of updates of the parameters made.
    Internal fields:
    • cache: Cached values. Implementations are free to use this for whatever they want.
    • objective_function: Objective function might be cached.
    Warning
    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.
    source
    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
+    ts::TrainState)
    Compute the gradients of the objective function wrt parameters stored in ts.
    Backends & AD Packages
    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl
    Arguments
    • ad: Backend (from ADTypes.jl) used to compute the gradients.
    • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.
    • data: Data used to compute the gradients.
    • ts: Current Training State. See TrainState.
    Return
    A 4-Tuple containing:
    • grads: Computed Gradients.
    • loss: Loss from the objective function.
    • stats: Any computed statistics from the objective function.
    • ts: Updated Training State.
    Known Limitations
    • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.
    Aliased Gradients
    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.
    source
    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.
    In most cases you should use single_train_step! instead of this function.
    Return
    Returned values are the same as compute_gradients.
    source
    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.
    Return
    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).
    source
    ',5))]),i[290]||(i[290]=t('
    Loss Functions ​
    Loss Functions Objects take 2 forms of inputs:
    1. ŷ and y where ŷ is the predicted output and y is the target output.
    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.
    Warning
    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.
    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)
    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.
    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> mseloss(y_model, 1:3) ≈ 0.01
-true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",c,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
+true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",E,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
     label_smoothing::Union{Nothing, Real}=nothing,
-    logits::Union{Bool, Val}=Val(false))
    Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
    Returns the binary cross entropy loss computed as:
    • If logits is either false or Val(false):
    `,4)),s("mjx-container",E,[(h(),n("svg",u,i[23]||(i[23]=[t('',1)]))),i[24]||(i[24]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[52]||(i[52]=s("ul",null,[s("li",null,[a("If "),s("code",null,"logits"),a(" is "),s("code",null,"true"),a(" or "),s("code",null,"Val(true)"),a(":")])],-1)),s("mjx-container",F,[(h(),n("svg",x,i[25]||(i[25]=[t('',1)]))),i[26]||(i[26]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"l"),s("mi",null,"o"),s("mi",null,"g"),s("mi",null,"σ"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[33]||(i[33]=a("The value of ")),s("mjx-container",C,[(h(),n("svg",L,i[27]||(i[27]=[t('',1)]))),i[28]||(i[28]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[34]||(i[34]=a(" is computed using label smoothing. If ")),i[35]||(i[35]=s("code",null,"label_smoothing",-1)),i[36]||(i[36]=a(" is ")),i[37]||(i[37]=s("code",null,"nothing",-1)),i[38]||(i[38]=a(", then no label smoothing is applied. If ")),i[39]||(i[39]=s("code",null,"label_smoothing",-1)),i[40]||(i[40]=a(" is a real number ")),s("mjx-container",f,[(h(),n("svg",b,i[29]||(i[29]=[t('',1)]))),i[30]||(i[30]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[41]||(i[41]=a(", then the value of ")),s("mjx-container",w,[(h(),n("svg",H,i[31]||(i[31]=[t('',1)]))),i[32]||(i[32]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[42]||(i[42]=a(" is:"))]),s("mjx-container",v,[(h(),n("svg",j,i[43]||(i[43]=[t('',1)]))),i[44]||(i[44]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mn",null,"0.5")])],-1))]),s("p",null,[i[47]||(i[47]=a("where ")),s("mjx-container",D,[(h(),n("svg",B,i[45]||(i[45]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[46]||(i[46]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[48]||(i[48]=a(" is the value of ")),i[49]||(i[49]=s("code",null,"label_smoothing",-1)),i[50]||(i[50]=a("."))]),i[53]||(i[53]=t(`
    Extended Help
    Example
    julia
    julia> bce = BinaryCrossEntropyLoss();
+    logits::Union{Bool, Val}=Val(false))
    Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
    Returns the binary cross entropy loss computed as:
    • If logits is either false or Val(false):
    `,4)),s("mjx-container",c,[(h(),n("svg",u,i[23]||(i[23]=[t('',1)]))),i[24]||(i[24]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[52]||(i[52]=s("ul",null,[s("li",null,[a("If "),s("code",null,"logits"),a(" is "),s("code",null,"true"),a(" or "),s("code",null,"Val(true)"),a(":")])],-1)),s("mjx-container",F,[(h(),n("svg",x,i[25]||(i[25]=[t('',1)]))),i[26]||(i[26]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"l"),s("mi",null,"o"),s("mi",null,"g"),s("mi",null,"σ"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[33]||(i[33]=a("The value of ")),s("mjx-container",C,[(h(),n("svg",L,i[27]||(i[27]=[t('',1)]))),i[28]||(i[28]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[34]||(i[34]=a(" is computed using label smoothing. If ")),i[35]||(i[35]=s("code",null,"label_smoothing",-1)),i[36]||(i[36]=a(" is ")),i[37]||(i[37]=s("code",null,"nothing",-1)),i[38]||(i[38]=a(", then no label smoothing is applied. If ")),i[39]||(i[39]=s("code",null,"label_smoothing",-1)),i[40]||(i[40]=a(" is a real number ")),s("mjx-container",f,[(h(),n("svg",b,i[29]||(i[29]=[t('',1)]))),i[30]||(i[30]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[41]||(i[41]=a(", then the value of ")),s("mjx-container",w,[(h(),n("svg",H,i[31]||(i[31]=[t('',1)]))),i[32]||(i[32]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[42]||(i[42]=a(" is:"))]),s("mjx-container",v,[(h(),n("svg",j,i[43]||(i[43]=[t('',1)]))),i[44]||(i[44]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mn",null,"0.5")])],-1))]),s("p",null,[i[47]||(i[47]=a("where ")),s("mjx-container",D,[(h(),n("svg",B,i[45]||(i[45]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[46]||(i[46]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[48]||(i[48]=a(" is the value of ")),i[49]||(i[49]=s("code",null,"label_smoothing",-1)),i[50]||(i[50]=a("."))]),i[53]||(i[53]=t(`
    Extended Help
    Example
    julia
    julia> bce = BinaryCrossEntropyLoss();
 
 julia> y_bin = Bool[1, 0, 1];
 
@@ -32,7 +32,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 julia> logitbce_ls = BinaryCrossEntropyLoss(label_smoothing=0.1, logits=Val(true));
 
 julia> logitbce_ls(y_model, y_bin) > logitbce(y_model, y_bin)
-true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
+true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
             1  0  1];
 
 julia> ŷ = [0.268941  0.5  0.268941
@@ -42,7 +42,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> BinaryFocalLoss(gamma=0)(ŷ, y) ≈ BinaryCrossEntropyLoss()(ŷ, y)
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
     label_smoothing::Union{Nothing, Real}=nothing)
    `,1)),s("p",null,[i[72]||(i[72]=a("Return the cross entropy loss which is used in multi-class classification tasks. The input, ")),s("mjx-container",S,[(h(),n("svg",N,i[70]||(i[70]=[t('',1)]))),i[71]||(i[71]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[73]||(i[73]=a(", is expected to be normalized (i.e. ")),i[74]||(i[74]=s("code",null,"softmax",-1)),i[75]||(i[75]=a(" output) if ")),i[76]||(i[76]=s("code",null,"logits",-1)),i[77]||(i[77]=a(" is ")),i[78]||(i[78]=s("code",null,"false",-1)),i[79]||(i[79]=a(" or ")),i[80]||(i[80]=s("code",null,"Val(false)",-1)),i[81]||(i[81]=a("."))]),i[112]||(i[112]=s("p",null,"The loss is calculated as:",-1)),s("mjx-container",P,[(h(),n("svg",z,i[82]||(i[82]=[t('',1)]))),i[83]||(i[83]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[92]||(i[92]=a("where ")),s("mjx-container",I,[(h(),n("svg",q,i[84]||(i[84]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D716",d:"M227 -11Q149 -11 95 41T40 174Q40 262 87 322Q121 367 173 396T287 430Q289 431 329 431H367Q382 426 382 411Q382 385 341 385H325H312Q191 385 154 277L150 265H327Q340 256 340 246Q340 228 320 219H138V217Q128 187 128 143Q128 77 160 52T231 26Q258 26 284 36T326 57T343 68Q350 68 354 58T358 39Q358 36 357 35Q354 31 337 21T289 0T227 -11Z",style:{"stroke-width":"3"}})])])],-1)]))),i[85]||(i[85]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"ϵ")])],-1))]),i[93]||(i[93]=a(" is added for numerical stability. The value of ")),s("mjx-container",G,[(h(),n("svg",X,i[86]||(i[86]=[t('',1)]))),i[87]||(i[87]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[94]||(i[94]=a(" is computed using label smoothing. If ")),i[95]||(i[95]=s("code",null,"label_smoothing",-1)),i[96]||(i[96]=a(" is ")),i[97]||(i[97]=s("code",null,"nothing",-1)),i[98]||(i[98]=a(", then no label smoothing is applied. If ")),i[99]||(i[99]=s("code",null,"label_smoothing",-1)),i[100]||(i[100]=a(" is a real number ")),s("mjx-container",J,[(h(),n("svg",U,i[88]||(i[88]=[t('',1)]))),i[89]||(i[89]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[101]||(i[101]=a(", then the value of ")),s("mjx-container",W,[(h(),n("svg",K,i[90]||(i[90]=[t('',1)]))),i[91]||(i[91]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[102]||(i[102]=a(" is calculated as:"))]),s("mjx-container",Y,[(h(),n("svg",$,i[103]||(i[103]=[t('',1)]))),i[104]||(i[104]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mtext",null,"size along dim")])],-1))]),s("p",null,[i[107]||(i[107]=a("where ")),s("mjx-container",_,[(h(),n("svg",s1,i[105]||(i[105]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[106]||(i[106]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[108]||(i[108]=a(" is the value of ")),i[109]||(i[109]=s("code",null,"label_smoothing",-1)),i[110]||(i[110]=a("."))]),i[113]||(i[113]=t(`
    Extended Help
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
@@ -64,7 +64,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> CrossEntropyLoss(label_smoothing=0.15)(y_model, y) ≈ 1.5776052f0
-true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
 
 julia> DiceCoeffLoss()(y_pred, 1:3)  ≈ 0.000992391663909964
 true
@@ -73,7 +73,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> DiceCoeffLoss()(reshape(y_pred, 3, 1), reshape(1:3, 3, 1)) ≈ 0.000992391663909964
-true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
+true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
 3×5 Matrix{Int64}:
@@ -88,20 +88,20 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
  0.665241   0.665241   0.665241   0.665241   0.665241
 
 julia> FocalLoss()(ŷ, y) ≈ 1.1277556f0
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.55
-true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",c1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",E1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",E1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",c1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
 
 julia> HuberLoss()(y_model, 1:3) ≈ 0.005000000000000009
 true
 
 julia> HuberLoss(delta=0.05)(y_model, 1:3) ≈ 0.003750000000000005
-true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
+true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
 2×2 Matrix{Int64}:
  1  0
  0  1
@@ -121,44 +121,44 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 0.0
 
 julia> KLDivergenceLoss(; epsilon=0)(p1, p2)
-Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
+Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.1
-true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
+true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.01
-true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
+true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
 
 julia> loss(Float32[1.1, 2.2, 3.3], 1:3) ≈ 0.009084041f0
 true
 
 julia> loss(Float32[0.9, 1.8, 2.7], 1:3) ≈ 0.011100831f0
-true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",P1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
+true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",P1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
 
 julia> PoissonLoss()(y_model, 1:3) ≈ 0.502312852219817
-true
    source
    `,3))]),s("details",z1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
+true
    source
    `,3))]),s("details",z1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
 
 julia> SiameseContrastiveLoss()(ŷ, 1:3) ≈ -4.833333333333333
 true
 
 julia> SiameseContrastiveLoss(margin=2)(ŷ, 1:3) ≈ -4.0
-true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
+true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.625
-true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
-foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
-getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
+true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
+foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
+getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
 istraining(::StaticBool)
 istraining(::Bool)
-istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
+istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
     ...
     @return y # optional (but recommended for best performance)
 end
@@ -274,7 +274,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
    You may also specify a name for the model, which will be used instead of the default printout, which gives a verbatim representation of the code used to construct the model:
    julia
    julia> model = @compact(w=rand(3), name="Linear(3 => 1)") do x
            @return sum(w .* x)
        end
-Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
+Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=@init_fn(rng->randn32(rng, 3, 2)),
            b=@init_fn(rng->randn32(rng, 3), :state)) do x
@@ -290,7 +290,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 (3,)
 
 julia> size(r([1, 2], ps, st)[1])
-(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
+(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=ones(3), w_fixed=@non_trainable(rand(3))) do x
            @return sum(w .* x .+ w_fixed)
@@ -310,5 +310,5 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> res isa Number
-true
    source
    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
-set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")
    Set the dispatch doctor preference for LuxCore and LuxLib packages.
    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.
    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.
    source
    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default};
+true

    source

    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
+set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")

    Set the dispatch doctor preference for LuxCore and LuxLib packages.

    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.

    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.

    source

    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default}; diff --git a/previews/PR1000/assets/api_Lux_utilities.md.DwOGkpib.lean.js b/previews/PR1000/assets/api_Lux_utilities.md.C5yiud75.lean.js similarity index 98% rename from previews/PR1000/assets/api_Lux_utilities.md.DwOGkpib.lean.js rename to previews/PR1000/assets/api_Lux_utilities.md.C5yiud75.lean.js index 69ad3919af..3b3a8f6f16 100644 --- a/previews/PR1000/assets/api_Lux_utilities.md.DwOGkpib.lean.js +++ b/previews/PR1000/assets/api_Lux_utilities.md.C5yiud75.lean.js @@ -1,12 +1,12 @@ -import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.DjZDIZsN.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},c1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},E1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(c2,i,E2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState

    Training State containing:

    Internal fields:

    Warning

    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.

    source

    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
-    ts::TrainState)

    Compute the gradients of the objective function wrt parameters stored in ts.

    Backends & AD Packages

    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl

    Arguments

    Return

    A 4-Tuple containing:

    Known Limitations

    Aliased Gradients

    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.

    source

    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)

    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.

    Arguments

    Returns

    Updated TrainState object.

    source

    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.

    In most cases you should use single_train_step! instead of this function.

    Return

    Returned values are the same as compute_gradients.

    source

    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)

    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.

    Return

    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).

    source

    ',5))]),i[290]||(i[290]=t('

    Loss Functions ​

    Loss Functions Objects take 2 forms of inputs:

    1. ŷ and y where ŷ is the predicted output and y is the target output.

    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.

    Warning

    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.

    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)

    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.

    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
+import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framework.DjZDIZsN.js";const L2=JSON.parse('{"title":"Utilities","description":"","frontmatter":{},"headers":[],"relativePath":"api/Lux/utilities.md","filePath":"api/Lux/utilities.md","lastUpdated":null}'),d={name:"api/Lux/utilities.md"},k={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},T={class:"jldocstring custom-block"},Q={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},m={class:"jldocstring custom-block"},y={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"},c={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"46.681ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 20633.1 1199","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"25.631ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 11329 1199","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},L={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},w={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},H={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},v={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},j={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.718ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10483.3 1000","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},M={class:"jldocstring custom-block"},V={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},A={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},Z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},R={class:"jldocstring custom-block"},S={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},N={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"23.184ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 10247.1 1799","aria-hidden":"true"},I={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},q={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"0.919ex",height:"1ex",role:"img",focusable:"false",viewBox:"0 -431 406 442","aria-hidden":"true"},G={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},X={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},J={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"6.664ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 2945.4 1000","aria-hidden":"true"},W={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.161ex",role:"img",focusable:"false",viewBox:"0 -750 490 955","aria-hidden":"true"},Y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},$={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"34.539ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 15266.3 1000","aria-hidden":"true"},_={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},s1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},i1={class:"jldocstring custom-block"},a1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},t1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.308ex"},xmlns:"http://www.w3.org/2000/svg",width:"28.659ex",height:"5.747ex",role:"img",focusable:"false",viewBox:"0 -1520 12667.4 2540","aria-hidden":"true"},e1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},l1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.448ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 640 453","aria-hidden":"true"},n1={class:"jldocstring custom-block"},h1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},r1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.229ex",height:"1.486ex",role:"img",focusable:"false",viewBox:"0 -441 543 657","aria-hidden":"true"},k1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.489ex"},xmlns:"http://www.w3.org/2000/svg",width:"5.377ex",height:"1.995ex",role:"img",focusable:"false",viewBox:"0 -666 2376.6 882","aria-hidden":"true"},T1={class:"jldocstring custom-block"},Q1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},g1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"20.065ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8868.8 1199","aria-hidden":"true"},m1={class:"jldocstring custom-block"},y1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},E1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.148ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"5.428ex",role:"img",focusable:"false",viewBox:"0 -1449.5 17948.3 2399","aria-hidden":"true"},c1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.023ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.005ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -717 444 727","aria-hidden":"true"},F1={class:"jldocstring custom-block"},x1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},C1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},L1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},b1={class:"jldocstring custom-block"},w1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},H1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"12.333ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5451.1 1199","aria-hidden":"true"},v1={class:"jldocstring custom-block"},j1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},D1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"14.515ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 6415.7 1799","aria-hidden":"true"},B1={class:"jldocstring custom-block"},M1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-1.469ex"},xmlns:"http://www.w3.org/2000/svg",width:"32.253ex",height:"4.07ex",role:"img",focusable:"false",viewBox:"0 -1149.5 14255.9 1799","aria-hidden":"true"},A1={class:"jldocstring custom-block"},Z1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},O1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"2.296ex",role:"img",focusable:"false",viewBox:"0 -810 490 1015","aria-hidden":"true"},R1={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},S1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.464ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.109ex",height:"1.464ex",role:"img",focusable:"false",viewBox:"0 -442 490 647","aria-hidden":"true"},N1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},P1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"18.723ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 8275.6 1199","aria-hidden":"true"},z1={class:"jldocstring custom-block"},I1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},q1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"40.607ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 17948.2 1233.4","aria-hidden":"true"},G1={class:"jldocstring custom-block"},X1={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},J1={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.053ex",height:"2.791ex",role:"img",focusable:"false",viewBox:"0 -883.9 9305.3 1233.4","aria-hidden":"true"},U1={class:"jldocstring custom-block"},W1={class:"jldocstring custom-block"},K1={class:"jldocstring custom-block"},Y1={class:"jldocstring custom-block"},$1={class:"jldocstring custom-block"},_1={class:"jldocstring custom-block"},s2={class:"jldocstring custom-block"},i2={class:"jldocstring custom-block"},a2={class:"jldocstring custom-block"},t2={class:"jldocstring custom-block"},e2={class:"jldocstring custom-block"},l2={class:"jldocstring custom-block"},n2={class:"jldocstring custom-block"},h2={class:"jldocstring custom-block"},p2={class:"jldocstring custom-block"},r2={class:"jldocstring custom-block"},d2={class:"jldocstring custom-block"},k2={class:"jldocstring custom-block"},o2={class:"jldocstring custom-block"},T2={class:"jldocstring custom-block"},Q2={class:"jldocstring custom-block"},g2={class:"jldocstring custom-block"},m2={class:"jldocstring custom-block"};function y2(E2,i,c2,u2,F2,x2){const e=r("Badge");return h(),n("div",null,[i[286]||(i[286]=s("h1",{id:"utilities",tabindex:"-1"},[a("Utilities "),s("a",{class:"header-anchor",href:"#utilities","aria-label":'Permalink to "Utilities"'},"​")],-1)),i[287]||(i[287]=s("h2",{id:"Training-API",tabindex:"-1"},[a("Training API "),s("a",{class:"header-anchor",href:"#Training-API","aria-label":'Permalink to "Training API {#Training-API}"'},"​")],-1)),i[288]||(i[288]=s("p",null,[a("Helper Functions making it easier to train "),s("code",null,"Lux.jl"),a(" models.")],-1)),i[289]||(i[289]=s("p",null,"Training is meant to be simple and provide extremely basic functionality. We provide basic building blocks which can be seamlessly composed to create complex training pipelines.",-1)),s("details",k,[s("summary",null,[i[0]||(i[0]=s("a",{id:"Lux.Training.TrainState",href:"#Lux.Training.TrainState"},[s("span",{class:"jlbinding"},"Lux.Training.TrainState")],-1)),i[1]||(i[1]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[2]||(i[2]=t('
    julia
    TrainState
    Training State containing:
    • model: Lux model.
    • parameters: Trainable Variables of the model.
    • states: Non-trainable Variables of the model.
    • optimizer: Optimizer from Optimisers.jl.
    • optimizer_state: Optimizer State.
    • step: Number of updates of the parameters made.
    Internal fields:
    • cache: Cached values. Implementations are free to use this for whatever they want.
    • objective_function: Objective function might be cached.
    Warning
    Constructing this object directly shouldn't be considered a stable API. Use the version with the Optimisers API.
    source
    ',7))]),s("details",o,[s("summary",null,[i[3]||(i[3]=s("a",{id:"Lux.Training.compute_gradients",href:"#Lux.Training.compute_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.compute_gradients")],-1)),i[4]||(i[4]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[5]||(i[5]=t(`
    julia
    compute_gradients(ad::AbstractADType, objective_function::Function, data,
+    ts::TrainState)
    Compute the gradients of the objective function wrt parameters stored in ts.
    Backends & AD Packages
    Supported BackendsPackages Needed
    AutoZygoteZygote.jl
    AutoReverseDiff(; compile)ReverseDiff.jl
    AutoTrackerTracker.jl
    AutoEnzymeEnzyme.jl
    Arguments
    • ad: Backend (from ADTypes.jl) used to compute the gradients.
    • objective_function: Objective function. The function must take 4 inputs – model, parameters, states and data. The function must return 3 values – loss, updated_state, and any computed statistics.
    • data: Data used to compute the gradients.
    • ts: Current Training State. See TrainState.
    Return
    A 4-Tuple containing:
    • grads: Computed Gradients.
    • loss: Loss from the objective function.
    • stats: Any computed statistics from the objective function.
    • ts: Updated Training State.
    Known Limitations
    • AutoReverseDiff(; compile=true) is not supported for Lux models with non-empty state st. Additionally the returned stats must be empty (NamedTuple()). We catch these issues in most cases and throw an error.
    Aliased Gradients
    grads returned by this function might be aliased by the implementation of the gradient backend. For example, if you cache the grads from step i, the new gradients returned in step i + 1 might be aliased by the old gradients. If you want to prevent this, simply use copy(grads) or deepcopy(grads) to make a copy of the gradients.
    source
    `,13))]),s("details",T,[s("summary",null,[i[6]||(i[6]=s("a",{id:"Lux.Training.apply_gradients",href:"#Lux.Training.apply_gradients"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients")],-1)),i[7]||(i[7]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[8]||(i[8]=t('
    julia
    apply_gradients(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",Q,[s("summary",null,[i[9]||(i[9]=s("a",{id:"Lux.Training.apply_gradients!",href:"#Lux.Training.apply_gradients!"},[s("span",{class:"jlbinding"},"Lux.Training.apply_gradients!")],-1)),i[10]||(i[10]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[11]||(i[11]=t('
    julia
    apply_gradients!(ts::TrainState, grads)
    Update the parameters stored in ts using the gradients grads. This is an inplace version of apply_gradients.
    Arguments
    • ts: TrainState object.
    • grads: Gradients of the loss function wrt ts.params.
    Returns
    Updated TrainState object.
    source
    ',7))]),s("details",g,[s("summary",null,[i[12]||(i[12]=s("a",{id:"Lux.Training.single_train_step",href:"#Lux.Training.single_train_step"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step")],-1)),i[13]||(i[13]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[14]||(i[14]=t('
    julia
    single_train_step(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients. All backends supported via compute_gradients are supported here.
    In most cases you should use single_train_step! instead of this function.
    Return
    Returned values are the same as compute_gradients.
    source
    ',6))]),s("details",m,[s("summary",null,[i[15]||(i[15]=s("a",{id:"Lux.Training.single_train_step!",href:"#Lux.Training.single_train_step!"},[s("span",{class:"jlbinding"},"Lux.Training.single_train_step!")],-1)),i[16]||(i[16]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[17]||(i[17]=t('
    julia
    single_train_step!(backend, obj_fn::F, data, ts::TrainState)
    Perform a single training step. Computes the gradients using compute_gradients and updates the parameters using apply_gradients!. All backends supported via compute_gradients are supported here.
    Return
    Returned values are the same as compute_gradients. Note that despite the !, only the parameters in ts are updated inplace. Users should be using the returned ts object for further training steps, else there is no caching and performance will be suboptimal (and absolutely terrible for backends like AutoReactant).
    source
    ',5))]),i[290]||(i[290]=t('
    Loss Functions ​
    Loss Functions Objects take 2 forms of inputs:
    1. ŷ and y where ŷ is the predicted output and y is the target output.
    2. model, ps, st, (x, y) where model is the model, ps are the parameters, st are the states and (x, y) are the input and target pair. Then it returns the loss, updated states, and an empty named tuple. This makes them compatible with the Training API.
    Warning
    When using ChainRules.jl compatible AD (like Zygote), we only compute the gradients wrt the inputs and drop any gradients wrt the targets.
    ',4)),s("details",y,[s("summary",null,[i[18]||(i[18]=s("a",{id:"Lux.GenericLossFunction",href:"#Lux.GenericLossFunction"},[s("span",{class:"jlbinding"},"Lux.GenericLossFunction")],-1)),i[19]||(i[19]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[20]||(i[20]=t(`
    julia
    GenericLossFunction(loss_fn; agg = mean)
    Takes any function loss_fn that maps 2 number inputs to a single number output. Additionally, array inputs are efficiently broadcasted and aggregated using agg.
    julia
    julia> mseloss = GenericLossFunction((ŷ, y) -> abs2(ŷ - y));
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> mseloss(y_model, 1:3) ≈ 0.01
-true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",c,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
+true
    Special Note
    This function takes any of the LossFunctions.jl public functions into the Lux Losses API with efficient aggregation.
    source
    `,6))]),s("details",E,[s("summary",null,[i[21]||(i[21]=s("a",{id:"Lux.BinaryCrossEntropyLoss",href:"#Lux.BinaryCrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryCrossEntropyLoss")],-1)),i[22]||(i[22]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[51]||(i[51]=t(`
    julia
    BinaryCrossEntropyLoss(; agg = mean, epsilon = nothing,
     label_smoothing::Union{Nothing, Real}=nothing,
-    logits::Union{Bool, Val}=Val(false))
    Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
    Returns the binary cross entropy loss computed as:
    • If logits is either false or Val(false):
    `,4)),s("mjx-container",E,[(h(),n("svg",u,i[23]||(i[23]=[t('',1)]))),i[24]||(i[24]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[52]||(i[52]=s("ul",null,[s("li",null,[a("If "),s("code",null,"logits"),a(" is "),s("code",null,"true"),a(" or "),s("code",null,"Val(true)"),a(":")])],-1)),s("mjx-container",F,[(h(),n("svg",x,i[25]||(i[25]=[t('',1)]))),i[26]||(i[26]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"l"),s("mi",null,"o"),s("mi",null,"g"),s("mi",null,"σ"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[33]||(i[33]=a("The value of ")),s("mjx-container",C,[(h(),n("svg",L,i[27]||(i[27]=[t('',1)]))),i[28]||(i[28]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[34]||(i[34]=a(" is computed using label smoothing. If ")),i[35]||(i[35]=s("code",null,"label_smoothing",-1)),i[36]||(i[36]=a(" is ")),i[37]||(i[37]=s("code",null,"nothing",-1)),i[38]||(i[38]=a(", then no label smoothing is applied. If ")),i[39]||(i[39]=s("code",null,"label_smoothing",-1)),i[40]||(i[40]=a(" is a real number ")),s("mjx-container",f,[(h(),n("svg",b,i[29]||(i[29]=[t('',1)]))),i[30]||(i[30]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[41]||(i[41]=a(", then the value of ")),s("mjx-container",w,[(h(),n("svg",H,i[31]||(i[31]=[t('',1)]))),i[32]||(i[32]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[42]||(i[42]=a(" is:"))]),s("mjx-container",v,[(h(),n("svg",j,i[43]||(i[43]=[t('',1)]))),i[44]||(i[44]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mn",null,"0.5")])],-1))]),s("p",null,[i[47]||(i[47]=a("where ")),s("mjx-container",D,[(h(),n("svg",B,i[45]||(i[45]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[46]||(i[46]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[48]||(i[48]=a(" is the value of ")),i[49]||(i[49]=s("code",null,"label_smoothing",-1)),i[50]||(i[50]=a("."))]),i[53]||(i[53]=t(`
    Extended Help
    Example
    julia
    julia> bce = BinaryCrossEntropyLoss();
+    logits::Union{Bool, Val}=Val(false))
    Binary Cross Entropy Loss with optional label smoothing and fused logit computation.
    Returns the binary cross entropy loss computed as:
    • If logits is either false or Val(false):
    `,4)),s("mjx-container",c,[(h(),n("svg",u,i[23]||(i[23]=[t('',1)]))),i[24]||(i[24]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[52]||(i[52]=s("ul",null,[s("li",null,[a("If "),s("code",null,"logits"),a(" is "),s("code",null,"true"),a(" or "),s("code",null,"Val(true)"),a(":")])],-1)),s("mjx-container",F,[(h(),n("svg",x,i[25]||(i[25]=[t('',1)]))),i[26]||(i[26]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"l"),s("mi",null,"o"),s("mi",null,"g"),s("mi",null,"σ"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[33]||(i[33]=a("The value of ")),s("mjx-container",C,[(h(),n("svg",L,i[27]||(i[27]=[t('',1)]))),i[28]||(i[28]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[34]||(i[34]=a(" is computed using label smoothing. If ")),i[35]||(i[35]=s("code",null,"label_smoothing",-1)),i[36]||(i[36]=a(" is ")),i[37]||(i[37]=s("code",null,"nothing",-1)),i[38]||(i[38]=a(", then no label smoothing is applied. If ")),i[39]||(i[39]=s("code",null,"label_smoothing",-1)),i[40]||(i[40]=a(" is a real number ")),s("mjx-container",f,[(h(),n("svg",b,i[29]||(i[29]=[t('',1)]))),i[30]||(i[30]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[41]||(i[41]=a(", then the value of ")),s("mjx-container",w,[(h(),n("svg",H,i[31]||(i[31]=[t('',1)]))),i[32]||(i[32]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[42]||(i[42]=a(" is:"))]),s("mjx-container",v,[(h(),n("svg",j,i[43]||(i[43]=[t('',1)]))),i[44]||(i[44]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mn",null,"0.5")])],-1))]),s("p",null,[i[47]||(i[47]=a("where ")),s("mjx-container",D,[(h(),n("svg",B,i[45]||(i[45]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[46]||(i[46]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[48]||(i[48]=a(" is the value of ")),i[49]||(i[49]=s("code",null,"label_smoothing",-1)),i[50]||(i[50]=a("."))]),i[53]||(i[53]=t(`
    Extended Help
    Example
    julia
    julia> bce = BinaryCrossEntropyLoss();
 
 julia> y_bin = Bool[1, 0, 1];
 
@@ -32,7 +32,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 julia> logitbce_ls = BinaryCrossEntropyLoss(label_smoothing=0.1, logits=Val(true));
 
 julia> logitbce_ls(y_model, y_bin) > logitbce(y_model, y_bin)
-true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
+true
    source
    `,4))]),s("details",M,[s("summary",null,[i[54]||(i[54]=s("a",{id:"Lux.BinaryFocalLoss",href:"#Lux.BinaryFocalLoss"},[s("span",{class:"jlbinding"},"Lux.BinaryFocalLoss")],-1)),i[55]||(i[55]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[66]||(i[66]=t('
    julia
    BinaryFocalLoss(; gamma = 2, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[58]||(i[58]=a("Return the binary focal loss [1]. The model input, ")),s("mjx-container",V,[(h(),n("svg",A,i[56]||(i[56]=[t('',1)]))),i[57]||(i[57]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[59]||(i[59]=a(", is expected to be normalized (i.e. softmax output)."))]),s("p",null,[i[62]||(i[62]=a("For ")),s("mjx-container",Z,[(h(),n("svg",O,i[60]||(i[60]=[t('',1)]))),i[61]||(i[61]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[63]||(i[63]=a(" this is equivalent to ")),i[64]||(i[64]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.BinaryCrossEntropyLoss"},[s("code",null,"BinaryCrossEntropyLoss")],-1)),i[65]||(i[65]=a("."))]),i[67]||(i[67]=t(`
    Example
    julia
    julia> y = [0  1  0
             1  0  1];
 
 julia> ŷ = [0.268941  0.5  0.268941
@@ -42,7 +42,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> BinaryFocalLoss(gamma=0)(ŷ, y) ≈ BinaryCrossEntropyLoss()(ŷ, y)
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",R,[s("summary",null,[i[68]||(i[68]=s("a",{id:"Lux.CrossEntropyLoss",href:"#Lux.CrossEntropyLoss"},[s("span",{class:"jlbinding"},"Lux.CrossEntropyLoss")],-1)),i[69]||(i[69]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[111]||(i[111]=t(`
    julia
    CrossEntropyLoss(; agg=mean, epsilon=nothing, dims=1,
     label_smoothing::Union{Nothing, Real}=nothing)
    `,1)),s("p",null,[i[72]||(i[72]=a("Return the cross entropy loss which is used in multi-class classification tasks. The input, ")),s("mjx-container",S,[(h(),n("svg",N,i[70]||(i[70]=[t('',1)]))),i[71]||(i[71]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[73]||(i[73]=a(", is expected to be normalized (i.e. ")),i[74]||(i[74]=s("code",null,"softmax",-1)),i[75]||(i[75]=a(" output) if ")),i[76]||(i[76]=s("code",null,"logits",-1)),i[77]||(i[77]=a(" is ")),i[78]||(i[78]=s("code",null,"false",-1)),i[79]||(i[79]=a(" or ")),i[80]||(i[80]=s("code",null,"Val(false)",-1)),i[81]||(i[81]=a("."))]),i[112]||(i[112]=s("p",null,"The loss is calculated as:",-1)),s("mjx-container",P,[(h(),n("svg",z,i[82]||(i[82]=[t('',1)]))),i[83]||(i[83]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",null,"−"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[92]||(i[92]=a("where ")),s("mjx-container",I,[(h(),n("svg",q,i[84]||(i[84]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D716",d:"M227 -11Q149 -11 95 41T40 174Q40 262 87 322Q121 367 173 396T287 430Q289 431 329 431H367Q382 426 382 411Q382 385 341 385H325H312Q191 385 154 277L150 265H327Q340 256 340 246Q340 228 320 219H138V217Q128 187 128 143Q128 77 160 52T231 26Q258 26 284 36T326 57T343 68Q350 68 354 58T358 39Q358 36 357 35Q354 31 337 21T289 0T227 -11Z",style:{"stroke-width":"3"}})])])],-1)]))),i[85]||(i[85]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"ϵ")])],-1))]),i[93]||(i[93]=a(" is added for numerical stability. The value of ")),s("mjx-container",G,[(h(),n("svg",X,i[86]||(i[86]=[t('',1)]))),i[87]||(i[87]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[94]||(i[94]=a(" is computed using label smoothing. If ")),i[95]||(i[95]=s("code",null,"label_smoothing",-1)),i[96]||(i[96]=a(" is ")),i[97]||(i[97]=s("code",null,"nothing",-1)),i[98]||(i[98]=a(", then no label smoothing is applied. If ")),i[99]||(i[99]=s("code",null,"label_smoothing",-1)),i[100]||(i[100]=a(" is a real number ")),s("mjx-container",J,[(h(),n("svg",U,i[88]||(i[88]=[t('',1)]))),i[89]||(i[89]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mo",null,"∈"),s("mo",{stretchy:"false"},"["),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",{stretchy:"false"},"]")])],-1))]),i[101]||(i[101]=a(", then the value of ")),s("mjx-container",W,[(h(),n("svg",K,i[90]||(i[90]=[t('',1)]))),i[91]||(i[91]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])])])],-1))]),i[102]||(i[102]=a(" is calculated as:"))]),s("mjx-container",Y,[(h(),n("svg",$,i[103]||(i[103]=[t('',1)]))),i[104]||(i[104]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"~")])]),s("mo",null,"="),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"α"),s("mo",{stretchy:"false"},")"),s("mo",null,"∗"),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"α"),s("mo",null,"∗"),s("mtext",null,"size along dim")])],-1))]),s("p",null,[i[107]||(i[107]=a("where ")),s("mjx-container",_,[(h(),n("svg",s1,i[105]||(i[105]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[106]||(i[106]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[108]||(i[108]=a(" is the value of ")),i[109]||(i[109]=s("code",null,"label_smoothing",-1)),i[110]||(i[110]=a("."))]),i[113]||(i[113]=t(`
    Extended Help
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
@@ -64,7 +64,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> CrossEntropyLoss(label_smoothing=0.15)(y_model, y) ≈ 1.5776052f0
-true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",i1,[s("summary",null,[i[114]||(i[114]=s("a",{id:"Lux.DiceCoeffLoss",href:"#Lux.DiceCoeffLoss"},[s("span",{class:"jlbinding"},"Lux.DiceCoeffLoss")],-1)),i[115]||(i[115]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[124]||(i[124]=t('
    julia
    DiceCoeffLoss(; smooth = true, agg = mean)
    Return the Dice Coefficient loss [1] which is used in segmentation tasks. The dice coefficient is similar to the F1_score. Loss calculated as:
    ',2)),s("mjx-container",a1,[(h(),n("svg",t1,i[116]||(i[116]=[t('',1)]))),i[117]||(i[117]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"a"),s("mi",null,"g"),s("mi",null,"g"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mfrac",null,[s("mrow",null,[s("mn",null,"2"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"α")]),s("mrow",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mi",null,"y"),s("mn",null,"2")]),s("mo",null,"+"),s("mo",{"data-mjx-texclass":"OP"},"∑"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"α")])]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),s("p",null,[i[120]||(i[120]=a("where ")),s("mjx-container",e1,[(h(),n("svg",l1,i[118]||(i[118]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FC",d:"M34 156Q34 270 120 356T309 442Q379 442 421 402T478 304Q484 275 485 237V208Q534 282 560 374Q564 388 566 390T582 393Q603 393 603 385Q603 376 594 346T558 261T497 161L486 147L487 123Q489 67 495 47T514 26Q528 28 540 37T557 60Q559 67 562 68T577 70Q597 70 597 62Q597 56 591 43Q579 19 556 5T512 -10H505Q438 -10 414 62L411 69L400 61Q390 53 370 41T325 18T267 -2T203 -11Q124 -11 79 39T34 156ZM208 26Q257 26 306 47T379 90L403 112Q401 255 396 290Q382 405 304 405Q235 405 183 332Q156 292 139 224T121 120Q121 71 146 49T208 26Z",style:{"stroke-width":"3"}})])])],-1)]))),i[119]||(i[119]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"α")])],-1))]),i[121]||(i[121]=a(" is the smoothing factor (")),i[122]||(i[122]=s("code",null,"smooth",-1)),i[123]||(i[123]=a(")."))]),i[125]||(i[125]=t(`
    Example
    julia
    julia> y_pred = [1.1, 2.1, 3.1];
 
 julia> DiceCoeffLoss()(y_pred, 1:3)  ≈ 0.000992391663909964
 true
@@ -73,7 +73,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> DiceCoeffLoss()(reshape(y_pred, 3, 1), reshape(1:3, 3, 1)) ≈ 0.000992391663909964
-true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
+true
    References
    [1] Milletari, Fausto, Nassir Navab, and Seyed-Ahmad Ahmadi. "V-net: Fully convolutional neural networks for volumetric medical image segmentation." 2016 fourth international conference on 3D vision (3DV). Ieee, 2016.
    source
    `,5))]),s("details",n1,[s("summary",null,[i[126]||(i[126]=s("a",{id:"Lux.FocalLoss",href:"#Lux.FocalLoss"},[s("span",{class:"jlbinding"},"Lux.FocalLoss")],-1)),i[127]||(i[127]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[143]||(i[143]=t('
    julia
    FocalLoss(; gamma = 2, dims = 1, agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[130]||(i[130]=a("Return the focal loss [1] which can be used in classification tasks with highly imbalanced classes. It down-weights well-classified examples and focuses on hard examples. The input, ")),s("mjx-container",h1,[(h(),n("svg",p1,i[128]||(i[128]=[t('',1)]))),i[129]||(i[129]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[131]||(i[131]=a(", is expected to be normalized (i.e. ")),i[132]||(i[132]=s("code",null,"softmax",-1)),i[133]||(i[133]=a(" output)."))]),s("p",null,[i[138]||(i[138]=a("The modulating factor ")),s("mjx-container",r1,[(h(),n("svg",d1,i[134]||(i[134]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FE",d:"M31 249Q11 249 11 258Q11 275 26 304T66 365T129 418T206 441Q233 441 239 440Q287 429 318 386T371 255Q385 195 385 170Q385 166 386 166L398 193Q418 244 443 300T486 391T508 430Q510 431 524 431H537Q543 425 543 422Q543 418 522 378T463 251T391 71Q385 55 378 6T357 -100Q341 -165 330 -190T303 -216Q286 -216 286 -188Q286 -138 340 32L346 51L347 69Q348 79 348 100Q348 257 291 317Q251 355 196 355Q148 355 108 329T51 260Q49 251 47 251Q45 249 31 249Z",style:{"stroke-width":"3"}})])])],-1)]))),i[135]||(i[135]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ")])],-1))]),i[139]||(i[139]=a(", controls the down-weighting strength. For ")),s("mjx-container",k1,[(h(),n("svg",o1,i[136]||(i[136]=[t('',1)]))),i[137]||(i[137]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"γ"),s("mo",null,"="),s("mn",null,"0")])],-1))]),i[140]||(i[140]=a(" this is equivalent to ")),i[141]||(i[141]=s("a",{href:"/previews/PR1000/api/Lux/utilities#Lux.CrossEntropyLoss"},[s("code",null,"CrossEntropyLoss")],-1)),i[142]||(i[142]=a("."))]),i[144]||(i[144]=t(`
    Example
    julia
    julia> y = [1  0  0  0  1
             0  1  0  1  0
             0  0  1  0  0]
 3×5 Matrix{Int64}:
@@ -88,20 +88,20 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
  0.665241   0.665241   0.665241   0.665241   0.665241
 
 julia> FocalLoss()(ŷ, y) ≈ 1.1277556f0
-true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
+true
    References
    [1] Lin, Tsung-Yi, et al. "Focal loss for dense object detection." Proceedings of the IEEE international conference on computer vision. 2017.
    source
    `,5))]),s("details",T1,[s("summary",null,[i[145]||(i[145]=s("a",{id:"Lux.HingeLoss",href:"#Lux.HingeLoss"},[s("span",{class:"jlbinding"},"Lux.HingeLoss")],-1)),i[146]||(i[146]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[149]||(i[149]=t('
    julia
    HingeLoss(; agg = mean)
    Return the hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",Q1,[(h(),n("svg",g1,i[147]||(i[147]=[t('',1)]))),i[148]||(i[148]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[150]||(i[150]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = HingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.55
-true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",c1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",E1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
+true
    source
    `,4))]),s("details",m1,[s("summary",null,[i[151]||(i[151]=s("a",{id:"Lux.HuberLoss",href:"#Lux.HuberLoss"},[s("span",{class:"jlbinding"},"Lux.HuberLoss")],-1)),i[152]||(i[152]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[161]||(i[161]=t('
    julia
    HuberLoss(; delta = 1, agg = mean)
    Returns the Huber loss, calculated as:
    ',2)),s("mjx-container",y1,[(h(),n("svg",E1,i[153]||(i[153]=[t('',1)]))),i[154]||(i[154]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mi",null,"L"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"{"),s("mtable",{columnalign:"left left",columnspacing:"1em",rowspacing:".2em"},[s("mtr",null,[s("mtd",null,[s("mn",null,"0.5"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mn",null,"2")])]),s("mtd",null,[s("mtext",null,"if "),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"≤"),s("mi",null,"δ")])]),s("mtr",null,[s("mtd",null,[s("mi",null,"δ"),s("mo",null,"∗"),s("mo",{stretchy:"false"},"("),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mi",null,"y"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{"data-mjx-texclass":"ORD",stretchy:"false"},"|"),s("mo",null,"−"),s("mn",null,"0.5"),s("mo",null,"∗"),s("mi",null,"δ"),s("mo",{stretchy:"false"},")")]),s("mtd",null,[s("mtext",null,"otherwise")])])]),s("mo",{"data-mjx-texclass":"CLOSE",fence:"true",stretchy:"true",symmetric:"true"})])])],-1))]),s("p",null,[i[157]||(i[157]=a("where ")),s("mjx-container",c1,[(h(),n("svg",u1,i[155]||(i[155]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D6FF",d:"M195 609Q195 656 227 686T302 717Q319 716 351 709T407 697T433 690Q451 682 451 662Q451 644 438 628T403 612Q382 612 348 641T288 671T249 657T235 628Q235 584 334 463Q401 379 401 292Q401 169 340 80T205 -10H198Q127 -10 83 36T36 153Q36 286 151 382Q191 413 252 434Q252 435 245 449T230 481T214 521T201 566T195 609ZM112 130Q112 83 136 55T204 27Q233 27 256 51T291 111T309 178T316 232Q316 267 309 298T295 344T269 400L259 396Q215 381 183 342T137 256T118 179T112 130Z",style:{"stroke-width":"3"}})])])],-1)]))),i[156]||(i[156]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"δ")])],-1))]),i[158]||(i[158]=a(" is the ")),i[159]||(i[159]=s("code",null,"delta",-1)),i[160]||(i[160]=a(" parameter."))]),i[162]||(i[162]=t(`
    Example
    julia
    julia> y_model = [1.1, 2.1, 3.1];
 
 julia> HuberLoss()(y_model, 1:3) ≈ 0.005000000000000009
 true
 
 julia> HuberLoss(delta=0.05)(y_model, 1:3) ≈ 0.003750000000000005
-true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
+true
    source
    `,3))]),s("details",F1,[s("summary",null,[i[163]||(i[163]=s("a",{id:"Lux.KLDivergenceLoss",href:"#Lux.KLDivergenceLoss"},[s("span",{class:"jlbinding"},"Lux.KLDivergenceLoss")],-1)),i[164]||(i[164]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[172]||(i[172]=t('
    julia
    KLDivergenceLoss(; dims = 1, agg = mean, epsilon = nothing, label_smoothing = nothing)
    ',1)),s("p",null,[i[169]||(i[169]=a("Return the Kullback-Leibler Divergence loss between the predicted distribution ")),s("mjx-container",x1,[(h(),n("svg",C1,i[165]||(i[165]=[t('',1)]))),i[166]||(i[166]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[170]||(i[170]=a(" and the true distribution ")),s("mjx-container",L1,[(h(),n("svg",f1,i[167]||(i[167]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[168]||(i[168]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[171]||(i[171]=a(":"))]),i[173]||(i[173]=t(`
    The KL divergence is a measure of how much one probability distribution is different from the other. It is always non-negative, and zero only when both the distributions are equal.
    For epsilon and label_smoothing, see CrossEntropyLoss.
    Example
    julia
    julia> p1 = [1 0; 0 1]
 2×2 Matrix{Int64}:
  1  0
  0  1
@@ -121,44 +121,44 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 0.0
 
 julia> KLDivergenceLoss(; epsilon=0)(p1, p2)
-Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
+Inf
    source
    `,5))]),s("details",b1,[s("summary",null,[i[174]||(i[174]=s("a",{id:"Lux.MAELoss",href:"#Lux.MAELoss"},[s("span",{class:"jlbinding"},"Lux.MAELoss")],-1)),i[175]||(i[175]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[178]||(i[178]=t('
    julia
    MAELoss(; agg = mean)
    Returns the loss corresponding to mean absolute error:
    ',2)),s("mjx-container",w1,[(h(),n("svg",H1,i[176]||(i[176]=[t('',1)]))),i[177]||(i[177]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"|"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},"|")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[179]||(i[179]=t(`
    Example
    julia
    julia> loss = MAELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.1
-true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
+true
    source
    `,3))]),s("details",v1,[s("summary",null,[i[180]||(i[180]=s("a",{id:"Lux.MSELoss",href:"#Lux.MSELoss"},[s("span",{class:"jlbinding"},"Lux.MSELoss")],-1)),i[181]||(i[181]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[184]||(i[184]=t('
    julia
    MSELoss(; agg = mean)
    Returns the loss corresponding to mean squared error:
    ',2)),s("mjx-container",j1,[(h(),n("svg",D1,i[182]||(i[182]=[t('',1)]))),i[183]||(i[183]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[185]||(i[185]=t(`
    Example
    julia
    julia> loss = MSELoss();
 
 julia> y_model = [1.1, 1.9, 3.1];
 
 julia> loss(y_model, 1:3) ≈ 0.01
-true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
+true
    source
    `,3))]),s("details",B1,[s("summary",null,[i[186]||(i[186]=s("a",{id:"Lux.MSLELoss",href:"#Lux.MSLELoss"},[s("span",{class:"jlbinding"},"Lux.MSLELoss")],-1)),i[187]||(i[187]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[190]||(i[190]=t('
    julia
    MSLELoss(; agg = mean, epsilon = nothing)
    Returns the loss corresponding to mean squared logarithmic error:
    ',2)),s("mjx-container",M1,[(h(),n("svg",V1,i[188]||(i[188]=[t('',1)]))),i[189]||(i[189]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("msup",null,[s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",null,"−"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mi",null,"y"),s("mo",null,"+"),s("mi",null,"ϵ"),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")]),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[191]||(i[191]=t(`
    epsilon is added to both y and ŷ to prevent taking the logarithm of zero. If epsilon is nothing, then we set it to eps(<type of y and ŷ>).
    Example
    julia
    julia> loss = MSLELoss();
 
 julia> loss(Float32[1.1, 2.2, 3.3], 1:3) ≈ 0.009084041f0
 true
 
 julia> loss(Float32[0.9, 1.8, 2.7], 1:3) ≈ 0.011100831f0
-true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",P1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
+true
    source
    `,4))]),s("details",A1,[s("summary",null,[i[192]||(i[192]=s("a",{id:"Lux.PoissonLoss",href:"#Lux.PoissonLoss"},[s("span",{class:"jlbinding"},"Lux.PoissonLoss")],-1)),i[193]||(i[193]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[203]||(i[203]=t('
    julia
    PoissonLoss(; agg = mean, epsilon = nothing)
    ',1)),s("p",null,[i[198]||(i[198]=a("Return how much the predicted distribution ")),s("mjx-container",Z1,[(h(),n("svg",O1,i[194]||(i[194]=[t('',1)]))),i[195]||(i[195]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])])])],-1))]),i[199]||(i[199]=a(" diverges from the expected Poisson distribution ")),s("mjx-container",R1,[(h(),n("svg",S1,i[196]||(i[196]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D466",d:"M21 287Q21 301 36 335T84 406T158 442Q199 442 224 419T250 355Q248 336 247 334Q247 331 231 288T198 191T182 105Q182 62 196 45T238 27Q261 27 281 38T312 61T339 94Q339 95 344 114T358 173T377 247Q415 397 419 404Q432 431 462 431Q475 431 483 424T494 412T496 403Q496 390 447 193T391 -23Q363 -106 294 -155T156 -205Q111 -205 77 -183T43 -117Q43 -95 50 -80T69 -58T89 -48T106 -45Q150 -45 150 -87Q150 -107 138 -122T115 -142T102 -147L99 -148Q101 -153 118 -160T152 -167H160Q177 -167 186 -165Q219 -156 247 -127T290 -65T313 -9T321 21L315 17Q309 13 296 6T270 -6Q250 -11 231 -11Q185 -11 150 11T104 82Q103 89 103 113Q103 170 138 262T173 379Q173 380 173 381Q173 390 173 393T169 400T158 404H154Q131 404 112 385T82 344T65 302T57 280Q55 278 41 278H27Q21 284 21 287Z",style:{"stroke-width":"3"}})])])],-1)]))),i[197]||(i[197]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"y")])],-1))]),i[200]||(i[200]=a(", calculated as:"))]),s("mjx-container",N1,[(h(),n("svg",P1,i[201]||(i[201]=[t('',1)]))),i[202]||(i[202]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",null,"−"),s("mi",null,"y"),s("mo",null,"∗"),s("mi",null,"log"),s("mo",{"data-mjx-texclass":"NONE"},"⁡"),s("mo",{stretchy:"false"},"("),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mo",{stretchy:"false"},")"),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[204]||(i[204]=t(`
    Example
    julia
    julia> y_model = [1, 3, 3];  # data should only take integral values
 
 julia> PoissonLoss()(y_model, 1:3) ≈ 0.502312852219817
-true
    source
    `,3))]),s("details",z1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
+true
    source
    `,3))]),s("details",z1,[s("summary",null,[i[205]||(i[205]=s("a",{id:"Lux.SiameseContrastiveLoss",href:"#Lux.SiameseContrastiveLoss"},[s("span",{class:"jlbinding"},"Lux.SiameseContrastiveLoss")],-1)),i[206]||(i[206]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[209]||(i[209]=t('
    julia
    SiameseContrastiveLoss(; margin = true, agg = mean)
    Return the contrastive loss [1] which can be useful for training Siamese Networks. It is given by:
    ',2)),s("mjx-container",I1,[(h(),n("svg",q1,i[207]||(i[207]=[t('',1)]))),i[208]||(i[208]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{stretchy:"false"},"("),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mo",{stretchy:"false"},")"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("mn",null,"2")]),s("mo",null,"+"),s("mi",null,"y"),s("mo",null,"∗"),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mtext",null,"margin"),s("mo",null,"−"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[210]||(i[210]=t(`
    Specify margin to set the baseline for distance at which pairs are dissimilar.
    Example
    julia
    julia> ŷ = [0.5, 1.5, 2.5];
 
 julia> SiameseContrastiveLoss()(ŷ, 1:3) ≈ -4.833333333333333
 true
 
 julia> SiameseContrastiveLoss(margin=2)(ŷ, 1:3) ≈ -4.0
-true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
+true
    References
    [1] Hadsell, Raia, Sumit Chopra, and Yann LeCun. "Dimensionality reduction by learning an invariant mapping." 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06). Vol. 2. IEEE, 2006.
    source
    `,6))]),s("details",G1,[s("summary",null,[i[211]||(i[211]=s("a",{id:"Lux.SquaredHingeLoss",href:"#Lux.SquaredHingeLoss"},[s("span",{class:"jlbinding"},"Lux.SquaredHingeLoss")],-1)),i[212]||(i[212]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[215]||(i[215]=t('
    julia
    SquaredHingeLoss(; agg = mean)
    Return the squared hinge loss loss given the prediction ŷ and true labels y (containing 1 or -1); calculated as:
    ',2)),s("mjx-container",X1,[(h(),n("svg",J1,i[213]||(i[213]=[t('',1)]))),i[214]||(i[214]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"agg"),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"("),s("mo",{"data-mjx-texclass":"OP",movablelimits:"true"},"max"),s("mo",{stretchy:"false"},"("),s("mn",null,"0"),s("mo",null,","),s("mn",null,"1"),s("mo",null,"−"),s("mi",null,"y"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mover",null,[s("mi",null,"y"),s("mo",{stretchy:"false"},"^")])]),s("msup",null,[s("mo",{stretchy:"false"},")"),s("mn",null,"2")]),s("mo",{"data-mjx-texclass":"CLOSE"},")")])])],-1))]),i[216]||(i[216]=t(`
    Usually used with classifiers like Support Vector Machines.
    Example
    julia
    julia> loss = SquaredHingeLoss();
 
 julia> y_true = [1, -1, 1, 1];
 
 julia> y_pred = [0.1, 0.3, 1, 1.5];
 
 julia> loss(y_pred, y_true) ≈ 0.625
-true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
-foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
-getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
+true
    source
    `,4))]),i[291]||(i[291]=s("h2",{id:"LuxOps-Module",tabindex:"-1"},[a("LuxOps Module "),s("a",{class:"header-anchor",href:"#LuxOps-Module","aria-label":'Permalink to "LuxOps Module {#LuxOps-Module}"'},"​")],-1)),s("details",U1,[s("summary",null,[i[217]||(i[217]=s("a",{id:"Lux.LuxOps",href:"#Lux.LuxOps"},[s("span",{class:"jlbinding"},"Lux.LuxOps")],-1)),i[218]||(i[218]=a()),l(e,{type:"info",class:"jlObjectType jlModule",text:"Module"})]),i[219]||(i[219]=t('
    julia
    LuxOps
    This module is a part of Lux.jl. It contains operations that are useful in DL context. Additionally certain operations here alias Base functions to behave more sensibly with GPUArrays.
    source
    ',3))]),s("details",W1,[s("summary",null,[i[220]||(i[220]=s("a",{id:"Lux.LuxOps.eachslice",href:"#Lux.LuxOps.eachslice"},[s("span",{class:"jlbinding"},"Lux.LuxOps.eachslice")],-1)),i[221]||(i[221]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[222]||(i[222]=t('
    julia
    eachslice(x, dims::Val)
    Same as Base.eachslice but doesn't produce a SubArray for the slices if x is a GPUArray.
    Additional dispatches for RNN helpers are also provided for TimeLastIndex and BatchLastIndex.
    source
    ',4))]),s("details",K1,[s("summary",null,[i[223]||(i[223]=s("a",{id:"Lux.LuxOps.foldl_init",href:"#Lux.LuxOps.foldl_init"},[s("span",{class:"jlbinding"},"Lux.LuxOps.foldl_init")],-1)),i[224]||(i[224]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[225]||(i[225]=t(`
    julia
    foldl_init(op, x)
+foldl_init(op, x, init)
    Exactly same as foldl(op, x; init) in the forward pass. But, gives gradients wrt init in the backward pass.
    source
    `,3))]),s("details",Y1,[s("summary",null,[i[226]||(i[226]=s("a",{id:"Lux.LuxOps.getproperty",href:"#Lux.LuxOps.getproperty"},[s("span",{class:"jlbinding"},"Lux.LuxOps.getproperty")],-1)),i[227]||(i[227]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[228]||(i[228]=t(`
    julia
    getproperty(x, ::Val{v})
+getproperty(x, ::StaticSymbol{v})
    Similar to Base.getproperty but requires a Val (or Static.StaticSymbol). Additionally, if v is not present in x, then nothing is returned.
    source
    `,3))]),s("details",$1,[s("summary",null,[i[229]||(i[229]=s("a",{id:"Lux.LuxOps.xlogx",href:"#Lux.LuxOps.xlogx"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogx")],-1)),i[230]||(i[230]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[231]||(i[231]=t('
    julia
    xlogx(x::Number)
    Return x * log(x) for x ≥ 0, handling x == 0 by taking the limit from above, to get zero.
    source
    ',3))]),s("details",_1,[s("summary",null,[i[232]||(i[232]=s("a",{id:"Lux.LuxOps.xlogy",href:"#Lux.LuxOps.xlogy"},[s("span",{class:"jlbinding"},"Lux.LuxOps.xlogy")],-1)),i[233]||(i[233]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[234]||(i[234]=t('
    julia
    xlogy(x::Number, y::Number)
    Return x * log(y) for y > 0, and zero when x == 0.
    source
    ',3))]),s("details",s2,[s("summary",null,[i[235]||(i[235]=s("a",{id:"Lux.LuxOps.istraining",href:"#Lux.LuxOps.istraining"},[s("span",{class:"jlbinding"},"Lux.LuxOps.istraining")],-1)),i[236]||(i[236]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[237]||(i[237]=t(`
    julia
    istraining(::Val{training})
 istraining(::StaticBool)
 istraining(::Bool)
-istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
+istraining(st::NamedTuple)
    Returns true if training is true or if st contains a training field with value true. Else returns false.
    source
    `,3))]),s("details",i2,[s("summary",null,[i[238]||(i[238]=s("a",{id:"Lux.LuxOps.multigate",href:"#Lux.LuxOps.multigate"},[s("span",{class:"jlbinding"},"Lux.LuxOps.multigate")],-1)),i[239]||(i[239]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[240]||(i[240]=t('
    julia
    multigate(x::AbstractArray, ::Val{N})
    Split up x into N equally sized chunks (along dimension 1).
    source
    ',3))]),i[292]||(i[292]=s("h2",{id:"Recursive-Operations",tabindex:"-1"},[a("Recursive Operations "),s("a",{class:"header-anchor",href:"#Recursive-Operations","aria-label":'Permalink to "Recursive Operations {#Recursive-Operations}"'},"​")],-1)),s("details",a2,[s("summary",null,[i[241]||(i[241]=s("a",{id:"Lux.recursive_map",href:"#Lux.recursive_map"},[s("span",{class:"jlbinding"},"Lux.recursive_map")],-1)),i[242]||(i[242]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[243]||(i[243]=t('
    julia
    recursive_map(f, x, args...)
    Similar to fmap(f, args...) but with restricted support for the notion of "leaf" types. However, this allows for more efficient and type stable implementations of recursive operations.
    How this works?
    For the following types it directly defines recursion rules:
    1. AbstractArray: If eltype is isbitstype, then f is applied to the array, else we recurse on the array.
    2. Tuple/NamedTuple: We recurse on the values.
    3. Number/Val/Nothing: We directly apply f.
    4. For all other types, we recurse on the fields using Functors.fmap.
    Note
    In most cases, users should gravitate towards Functors.fmap if it is being used outside of hot loops. Even for other cases, it is always recommended to verify the correctness of this implementation for specific usecases.
    source
    ',7))]),s("details",t2,[s("summary",null,[i[244]||(i[244]=s("a",{id:"Lux.recursive_add!!",href:"#Lux.recursive_add!!"},[s("span",{class:"jlbinding"},"Lux.recursive_add!!")],-1)),i[245]||(i[245]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[246]||(i[246]=t('
    julia
    recursive_add!!(x, y)
    Recursively add the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(+, x, y), but this implementation uses type stable code for common cases.
    Any leaves of x that are arrays and allow in-place addition will be modified in place.
    source
    ',4))]),s("details",e2,[s("summary",null,[i[247]||(i[247]=s("a",{id:"Lux.recursive_copyto!",href:"#Lux.recursive_copyto!"},[s("span",{class:"jlbinding"},"Lux.recursive_copyto!")],-1)),i[248]||(i[248]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[249]||(i[249]=t('
    julia
    recursive_copyto!(x, y)
    Recursively copy the leaves of two nested structures x and y. In Functor language, this is equivalent to doing fmap(copyto!, x, y), but this implementation uses type stable code for common cases. Note that any immutable leaf will lead to an error.
    source
    ',3))]),s("details",l2,[s("summary",null,[i[250]||(i[250]=s("a",{id:"Lux.recursive_eltype",href:"#Lux.recursive_eltype"},[s("span",{class:"jlbinding"},"Lux.recursive_eltype")],-1)),i[251]||(i[251]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[252]||(i[252]=t('
    julia
    recursive_eltype(x, unwrap_ad_types = Val(false))
    Recursively determine the element type of a nested structure x. This is equivalent to doing fmap(Lux.Utils.eltype, x), but this implementation uses type stable code for common cases.
    For ambiguous inputs like nothing and Val types we return Bool as the eltype.
    If unwrap_ad_types is set to Val(true) then for tracing and operator overloading based ADs (ForwardDiff, ReverseDiff, Tracker), this function will return the eltype of the unwrapped value.
    source
    ',5))]),s("details",n2,[s("summary",null,[i[253]||(i[253]=s("a",{id:"Lux.recursive_make_zero",href:"#Lux.recursive_make_zero"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero")],-1)),i[254]||(i[254]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[255]||(i[255]=t('
    julia
    recursive_make_zero(x)
    Recursively create a zero value for a nested structure x. This is equivalent to doing fmap(zero, x), but this implementation uses type stable code for common cases.
    See also Lux.recursive_make_zero!!.
    source
    ',4))]),s("details",h2,[s("summary",null,[i[256]||(i[256]=s("a",{id:"Lux.recursive_make_zero!!",href:"#Lux.recursive_make_zero!!"},[s("span",{class:"jlbinding"},"Lux.recursive_make_zero!!")],-1)),i[257]||(i[257]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[258]||(i[258]=t('
    julia
    recursive_make_zero!!(x)
    Recursively create a zero value for a nested structure x. Leaves that can be mutated with in-place zeroing will be modified in place.
    See also Lux.recursive_make_zero for fully out-of-place version.
    source
    ',4))]),i[293]||(i[293]=s("h2",{id:"Updating-Floating-Point-Precision",tabindex:"-1"},[a("Updating Floating Point Precision "),s("a",{class:"header-anchor",href:"#Updating-Floating-Point-Precision","aria-label":'Permalink to "Updating Floating Point Precision {#Updating-Floating-Point-Precision}"'},"​")],-1)),i[294]||(i[294]=s("p",null,"By default, Lux uses Float32 for all parameters and states. To update the precision simply pass the parameters / states / arrays into one of the following functions.",-1)),s("details",p2,[s("summary",null,[i[259]||(i[259]=s("a",{id:"Lux.f16",href:"#Lux.f16"},[s("span",{class:"jlbinding"},"Lux.f16")],-1)),i[260]||(i[260]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[261]||(i[261]=t('
    julia
    f16(m)
    Converts the eltype of m floating point values to Float16. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",r2,[s("summary",null,[i[262]||(i[262]=s("a",{id:"Lux.f32",href:"#Lux.f32"},[s("span",{class:"jlbinding"},"Lux.f32")],-1)),i[263]||(i[263]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[264]||(i[264]=t('
    julia
    f32(m)
    Converts the eltype of m floating point values to Float32. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),s("details",d2,[s("summary",null,[i[265]||(i[265]=s("a",{id:"Lux.f64",href:"#Lux.f64"},[s("span",{class:"jlbinding"},"Lux.f64")],-1)),i[266]||(i[266]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[267]||(i[267]=t('
    julia
    f64(m)
    Converts the eltype of m floating point values to Float64. Recurses into structs marked with Functors.@functor.
    source
    ',3))]),i[295]||(i[295]=s("h2",{id:"Element-Type-Matching",tabindex:"-1"},[a("Element Type Matching "),s("a",{class:"header-anchor",href:"#Element-Type-Matching","aria-label":'Permalink to "Element Type Matching {#Element-Type-Matching}"'},"​")],-1)),s("details",k2,[s("summary",null,[i[268]||(i[268]=s("a",{id:"Lux.match_eltype",href:"#Lux.match_eltype"},[s("span",{class:"jlbinding"},"Lux.match_eltype")],-1)),i[269]||(i[269]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[270]||(i[270]=t('
    julia
    match_eltype(layer, ps, st, args...)
    Helper function to "maybe" (see below) match the element type of args... with the element type of the layer's parameters and states. This is useful for debugging purposes, to track down accidental type-promotions inside Lux layers.
    Extended Help
    Controlling the Behavior via Preferences
    Behavior of this function is controlled via the eltype_mismatch_handling preference. The following options are supported:
    • "none": This is the default behavior. In this case, this function is a no-op, i.e., it simply returns args....
    • "warn": This option will issue a warning if the element type of args... does not match the element type of the layer's parameters and states. The warning will contain information about the layer and the element type mismatch.
    • "convert": This option is same as "warn", but it will also convert the element type of args... to match the element type of the layer's parameters and states (for the cases listed below).
    • "error": Same as "warn", but instead of issuing a warning, it will throw an error.
    Warning
    We print the warning for type-mismatch only once.
    Element Type Conversions
    For "convert" only the following conversions are done:
    Element Type of parameters/statesElement Type of args...Converted to
    Float64IntegerFloat64
    Float32Float64Float32
    Float32IntegerFloat32
    Float16Float64Float16
    Float16Float32Float16
    Float16IntegerFloat16
    source
    ',11))]),i[296]||(i[296]=s("h2",{id:"Stateful-Layer",tabindex:"-1"},[a("Stateful Layer "),s("a",{class:"header-anchor",href:"#Stateful-Layer","aria-label":'Permalink to "Stateful Layer {#Stateful-Layer}"'},"​")],-1)),s("details",o2,[s("summary",null,[i[271]||(i[271]=s("a",{id:"Lux.StatefulLuxLayer",href:"#Lux.StatefulLuxLayer"},[s("span",{class:"jlbinding"},"Lux.StatefulLuxLayer")],-1)),i[272]||(i[272]=a()),l(e,{type:"info",class:"jlObjectType jlType",text:"Type"})]),i[273]||(i[273]=t('
    julia
    StatefulLuxLayer{FT}(model, ps, st)
    Warning
    This is not a Lux.AbstractLuxLayer
    A convenience wrapper over Lux layers which stores the parameters and states internally. This is meant to be used in internal implementation of layers.
    Usecases
    • Internal implementation of @compact heavily uses this layer.
    • In SciML codebases where propagating state might involving Boxing. For a motivating example, see the Neural ODE tutorial.
    • Facilitates Nested AD support in Lux. For more details on this feature, see the Nested AD Manual Page.
    Static Parameters
    • If FT = true then the type of the state is fixed, i.e., typeof(last(model(x, ps, st))) == st.
    • If FT = false then type of the state might change. Note that while this works in all cases, it will introduce type instability.
    Arguments
    • model: A Lux layer
    • ps: The parameters of the layer. This can be set to nothing, if the user provides the parameters on function call
    • st: The state of the layer
    Inputs
    • x: The input to the layer
    • ps: The parameters of the layer. Optional, defaults to s.ps
    Outputs
    • y: The output of the layer
    source
    ',14))]),i[297]||(i[297]=s("h2",{id:"Compact-Layer",tabindex:"-1"},[a("Compact Layer "),s("a",{class:"header-anchor",href:"#Compact-Layer","aria-label":'Permalink to "Compact Layer {#Compact-Layer}"'},"​")],-1)),s("details",T2,[s("summary",null,[i[274]||(i[274]=s("a",{id:"Lux.@compact",href:"#Lux.@compact"},[s("span",{class:"jlbinding"},"Lux.@compact")],-1)),i[275]||(i[275]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[276]||(i[276]=t(`
    julia
    @compact(kw...) do x
     ...
     @return y # optional (but recommended for best performance)
 end
@@ -274,7 +274,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
    You may also specify a name for the model, which will be used instead of the default printout, which gives a verbatim representation of the code used to construct the model:
    julia
    julia> model = @compact(w=rand(3), name="Linear(3 => 1)") do x
            @return sum(w .* x)
        end
-Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
+Linear(3 => 1)      # 3 parameters
    This can be useful when using @compact to hierarchically construct complex models to be used inside a Chain.
    Type Stability
    If your input function f is type-stable but the generated model is not type stable, it should be treated as a bug. We will appreciate issues if you find such cases.
    Parameter Count
    Array Parameter don't print the number of parameters on the side. However, they do account for the total number of parameters printed at the bottom.
    source
    `,23))]),s("details",Q2,[s("summary",null,[i[277]||(i[277]=s("a",{id:"Lux.@init_fn",href:"#Lux.@init_fn"},[s("span",{class:"jlbinding"},"Lux.@init_fn")],-1)),i[278]||(i[278]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[279]||(i[279]=t(`
    julia
    @init_fn(fn, kind::Symbol = :parameter)
    Create an initializer function for a parameter or state to be used for in a Compact Lux Layer created using @compact.
    Arguments
    • fn: The function to be used for initializing the parameter or state. This only takes a single argument rng.
    • kind: If set to :parameter, the initializer function will be used to initialize the parameters of the layer. If set to :state, the initializer function will be used to initialize the states of the layer.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=@init_fn(rng->randn32(rng, 3, 2)),
            b=@init_fn(rng->randn32(rng, 3), :state)) do x
@@ -290,7 +290,7 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 (3,)
 
 julia> size(r([1, 2], ps, st)[1])
-(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
+(3,)
    source
    `,7))]),s("details",g2,[s("summary",null,[i[280]||(i[280]=s("a",{id:"Lux.@non_trainable",href:"#Lux.@non_trainable"},[s("span",{class:"jlbinding"},"Lux.@non_trainable")],-1)),i[281]||(i[281]=a()),l(e,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),i[282]||(i[282]=t(`
    julia
    @non_trainable(x)
    Mark a value as non-trainable. This bypasses the regular checks and places the value into the state of the layer.
    Arguments
    • x: The value to be marked as non-trainable.
    Examples
    julia
    julia> using Lux, Random
 
 julia> r = @compact(w=ones(3), w_fixed=@non_trainable(rand(3))) do x
            @return sum(w .* x .+ w_fixed)
@@ -310,5 +310,5 @@ import{_ as p,c as n,j as s,a,G as l,a2 as t,B as r,o as h}from"./chunks/framewo
 true
 
 julia> res isa Number
-true
    source
    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
-set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")
    Set the dispatch doctor preference for LuxCore and LuxLib packages.
    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.
    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.
    source
    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default};
+true

    source

    `,7))]),i[298]||(i[298]=s("h2",{id:"miscellaneous",tabindex:"-1"},[a("Miscellaneous "),s("a",{class:"header-anchor",href:"#miscellaneous","aria-label":'Permalink to "Miscellaneous"'},"​")],-1)),s("details",m2,[s("summary",null,[i[283]||(i[283]=s("a",{id:"Lux.set_dispatch_doctor_preferences!",href:"#Lux.set_dispatch_doctor_preferences!"},[s("span",{class:"jlbinding"},"Lux.set_dispatch_doctor_preferences!")],-1)),i[284]||(i[284]=a()),l(e,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),i[285]||(i[285]=t(`
    julia
    set_dispatch_doctor_preferences!(mode::String)
+set_dispatch_doctor_preferences!(; luxcore::String="disable", luxlib::String="disable")

    Set the dispatch doctor preference for LuxCore and LuxLib packages.

    mode can be "disable", "warn", or "error". For details on the different modes, see the DispatchDoctor.jl documentation.

    If the preferences are already set, then no action is taken. Otherwise the preference is set. For changes to take effect, the Julia session must be restarted.

    source

    `,5))])])}const f2=p(d,[["render",y2]]);export{L2 as __pageData,f2 as default}; diff --git a/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.D4Wj3KkE.js b/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.D4Wj3KkE.js deleted file mode 100644 index 3365439777..0000000000 --- a/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.D4Wj3KkE.js +++ /dev/null @@ -1,12 +0,0 @@ -import{_ as n,c as h,a2 as e,j as t,a as i,G as l,B as p,o as d}from"./chunks/framework.DjZDIZsN.js";const C=JSON.parse('{"title":"LuxTestUtils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Testing_Functionality/LuxTestUtils.md","filePath":"api/Testing_Functionality/LuxTestUtils.md","lastUpdated":null}'),k={name:"api/Testing_Functionality/LuxTestUtils.md"},r={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"};function u(y,s,f,b,F,x){const a=p("Badge");return d(),h("div",null,[s[15]||(s[15]=e('

    LuxTestUtils ​

    Warning

    This is a testing package. Hence, we don't use features like weak dependencies to reduce load times. It is recommended that you exclusively use this package for testing and not add a dependency to it in your main package Project.toml.

    Implements utilities for testing gradient correctness and dynamic dispatch of Lux.jl models.

    Testing using JET.jl ​

    ',4)),t("details",r,[t("summary",null,[s[0]||(s[0]=t("a",{id:"LuxTestUtils.@jet",href:"#LuxTestUtils.@jet"},[t("span",{class:"jlbinding"},"LuxTestUtils.@jet")],-1)),s[1]||(s[1]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[2]||(s[2]=e(`
    julia
    @jet f(args...) call_broken=false opt_broken=false

    Run JET tests on the function f with the arguments args.... If JET.jl fails to compile, then the macro will be a no-op.

    Keyword Arguments

    All additional arguments will be forwarded to JET.@test_call and JET.@test_opt.

    Tip

    Instead of specifying target_modules with every call, you can set global target modules using jet_target_modules!.

    julia
    using LuxTestUtils
-
-jet_target_modules!(["Lux", "LuxLib"]) # Expects Lux and LuxLib to be present in the module calling \`@jet\`

    Example

    julia
    julia> @jet sum([1, 2, 3]) target_modules=(Base, Core)
-Test Passed
-
-julia> @jet sum(1, 1) target_modules=(Base, Core) opt_broken=true call_broken=true
-Test Broken
-  Expression: #= REPL[21]:1 =# JET.@test_opt target_modules = (Base, Core) sum(1, 1)

    source

    `,9))]),t("details",o,[t("summary",null,[s[3]||(s[3]=t("a",{id:"LuxTestUtils.jet_target_modules!",href:"#LuxTestUtils.jet_target_modules!"},[t("span",{class:"jlbinding"},"LuxTestUtils.jet_target_modules!")],-1)),s[4]||(s[4]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e('
    julia
    jet_target_modules!(list::Vector{String}; force::Bool=false)

    This sets target_modules for all JET tests when using @jet.

    source

    ',3))]),s[16]||(s[16]=t("h2",{id:"Gradient-Correctness",tabindex:"-1"},[i("Gradient Correctness "),t("a",{class:"header-anchor",href:"#Gradient-Correctness","aria-label":'Permalink to "Gradient Correctness {#Gradient-Correctness}"'},"​")],-1)),t("details",g,[t("summary",null,[s[6]||(s[6]=t("a",{id:"LuxTestUtils.test_gradients",href:"#LuxTestUtils.test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.test_gradients")],-1)),s[7]||(s[7]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e(`
    julia
    test_gradients(f, args...; skip_backends=[], broken_backends=[], kwargs...)

    Test the gradients of f with respect to args using the specified backends.

    BackendADTypeCPUGPUNotes
    Zygote.jlAutoZygote()✔✔
    Tracker.jlAutoTracker()✔✔
    ReverseDiff.jlAutoReverseDiff()✔✖
    ForwardDiff.jlAutoForwardDiff()✔✖len ≤ 100
    FiniteDiff.jlAutoFiniteDiff()✔✖len ≤ 100
    Enzyme.jlAutoEnzyme()✔✖Only Reverse Mode

    Arguments

    Keyword Arguments

    Example

    julia
    julia> f(x, y, z) = x .+ sum(abs2, y.t) + sum(y.x.z)
-
-julia> x = (; t=rand(10), x=(z=[2.0],))
-
-julia> test_gradients(f, 1.0, x, nothing)

    source

    `,10))]),t("details",c,[t("summary",null,[s[9]||(s[9]=t("a",{id:"LuxTestUtils.@test_gradients",href:"#LuxTestUtils.@test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_gradients")],-1)),s[10]||(s[10]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[11]||(s[11]=e('
    julia
    @test_gradients(f, args...; kwargs...)

    See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

    source

    ',3))]),s[17]||(s[17]=t("h2",{id:"Extensions-to-@test",tabindex:"-1"},[i("Extensions to "),t("code",null,"@test"),i(),t("a",{class:"header-anchor",href:"#Extensions-to-@test","aria-label":'Permalink to "Extensions to `@test` {#Extensions-to-@test}"'},"​")],-1)),t("details",E,[t("summary",null,[s[12]||(s[12]=t("a",{id:"LuxTestUtils.@test_softfail",href:"#LuxTestUtils.@test_softfail"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_softfail")],-1)),s[13]||(s[13]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @test_softfail expr

    Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

    If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

    source

    ',4))])])}const j=n(k,[["render",u]]);export{C as __pageData,j as default}; diff --git a/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.D4Wj3KkE.lean.js b/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.D4Wj3KkE.lean.js deleted file mode 100644 index 3365439777..0000000000 --- a/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.D4Wj3KkE.lean.js +++ /dev/null @@ -1,12 +0,0 @@ -import{_ as n,c as h,a2 as e,j as t,a as i,G as l,B as p,o as d}from"./chunks/framework.DjZDIZsN.js";const C=JSON.parse('{"title":"LuxTestUtils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Testing_Functionality/LuxTestUtils.md","filePath":"api/Testing_Functionality/LuxTestUtils.md","lastUpdated":null}'),k={name:"api/Testing_Functionality/LuxTestUtils.md"},r={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"};function u(y,s,f,b,F,x){const a=p("Badge");return d(),h("div",null,[s[15]||(s[15]=e('

    LuxTestUtils ​

    Warning

    This is a testing package. Hence, we don't use features like weak dependencies to reduce load times. It is recommended that you exclusively use this package for testing and not add a dependency to it in your main package Project.toml.

    Implements utilities for testing gradient correctness and dynamic dispatch of Lux.jl models.

    Testing using JET.jl ​

    ',4)),t("details",r,[t("summary",null,[s[0]||(s[0]=t("a",{id:"LuxTestUtils.@jet",href:"#LuxTestUtils.@jet"},[t("span",{class:"jlbinding"},"LuxTestUtils.@jet")],-1)),s[1]||(s[1]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[2]||(s[2]=e(`
    julia
    @jet f(args...) call_broken=false opt_broken=false

    Run JET tests on the function f with the arguments args.... If JET.jl fails to compile, then the macro will be a no-op.

    Keyword Arguments

    All additional arguments will be forwarded to JET.@test_call and JET.@test_opt.

    Tip

    Instead of specifying target_modules with every call, you can set global target modules using jet_target_modules!.

    julia
    using LuxTestUtils
-
-jet_target_modules!(["Lux", "LuxLib"]) # Expects Lux and LuxLib to be present in the module calling \`@jet\`

    Example

    julia
    julia> @jet sum([1, 2, 3]) target_modules=(Base, Core)
-Test Passed
-
-julia> @jet sum(1, 1) target_modules=(Base, Core) opt_broken=true call_broken=true
-Test Broken
-  Expression: #= REPL[21]:1 =# JET.@test_opt target_modules = (Base, Core) sum(1, 1)

    source

    `,9))]),t("details",o,[t("summary",null,[s[3]||(s[3]=t("a",{id:"LuxTestUtils.jet_target_modules!",href:"#LuxTestUtils.jet_target_modules!"},[t("span",{class:"jlbinding"},"LuxTestUtils.jet_target_modules!")],-1)),s[4]||(s[4]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e('
    julia
    jet_target_modules!(list::Vector{String}; force::Bool=false)

    This sets target_modules for all JET tests when using @jet.

    source

    ',3))]),s[16]||(s[16]=t("h2",{id:"Gradient-Correctness",tabindex:"-1"},[i("Gradient Correctness "),t("a",{class:"header-anchor",href:"#Gradient-Correctness","aria-label":'Permalink to "Gradient Correctness {#Gradient-Correctness}"'},"​")],-1)),t("details",g,[t("summary",null,[s[6]||(s[6]=t("a",{id:"LuxTestUtils.test_gradients",href:"#LuxTestUtils.test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.test_gradients")],-1)),s[7]||(s[7]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e(`
    julia
    test_gradients(f, args...; skip_backends=[], broken_backends=[], kwargs...)

    Test the gradients of f with respect to args using the specified backends.

    BackendADTypeCPUGPUNotes
    Zygote.jlAutoZygote()✔✔
    Tracker.jlAutoTracker()✔✔
    ReverseDiff.jlAutoReverseDiff()✔✖
    ForwardDiff.jlAutoForwardDiff()✔✖len ≤ 100
    FiniteDiff.jlAutoFiniteDiff()✔✖len ≤ 100
    Enzyme.jlAutoEnzyme()✔✖Only Reverse Mode

    Arguments

    Keyword Arguments

    Example

    julia
    julia> f(x, y, z) = x .+ sum(abs2, y.t) + sum(y.x.z)
-
-julia> x = (; t=rand(10), x=(z=[2.0],))
-
-julia> test_gradients(f, 1.0, x, nothing)

    source

    `,10))]),t("details",c,[t("summary",null,[s[9]||(s[9]=t("a",{id:"LuxTestUtils.@test_gradients",href:"#LuxTestUtils.@test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_gradients")],-1)),s[10]||(s[10]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[11]||(s[11]=e('
    julia
    @test_gradients(f, args...; kwargs...)

    See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

    source

    ',3))]),s[17]||(s[17]=t("h2",{id:"Extensions-to-@test",tabindex:"-1"},[i("Extensions to "),t("code",null,"@test"),i(),t("a",{class:"header-anchor",href:"#Extensions-to-@test","aria-label":'Permalink to "Extensions to `@test` {#Extensions-to-@test}"'},"​")],-1)),t("details",E,[t("summary",null,[s[12]||(s[12]=t("a",{id:"LuxTestUtils.@test_softfail",href:"#LuxTestUtils.@test_softfail"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_softfail")],-1)),s[13]||(s[13]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @test_softfail expr

    Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

    If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

    source

    ',4))])])}const j=n(k,[["render",u]]);export{C as __pageData,j as default}; diff --git a/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.DpA6j4Xc.js b/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.DpA6j4Xc.js new file mode 100644 index 0000000000..4485b85ff6 --- /dev/null +++ b/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.DpA6j4Xc.js @@ -0,0 +1,12 @@ +import{_ as n,c as h,a2 as e,j as t,a as i,G as l,B as p,o as d}from"./chunks/framework.DjZDIZsN.js";const C=JSON.parse('{"title":"LuxTestUtils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Testing_Functionality/LuxTestUtils.md","filePath":"api/Testing_Functionality/LuxTestUtils.md","lastUpdated":null}'),k={name:"api/Testing_Functionality/LuxTestUtils.md"},r={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"};function u(y,s,f,b,x,F){const a=p("Badge");return d(),h("div",null,[s[15]||(s[15]=e('

    LuxTestUtils ​

    Warning

    This is a testing package. Hence, we don't use features like weak dependencies to reduce load times. It is recommended that you exclusively use this package for testing and not add a dependency to it in your main package Project.toml.

    Implements utilities for testing gradient correctness and dynamic dispatch of Lux.jl models.

    Testing using JET.jl ​

    ',4)),t("details",r,[t("summary",null,[s[0]||(s[0]=t("a",{id:"LuxTestUtils.@jet",href:"#LuxTestUtils.@jet"},[t("span",{class:"jlbinding"},"LuxTestUtils.@jet")],-1)),s[1]||(s[1]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[2]||(s[2]=e(`
    julia
    @jet f(args...) call_broken=false opt_broken=false

    Run JET tests on the function f with the arguments args.... If JET.jl fails to compile, then the macro will be a no-op.

    Keyword Arguments

    All additional arguments will be forwarded to JET.@test_call and JET.@test_opt.

    Tip

    Instead of specifying target_modules with every call, you can set global target modules using jet_target_modules!.

    julia
    using LuxTestUtils
+
+jet_target_modules!(["Lux", "LuxLib"]) # Expects Lux and LuxLib to be present in the module calling \`@jet\`

    Example

    julia
    julia> @jet sum([1, 2, 3]) target_modules=(Base, Core)
+Test Passed
+
+julia> @jet sum(1, 1) target_modules=(Base, Core) opt_broken=true call_broken=true
+Test Broken
+  Expression: #= REPL[21]:1 =# JET.@test_opt target_modules = (Base, Core) sum(1, 1)

    source

    `,9))]),t("details",o,[t("summary",null,[s[3]||(s[3]=t("a",{id:"LuxTestUtils.jet_target_modules!",href:"#LuxTestUtils.jet_target_modules!"},[t("span",{class:"jlbinding"},"LuxTestUtils.jet_target_modules!")],-1)),s[4]||(s[4]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e('
    julia
    jet_target_modules!(list::Vector{String}; force::Bool=false)

    This sets target_modules for all JET tests when using @jet.

    source

    ',3))]),s[16]||(s[16]=t("h2",{id:"Gradient-Correctness",tabindex:"-1"},[i("Gradient Correctness "),t("a",{class:"header-anchor",href:"#Gradient-Correctness","aria-label":'Permalink to "Gradient Correctness {#Gradient-Correctness}"'},"​")],-1)),t("details",g,[t("summary",null,[s[6]||(s[6]=t("a",{id:"LuxTestUtils.test_gradients",href:"#LuxTestUtils.test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.test_gradients")],-1)),s[7]||(s[7]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e(`
    julia
    test_gradients(f, args...; skip_backends=[], broken_backends=[], kwargs...)

    Test the gradients of f with respect to args using the specified backends.

    BackendADTypeCPUGPUNotes
    Zygote.jlAutoZygote()✔✔
    Tracker.jlAutoTracker()✔✔
    ReverseDiff.jlAutoReverseDiff()✔✖
    ForwardDiff.jlAutoForwardDiff()✔✖len ≤ 100
    FiniteDiff.jlAutoFiniteDiff()✔✖len ≤ 100
    Enzyme.jlAutoEnzyme()✔✖Only Reverse Mode

    Arguments

    Keyword Arguments

    Example

    julia
    julia> f(x, y, z) = x .+ sum(abs2, y.t) + sum(y.x.z)
+
+julia> x = (; t=rand(10), x=(z=[2.0],))
+
+julia> test_gradients(f, 1.0, x, nothing)

    source

    `,10))]),t("details",c,[t("summary",null,[s[9]||(s[9]=t("a",{id:"LuxTestUtils.@test_gradients",href:"#LuxTestUtils.@test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_gradients")],-1)),s[10]||(s[10]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[11]||(s[11]=e('
    julia
    @test_gradients(f, args...; kwargs...)

    See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

    source

    ',3))]),s[17]||(s[17]=t("h2",{id:"Extensions-to-@test",tabindex:"-1"},[i("Extensions to "),t("code",null,"@test"),i(),t("a",{class:"header-anchor",href:"#Extensions-to-@test","aria-label":'Permalink to "Extensions to `@test` {#Extensions-to-@test}"'},"​")],-1)),t("details",E,[t("summary",null,[s[12]||(s[12]=t("a",{id:"LuxTestUtils.@test_softfail",href:"#LuxTestUtils.@test_softfail"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_softfail")],-1)),s[13]||(s[13]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @test_softfail expr

    Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

    If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

    source

    ',4))])])}const j=n(k,[["render",u]]);export{C as __pageData,j as default}; diff --git a/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.DpA6j4Xc.lean.js b/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.DpA6j4Xc.lean.js new file mode 100644 index 0000000000..4485b85ff6 --- /dev/null +++ b/previews/PR1000/assets/api_Testing_Functionality_LuxTestUtils.md.DpA6j4Xc.lean.js @@ -0,0 +1,12 @@ +import{_ as n,c as h,a2 as e,j as t,a as i,G as l,B as p,o as d}from"./chunks/framework.DjZDIZsN.js";const C=JSON.parse('{"title":"LuxTestUtils","description":"","frontmatter":{},"headers":[],"relativePath":"api/Testing_Functionality/LuxTestUtils.md","filePath":"api/Testing_Functionality/LuxTestUtils.md","lastUpdated":null}'),k={name:"api/Testing_Functionality/LuxTestUtils.md"},r={class:"jldocstring custom-block"},o={class:"jldocstring custom-block"},g={class:"jldocstring custom-block"},c={class:"jldocstring custom-block"},E={class:"jldocstring custom-block"};function u(y,s,f,b,x,F){const a=p("Badge");return d(),h("div",null,[s[15]||(s[15]=e('

    LuxTestUtils ​

    Warning

    This is a testing package. Hence, we don't use features like weak dependencies to reduce load times. It is recommended that you exclusively use this package for testing and not add a dependency to it in your main package Project.toml.

    Implements utilities for testing gradient correctness and dynamic dispatch of Lux.jl models.

    Testing using JET.jl ​

    ',4)),t("details",r,[t("summary",null,[s[0]||(s[0]=t("a",{id:"LuxTestUtils.@jet",href:"#LuxTestUtils.@jet"},[t("span",{class:"jlbinding"},"LuxTestUtils.@jet")],-1)),s[1]||(s[1]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[2]||(s[2]=e(`
    julia
    @jet f(args...) call_broken=false opt_broken=false

    Run JET tests on the function f with the arguments args.... If JET.jl fails to compile, then the macro will be a no-op.

    Keyword Arguments

    All additional arguments will be forwarded to JET.@test_call and JET.@test_opt.

    Tip

    Instead of specifying target_modules with every call, you can set global target modules using jet_target_modules!.

    julia
    using LuxTestUtils
+
+jet_target_modules!(["Lux", "LuxLib"]) # Expects Lux and LuxLib to be present in the module calling \`@jet\`

    Example

    julia
    julia> @jet sum([1, 2, 3]) target_modules=(Base, Core)
+Test Passed
+
+julia> @jet sum(1, 1) target_modules=(Base, Core) opt_broken=true call_broken=true
+Test Broken
+  Expression: #= REPL[21]:1 =# JET.@test_opt target_modules = (Base, Core) sum(1, 1)

    source

    `,9))]),t("details",o,[t("summary",null,[s[3]||(s[3]=t("a",{id:"LuxTestUtils.jet_target_modules!",href:"#LuxTestUtils.jet_target_modules!"},[t("span",{class:"jlbinding"},"LuxTestUtils.jet_target_modules!")],-1)),s[4]||(s[4]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[5]||(s[5]=e('
    julia
    jet_target_modules!(list::Vector{String}; force::Bool=false)

    This sets target_modules for all JET tests when using @jet.

    source

    ',3))]),s[16]||(s[16]=t("h2",{id:"Gradient-Correctness",tabindex:"-1"},[i("Gradient Correctness "),t("a",{class:"header-anchor",href:"#Gradient-Correctness","aria-label":'Permalink to "Gradient Correctness {#Gradient-Correctness}"'},"​")],-1)),t("details",g,[t("summary",null,[s[6]||(s[6]=t("a",{id:"LuxTestUtils.test_gradients",href:"#LuxTestUtils.test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.test_gradients")],-1)),s[7]||(s[7]=i()),l(a,{type:"info",class:"jlObjectType jlFunction",text:"Function"})]),s[8]||(s[8]=e(`
    julia
    test_gradients(f, args...; skip_backends=[], broken_backends=[], kwargs...)

    Test the gradients of f with respect to args using the specified backends.

    BackendADTypeCPUGPUNotes
    Zygote.jlAutoZygote()✔✔
    Tracker.jlAutoTracker()✔✔
    ReverseDiff.jlAutoReverseDiff()✔✖
    ForwardDiff.jlAutoForwardDiff()✔✖len ≤ 100
    FiniteDiff.jlAutoFiniteDiff()✔✖len ≤ 100
    Enzyme.jlAutoEnzyme()✔✖Only Reverse Mode

    Arguments

    Keyword Arguments

    Example

    julia
    julia> f(x, y, z) = x .+ sum(abs2, y.t) + sum(y.x.z)
+
+julia> x = (; t=rand(10), x=(z=[2.0],))
+
+julia> test_gradients(f, 1.0, x, nothing)

    source

    `,10))]),t("details",c,[t("summary",null,[s[9]||(s[9]=t("a",{id:"LuxTestUtils.@test_gradients",href:"#LuxTestUtils.@test_gradients"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_gradients")],-1)),s[10]||(s[10]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[11]||(s[11]=e('
    julia
    @test_gradients(f, args...; kwargs...)

    See the documentation of test_gradients for more details. This macro provides correct line information for the failing tests.

    source

    ',3))]),s[17]||(s[17]=t("h2",{id:"Extensions-to-@test",tabindex:"-1"},[i("Extensions to "),t("code",null,"@test"),i(),t("a",{class:"header-anchor",href:"#Extensions-to-@test","aria-label":'Permalink to "Extensions to `@test` {#Extensions-to-@test}"'},"​")],-1)),t("details",E,[t("summary",null,[s[12]||(s[12]=t("a",{id:"LuxTestUtils.@test_softfail",href:"#LuxTestUtils.@test_softfail"},[t("span",{class:"jlbinding"},"LuxTestUtils.@test_softfail")],-1)),s[13]||(s[13]=i()),l(a,{type:"info",class:"jlObjectType jlMacro",text:"Macro"})]),s[14]||(s[14]=e('
    julia
    @test_softfail expr

    Evaluate expr and record a test result. If expr throws an exception, the test result will be recorded as an error. If expr returns a value, and it is not a boolean, the test result will be recorded as an error.

    If the test result is false then the test will be recorded as a broken test, else it will be recorded as a pass.

    source

    ',4))])])}const j=n(k,[["render",u]]);export{C as __pageData,j as default}; diff --git a/previews/PR1000/assets/app.BrJPXIEN.js b/previews/PR1000/assets/app.Bp2YcZtd.js similarity index 95% rename from previews/PR1000/assets/app.BrJPXIEN.js rename to previews/PR1000/assets/app.Bp2YcZtd.js index 9e1200aa68..ec4eee4120 100644 --- a/previews/PR1000/assets/app.BrJPXIEN.js +++ b/previews/PR1000/assets/app.Bp2YcZtd.js @@ -1 +1 @@ -import{R as p}from"./chunks/theme.DRSgyssM.js";import{R as o,a6 as u,a7 as c,a8 as l,a9 as f,aa as d,ab as m,ac as h,ad as g,ae as A,af as v,d as P,u as R,v as w,s as y,ag as C,ah as b,ai as E,a5 as S}from"./chunks/framework.DjZDIZsN.js";function i(e){if(e.extends){const a=i(e.extends);return{...a,...e,async enhanceApp(t){a.enhanceApp&&await a.enhanceApp(t),e.enhanceApp&&await e.enhanceApp(t)}}}return e}const s=i(p),T=P({name:"VitePressApp",setup(){const{site:e,lang:a,dir:t}=R();return w(()=>{y(()=>{document.documentElement.lang=a.value,document.documentElement.dir=t.value})}),e.value.router.prefetchLinks&&C(),b(),E(),s.setup&&s.setup(),()=>S(s.Layout)}});async function D(){globalThis.__VITEPRESS__=!0;const e=j(),a=_();a.provide(c,e);const t=l(e.route);return a.provide(f,t),a.component("Content",d),a.component("ClientOnly",m),Object.defineProperties(a.config.globalProperties,{$frontmatter:{get(){return t.frontmatter.value}},$params:{get(){return t.page.value.params}}}),s.enhanceApp&&await s.enhanceApp({app:a,router:e,siteData:h}),{app:a,router:e,data:t}}function _(){return g(T)}function j(){let e=o,a;return A(t=>{let n=v(t),r=null;return n&&(e&&(a=n),(e||a===n)&&(n=n.replace(/\.js$/,".lean.js")),r=import(n)),o&&(e=!1),r},s.NotFound)}o&&D().then(({app:e,router:a,data:t})=>{a.go().then(()=>{u(a.route,t.site),e.mount("#app")})});export{D as createApp}; +import{R as p}from"./chunks/theme.BD8xVYwz.js";import{R as o,a6 as u,a7 as c,a8 as l,a9 as f,aa as d,ab as m,ac as h,ad as g,ae as A,af as v,d as P,u as R,v as w,s as y,ag as C,ah as b,ai as E,a5 as S}from"./chunks/framework.DjZDIZsN.js";function i(e){if(e.extends){const a=i(e.extends);return{...a,...e,async enhanceApp(t){a.enhanceApp&&await a.enhanceApp(t),e.enhanceApp&&await e.enhanceApp(t)}}}return e}const s=i(p),T=P({name:"VitePressApp",setup(){const{site:e,lang:a,dir:t}=R();return w(()=>{y(()=>{document.documentElement.lang=a.value,document.documentElement.dir=t.value})}),e.value.router.prefetchLinks&&C(),b(),E(),s.setup&&s.setup(),()=>S(s.Layout)}});async function D(){globalThis.__VITEPRESS__=!0;const e=j(),a=_();a.provide(c,e);const t=l(e.route);return a.provide(f,t),a.component("Content",d),a.component("ClientOnly",m),Object.defineProperties(a.config.globalProperties,{$frontmatter:{get(){return t.frontmatter.value}},$params:{get(){return t.page.value.params}}}),s.enhanceApp&&await s.enhanceApp({app:a,router:e,siteData:h}),{app:a,router:e,data:t}}function _(){return g(T)}function j(){let e=o,a;return A(t=>{let n=v(t),r=null;return n&&(e&&(a=n),(e||a===n)&&(n=n.replace(/\.js$/,".lean.js")),r=import(n)),o&&(e=!1),r},s.NotFound)}o&&D().then(({app:e,router:a,data:t})=>{a.go().then(()=>{u(a.route,t.site),e.mount("#app")})});export{D as createApp}; diff --git a/previews/PR1000/assets/chunks/@localSearchIndexroot.1OabY8x9.js b/previews/PR1000/assets/chunks/@localSearchIndexroot.1OabY8x9.js new file mode 100644 index 0000000000..f04f4ee093 --- /dev/null +++ b/previews/PR1000/assets/chunks/@localSearchIndexroot.1OabY8x9.js @@ -0,0 +1 @@ +const e='{"documentCount":261,"nextId":261,"documentIds":{"0":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#MLDataDevices-API","1":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#preferences","2":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#Data-Transfer","3":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#miscellaneous","4":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#Multi-GPU-Support","5":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#iteration","6":"/previews/PR1000/api/Building_Blocks/WeightInitializers#WeightInitializers-API","7":"/previews/PR1000/api/Building_Blocks/WeightInitializers#Supported-RNG-Types-WeightInit","8":"/previews/PR1000/api/Building_Blocks/WeightInitializers#API-Reference","9":"/previews/PR1000/api/Building_Blocks/WeightInitializers#Main-Functions","10":"/previews/PR1000/api/Building_Blocks/WeightInitializers#Other-Convenience-Functions","11":"/previews/PR1000/api/Building_Blocks/LuxCore#luxcore","12":"/previews/PR1000/api/Building_Blocks/LuxCore#Abstract-Types","13":"/previews/PR1000/api/Building_Blocks/LuxCore#general","14":"/previews/PR1000/api/Building_Blocks/LuxCore#parameters","15":"/previews/PR1000/api/Building_Blocks/LuxCore#states","16":"/previews/PR1000/api/Building_Blocks/LuxCore#Layer-size","17":"/previews/PR1000/api/Building_Blocks/LuxLib#LuxLib-API","18":"/previews/PR1000/api/Building_Blocks/LuxLib#Apply-Activation","19":"/previews/PR1000/api/Building_Blocks/LuxLib#Batched-Operations","20":"/previews/PR1000/api/Building_Blocks/LuxLib#Bias-Activation","21":"/previews/PR1000/api/Building_Blocks/LuxLib#Convolutional-Layers","22":"/previews/PR1000/api/Building_Blocks/LuxLib#dropout","23":"/previews/PR1000/api/Building_Blocks/LuxLib#Fully-Connected-Layers","24":"/previews/PR1000/api/Building_Blocks/LuxLib#normalization","25":"/previews/PR1000/api/Building_Blocks/LuxLib#Helper-Functions","26":"/previews/PR1000/api/Lux/autodiff#autodiff-lux-helpers","27":"/previews/PR1000/api/Lux/autodiff#JVP-and-VJP-Wrappers","28":"/previews/PR1000/api/Lux/autodiff#Batched-AD","29":"/previews/PR1000/api/Lux/autodiff#Nested-2nd-Order-AD","30":"/previews/PR1000/api/Lux/contrib#Experimental-Features","31":"/previews/PR1000/api/Lux/contrib#Parameter-Freezing","32":"/previews/PR1000/api/Lux/contrib#Map-over-Layer","33":"/previews/PR1000/api/Lux/contrib#Debugging-Functionality","34":"/previews/PR1000/api/Lux/contrib#Tied-Parameters","35":"/previews/PR1000/api/Lux/distributed_utils#Distributed-Utils","36":"/previews/PR1000/api/Lux/distributed_utils#communication-backends","37":"/previews/PR1000/api/Lux/distributed_utils#initialization","38":"/previews/PR1000/api/Lux/distributed_utils#Helper-Functions","39":"/previews/PR1000/api/Lux/distributed_utils#Communication-Primitives","40":"/previews/PR1000/api/Lux/distributed_utils#Optimizers.jl-Integration","41":"/previews/PR1000/api/Lux/distributed_utils#MLUtils.jl-Integration","42":"/previews/PR1000/api/Lux/interop#Interoperability-between-Lux-and-other-packages","43":"/previews/PR1000/api/Lux/interop#Switching-from-older-frameworks","44":"/previews/PR1000/api/Lux/interop#flux-to-lux-migrate-api","45":"/previews/PR1000/api/Lux/interop#Using-a-different-backend-for-Lux","46":"/previews/PR1000/api/Lux/interop#Lux-Models-to-Simple-Chains","47":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#luxtestutils","48":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#Testing-using-JET.jl","49":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#Gradient-Correctness","50":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#Extensions-to-@test","51":"/previews/PR1000/api/Lux/utilities#utilities","52":"/previews/PR1000/api/Lux/utilities#Training-API","53":"/previews/PR1000/api/Lux/utilities#Loss-Functions","54":"/previews/PR1000/api/Lux/utilities#LuxOps-Module","55":"/previews/PR1000/api/Lux/utilities#Recursive-Operations","56":"/previews/PR1000/api/Lux/utilities#Updating-Floating-Point-Precision","57":"/previews/PR1000/api/Lux/utilities#Element-Type-Matching","58":"/previews/PR1000/api/Lux/utilities#Stateful-Layer","59":"/previews/PR1000/api/Lux/utilities#Compact-Layer","60":"/previews/PR1000/api/Lux/utilities#miscellaneous","61":"/previews/PR1000/#How-to-Install-Lux.jl?","62":"/previews/PR1000/#Want-GPU-Support?","63":"/previews/PR1000/#Want-XLA-Support?","64":"/previews/PR1000/api/Lux/layers#Built-In-Layers","65":"/previews/PR1000/api/Lux/layers#containers","66":"/previews/PR1000/api/Lux/layers#Convolutional-Layers","67":"/previews/PR1000/api/Lux/layers#Dropout-Layers","68":"/previews/PR1000/api/Lux/layers#Pooling-Layers","69":"/previews/PR1000/api/Lux/layers#Recurrent-Layers","70":"/previews/PR1000/api/Lux/layers#Linear-Layers","71":"/previews/PR1000/api/Lux/layers#Misc.-Helper-Layers","72":"/previews/PR1000/api/Lux/layers#Normalization-Layers","73":"/previews/PR1000/api/Lux/layers#upsampling","74":"/previews/PR1000/introduction/citation#citation","75":"/previews/PR1000/introduction/overview#Why-we-wrote-Lux?","76":"/previews/PR1000/introduction/overview#Design-Principles","77":"/previews/PR1000/introduction/overview#Why-use-Lux-over-Flux?","78":"/previews/PR1000/introduction/#getting-started","79":"/previews/PR1000/introduction/#installation","80":"/previews/PR1000/introduction/#quickstart","81":"/previews/PR1000/introduction/#Defining-Custom-Layers","82":"/previews/PR1000/introduction/#Additional-Packages","83":"/previews/PR1000/introduction/#GPU-Support","84":"/previews/PR1000/introduction/resources#Resources-to-Get-Started","85":"/previews/PR1000/manual/autodiff#autodiff-lux","86":"/previews/PR1000/manual/autodiff#overview","87":"/previews/PR1000/manual/autodiff#autodiff-recommendations","88":"/previews/PR1000/manual/autodiff#Support-Class","89":"/previews/PR1000/manual/autodiff#footnotes","90":"/previews/PR1000/introduction/updating_to_v1#updating-to-v1","91":"/previews/PR1000/introduction/updating_to_v1#LuxLib.jl","92":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes","93":"/previews/PR1000/introduction/updating_to_v1#New-Major-Features","94":"/previews/PR1000/introduction/updating_to_v1#LuxCore.jl","95":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-2","96":"/previews/PR1000/introduction/updating_to_v1#New-Major-Features-2","97":"/previews/PR1000/introduction/updating_to_v1#WeightInitializers.jl","98":"/previews/PR1000/introduction/updating_to_v1#MLDataDevices.jl","99":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-3","100":"/previews/PR1000/introduction/updating_to_v1#New-Major-Features-3","101":"/previews/PR1000/introduction/updating_to_v1#Lux.jl","102":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-(Removed-Functionality)","103":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-(Moved-Functionality)","104":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-(Changes-in-Defaults)","105":"/previews/PR1000/introduction/updating_to_v1#New-Features","106":"/previews/PR1000/manual/compiling_lux_models#reactant-compilation","107":"/previews/PR1000/manual/compiling_lux_models#compile_lux_model_trainstate","108":"/previews/PR1000/manual/debugging#debug-lux-layers","109":"/previews/PR1000/manual/debugging#Incorrect-Model-Specification:-Dimension-Mismatch-Problems","110":"/previews/PR1000/manual/debugging#Tracking-down-NaNs","111":"/previews/PR1000/manual/debugging#conclusion","112":"/previews/PR1000/manual/dispatch_custom_input#Dispatching-on-Custom-Input-Types","113":"/previews/PR1000/manual/dispatch_custom_input#Which-function-should-participate-in-dispatch?","114":"/previews/PR1000/manual/dispatch_custom_input#Concrete-Example","115":"/previews/PR1000/manual/dispatch_custom_input#Time-Dependent-Chain-Implementation","116":"/previews/PR1000/manual/dispatch_custom_input#Running-the-TDChain","117":"/previews/PR1000/manual/dispatch_custom_input#Writing-the-Correct-Dispatch-Rules","118":"/previews/PR1000/manual/dispatch_custom_input#Using-the-Same-Input-for-Non-TD-Models","119":"/previews/PR1000/manual/freezing_model_parameters#freezing-model-parameters","120":"/previews/PR1000/manual/freezing_model_parameters#Freezing-Layers-of-a-Particular-Kind","121":"/previews/PR1000/manual/freezing_model_parameters#Freezing-by-Layer-Name","122":"/previews/PR1000/manual/freezing_model_parameters#Freezing-Part-of-the-Parameters","123":"/previews/PR1000/manual/freezing_model_parameters#Freezing-Part-of-a-Chain","124":"/previews/PR1000/manual/gpu_management#GPU-Management","125":"/previews/PR1000/manual/gpu_management#Automatic-Backend-Management-(Recommended-Approach)","126":"/previews/PR1000/manual/gpu_management#Manual-Backend-Management","127":"/previews/PR1000/manual/interface#lux-interface","128":"/previews/PR1000/manual/interface#Layer-Interface","129":"/previews/PR1000/manual/interface#Singular-Layer","130":"/previews/PR1000/manual/interface#Container-Layer","131":"/previews/PR1000/manual/interface#Parameter-Interface","132":"/previews/PR1000/manual/interface#State-Interface","133":"/previews/PR1000/manual/nested_autodiff#nested_autodiff","134":"/previews/PR1000/manual/nested_autodiff#Loss-Function-containing-Jacobian-Computation","135":"/previews/PR1000/manual/nested_autodiff#Using-Batched-Jacobian-for-Multiple-Inputs","136":"/previews/PR1000/manual/nested_autodiff#Loss-Function-contains-Gradient-Computation","137":"/previews/PR1000/manual/nested_autodiff#Loss-Function-computing-the-Jacobian-of-the-Parameters","138":"/previews/PR1000/manual/nested_autodiff#Hutchinson-Trace-Estimation","139":"/previews/PR1000/manual/nested_autodiff#Computing-using-the-Vector-Jacobian-Product","140":"/previews/PR1000/manual/nested_autodiff#Computing-using-the-Jacobian-Vector-Product","141":"/previews/PR1000/manual/nested_autodiff#Computing-using-the-Full-Jacobian","142":"/previews/PR1000/manual/distributed_utils#Distributed-Data-Parallel-Training","143":"/previews/PR1000/manual/distributed_utils#Guide-to-Integrating-DistributedUtils-into-your-code","144":"/previews/PR1000/manual/distributed_utils#Migration-Guide-from-FluxMPI.jl","145":"/previews/PR1000/manual/distributed_utils#Removed-Functionality","146":"/previews/PR1000/manual/distributed_utils#Key-Differences","147":"/previews/PR1000/manual/distributed_utils#Known-Shortcomings","148":"/previews/PR1000/manual/performance_pitfalls#Performance-Pitfalls-and-How-to-Catch-Them","149":"/previews/PR1000/manual/performance_pitfalls#Spurious-Type-Promotion","150":"/previews/PR1000/manual/performance_pitfalls#Scalar-Indexing-on-GPU-Arrays","151":"/previews/PR1000/manual/performance_pitfalls#Type-Instabilities","152":"/previews/PR1000/manual/performance_pitfalls#Faster-Primitives","153":"/previews/PR1000/manual/performance_pitfalls#Optional-Dependencies-for-Performance","154":"/previews/PR1000/manual/performance_pitfalls#Data-Loading-and-Device-Transfer","155":"/previews/PR1000/manual/preferences#Preferences-for-Lux.jl","156":"/previews/PR1000/manual/preferences#Nested-Automatic-Differentiation","157":"/previews/PR1000/manual/preferences#gpu-aware-mpi-preferences","158":"/previews/PR1000/manual/preferences#GPU-Backend-Selection","159":"/previews/PR1000/manual/preferences#automatic-eltypes-preference","160":"/previews/PR1000/manual/preferences#dispatch-doctor-preference","161":"/previews/PR1000/manual/preferences#disable_loop_vectorization","162":"/previews/PR1000/manual/weight_initializers#Initializing-Weights","163":"/previews/PR1000/manual/weight_initializers#Quick-examples","164":"/previews/PR1000/manual/nn_inside_gpu_kernels#Neural-Networks-Inside-GPU-Kernels","165":"/previews/PR1000/tutorials/beginner/1_Basics#Julia-and-Lux-for-the-Uninitiated","166":"/previews/PR1000/tutorials/beginner/1_Basics#arrays","167":"/previews/PR1000/tutorials/beginner/1_Basics#CUDA-Arrays","168":"/previews/PR1000/tutorials/beginner/1_Basics#im-mutability","169":"/previews/PR1000/tutorials/beginner/1_Basics#Managing-Randomness","170":"/previews/PR1000/tutorials/beginner/1_Basics#Automatic-Differentiation","171":"/previews/PR1000/tutorials/beginner/1_Basics#gradients","172":"/previews/PR1000/tutorials/beginner/1_Basics#Jacobian-Vector-Product","173":"/previews/PR1000/tutorials/beginner/1_Basics#Vector-Jacobian-Product","174":"/previews/PR1000/tutorials/beginner/1_Basics#Linear-Regression","175":"/previews/PR1000/tutorials/beginner/1_Basics#appendix","176":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Training-a-Simple-LSTM","177":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Package-Imports","178":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#dataset","179":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Creating-a-Classifier","180":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Using-the-@compact-API","181":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Defining-Accuracy,-Loss-and-Optimiser","182":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Training-the-Model","183":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Saving-the-Model","184":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#appendix","185":"/previews/PR1000/tutorials/beginner/4_SimpleChains#MNIST-Classification-with-SimpleChains","186":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Package-Imports","187":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Loading-MNIST","188":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Define-the-Model","189":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Helper-Functions","190":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Define-the-Training-Loop","191":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Finally-Training-the-Model","192":"/previews/PR1000/tutorials/beginner/4_SimpleChains#appendix","193":"/previews/PR1000/manual/migrate_from_flux#migrate-from-flux","194":"/previews/PR1000/manual/migrate_from_flux#Implementing-Custom-Layers","195":"/previews/PR1000/manual/migrate_from_flux#Certain-Important-Implementation-Details","196":"/previews/PR1000/manual/migrate_from_flux#Training/Inference-Mode","197":"/previews/PR1000/manual/migrate_from_flux#Can-we-still-use-Flux-Layers?","198":"/previews/PR1000/tutorials/#tutorials","199":"/previews/PR1000/tutorials/#beginner-tutorials","200":"/previews/PR1000/tutorials/#intermediate-tutorials","201":"/previews/PR1000/tutorials/#advanced-tutorials","202":"/previews/PR1000/tutorials/#larger-models","203":"/previews/PR1000/tutorials/#selected-3rd-party-tutorials","204":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Optimization-Lux-Tutorial","205":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Imports-packages","206":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Generate-some-training-data","207":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Define-the-DataLoader","208":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Training-the-model","209":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Plotting-the-results","210":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#appendix","211":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Fitting-a-Polynomial-using-MLP","212":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Package-Imports","213":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#dataset","214":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Neural-Network","215":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#optimizer","216":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Loss-Function","217":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#training","218":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#appendix","219":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Training-a-HyperNetwork-on-MNIST-and-FashionMNIST","220":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Package-Imports","221":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Loading-Datasets","222":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Implement-a-HyperNet-Layer","223":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Create-and-Initialize-the-HyperNet","224":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Define-Utility-Functions","225":"/previews/PR1000/tutorials/intermediate/3_HyperNet#training","226":"/previews/PR1000/tutorials/intermediate/3_HyperNet#appendix","227":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#MNIST-Classification-using-Neural-ODEs","228":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Package-Imports","229":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Loading-MNIST","230":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Define-the-Neural-ODE-Layer","231":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Create-and-Initialize-the-Neural-ODE-Layer","232":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Define-Utility-Functions","233":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#training","234":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Alternate-Implementation-using-Stateful-Layer","235":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Train-the-new-Stateful-Neural-ODE","236":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Type-Stability","237":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#appendix","238":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Training-a-PINN-on-2D-PDE","239":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Package-Imports","240":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Problem-Definition","241":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Define-the-Neural-Networks","242":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Define-the-Loss-Functions","243":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Generate-the-Data","244":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#training","245":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Visualizing-the-Results","246":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#appendix","247":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Bayesian-Neural-Network","248":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Generating-data","249":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Building-the-Neural-Network","250":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Prediction-Visualization","251":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#appendix","252":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Training-a-Neural-ODE-to-Model-Gravitational-Waveforms","253":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Package-Imports","254":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Define-some-Utility-Functions","255":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Simulating-the-True-Model","256":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Defiing-a-Neural-Network-Model","257":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Setting-Up-for-Training-the-Neural-Network","258":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Training-the-Neural-Network","259":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Visualizing-the-Results","260":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#appendix"},"fieldIds":{"title":0,"titles":1,"text":2},"fieldLength":{"0":[1,1,14],"1":[1,1,46],"2":[2,1,156],"3":[1,1,194],"4":[3,1,93],"5":[1,1,114],"6":[1,1,16],"7":[3,1,27],"8":[2,1,1],"9":[2,3,342],"10":[3,3,92],"11":[1,1,38],"12":[2,1,160],"13":[1,1,214],"14":[1,1,19],"15":[1,1,62],"16":[2,1,77],"17":[1,1,5],"18":[2,1,100],"19":[2,1,44],"20":[2,1,68],"21":[2,1,128],"22":[1,1,160],"23":[3,1,132],"24":[1,1,232],"25":[2,1,91],"26":[3,1,1],"27":[4,3,72],"28":[2,3,86],"29":[4,3,13],"30":[2,1,54],"31":[2,2,131],"32":[3,2,124],"33":[2,2,140],"34":[2,2,118],"35":[2,1,11],"36":[1,2,29],"37":[1,2,81],"38":[2,2,18],"39":[2,2,58],"40":[3,2,31],"41":[3,2,35],"42":[6,1,1],"43":[4,6,1],"44":[4,10,172],"45":[6,6,1],"46":[5,11,188],"47":[1,1,48],"48":[4,1,91],"49":[2,1,127],"50":[3,1,33],"51":[1,1,1],"52":[2,1,228],"53":[2,1,449],"54":[2,1,134],"55":[2,1,164],"56":[4,1,45],"57":[3,1,108],"58":[2,1,112],"59":[2,1,358],"60":[1,1,54],"61":[6,1,47],"62":[4,1,34],"63":[4,1,29],"64":[3,1,1],"65":[1,3,292],"66":[2,3,257],"67":[2,3,115],"68":[2,3,168],"69":[2,3,361],"70":[2,3,191],"71":[3,3,243],"72":[2,3,294],"73":[1,3,165],"74":[1,1,67],"75":[5,1,63],"76":[2,5,58],"77":[6,5,218],"78":[2,1,1],"79":[1,2,51],"80":[1,2,402],"81":[3,2,181],"82":[2,2,36],"83":[2,2,17],"84":[4,1,48],"85":[2,1,35],"86":[1,2,34],"87":[1,2,50],"88":[2,2,65],"89":[1,2,67],"90":[4,1,41],"91":[2,4,1],"92":[2,6,27],"93":[3,6,22],"94":[2,4,1],"95":[2,6,119],"96":[3,6,19],"97":[2,4,30],"98":[2,4,48],"99":[2,6,22],"100":[3,6,30],"101":[2,4,1],"102":[5,5,117],"103":[5,5,54],"104":[5,5,68],"105":[2,5,63],"106":[6,1,493],"107":[4,6,156],"108":[3,1,76],"109":[6,3,156],"110":[3,3,190],"111":[1,3,51],"112":[5,1,1],"113":[7,5,36],"114":[2,5,40],"115":[4,7,72],"116":[3,7,70],"117":[5,7,47],"118":[8,7,46],"119":[3,1,27],"120":[6,3,123],"121":[4,3,53],"122":[5,3,43],"123":[5,3,49],"124":[2,1,71],"125":[6,2,89],"126":[3,2,44],"127":[2,1,80],"128":[2,2,1],"129":[2,3,239],"130":[2,3,158],"131":[2,2,151],"132":[2,2,47],"133":[3,1,169],"134":[5,3,262],"135":[6,8,155],"136":[5,3,219],"137":[7,3,216],"138":[3,3,88],"139":[6,6,46],"140":[6,6,31],"141":[5,6,113],"142":[4,1,27],"143":[7,4,122],"144":[5,4,42],"145":[2,9,28],"146":[2,9,79],"147":[2,4,52],"148":[7,1,18],"149":[3,7,149],"150":[5,7,32],"151":[2,7,39],"152":[2,7,43],"153":[4,7,39],"154":[5,7,75],"155":[4,1,49],"156":[3,4,21],"157":[4,4,40],"158":[3,4,33],"159":[3,4,23],"160":[2,4,61],"161":[4,4,58],"162":[2,1,133],"163":[2,2,64],"164":[5,1,286],"165":[6,1,77],"166":[1,6,286],"167":[2,7,86],"168":[3,6,110],"169":[2,6,130],"170":[2,6,96],"171":[1,8,72],"172":[3,8,106],"173":[3,8,30],"174":[2,6,253],"175":[1,6,150],"176":[4,1,38],"177":[2,4,12],"178":[1,4,95],"179":[3,4,160],"180":[4,4,74],"181":[5,4,56],"182":[3,4,530],"183":[3,4,48],"184":[1,4,150],"185":[4,1,33],"186":[2,4,15],"187":[2,4,60],"188":[3,4,64],"189":[2,4,38],"190":[4,4,73],"191":[4,4,127],"192":[1,4,110],"193":[5,1,67],"194":[3,5,164],"195":[4,5,1],"196":[3,9,49],"197":[7,5,34],"198":[1,1,1],"199":[2,1,1],"200":[2,1,1],"201":[2,1,1],"202":[2,1,41],"203":[4,1,65],"204":[6,1,78],"205":[2,6,29],"206":[4,6,58],"207":[3,6,70],"208":[3,6,242],"209":[3,6,47],"210":[1,6,150],"211":[5,1,16],"212":[2,5,12],"213":[1,5,274],"214":[2,5,37],"215":[1,5,18],"216":[2,5,112],"217":[1,5,150],"218":[1,5,150],"219":[7,1,1],"220":[2,7,18],"221":[2,7,41],"222":[4,7,76],"223":[5,7,42],"224":[3,7,39],"225":[1,7,268],"226":[1,7,150],"227":[5,1,24],"228":[2,5,23],"229":[2,5,64],"230":[5,5,139],"231":[7,5,60],"232":[3,5,37],"233":[1,5,238],"234":[5,5,60],"235":[6,5,70],"236":[2,5,179],"237":[1,5,150],"238":[6,1,45],"239":[2,6,31],"240":[2,6,26],"241":[4,6,55],"242":[4,6,71],"243":[3,6,86],"244":[1,6,1041],"245":[3,6,73],"246":[1,6,150],"247":[3,1,72],"248":[2,3,105],"249":[4,3,548],"250":[2,3,174],"251":[1,3,103],"252":[8,1,21],"253":[2,8,13],"254":[4,8,202],"255":[4,8,124],"256":[5,8,1354],"257":[7,8,73],"258":[4,8,1250],"259":[3,8,83],"260":[1,8,103]},"averageFieldLength":[2.9540229885057467,3.609195402298851,107.4521072796935],"storedFields":{"0":{"title":"MLDataDevices","titles":[]},"1":{"title":"Preferences","titles":["MLDataDevices"]},"2":{"title":"Data Transfer","titles":["MLDataDevices"]},"3":{"title":"Miscellaneous","titles":["MLDataDevices"]},"4":{"title":"Multi-GPU Support","titles":["MLDataDevices"]},"5":{"title":"Iteration","titles":["MLDataDevices"]},"6":{"title":"WeightInitializers","titles":[]},"7":{"title":"Supported RNG Types","titles":["WeightInitializers"]},"8":{"title":"API Reference","titles":["WeightInitializers"]},"9":{"title":"Main Functions","titles":["WeightInitializers","API Reference"]},"10":{"title":"Other Convenience Functions","titles":["WeightInitializers","API Reference"]},"11":{"title":"LuxCore","titles":[]},"12":{"title":"Abstract Types","titles":["LuxCore"]},"13":{"title":"General","titles":["LuxCore"]},"14":{"title":"Parameters","titles":["LuxCore"]},"15":{"title":"States","titles":["LuxCore"]},"16":{"title":"Layer size","titles":["LuxCore"]},"17":{"title":"LuxLib","titles":[]},"18":{"title":"Apply Activation","titles":["LuxLib"]},"19":{"title":"Batched Operations","titles":["LuxLib"]},"20":{"title":"Bias Activation","titles":["LuxLib"]},"21":{"title":"Convolutional Layers","titles":["LuxLib"]},"22":{"title":"Dropout","titles":["LuxLib"]},"23":{"title":"Fully Connected Layers","titles":["LuxLib"]},"24":{"title":"Normalization","titles":["LuxLib"]},"25":{"title":"Helper Functions","titles":["LuxLib"]},"26":{"title":"Automatic Differentiation Helpers","titles":[]},"27":{"title":"JVP & VJP Wrappers","titles":["Automatic Differentiation Helpers"]},"28":{"title":"Batched AD","titles":["Automatic Differentiation Helpers"]},"29":{"title":"Nested 2nd Order AD","titles":["Automatic Differentiation Helpers"]},"30":{"title":"Experimental Features","titles":[]},"31":{"title":"Parameter Freezing","titles":["Experimental Features"]},"32":{"title":"Map over Layer","titles":["Experimental Features"]},"33":{"title":"Debugging Functionality","titles":["Experimental Features"]},"34":{"title":"Tied Parameters","titles":["Experimental Features"]},"35":{"title":"Distributed Utils","titles":[]},"36":{"title":"Backends","titles":["Distributed Utils"]},"37":{"title":"Initialization","titles":["Distributed Utils"]},"38":{"title":"Helper Functions","titles":["Distributed Utils"]},"39":{"title":"Communication Primitives","titles":["Distributed Utils"]},"40":{"title":"Optimizers.jl Integration","titles":["Distributed Utils"]},"41":{"title":"MLUtils.jl Integration","titles":["Distributed Utils"]},"42":{"title":"Interoperability between Lux and other packages","titles":[]},"43":{"title":"Switching from older frameworks","titles":["Interoperability between Lux and other packages"]},"44":{"title":"Flux Models to Lux Models","titles":["Interoperability between Lux and other packages","Switching from older frameworks"]},"45":{"title":"Using a different backend for Lux","titles":["Interoperability between Lux and other packages"]},"46":{"title":"Lux Models to Simple Chains","titles":["Interoperability between Lux and other packages","Using a different backend for Lux"]},"47":{"title":"LuxTestUtils","titles":[]},"48":{"title":"Testing using JET.jl","titles":["LuxTestUtils"]},"49":{"title":"Gradient Correctness","titles":["LuxTestUtils"]},"50":{"title":"Extensions to @test","titles":["LuxTestUtils"]},"51":{"title":"Utilities","titles":[]},"52":{"title":"Training API","titles":["Utilities"]},"53":{"title":"Loss Functions","titles":["Utilities"]},"54":{"title":"LuxOps Module","titles":["Utilities"]},"55":{"title":"Recursive Operations","titles":["Utilities"]},"56":{"title":"Updating Floating Point Precision","titles":["Utilities"]},"57":{"title":"Element Type Matching","titles":["Utilities"]},"58":{"title":"Stateful Layer","titles":["Utilities"]},"59":{"title":"Compact Layer","titles":["Utilities"]},"60":{"title":"Miscellaneous","titles":["Utilities"]},"61":{"title":"How to Install Lux.jl?","titles":[]},"62":{"title":"Want GPU Support?","titles":[]},"63":{"title":"Want XLA Support?","titles":[]},"64":{"title":"Built-In Layers","titles":[]},"65":{"title":"Containers","titles":["Built-In Layers"]},"66":{"title":"Convolutional Layers","titles":["Built-In Layers"]},"67":{"title":"Dropout Layers","titles":["Built-In Layers"]},"68":{"title":"Pooling Layers","titles":["Built-In Layers"]},"69":{"title":"Recurrent Layers","titles":["Built-In Layers"]},"70":{"title":"Linear Layers","titles":["Built-In Layers"]},"71":{"title":"Misc. Helper Layers","titles":["Built-In Layers"]},"72":{"title":"Normalization Layers","titles":["Built-In Layers"]},"73":{"title":"Upsampling","titles":["Built-In Layers"]},"74":{"title":"Citation","titles":[]},"75":{"title":"Why we wrote Lux?","titles":[]},"76":{"title":"Design Principles","titles":["Why we wrote Lux?"]},"77":{"title":"Why use Lux over Flux?","titles":["Why we wrote Lux?"]},"78":{"title":"Getting Started","titles":[]},"79":{"title":"Installation","titles":["Getting Started"]},"80":{"title":"Quickstart","titles":["Getting Started"]},"81":{"title":"Defining Custom Layers","titles":["Getting Started"]},"82":{"title":"Additional Packages","titles":["Getting Started"]},"83":{"title":"GPU Support","titles":["Getting Started"]},"84":{"title":"Resources to Get Started","titles":[]},"85":{"title":"Automatic Differentiation","titles":[]},"86":{"title":"Overview","titles":["Automatic Differentiation"]},"87":{"title":"Recommendations","titles":["Automatic Differentiation"]},"88":{"title":"Support Class","titles":["Automatic Differentiation"]},"89":{"title":"Footnotes","titles":["Automatic Differentiation"]},"90":{"title":"Updating to Lux v1","titles":[]},"91":{"title":"LuxLib.jl","titles":["Updating to Lux v1"]},"92":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxLib.jl"]},"93":{"title":"New Major Features","titles":["Updating to Lux v1","LuxLib.jl"]},"94":{"title":"LuxCore.jl","titles":["Updating to Lux v1"]},"95":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxCore.jl"]},"96":{"title":"New Major Features","titles":["Updating to Lux v1","LuxCore.jl"]},"97":{"title":"WeightInitializers.jl","titles":["Updating to Lux v1"]},"98":{"title":"MLDataDevices.jl","titles":["Updating to Lux v1"]},"99":{"title":"Breaking Changes","titles":["Updating to Lux v1","MLDataDevices.jl"]},"100":{"title":"New Major Features","titles":["Updating to Lux v1","MLDataDevices.jl"]},"101":{"title":"Lux.jl","titles":["Updating to Lux v1"]},"102":{"title":"Breaking Changes (Removed Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"103":{"title":"Breaking Changes (Moved Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"104":{"title":"Breaking Changes (Changes in Defaults)","titles":["Updating to Lux v1","Lux.jl"]},"105":{"title":"New Features","titles":["Updating to Lux v1","Lux.jl"]},"106":{"title":"Compiling Lux Models using Reactant.jl","titles":[]},"107":{"title":"Using the TrainState API","titles":["Compiling Lux Models using Reactant.jl"]},"108":{"title":"Debugging Lux Models","titles":[]},"109":{"title":"Incorrect Model Specification: Dimension Mismatch Problems","titles":["Debugging Lux Models"]},"110":{"title":"Tracking down NaNs","titles":["Debugging Lux Models"]},"111":{"title":"Conclusion","titles":["Debugging Lux Models"]},"112":{"title":"Dispatching on Custom Input Types","titles":[]},"113":{"title":"Which function should participate in dispatch?","titles":["Dispatching on Custom Input Types"]},"114":{"title":"Concrete Example","titles":["Dispatching on Custom Input Types"]},"115":{"title":"Time-Dependent Chain Implementation","titles":["Dispatching on Custom Input Types","Concrete Example"]},"116":{"title":"Running the TDChain","titles":["Dispatching on Custom Input Types","Concrete Example"]},"117":{"title":"Writing the Correct Dispatch Rules","titles":["Dispatching on Custom Input Types","Concrete Example"]},"118":{"title":"Using the Same Input for Non-TD Models","titles":["Dispatching on Custom Input Types","Concrete Example"]},"119":{"title":"Freezing Model Parameters","titles":[]},"120":{"title":"Freezing Layers of a Particular Kind","titles":["Freezing Model Parameters"]},"121":{"title":"Freezing by Layer Name","titles":["Freezing Model Parameters"]},"122":{"title":"Freezing Part of the Parameters","titles":["Freezing Model Parameters"]},"123":{"title":"Freezing Part of a Chain","titles":["Freezing Model Parameters"]},"124":{"title":"GPU Management","titles":[]},"125":{"title":"Automatic Backend Management (Recommended Approach)","titles":["GPU Management"]},"126":{"title":"Manual Backend Management","titles":["GPU Management"]},"127":{"title":"Lux Interface","titles":[]},"128":{"title":"Layer Interface","titles":["Lux Interface"]},"129":{"title":"Singular Layer","titles":["Lux Interface","Layer Interface"]},"130":{"title":"Container Layer","titles":["Lux Interface","Layer Interface"]},"131":{"title":"Parameter Interface","titles":["Lux Interface"]},"132":{"title":"State Interface","titles":["Lux Interface"]},"133":{"title":"Nested Automatic Differentiation","titles":[]},"134":{"title":"Loss Function containing Jacobian Computation","titles":["Nested Automatic Differentiation"]},"135":{"title":"Using Batched Jacobian for Multiple Inputs","titles":["Nested Automatic Differentiation","Loss Function containing Jacobian Computation"]},"136":{"title":"Loss Function contains Gradient Computation","titles":["Nested Automatic Differentiation"]},"137":{"title":"Loss Function computing the Jacobian of the Parameters","titles":["Nested Automatic Differentiation"]},"138":{"title":"Hutchinson Trace Estimation","titles":["Nested Automatic Differentiation"]},"139":{"title":"Computing using the Vector-Jacobian Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"140":{"title":"Computing using the Jacobian-Vector Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"141":{"title":"Computing using the Full Jacobian","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"142":{"title":"Distributed Data Parallel Training","titles":[]},"143":{"title":"Guide to Integrating DistributedUtils into your code","titles":["Distributed Data Parallel Training"]},"144":{"title":"Migration Guide from FluxMPI.jl","titles":["Distributed Data Parallel Training"]},"145":{"title":"Removed Functionality","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"146":{"title":"Key Differences","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"147":{"title":"Known Shortcomings","titles":["Distributed Data Parallel Training"]},"148":{"title":"Performance Pitfalls & How to Catch Them","titles":[]},"149":{"title":"Spurious Type-Promotion","titles":["Performance Pitfalls & How to Catch Them"]},"150":{"title":"Scalar Indexing on GPU Arrays","titles":["Performance Pitfalls & How to Catch Them"]},"151":{"title":"Type Instabilities","titles":["Performance Pitfalls & How to Catch Them"]},"152":{"title":"Faster Primitives","titles":["Performance Pitfalls & How to Catch Them"]},"153":{"title":"Optional Dependencies for Performance","titles":["Performance Pitfalls & How to Catch Them"]},"154":{"title":"Data Loading and Device Transfer","titles":["Performance Pitfalls & How to Catch Them"]},"155":{"title":"Preferences for Lux.jl","titles":[]},"156":{"title":"Nested Automatic Differentiation","titles":["Preferences for Lux.jl"]},"157":{"title":"GPU-Aware MPI Support","titles":["Preferences for Lux.jl"]},"158":{"title":"GPU Backend Selection","titles":["Preferences for Lux.jl"]},"159":{"title":"Automatic Eltype Conversion","titles":["Preferences for Lux.jl"]},"160":{"title":"Dispatch Doctor","titles":["Preferences for Lux.jl"]},"161":{"title":"Disabling Loop Vectorization / Octavian","titles":["Preferences for Lux.jl"]},"162":{"title":"Initializing Weights","titles":[]},"163":{"title":"Quick examples","titles":["Initializing Weights"]},"164":{"title":"Neural Networks Inside GPU Kernels","titles":[]},"165":{"title":"Julia & Lux for the Uninitiated","titles":[]},"166":{"title":"Arrays","titles":["Julia & Lux for the Uninitiated"]},"167":{"title":"CUDA Arrays","titles":["Julia & Lux for the Uninitiated","Arrays"]},"168":{"title":"(Im)mutability","titles":["Julia & Lux for the Uninitiated"]},"169":{"title":"Managing Randomness","titles":["Julia & Lux for the Uninitiated"]},"170":{"title":"Automatic Differentiation","titles":["Julia & Lux for the Uninitiated"]},"171":{"title":"Gradients","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"172":{"title":"Jacobian-Vector Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"173":{"title":"Vector-Jacobian Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"174":{"title":"Linear Regression","titles":["Julia & Lux for the Uninitiated"]},"175":{"title":"Appendix","titles":["Julia & Lux for the Uninitiated"]},"176":{"title":"Training a Simple LSTM","titles":[]},"177":{"title":"Package Imports","titles":["Training a Simple LSTM"]},"178":{"title":"Dataset","titles":["Training a Simple LSTM"]},"179":{"title":"Creating a Classifier","titles":["Training a Simple LSTM"]},"180":{"title":"Using the @compact API","titles":["Training a Simple LSTM"]},"181":{"title":"Defining Accuracy, Loss and Optimiser","titles":["Training a Simple LSTM"]},"182":{"title":"Training the Model","titles":["Training a Simple LSTM"]},"183":{"title":"Saving the Model","titles":["Training a Simple LSTM"]},"184":{"title":"Appendix","titles":["Training a Simple LSTM"]},"185":{"title":"MNIST Classification with SimpleChains","titles":[]},"186":{"title":"Package Imports","titles":["MNIST Classification with SimpleChains"]},"187":{"title":"Loading MNIST","titles":["MNIST Classification with SimpleChains"]},"188":{"title":"Define the Model","titles":["MNIST Classification with SimpleChains"]},"189":{"title":"Helper Functions","titles":["MNIST Classification with SimpleChains"]},"190":{"title":"Define the Training Loop","titles":["MNIST Classification with SimpleChains"]},"191":{"title":"Finally Training the Model","titles":["MNIST Classification with SimpleChains"]},"192":{"title":"Appendix","titles":["MNIST Classification with SimpleChains"]},"193":{"title":"Migrating from Flux to Lux","titles":[]},"194":{"title":"Implementing Custom Layers","titles":["Migrating from Flux to Lux"]},"195":{"title":"Certain Important Implementation Details","titles":["Migrating from Flux to Lux"]},"196":{"title":"Training/Inference Mode","titles":["Migrating from Flux to Lux","Certain Important Implementation Details"]},"197":{"title":"Can we still use Flux Layers?","titles":["Migrating from Flux to Lux"]},"198":{"title":"Tutorials","titles":[]},"199":{"title":"Beginner Tutorials","titles":["Tutorials"]},"200":{"title":"Intermediate Tutorials","titles":["Tutorials"]},"201":{"title":"Advanced Tutorials","titles":["Tutorials"]},"202":{"title":"Larger Models","titles":["Tutorials"]},"203":{"title":"Selected 3rd Party Tutorials","titles":["Tutorials"]},"204":{"title":"Training Lux Models using Optimization.jl","titles":[]},"205":{"title":"Imports packages","titles":["Training Lux Models using Optimization.jl"]},"206":{"title":"Generate some training data","titles":["Training Lux Models using Optimization.jl"]},"207":{"title":"Define the DataLoader","titles":["Training Lux Models using Optimization.jl"]},"208":{"title":"Training the model","titles":["Training Lux Models using Optimization.jl"]},"209":{"title":"Plotting the results","titles":["Training Lux Models using Optimization.jl"]},"210":{"title":"Appendix","titles":["Training Lux Models using Optimization.jl"]},"211":{"title":"Fitting a Polynomial using MLP","titles":[]},"212":{"title":"Package Imports","titles":["Fitting a Polynomial using MLP"]},"213":{"title":"Dataset","titles":["Fitting a Polynomial using MLP"]},"214":{"title":"Neural Network","titles":["Fitting a Polynomial using MLP"]},"215":{"title":"Optimizer","titles":["Fitting a Polynomial using MLP"]},"216":{"title":"Loss Function","titles":["Fitting a Polynomial using MLP"]},"217":{"title":"Training","titles":["Fitting a Polynomial using MLP"]},"218":{"title":"Appendix","titles":["Fitting a Polynomial using MLP"]},"219":{"title":"Training a HyperNetwork on MNIST and FashionMNIST","titles":[]},"220":{"title":"Package Imports","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"221":{"title":"Loading Datasets","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"222":{"title":"Implement a HyperNet Layer","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"223":{"title":"Create and Initialize the HyperNet","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"224":{"title":"Define Utility Functions","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"225":{"title":"Training","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"226":{"title":"Appendix","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"227":{"title":"MNIST Classification using Neural ODEs","titles":[]},"228":{"title":"Package Imports","titles":["MNIST Classification using Neural ODEs"]},"229":{"title":"Loading MNIST","titles":["MNIST Classification using Neural ODEs"]},"230":{"title":"Define the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"231":{"title":"Create and Initialize the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"232":{"title":"Define Utility Functions","titles":["MNIST Classification using Neural ODEs"]},"233":{"title":"Training","titles":["MNIST Classification using Neural ODEs"]},"234":{"title":"Alternate Implementation using Stateful Layer","titles":["MNIST Classification using Neural ODEs"]},"235":{"title":"Train the new Stateful Neural ODE","titles":["MNIST Classification using Neural ODEs"]},"236":{"title":"Type Stability","titles":["MNIST Classification using Neural ODEs"]},"237":{"title":"Appendix","titles":["MNIST Classification using Neural ODEs"]},"238":{"title":"Training a PINN on 2D PDE","titles":[]},"239":{"title":"Package Imports","titles":["Training a PINN on 2D PDE"]},"240":{"title":"Problem Definition","titles":["Training a PINN on 2D PDE"]},"241":{"title":"Define the Neural Networks","titles":["Training a PINN on 2D PDE"]},"242":{"title":"Define the Loss Functions","titles":["Training a PINN on 2D PDE"]},"243":{"title":"Generate the Data","titles":["Training a PINN on 2D PDE"]},"244":{"title":"Training","titles":["Training a PINN on 2D PDE"]},"245":{"title":"Visualizing the Results","titles":["Training a PINN on 2D PDE"]},"246":{"title":"Appendix","titles":["Training a PINN on 2D PDE"]},"247":{"title":"Bayesian Neural Network","titles":[]},"248":{"title":"Generating data","titles":["Bayesian Neural Network"]},"249":{"title":"Building the Neural Network","titles":["Bayesian Neural Network"]},"250":{"title":"Prediction Visualization","titles":["Bayesian Neural Network"]},"251":{"title":"Appendix","titles":["Bayesian Neural Network"]},"252":{"title":"Training a Neural ODE to Model Gravitational Waveforms","titles":[]},"253":{"title":"Package Imports","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"254":{"title":"Define some Utility Functions","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"255":{"title":"Simulating the True Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"256":{"title":"Defiing a Neural Network Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"257":{"title":"Setting Up for Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"258":{"title":"Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"259":{"title":"Visualizing the Results","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"260":{"title":"Appendix","titles":["Training a Neural ODE to Model Gravitational Waveforms"]}},"dirtCount":0,"index":[["π",{"2":{"254":2,"255":1}}],["√",{"2":{"254":2}}],["√t",{"2":{"9":2}}],["↦",{"2":{"254":1}}],["ϕ̇",{"2":{"255":2,"256":2}}],["ϕ",{"2":{"254":5,"255":1,"256":1}}],["χ̇",{"2":{"255":2,"256":2}}],["χ",{"2":{"254":4,"255":4,"256":2}}],["~",{"2":{"249":2}}],["₋₋₋kwargs₋₋₋",{"2":{"236":4}}],["₋₋₋no",{"2":{"236":3}}],["│",{"2":{"236":3,"244":4}}],["─",{"2":{"236":3}}],["θ|x",{"2":{"250":2}}],["θ",{"2":{"208":2,"249":2,"250":9,"257":3}}],["÷",{"2":{"178":4}}],["∥22we",{"2":{"174":1}}],["⟶∑i=1k12∥yi−fw",{"2":{"174":1}}],["∇f",{"2":{"171":3}}],["∇offending",{"2":{"110":2}}],["$",{"2":{"245":1}}],["$i",{"2":{"169":2,"250":1}}],["$name",{"2":{"32":1}}],["⋮",{"2":{"164":4}}],["∞",{"2":{"134":4,"135":4,"136":4,"137":4,"141":8}}],["∘",{"2":{"133":4,"136":1,"242":3}}],["└──",{"2":{"236":3}}],["└",{"2":{"109":1,"134":2,"244":4}}],["┌",{"2":{"109":1,"134":2,"244":4}}],["∂w",{"2":{"242":2}}],["∂v",{"2":{"242":2}}],["∂y",{"2":{"242":4}}],["∂u",{"2":{"242":10}}],["∂t",{"2":{"136":7,"242":2}}],["∂xyt",{"2":{"242":4}}],["∂x",{"2":{"134":7,"135":7,"137":7,"141":11,"242":4}}],["∂ps",{"2":{"106":4,"134":8,"135":8,"136":8,"137":8,"141":11}}],["∂f∂x",{"2":{"27":2}}],["↩︎",{"2":{"89":8}}],["❌",{"2":{"86":5}}],["❓",{"2":{"86":6}}],["\\ttime",{"2":{"233":45,"235":9}}],["\\ttest",{"2":{"225":102,"233":45,"235":9}}],["\\tfashionmnist\\ttraining",{"2":{"225":1}}],["\\tfashionmnist\\ttime",{"2":{"225":50}}],["\\tloss",{"2":{"107":11}}],["\\t",{"2":{"81":11,"191":60,"217":6,"225":51,"244":400}}],["quadrupole",{"2":{"254":10}}],["quadratic",{"2":{"213":1,"217":1}}],["quantiles",{"2":{"249":1}}],["questions",{"2":{"84":3,"133":1,"170":1}}],["quick",{"0":{"163":1},"2":{"165":1}}],["quickstart",{"0":{"80":1},"2":{"84":1}}],["quite",{"2":{"75":1,"168":1,"170":1}}],["quoting",{"2":{"106":1}}],["quot",{"2":{"1":2,"9":12,"13":2,"22":4,"23":1,"24":8,"53":8,"55":2,"57":16,"59":2,"60":6,"65":4,"71":2,"72":2,"80":4,"146":2,"149":2,"158":8,"194":4}}],["⋅n+z⋅hprevarguments",{"2":{"69":1}}],["↗",{"2":{"65":2}}],["↘",{"2":{"65":2}}],["→",{"2":{"65":5}}],["8c79",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["831089962910616e",{"2":{"258":1}}],["8317274f",{"2":{"174":1}}],["835987097737537e",{"2":{"258":1}}],["832680953664729e",{"2":{"258":1}}],["832331f",{"2":{"256":1}}],["8328976f",{"2":{"256":1}}],["8325594f",{"2":{"256":1}}],["8321s\\ttraining",{"2":{"233":1}}],["839174579736624e",{"2":{"258":1}}],["8391583189296476e",{"2":{"258":1}}],["839207f",{"2":{"256":1}}],["8397034f",{"2":{"256":1}}],["8393",{"2":{"166":1}}],["8393034",{"2":{"166":1}}],["839303",{"2":{"166":2}}],["837145376351397e",{"2":{"258":1}}],["837898211029819e",{"2":{"258":1}}],["8372574f",{"2":{"256":1}}],["837018f",{"2":{"256":1}}],["837797f",{"2":{"256":1}}],["8376",{"2":{"249":1}}],["8374606",{"2":{"216":1}}],["837991",{"2":{"137":1}}],["8346457",{"2":{"213":2}}],["8340728",{"2":{"137":1}}],["83",{"2":{"191":4,"233":6,"235":1}}],["836",{"2":{"188":2}}],["833150327334844e",{"2":{"258":1}}],["833168f",{"2":{"256":1}}],["833518",{"2":{"167":1}}],["8337032",{"2":{"120":1}}],["833333333333333",{"2":{"53":1}}],["893228649103207e",{"2":{"258":1}}],["8931157f",{"2":{"256":1}}],["892866574317183e",{"2":{"258":1}}],["892634f",{"2":{"256":1}}],["8950031747943242e",{"2":{"258":1}}],["8950981f",{"2":{"256":1}}],["8956723722780767e",{"2":{"258":1}}],["894742336531445e",{"2":{"258":1}}],["894702f",{"2":{"256":1}}],["8946",{"2":{"249":1}}],["894389e",{"2":{"208":1}}],["8904",{"2":{"249":1}}],["898699471965577e",{"2":{"258":1}}],["898851f",{"2":{"256":1}}],["8984",{"2":{"249":1}}],["89858943",{"2":{"136":1}}],["89",{"2":{"225":3}}],["8971257737095413e",{"2":{"258":1}}],["897193248943069e",{"2":{"258":1}}],["897066f",{"2":{"256":1}}],["8972864",{"2":{"213":1}}],["8976378",{"2":{"213":2}}],["89s",{"2":{"191":1}}],["8969011",{"2":{"213":1}}],["896",{"2":{"188":2}}],["8990",{"2":{"249":1}}],["899766",{"2":{"166":1}}],["8999977",{"2":{"137":1}}],["819267667109008e",{"2":{"258":1}}],["819155693737988e",{"2":{"258":1}}],["8191115f",{"2":{"256":1}}],["81914f",{"2":{"256":1}}],["812307360893733e",{"2":{"258":1}}],["8122438f",{"2":{"256":1}}],["8181799097218416e",{"2":{"258":1}}],["818170232392436e",{"2":{"258":1}}],["818207f",{"2":{"256":1}}],["8189028f",{"2":{"256":1}}],["8109587064243945e",{"2":{"258":1}}],["81021370076556e",{"2":{"258":1}}],["81069772469319e",{"2":{"258":1}}],["8101463154730067e",{"2":{"258":1}}],["810577744266379e",{"2":{"258":1}}],["810738f",{"2":{"256":1}}],["8104418f",{"2":{"256":1}}],["8104494f",{"2":{"256":1}}],["8100424f",{"2":{"256":1}}],["8100",{"2":{"249":1}}],["815132181910655e",{"2":{"258":1}}],["8151837f",{"2":{"256":1}}],["815336",{"2":{"137":1}}],["816281",{"2":{"213":1}}],["8110236",{"2":{"213":2}}],["81",{"2":{"191":2,"225":28,"233":7,"235":1,"249":1}}],["81s",{"2":{"191":1}}],["8173783982243132",{"2":{"164":2}}],["81481",{"2":{"233":2,"235":1}}],["81446123",{"2":{"216":1}}],["8146973e",{"2":{"136":2}}],["8147778",{"2":{"106":1}}],["881843239953348e",{"2":{"258":1}}],["881772291117934e",{"2":{"258":1}}],["8812",{"2":{"249":1}}],["88329711031072e",{"2":{"258":1}}],["883044f",{"2":{"256":1}}],["883664f0",{"2":{"134":1}}],["885345082923397e",{"2":{"258":1}}],["88502f",{"2":{"256":1}}],["885141f",{"2":{"256":1}}],["8852221",{"2":{"137":1}}],["886728755885818e",{"2":{"258":1}}],["886729f",{"2":{"256":1}}],["8862s\\ttraining",{"2":{"233":1}}],["8805s\\ttraining",{"2":{"235":1}}],["88889",{"2":{"233":1}}],["88",{"2":{"225":14}}],["8821303",{"2":{"213":1}}],["88214725",{"2":{"137":1}}],["88951594",{"2":{"213":1}}],["8652516581687163e",{"2":{"258":1}}],["8653448f",{"2":{"256":1}}],["864511026744694e",{"2":{"258":1}}],["864171015696166e",{"2":{"258":1}}],["8643836f",{"2":{"256":1}}],["864298f",{"2":{"256":1}}],["864081e",{"2":{"208":1}}],["869451049973391e",{"2":{"258":1}}],["8694303f",{"2":{"256":1}}],["869967270719461e",{"2":{"258":1}}],["869763f",{"2":{"256":1}}],["869632",{"2":{"134":1}}],["8689",{"2":{"249":1}}],["8688824",{"2":{"213":1}}],["8638",{"2":{"249":1}}],["86614174",{"2":{"213":2}}],["86",{"2":{"191":2,"225":1}}],["8601828553599953",{"2":{"169":1}}],["8601870861800127",{"2":{"164":2}}],["867334374931775e",{"2":{"258":1}}],["8678929363374415e",{"2":{"258":1}}],["867486f",{"2":{"256":1}}],["8674763752817414",{"2":{"164":1}}],["8674763752817416",{"2":{"164":1}}],["867762f",{"2":{"256":1}}],["8672",{"2":{"249":1}}],["867202",{"2":{"134":1}}],["862447f",{"2":{"256":1}}],["862480861009647",{"2":{"164":2}}],["86290836",{"2":{"120":1}}],["8626451f",{"2":{"106":5}}],["858580723254803e",{"2":{"258":1}}],["858364f",{"2":{"256":1}}],["856513863138032e",{"2":{"258":1}}],["856615f",{"2":{"256":1}}],["8567588f",{"2":{"256":1}}],["859565073292652e",{"2":{"258":1}}],["859555f",{"2":{"256":1}}],["85945405234782e",{"2":{"258":1}}],["859405",{"2":{"136":1,"137":1}}],["859231075038756e",{"2":{"258":1}}],["8592405f",{"2":{"256":1}}],["8534",{"2":{"249":1}}],["85344f",{"2":{"80":1}}],["85185",{"2":{"233":2}}],["85",{"2":{"191":3,"225":1,"233":4}}],["8501",{"2":{"244":1}}],["85050875",{"2":{"213":1}}],["850",{"2":{"188":2}}],["857484414169174e",{"2":{"258":1}}],["8574s\\ttraining",{"2":{"233":1}}],["8570116978216505e",{"2":{"258":1}}],["857521f",{"2":{"256":1}}],["8576512",{"2":{"213":1}}],["857257",{"2":{"167":1}}],["857126",{"2":{"167":1}}],["8573820474674845",{"2":{"166":1}}],["8541596",{"2":{"134":1}}],["807031263771555e",{"2":{"258":1}}],["802835f",{"2":{"256":1}}],["8024",{"2":{"249":1}}],["808523581943265e",{"2":{"258":1}}],["808188192255654e",{"2":{"258":1}}],["808402f",{"2":{"256":1}}],["808087f",{"2":{"256":1}}],["809125507369087e",{"2":{"258":1}}],["8097627f",{"2":{"256":1}}],["809247f",{"2":{"256":1}}],["809343f",{"2":{"256":1}}],["8094946705911407e",{"2":{"258":1}}],["8094",{"2":{"249":1}}],["809456",{"2":{"166":1}}],["803506896010253e",{"2":{"258":1}}],["8038628844860654e",{"2":{"258":1}}],["803355f",{"2":{"256":1}}],["8037993f",{"2":{"256":1}}],["8031547",{"2":{"213":1}}],["8031496",{"2":{"213":2}}],["8069002f",{"2":{"256":1}}],["8067698",{"2":{"136":1}}],["8014057053713016e",{"2":{"258":1}}],["801946592103462e",{"2":{"258":1}}],["8019178f",{"2":{"256":1}}],["801173f",{"2":{"256":1}}],["8013359",{"2":{"213":1}}],["80557334859591e",{"2":{"258":1}}],["8056135f",{"2":{"256":1}}],["80500405533108e",{"2":{"258":1}}],["8050",{"2":{"249":1}}],["805853e",{"2":{"208":1}}],["80",{"2":{"175":1,"184":1,"191":2,"210":1,"218":1,"225":2,"226":1,"233":10,"235":2,"237":1,"246":1,"248":1}}],["8045s\\ttraining",{"2":{"233":1}}],["8048109",{"2":{"216":1}}],["80465555",{"2":{"174":1}}],["80423f",{"2":{"80":1}}],["8001",{"2":{"174":1,"244":1}}],["800792614736468",{"2":{"164":2}}],["8003627",{"2":{"136":1}}],["800",{"2":{"107":1}}],["8f",{"2":{"107":1,"174":1}}],["873553148078214e",{"2":{"258":1}}],["873516f",{"2":{"256":1}}],["872884706412406e",{"2":{"258":1}}],["8729362f",{"2":{"256":1}}],["879130314243693e",{"2":{"258":1}}],["879042369972584e",{"2":{"258":1}}],["879363f",{"2":{"256":1}}],["878988f",{"2":{"256":1}}],["8787",{"2":{"249":1}}],["8701",{"2":{"249":1}}],["875388162700245e",{"2":{"258":1}}],["8759535924339176e",{"2":{"258":1}}],["8754206f",{"2":{"256":1}}],["875883f",{"2":{"256":1}}],["8755s\\ttraining",{"2":{"233":1}}],["875082",{"2":{"213":1}}],["8740157",{"2":{"213":2}}],["87431f",{"2":{"80":1}}],["877992978169433e",{"2":{"258":1}}],["877776f",{"2":{"256":1}}],["8771",{"2":{"249":1}}],["877497",{"2":{"172":1,"173":1}}],["87702435",{"2":{"106":1}}],["87",{"2":{"109":1,"191":1,"225":5}}],["87143785f0",{"2":{"80":1}}],["82363948355292e",{"2":{"258":1}}],["823387f",{"2":{"256":1}}],["8247074246699347e",{"2":{"258":1}}],["824178304225082e",{"2":{"258":1}}],["8245557f",{"2":{"256":1}}],["82423f",{"2":{"256":1}}],["8267525f",{"2":{"256":1}}],["822139041115574e",{"2":{"258":1}}],["822381263758857e",{"2":{"258":1}}],["822046f",{"2":{"256":1}}],["822452f",{"2":{"256":1}}],["82293516",{"2":{"213":1}}],["829450274425075e",{"2":{"258":1}}],["82945824",{"2":{"80":1}}],["82938f",{"2":{"256":1}}],["828026639213363e",{"2":{"258":1}}],["82805836",{"2":{"120":1}}],["82827676214142e",{"2":{"258":1}}],["8281497f",{"2":{"256":1}}],["8277",{"2":{"249":1}}],["8271f",{"2":{"80":1}}],["82030004",{"2":{"213":1}}],["82",{"2":{"191":2,"225":3,"233":6,"235":2}}],["82s",{"2":{"191":2}}],["8",{"2":{"53":1,"65":2,"71":1,"80":27,"106":28,"134":1,"136":2,"137":3,"166":2,"168":4,"178":1,"182":15,"191":2,"208":3,"213":1,"215":1,"217":1,"225":2,"233":5,"235":1,"249":8,"251":1,"256":67,"258":65,"260":1}}],["844121978835604e",{"2":{"258":1}}],["848885813975649e",{"2":{"258":1}}],["84303728698095e",{"2":{"258":1}}],["843058f",{"2":{"256":1}}],["843955090580263e",{"2":{"258":1}}],["843995f",{"2":{"256":1}}],["8409362356176015e",{"2":{"258":1}}],["8404866926718095e",{"2":{"258":1}}],["84041f",{"2":{"256":1}}],["840369863028096e",{"2":{"258":1}}],["8408233f",{"2":{"256":1}}],["840527f",{"2":{"256":1}}],["840588",{"2":{"134":1}}],["847409772281022e",{"2":{"258":1}}],["847454513714503e",{"2":{"258":1}}],["847962927283407e",{"2":{"258":1}}],["8475592587357495e",{"2":{"258":1}}],["8475313f",{"2":{"256":1}}],["8475675f",{"2":{"256":1}}],["8478355f",{"2":{"256":1}}],["8478474f",{"2":{"256":1}}],["847106",{"2":{"137":1}}],["849695227782469e",{"2":{"258":1}}],["849716f",{"2":{"256":1}}],["8491387f",{"2":{"256":1}}],["8499767",{"2":{"213":1}}],["8490221",{"2":{"131":1}}],["842894007878977e",{"2":{"258":1}}],["8428395f",{"2":{"256":1}}],["8424188",{"2":{"213":1}}],["8425196",{"2":{"213":2}}],["8454118",{"2":{"137":1}}],["845766f",{"2":{"256":1}}],["84576",{"2":{"80":1}}],["841231",{"2":{"137":1}}],["846457",{"2":{"137":1}}],["84",{"2":{"46":2,"188":6,"225":19,"233":1,"235":1}}],["δ",{"2":{"53":1,"110":3,"206":2}}],["ϵ",{"2":{"53":1}}],["ŷ",{"2":{"53":6}}],["∈",{"2":{"53":2,"254":1}}],["∗y+α∗size",{"2":{"53":1}}],["∗y+α∗0",{"2":{"53":1}}],["∗y^−logσ",{"2":{"53":1}}],["∗log⁡",{"2":{"53":1}}],["−log⁡",{"2":{"53":1}}],["−∑y~log⁡",{"2":{"53":1}}],["−",{"2":{"53":1}}],["−y~∗log⁡",{"2":{"53":1}}],["−x",{"2":{"9":1}}],["≈∑θ∼p",{"2":{"250":1}}],["≈",{"2":{"53":25,"59":1,"125":1,"126":1,"169":2}}],["9f",{"2":{"244":8}}],["932340121209963e",{"2":{"258":1}}],["931653767119681e",{"2":{"258":1}}],["931654f",{"2":{"256":1}}],["933901033906554e",{"2":{"258":1}}],["9336143f",{"2":{"256":1}}],["9338465f",{"2":{"256":1}}],["9338737",{"2":{"137":1}}],["9300596f",{"2":{"256":1}}],["9307",{"2":{"249":1}}],["9345",{"2":{"249":1}}],["9347s\\ttraining",{"2":{"235":1}}],["9340093",{"2":{"136":1}}],["93",{"2":{"225":4}}],["9370079",{"2":{"213":2}}],["919753489032578e",{"2":{"258":1}}],["911820979507831e",{"2":{"258":1}}],["9147372703658136e",{"2":{"258":1}}],["9146099f",{"2":{"256":1}}],["915055148453373e",{"2":{"258":1}}],["915397136620633e",{"2":{"258":1}}],["915148f",{"2":{"256":1}}],["915949f",{"2":{"256":1}}],["9120237f",{"2":{"256":1}}],["912781",{"2":{"141":1}}],["9127817",{"2":{"141":1}}],["912782",{"2":{"141":1}}],["91717624114733e",{"2":{"258":1}}],["917144f",{"2":{"256":1}}],["917755001005653e",{"2":{"258":1}}],["9175829974698705e",{"2":{"258":1}}],["9176105f",{"2":{"256":1}}],["917452f",{"2":{"256":1}}],["91",{"2":{"225":6}}],["916295256675445e",{"2":{"258":1}}],["9165122f",{"2":{"256":1}}],["9166674f",{"2":{"256":1}}],["9166",{"2":{"249":1}}],["9161865",{"2":{"213":1}}],["9164896",{"2":{"137":1}}],["9109",{"2":{"249":1}}],["9108207",{"2":{"213":1}}],["910251",{"2":{"136":1}}],["918918",{"2":{"162":1}}],["9139614",{"2":{"137":1}}],["9130077",{"2":{"106":2}}],["9521455853597164e",{"2":{"258":1}}],["9522387f",{"2":{"256":1}}],["9539240488211657e",{"2":{"258":1}}],["9536714f",{"2":{"256":1}}],["955887999559289e",{"2":{"258":1}}],["9557868f",{"2":{"256":1}}],["958926f",{"2":{"256":1}}],["951167408783001e",{"2":{"258":1}}],["9511775",{"2":{"136":1}}],["951066f",{"2":{"256":1}}],["9515",{"2":{"249":1}}],["9517",{"2":{"249":1}}],["9598630492229406e",{"2":{"258":1}}],["9598s\\ttraining",{"2":{"233":1}}],["959221400743762e",{"2":{"258":1}}],["9596104f",{"2":{"256":1}}],["959182",{"2":{"137":1}}],["95",{"2":{"225":6}}],["954497048078123e",{"2":{"258":1}}],["9541194621509316e",{"2":{"258":1}}],["954174f",{"2":{"256":1}}],["9547",{"2":{"249":1}}],["9545",{"2":{"249":1}}],["95401394",{"2":{"213":1}}],["9543",{"2":{"162":1}}],["9500",{"2":{"249":1}}],["9500444",{"2":{"136":1}}],["9501",{"2":{"244":1}}],["9501405",{"2":{"137":1}}],["957240402024154e",{"2":{"258":1}}],["957273f",{"2":{"256":1}}],["9572f",{"2":{"80":1}}],["9571",{"2":{"249":1}}],["9577397",{"2":{"136":1}}],["978239208939439e",{"2":{"258":1}}],["978068f",{"2":{"256":1}}],["977641412069611e",{"2":{"258":1}}],["9776292f",{"2":{"256":1}}],["977304667682311e",{"2":{"258":1}}],["9773887f",{"2":{"256":1}}],["971269564295335e",{"2":{"258":1}}],["9713708f",{"2":{"256":1}}],["971899",{"2":{"109":1}}],["970483450145065e",{"2":{"258":1}}],["9702306f",{"2":{"256":1}}],["9708",{"2":{"249":1}}],["974130119097646e",{"2":{"258":1}}],["974424345790755e",{"2":{"258":1}}],["9747",{"2":{"249":1}}],["9742292",{"2":{"213":1}}],["97",{"2":{"225":3,"249":1}}],["9738056f",{"2":{"256":1}}],["973879f",{"2":{"256":1}}],["97378695",{"2":{"213":1}}],["9734574",{"2":{"134":1}}],["9767632",{"2":{"213":1}}],["976634f",{"2":{"256":1}}],["9766699015845924",{"2":{"169":4}}],["97667",{"2":{"166":1}}],["975649",{"2":{"167":1}}],["9721554734769875",{"2":{"164":2}}],["9727281",{"2":{"136":1}}],["903685552249666e",{"2":{"258":1}}],["903475528666147e",{"2":{"258":1}}],["9043554629668638e",{"2":{"258":1}}],["9043228",{"2":{"134":1}}],["9041228f",{"2":{"256":1}}],["9045",{"2":{"249":1}}],["9028980128354384e",{"2":{"258":1}}],["90291f",{"2":{"256":1}}],["9029382f",{"2":{"256":1}}],["902982902377702",{"2":{"164":2}}],["9021",{"2":{"249":1}}],["90",{"2":{"225":6}}],["905398f",{"2":{"256":1}}],["9055119",{"2":{"213":2}}],["90508",{"2":{"120":1,"123":1}}],["9019142280758281",{"2":{"164":2}}],["909560581959909e",{"2":{"258":1}}],["9099054854809883e",{"2":{"258":1}}],["909601f",{"2":{"256":1}}],["9097244f",{"2":{"256":1}}],["90910274",{"2":{"137":1}}],["9094505",{"2":{"134":1}}],["9071",{"2":{"249":1}}],["9076505",{"2":{"216":1}}],["9076533",{"2":{"134":1}}],["9079035",{"2":{"213":1}}],["90779305",{"2":{"213":1}}],["90772897",{"2":{"120":1}}],["9002s\\ttraining",{"2":{"233":1}}],["9001",{"2":{"174":1,"244":1}}],["900",{"2":{"107":1}}],["963713681034698e",{"2":{"258":1}}],["9636124f",{"2":{"256":1}}],["9611733534690534e",{"2":{"258":1}}],["9616315447848574e",{"2":{"258":1}}],["961775925591352e",{"2":{"258":1}}],["9612102f",{"2":{"256":1}}],["9615041f",{"2":{"256":1}}],["964991176283433e",{"2":{"258":1}}],["964992f",{"2":{"256":1}}],["9640",{"2":{"249":1}}],["964663+0",{"2":{"162":1}}],["9697625605151052e",{"2":{"258":1}}],["9695848f",{"2":{"256":1}}],["9695099f",{"2":{"256":1}}],["9694",{"2":{"249":1}}],["9693878",{"2":{"137":1}}],["96289262864811e",{"2":{"258":1}}],["962125107938336e",{"2":{"258":1}}],["962925f",{"2":{"256":1}}],["96296",{"2":{"235":1}}],["962413f",{"2":{"256":1}}],["9626998",{"2":{"136":1}}],["96",{"2":{"225":4}}],["965702457271244e",{"2":{"258":1}}],["96567f",{"2":{"256":1}}],["9651182f",{"2":{"256":1}}],["965117",{"2":{"120":1}}],["965008",{"2":{"216":1}}],["9679338",{"2":{"213":1}}],["967811",{"2":{"167":1}}],["960662267133479e",{"2":{"258":1}}],["96062994",{"2":{"213":2}}],["960599f",{"2":{"256":1}}],["9604645f",{"2":{"106":4}}],["968504",{"2":{"213":2}}],["9812624844068647e",{"2":{"258":1}}],["981339",{"2":{"166":1}}],["98817959074871e",{"2":{"258":1}}],["988372814958256e",{"2":{"258":1}}],["988504f",{"2":{"256":1}}],["986637222118892e",{"2":{"258":1}}],["986819156847103e",{"2":{"258":1}}],["9867588f",{"2":{"256":1}}],["9879985f",{"2":{"256":1}}],["9871407f",{"2":{"256":1}}],["987689",{"2":{"134":1}}],["9837260326726308e",{"2":{"258":1}}],["983743f",{"2":{"256":1}}],["983938040906761e",{"2":{"258":1}}],["98396176",{"2":{"110":1}}],["983421208855968e",{"2":{"258":1}}],["9834733f",{"2":{"256":1}}],["9853",{"2":{"249":1}}],["9856883",{"2":{"213":1}}],["982162476",{"2":{"244":2}}],["982390e",{"2":{"208":1}}],["9840825f",{"2":{"256":1}}],["9847",{"2":{"249":1}}],["9844s\\ttraining",{"2":{"233":1}}],["9849229",{"2":{"216":1}}],["9849193f",{"2":{"106":1}}],["98",{"2":{"225":2}}],["980705549109859e",{"2":{"258":1}}],["980967f",{"2":{"256":1}}],["9804526f",{"2":{"256":1}}],["9803549",{"2":{"213":1}}],["980",{"2":{"153":1}}],["9802322f",{"2":{"106":2}}],["98023f",{"2":{"106":4}}],["947066717177559e",{"2":{"258":1}}],["9470896",{"2":{"123":1}}],["9496953879748424e",{"2":{"258":1}}],["949551f",{"2":{"256":1}}],["9438787707441644e",{"2":{"258":1}}],["9436797",{"2":{"174":1}}],["942554189830378e",{"2":{"258":1}}],["94259596",{"2":{"216":1}}],["942706f",{"2":{"256":1}}],["9440233f",{"2":{"256":1}}],["948348891458086e",{"2":{"258":1}}],["948349f",{"2":{"256":1}}],["9489941",{"2":{"213":1}}],["9460",{"2":{"249":1}}],["941009565607624e",{"2":{"258":1}}],["941131f",{"2":{"256":1}}],["9414",{"2":{"249":1}}],["9413",{"2":{"249":1}}],["9418648",{"2":{"213":1}}],["94",{"2":{"225":6}}],["9452923791531558",{"2":{"164":2}}],["94557f",{"2":{"80":1}}],["9403538861096767e",{"2":{"258":1}}],["9402",{"2":{"249":1}}],["9405848223512736",{"2":{"169":4}}],["940585",{"2":{"166":1}}],["940697f",{"2":{"106":1}}],["9407f",{"2":{"106":1}}],["9g",{"2":{"81":1}}],["92288854255075e",{"2":{"258":1}}],["922868f",{"2":{"256":1}}],["922447443302018e",{"2":{"258":1}}],["922391f",{"2":{"256":1}}],["922303e",{"2":{"208":1}}],["922069f",{"2":{"256":1}}],["9276125f",{"2":{"256":1}}],["9230",{"2":{"249":1}}],["923950547913545",{"2":{"164":2}}],["920653601898014e",{"2":{"258":1}}],["9206433",{"2":{"134":1}}],["920781f",{"2":{"256":1}}],["920",{"2":{"237":1}}],["92593",{"2":{"233":1,"235":1}}],["9253495",{"2":{"213":1}}],["92",{"2":{"225":6}}],["9248211",{"2":{"213":1}}],["929855401085215e",{"2":{"258":1}}],["92913383",{"2":{"213":2}}],["929f",{"2":{"80":1}}],["928609",{"2":{"134":1}}],["92832f",{"2":{"80":1}}],["921917441772954e",{"2":{"258":1}}],["921982f",{"2":{"256":1}}],["92166064731826e",{"2":{"258":1}}],["921466f",{"2":{"256":1}}],["9214196",{"2":{"137":1}}],["92155594",{"2":{"171":4}}],["921",{"2":{"59":1}}],["998503837584602e",{"2":{"258":1}}],["998411f",{"2":{"256":1}}],["990932f",{"2":{"256":1}}],["996500405525111e",{"2":{"258":1}}],["9968596487828436e",{"2":{"258":1}}],["996831f",{"2":{"256":1}}],["996796f",{"2":{"256":1}}],["991045092290287e",{"2":{"258":1}}],["9914954854560117e",{"2":{"258":1}}],["99155f",{"2":{"256":1}}],["9913",{"2":{"249":1}}],["9916",{"2":{"249":1}}],["99799860578801e",{"2":{"258":1}}],["997962f",{"2":{"256":1}}],["9970157145265214e",{"2":{"258":1}}],["9970723f",{"2":{"256":1}}],["997003",{"2":{"80":6}}],["9971247",{"2":{"216":1}}],["9934639",{"2":{"213":1}}],["9930804",{"2":{"213":1}}],["992126",{"2":{"213":2}}],["992662",{"2":{"134":1}}],["99",{"2":{"137":1,"225":8}}],["999448100896822e",{"2":{"258":1}}],["9992154f",{"2":{"256":1}}],["9999881",{"2":{"110":2}}],["999986",{"2":{"109":1}}],["99998605",{"2":{"109":1}}],["999",{"2":{"80":7,"215":1,"217":1}}],["99900760833609",{"2":{"53":1}}],["9",{"2":{"53":4,"65":2,"80":15,"102":1,"106":25,"120":2,"123":2,"134":1,"136":1,"137":2,"141":1,"166":1,"174":1,"182":14,"187":1,"190":2,"191":2,"208":1,"215":1,"217":1,"221":2,"225":2,"229":1,"233":53,"235":10,"249":3,"256":81,"258":95}}],["✖",{"2":{"49":4}}],["✔️",{"2":{"86":13}}],["✔",{"2":{"49":8}}],["`θ`",{"2":{"250":1}}],["`x`",{"2":{"250":1}}],["`circbuff",{"2":{"244":4}}],["`carry`",{"2":{"179":2}}],["`p",{"2":{"230":1}}],["`ps",{"2":{"130":1}}],["`a`",{"2":{"194":3}}],["`autoforwarddiff",{"2":{"135":1}}],["`autozygote",{"2":{"135":1}}],["`b`",{"2":{"194":2}}],["`iterators",{"2":{"179":1}}],["`eachslice`",{"2":{"179":1}}],["`∇`",{"2":{"171":1}}],["`val",{"2":{"134":2}}],["`training`",{"2":{"134":2}}],["`zygote",{"2":{"134":1}}],["`denselayerparameters`",{"2":{"131":2}}],["`namedtuple`",{"2":{"131":2}}],["`model",{"2":{"130":1}}],["`st`",{"2":{"130":1,"256":2}}],["`st",{"2":{"130":1}}],["`lag",{"2":{"244":4}}],["`lstm",{"2":{"179":1}}],["`linear",{"2":{"130":2}}],["`l",{"2":{"129":1}}],["`luxcore",{"2":{"134":2}}],["`luxcore`",{"2":{"129":1}}],["`lux",{"2":{"134":2}}],["`lux``",{"2":{"167":1}}],["`lux`",{"2":{"129":1}}],["`",{"2":{"48":1,"80":2,"129":1,"130":1,"134":2,"135":2,"171":1,"244":8}}],["`octavian",{"2":{"23":1}}],["×",{"2":{"28":6,"175":1,"178":2,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["×dn−2×1×1`",{"2":{"72":1}}],["×dn−2×1×1",{"2":{"24":1}}],["×dn−2×1×dn",{"2":{"24":1,"72":1}}],["≥",{"2":{"28":2,"54":1,"244":1}}],["znver2",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["zip",{"2":{"107":1,"244":1}}],["zips",{"2":{"32":1}}],["zyg",{"2":{"106":2}}],["zygotevjp",{"2":{"236":13}}],["zygote",{"2":{"27":1,"28":1,"49":1,"52":1,"53":1,"59":2,"77":1,"80":3,"81":1,"86":1,"87":3,"106":3,"110":2,"133":3,"134":1,"135":1,"136":3,"137":3,"141":3,"165":1,"168":1,"170":3,"171":4,"176":1,"177":1,"186":1,"193":2,"194":2,"212":1,"217":1,"220":1,"228":1,"239":1,"242":3}}],["zenodo",{"2":{"74":3}}],["zeroing",{"2":{"32":4,"55":1}}],["zero",{"2":{"9":3,"32":3,"53":2,"54":2,"55":9,"66":1,"67":3,"70":1,"107":1,"135":1,"166":1}}],["zeroed",{"2":{"9":3}}],["zerosc64",{"2":{"10":1}}],["zerosc32",{"2":{"10":1}}],["zerosc16",{"2":{"10":1}}],["zeros64",{"2":{"10":1}}],["zeros32",{"2":{"10":1,"129":2}}],["zeros16",{"2":{"10":1}}],["zeros",{"2":{"9":3,"10":6,"164":2,"166":1,"243":3,"248":1,"249":1}}],["z=σ",{"2":{"69":1}}],["z=",{"2":{"49":1}}],["z",{"2":{"49":2,"70":3,"249":2,"250":8}}],["β",{"2":{"24":4,"72":1,"206":2}}],["γ=0",{"2":{"53":2}}],["γ",{"2":{"24":4,"53":1,"72":1,"206":2}}],["7666128507673295e",{"2":{"258":1}}],["7666694f",{"2":{"256":1}}],["7663455",{"2":{"213":1}}],["7657664391610264e",{"2":{"258":1}}],["765442f",{"2":{"256":1}}],["7652s\\ttraining",{"2":{"233":1}}],["7681201586580233e",{"2":{"258":1}}],["7680657f",{"2":{"256":1}}],["7689272356710357",{"2":{"164":2}}],["76158035",{"2":{"213":1}}],["76425457",{"2":{"213":1}}],["7649314",{"2":{"137":1}}],["76",{"2":{"208":2,"233":6,"235":2}}],["7673453",{"2":{"120":1}}],["7672513",{"2":{"106":1}}],["739221404019534e",{"2":{"258":1}}],["739224497562247e",{"2":{"258":1}}],["733199925496392e",{"2":{"258":1}}],["73341",{"2":{"213":1}}],["7367696618746988e",{"2":{"258":1}}],["7363671075209743e",{"2":{"258":1}}],["7365168f",{"2":{"256":1}}],["7374953975530736e",{"2":{"258":1}}],["7376667f",{"2":{"256":1}}],["7370",{"2":{"249":1}}],["732947f",{"2":{"256":1}}],["735962678692488e",{"2":{"258":1}}],["73548986477575e",{"2":{"258":1}}],["735494988039545e",{"2":{"258":1}}],["7354497f",{"2":{"256":1}}],["735818f",{"2":{"256":1}}],["735626f",{"2":{"256":1}}],["7352",{"2":{"249":1}}],["7353s\\ttraining",{"2":{"233":1}}],["7318s\\ttraining",{"2":{"233":1}}],["731059",{"2":{"53":2}}],["730883105088676e",{"2":{"258":1}}],["730149137131014e",{"2":{"258":1}}],["730133f",{"2":{"256":1}}],["7306304f",{"2":{"256":1}}],["7304446f",{"2":{"256":1}}],["730291e",{"2":{"208":1}}],["730941",{"2":{"137":1}}],["73",{"2":{"191":2,"233":1,"235":1}}],["7389f",{"2":{"256":1}}],["7384",{"2":{"249":1}}],["7384486",{"2":{"213":1}}],["73806",{"2":{"167":1}}],["738356",{"2":{"166":1}}],["7531731863723377e",{"2":{"258":1}}],["7533444f",{"2":{"256":1}}],["7533",{"2":{"249":1}}],["757903300690267e",{"2":{"258":1}}],["757490643817917e",{"2":{"258":1}}],["757703004370722e",{"2":{"258":1}}],["7577355f",{"2":{"256":1}}],["757772f",{"2":{"256":1}}],["757812f",{"2":{"256":1}}],["759294799648488e",{"2":{"258":1}}],["7592948f",{"2":{"256":1}}],["759412",{"2":{"137":1}}],["75148054747745e",{"2":{"258":1}}],["75148f",{"2":{"256":1}}],["751377",{"2":{"136":1}}],["756840623890042e",{"2":{"258":1}}],["7568614f",{"2":{"256":1}}],["7560",{"2":{"249":1}}],["756944",{"2":{"162":1}}],["7559453",{"2":{"213":1}}],["75",{"2":{"191":2,"225":18,"233":2,"235":1,"249":1,"255":1,"256":2,"259":4}}],["7502",{"2":{"251":1,"260":1}}],["7501",{"2":{"244":1}}],["750",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["75487764716636e",{"2":{"258":1}}],["7541148531797e",{"2":{"258":1}}],["7541456",{"2":{"213":1}}],["754733f",{"2":{"256":1}}],["754747",{"2":{"137":1}}],["754051f",{"2":{"256":1}}],["754463",{"2":{"137":1}}],["7580993408473766",{"2":{"22":1}}],["773799045844129e",{"2":{"258":1}}],["7734745f",{"2":{"256":1}}],["7735352f",{"2":{"256":1}}],["7733535276924052",{"2":{"169":4}}],["773354",{"2":{"166":1}}],["771278900507687e",{"2":{"258":1}}],["771379f",{"2":{"256":1}}],["771308f",{"2":{"256":1}}],["7715",{"2":{"249":1}}],["77165353",{"2":{"213":2}}],["7789",{"2":{"249":1}}],["7720901543366787e",{"2":{"258":1}}],["77207",{"2":{"137":1}}],["7721s\\ttraining",{"2":{"233":1}}],["77778",{"2":{"233":2}}],["7792964603229816e",{"2":{"258":1}}],["7791447f",{"2":{"256":1}}],["7795275",{"2":{"213":2}}],["7794657",{"2":{"120":1}}],["77",{"2":{"191":1,"225":1,"233":1}}],["7746656838366666",{"2":{"169":1}}],["7766",{"2":{"166":1}}],["776598",{"2":{"166":2}}],["770976854049545e",{"2":{"258":1}}],["7703",{"2":{"249":1}}],["77037",{"2":{"136":1}}],["7701346738750905",{"2":{"164":2}}],["7700396",{"2":{"136":1}}],["7152006424891324e",{"2":{"258":1}}],["7170749f",{"2":{"256":1}}],["717863343114228",{"2":{"164":2}}],["719009847552347e",{"2":{"258":1}}],["7190664f",{"2":{"256":1}}],["7199",{"2":{"249":1}}],["7195462169922173",{"2":{"164":1}}],["7195462169922175",{"2":{"164":1}}],["7149052f",{"2":{"256":1}}],["7149",{"2":{"249":1}}],["7125428650239895e",{"2":{"258":1}}],["7121",{"2":{"249":1}}],["7127",{"2":{"249":1}}],["7123551",{"2":{"213":1}}],["712979078066394",{"2":{"164":2}}],["718531017366678e",{"2":{"258":1}}],["718644f",{"2":{"256":1}}],["7183194",{"2":{"213":1}}],["7181164",{"2":{"134":1}}],["71",{"2":{"191":1,"225":12,"233":2}}],["716893762212287e",{"2":{"258":1}}],["7161783188722274e",{"2":{"258":1}}],["716134755899883",{"2":{"164":2}}],["716242f",{"2":{"256":1}}],["7165354",{"2":{"213":2}}],["7165512278088038",{"2":{"164":2}}],["713031f",{"2":{"256":1}}],["713",{"2":{"217":1}}],["713567516238075",{"2":{"164":2}}],["713316",{"2":{"106":1}}],["711329",{"2":{"162":1}}],["7446948785966035e",{"2":{"258":1}}],["7444965328947305e",{"2":{"258":1}}],["744489",{"2":{"134":1}}],["745934037136632e",{"2":{"258":1}}],["7456613895707984e",{"2":{"258":1}}],["7457823f",{"2":{"256":1}}],["7457888f",{"2":{"256":1}}],["7438878875030686e",{"2":{"258":1}}],["747220297384205e",{"2":{"258":1}}],["748175266906924e",{"2":{"258":1}}],["748175f",{"2":{"256":1}}],["7480315",{"2":{"213":2}}],["742596203254984e",{"2":{"258":1}}],["7424688f",{"2":{"256":1}}],["7426",{"2":{"249":1}}],["7429947",{"2":{"123":1}}],["7416",{"2":{"249":1}}],["741348",{"2":{"213":1}}],["740610020494017e",{"2":{"258":1}}],["74074",{"2":{"233":3}}],["7401575",{"2":{"213":2}}],["7402",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["74",{"2":{"191":1,"225":1,"233":5}}],["746134534943557e",{"2":{"258":1}}],["7460216f",{"2":{"256":1}}],["7468008914093891",{"2":{"169":4}}],["746801",{"2":{"166":1}}],["74694291453392",{"2":{"169":4}}],["746943",{"2":{"166":1}}],["705188f",{"2":{"256":1}}],["7054554",{"2":{"136":1}}],["701272574869138e",{"2":{"258":1}}],["7014895f",{"2":{"256":1}}],["701388f",{"2":{"256":1}}],["7018592",{"2":{"134":1}}],["7063366256061072e",{"2":{"258":1}}],["706454964950166e",{"2":{"258":1}}],["7066754884979796e",{"2":{"258":1}}],["7066755f",{"2":{"256":1}}],["706114f",{"2":{"256":1}}],["7061304f",{"2":{"256":1}}],["7065893f",{"2":{"256":1}}],["7062",{"2":{"249":1}}],["702082171712219e",{"2":{"258":1}}],["7020614f",{"2":{"256":1}}],["7024",{"2":{"249":1}}],["7023s\\ttraining",{"2":{"233":1}}],["707818629296995e",{"2":{"258":1}}],["7076144f",{"2":{"256":1}}],["7070",{"2":{"249":1}}],["7074s\\ttraining",{"2":{"233":1}}],["707534",{"2":{"166":1}}],["70",{"2":{"225":2,"233":1}}],["70866144",{"2":{"213":2}}],["7089496198123055",{"2":{"164":2}}],["70370",{"2":{"233":1}}],["70390546",{"2":{"120":1}}],["70330536",{"2":{"80":1}}],["7001468",{"2":{"213":1}}],["7001",{"2":{"174":1,"244":1}}],["700",{"2":{"107":1}}],["7215132149372895e",{"2":{"258":1}}],["721716f",{"2":{"256":1}}],["7223255048506476e",{"2":{"258":1}}],["722093f",{"2":{"256":1}}],["7270469192383486e",{"2":{"258":1}}],["7275808177650394e",{"2":{"258":1}}],["7272497f",{"2":{"256":1}}],["7274678f",{"2":{"256":1}}],["7273",{"2":{"249":1}}],["7256763730982073e",{"2":{"258":1}}],["7254235f",{"2":{"256":1}}],["7252903f",{"2":{"106":8}}],["7200",{"2":{"249":1}}],["72",{"2":{"191":3,"233":3}}],["7237308297251812",{"2":{"169":1}}],["72373",{"2":{"166":1}}],["723731",{"2":{"166":2}}],["7262075",{"2":{"136":1}}],["724099",{"2":{"162":1}}],["72471225",{"2":{"136":1}}],["72428167",{"2":{"120":1}}],["729900034517088e",{"2":{"258":1}}],["72932799938929e",{"2":{"258":1}}],["729075f",{"2":{"256":1}}],["72908926",{"2":{"137":1}}],["7296474",{"2":{"136":1}}],["7292344",{"2":{"134":1}}],["729",{"2":{"80":6}}],["795901996156597e",{"2":{"258":1}}],["795943im",{"2":{"162":1}}],["795865f",{"2":{"256":1}}],["7966631f",{"2":{"256":1}}],["79656f",{"2":{"80":1}}],["792073158528541e",{"2":{"258":1}}],["7922774f",{"2":{"256":1}}],["79250443",{"2":{"136":1}}],["7978788815051777e",{"2":{"258":1}}],["7977",{"2":{"249":1}}],["797500e",{"2":{"208":1}}],["7914s\\ttraining",{"2":{"233":1}}],["7911112",{"2":{"136":1}}],["7990667",{"2":{"216":1}}],["7942",{"2":{"249":1}}],["794352e",{"2":{"208":1}}],["794908",{"2":{"136":1}}],["79",{"2":{"191":2,"233":4,"249":1}}],["798027",{"2":{"136":1}}],["793842635442057e",{"2":{"258":1}}],["793578080352111e",{"2":{"258":1}}],["79361f",{"2":{"256":1}}],["793325f",{"2":{"256":1}}],["793159",{"2":{"166":1}}],["7939677f",{"2":{"106":1}}],["793463f",{"2":{"80":1}}],["786641433142276e",{"2":{"258":1}}],["786119776188296e",{"2":{"258":1}}],["7867932f",{"2":{"256":1}}],["78538332196153e",{"2":{"258":1}}],["785060342285634e",{"2":{"258":1}}],["7856f",{"2":{"256":1}}],["7851914f",{"2":{"256":1}}],["7838948f",{"2":{"256":1}}],["7834",{"2":{"249":1}}],["7834251",{"2":{"136":1}}],["7871654195867056e",{"2":{"258":1}}],["787159",{"2":{"134":1}}],["78722f",{"2":{"256":1}}],["782299164798104e",{"2":{"258":1}}],["78226817",{"2":{"129":1}}],["782373255445095e",{"2":{"258":1}}],["7823s\\ttraining",{"2":{"235":1}}],["7824436f",{"2":{"256":1}}],["7821406f",{"2":{"256":1}}],["7891",{"2":{"249":1}}],["7894s\\ttraining",{"2":{"233":1}}],["7897024",{"2":{"213":1}}],["78818554",{"2":{"213":1}}],["78814f",{"2":{"106":1}}],["78",{"2":{"191":2,"225":5,"233":4,"235":1}}],["7817315740767116",{"2":{"169":1}}],["7811481",{"2":{"134":1}}],["78000563",{"2":{"130":1}}],["7808904",{"2":{"74":2}}],["784216390754152e",{"2":{"258":1}}],["784768",{"2":{"136":1,"137":1}}],["784",{"2":{"72":6,"223":1,"231":1,"236":6}}],["7",{"2":{"24":9,"53":5,"65":4,"80":17,"106":10,"134":5,"136":2,"137":5,"141":1,"168":3,"174":1,"175":4,"182":14,"184":4,"191":2,"192":1,"208":1,"210":4,"213":5,"218":4,"225":2,"226":4,"233":6,"235":1,"237":4,"244":4,"246":4,"249":6,"251":1,"256":81,"258":80,"260":1}}],["^2",{"2":{"255":2,"256":1}}],["^",{"2":{"24":9,"59":1,"81":1,"106":1,"107":1,"254":2}}],["λβ",{"2":{"22":1}}],["α",{"2":{"22":3,"53":3,"206":2,"250":3}}],["σ=identity",{"2":{"24":1}}],["σ",{"2":{"18":10,"20":5,"21":3,"23":3,"24":7,"152":3}}],["≤",{"2":{"9":2,"49":2,"70":2}}],["ys",{"2":{"245":6}}],["y∈",{"2":{"243":1}}],["ylabel=",{"2":{"213":1,"217":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["y=x2−2x",{"2":{"213":1}}],["y=x−e",{"2":{"24":1,"72":1}}],["yᵢ",{"2":{"107":2}}],["yes",{"2":{"76":1}}],["year",{"2":{"74":2}}],["yet",{"2":{"44":1,"106":1}}],["y3",{"2":{"65":1}}],["y2",{"2":{"65":1,"248":2}}],["y1",{"2":{"65":1,"248":2}}],["yann",{"2":{"53":1}}],["y+ϵ",{"2":{"53":1}}],["yi∈rm",{"2":{"174":1}}],["yi",{"2":{"53":2,"174":1}}],["y~=",{"2":{"53":2}}],["y~",{"2":{"53":4}}],["y^2+y∗max",{"2":{"53":1}}],["y^−y∗log⁡",{"2":{"53":1}}],["y^−y",{"2":{"53":1}}],["y^",{"2":{"53":7}}],["y^+ϵ",{"2":{"53":3}}],["ŷ",{"2":{"53":11,"134":2,"135":2,"136":2,"181":2,"182":3}}],["yuxin",{"2":{"24":1}}],["y",{"2":{"9":3,"19":2,"20":2,"24":1,"28":5,"49":3,"53":63,"54":3,"55":6,"58":1,"59":10,"65":8,"66":2,"68":9,"70":10,"71":6,"72":4,"80":1,"81":4,"106":11,"107":2,"110":2,"117":4,"129":3,"130":4,"134":7,"135":7,"154":5,"164":2,"174":11,"178":4,"179":5,"180":3,"181":12,"182":5,"187":6,"189":2,"190":4,"206":4,"213":5,"217":5,"221":4,"222":2,"224":2,"225":2,"229":6,"232":2,"233":2,"243":7,"245":1,"248":1,"254":6,"256":3}}],["yoshua",{"2":{"9":2}}],["yourself",{"2":{"203":1}}],["your",{"0":{"143":1},"2":{"44":2,"47":1,"59":2,"72":3,"90":1,"108":2,"110":1,"127":1,"129":2,"130":2,"131":3,"133":1,"134":2,"147":1,"151":1,"155":1,"157":2,"183":1,"197":2}}],["you",{"2":{"2":1,"11":2,"33":2,"44":1,"47":1,"48":1,"52":3,"59":5,"61":3,"74":2,"79":3,"80":1,"81":1,"82":1,"89":1,"102":1,"106":2,"107":1,"108":3,"109":2,"110":2,"111":1,"121":1,"127":1,"129":8,"130":6,"131":5,"133":5,"134":4,"135":1,"143":2,"147":1,"150":1,"151":2,"157":2,"162":1,"164":1,"166":1,"167":3,"168":1,"170":1,"176":1,"180":1,"183":1,"193":2,"197":2,"202":1,"203":1,"214":1,"222":1,"233":2,"243":1,"256":1}}],["hₓ",{"2":{"254":4}}],["h₊",{"2":{"254":4}}],["h12",{"2":{"254":12}}],["h11",{"2":{"254":12}}],["h22",{"2":{"254":12}}],["hmc",{"2":{"249":2}}],["hcat",{"2":{"243":1,"249":1}}],["hn",{"2":{"222":2}}],["hnew",{"2":{"69":6}}],["hnew=activation",{"2":{"69":1}}],["hnew=",{"2":{"69":1}}],["hypernet",{"0":{"222":1,"223":1},"2":{"222":4,"223":1}}],["hypernetwork",{"0":{"219":1},"1":{"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1}}],["hh",{"2":{"69":6,"105":1}}],["h",{"2":{"68":6,"69":3,"73":4,"187":1,"229":1,"254":14}}],["home",{"2":{"131":1}}],["hosts",{"2":{"82":1}}],["hold",{"2":{"76":1,"80":1}}],["hot",{"2":{"55":1}}],["how",{"0":{"61":1,"148":1},"1":{"149":1,"150":1,"151":1,"152":1,"153":1,"154":1},"2":{"46":1,"53":2,"55":1,"65":2,"69":3,"72":8,"77":1,"79":1,"80":1,"106":1,"107":1,"108":1,"110":2,"119":1,"129":1,"131":1,"133":1,"137":1,"151":1,"155":2,"164":1,"166":2,"172":1,"185":1,"194":1,"204":1,"222":1,"230":1,"247":1,"249":1,"250":1}}],["however",{"2":{"13":1,"16":1,"55":1,"59":1,"65":1,"75":1,"80":1,"89":1,"104":1,"111":1,"137":1,"146":1,"161":1,"168":2,"170":1,"202":1,"204":3,"230":1,"238":1,"254":1,"256":2}}],["hutchinson",{"0":{"138":1},"1":{"139":1,"140":1,"141":1},"2":{"138":2,"139":3,"140":2,"141":8}}],["huber",{"2":{"53":1}}],["huberloss",{"2":{"53":3}}],["human",{"2":{"9":2}}],["hamiltonian",{"2":{"249":2}}],["hat",{"2":{"209":2}}],["had",{"2":{"170":1}}],["hadsell",{"2":{"53":1}}],["hand",{"2":{"166":1}}],["handles",{"2":{"77":1,"180":1}}],["handle",{"2":{"69":1,"124":1,"164":1,"230":1,"249":1}}],["handling",{"2":{"54":1,"57":1,"127":1,"149":1,"159":1}}],["happened",{"2":{"149":1}}],["happening",{"2":{"135":1,"154":1}}],["happens",{"2":{"109":1,"133":1,"154":1,"159":1}}],["harder",{"2":{"77":1,"108":1}}],["hardware",{"2":{"76":1}}],["hard",{"2":{"16":1,"53":1,"110":1,"129":1,"175":2,"184":2,"192":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"260":1}}],["has",{"2":{"13":3,"31":1,"44":2,"59":1,"66":1,"68":3,"73":1,"75":1,"77":1,"92":2,"95":6,"99":1,"102":7,"103":4,"104":1,"105":1,"108":1,"109":4,"124":1,"130":1,"137":1,"142":1,"145":1,"165":1,"166":2,"170":1,"249":1,"252":1,"256":1}}],["having",{"2":{"11":1,"34":1,"59":1}}],["haven",{"2":{"44":1}}],["have",{"2":{"9":1,"28":1,"30":2,"66":1,"69":1,"71":1,"77":2,"84":1,"88":3,"89":1,"95":1,"99":2,"102":3,"103":1,"104":2,"105":1,"109":1,"110":2,"111":2,"129":1,"130":1,"131":1,"133":2,"134":1,"135":1,"141":1,"147":1,"149":1,"167":1,"168":1,"169":1,"170":3,"179":3,"193":2,"197":1,"208":1,"223":1,"249":1,"250":2,"254":1}}],["https",{"2":{"9":1,"61":1,"69":1,"71":1,"74":1,"172":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"254":2,"260":1}}],["historical",{"2":{"230":1}}],["historically",{"2":{"30":1}}],["hidden",{"2":{"69":43,"179":6,"180":3,"241":11,"244":2,"249":1}}],["hierarchically",{"2":{"59":1}}],["hinge",{"2":{"53":2}}],["hingeloss",{"2":{"53":2}}],["hinton",{"2":{"24":1}}],["highlight",{"2":{"146":1}}],["highly",{"2":{"53":1}}],["higher",{"2":{"73":1,"136":1,"149":1}}],["highest",{"2":{"21":1,"88":1,"149":1,"250":1}}],["high",{"2":{"65":1,"164":1,"166":1,"172":1}}],["hit",{"2":{"13":1}}],["hi",{"2":{"9":2}}],["heatmap",{"2":{"245":1}}],["head",{"2":{"77":1,"179":1}}],["heavily",{"2":{"58":1}}],["heavy",{"2":{"11":1}}],["height",{"2":{"66":1}}],["hence",{"2":{"44":1,"47":1,"139":1,"169":1,"170":1,"191":1,"194":1,"196":1,"208":1,"230":1,"256":1}}],["helps",{"2":{"230":1}}],["help",{"2":{"30":1,"31":1,"32":1,"33":2,"53":2,"57":1,"59":1,"65":6,"66":2,"68":3,"69":2,"71":1,"72":4,"73":1,"108":1,"151":1}}],["helpers",{"0":{"26":1},"1":{"27":1,"28":1,"29":1},"2":{"54":1}}],["helper",{"0":{"25":1,"38":1,"71":1,"189":1},"2":{"52":1,"57":1,"250":1}}],["here",{"2":{"30":1,"33":1,"52":2,"54":1,"59":3,"75":1,"80":1,"88":1,"114":1,"130":1,"134":3,"135":1,"136":1,"143":1,"154":1,"155":1,"166":3,"172":2,"208":1,"230":1,"241":1,"248":1,"249":2}}],["hessian",{"2":{"9":1}}],["he",{"2":{"9":2,"24":1}}],["5g",{"2":{"217":1}}],["5gb",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["5fs",{"2":{"225":1}}],["5f",{"2":{"182":3,"233":2}}],["5f0",{"2":{"120":1,"123":1,"141":1,"181":1,"248":12}}],["542232323009273e",{"2":{"258":1}}],["5424134f",{"2":{"256":1}}],["5424696",{"2":{"213":1}}],["545989582085756e",{"2":{"258":1}}],["545392813903837e",{"2":{"258":1}}],["545845f",{"2":{"256":1}}],["5483473f",{"2":{"256":1}}],["548101+0",{"2":{"162":1}}],["54333233557773e",{"2":{"258":1}}],["543387f",{"2":{"256":1}}],["5438",{"2":{"249":1}}],["543880e",{"2":{"208":1}}],["540",{"2":{"236":8}}],["546639061704991e",{"2":{"258":1}}],["5466956f",{"2":{"256":1}}],["5467s\\ttraining",{"2":{"233":1}}],["54630",{"2":{"182":1}}],["54",{"2":{"225":1,"233":1}}],["54s",{"2":{"191":2}}],["54452",{"2":{"167":1}}],["5476424498276177",{"2":{"169":4}}],["547642",{"2":{"166":1}}],["5×3",{"2":{"166":8,"167":1}}],["5×5",{"2":{"9":1,"166":3}}],["5866286648817994e",{"2":{"258":1}}],["5865919f",{"2":{"256":1}}],["5865265807660498",{"2":{"164":2}}],["5831630774748768e",{"2":{"258":1}}],["5834",{"2":{"249":1}}],["58443632322203e",{"2":{"258":1}}],["584373f",{"2":{"256":1}}],["5840",{"2":{"249":1}}],["5810668587742027e",{"2":{"258":1}}],["5819551063395286e",{"2":{"258":1}}],["5811183f",{"2":{"256":1}}],["5812",{"2":{"249":1}}],["5821",{"2":{"249":1}}],["58263564",{"2":{"216":1}}],["5826772",{"2":{"213":2}}],["5800279434907896e",{"2":{"258":1}}],["5807853f",{"2":{"256":1}}],["5806",{"2":{"249":1}}],["580913592300915e",{"2":{"258":1}}],["5809",{"2":{"249":1}}],["5852",{"2":{"249":1}}],["58",{"2":{"233":3,"235":1}}],["58s",{"2":{"191":1}}],["58721",{"2":{"166":1}}],["58720636",{"2":{"166":1}}],["58720636f0",{"2":{"166":1}}],["587206",{"2":{"166":2}}],["5878153",{"2":{"136":1}}],["5879354f",{"2":{"106":1}}],["523079677439581e",{"2":{"258":1}}],["523252587338435e",{"2":{"258":1}}],["523307f",{"2":{"256":1}}],["5299496f",{"2":{"256":1}}],["529878e",{"2":{"208":1}}],["5289934987238694e",{"2":{"258":1}}],["5284858218362233e",{"2":{"258":1}}],["5287605465845612e",{"2":{"258":1}}],["5285226f",{"2":{"256":1}}],["5286476f",{"2":{"256":1}}],["5281",{"2":{"74":2}}],["5218094f",{"2":{"256":1}}],["5215s\\ttraining",{"2":{"233":1}}],["5210742834996898",{"2":{"164":2}}],["5255s\\ttraining",{"2":{"233":1}}],["52",{"2":{"225":2}}],["5269332221808443e",{"2":{"258":1}}],["5269046100464052e",{"2":{"258":1}}],["526848f",{"2":{"256":1}}],["526762f",{"2":{"256":1}}],["5267549745189349",{"2":{"164":2}}],["52628386",{"2":{"213":1}}],["5201509365519844e",{"2":{"258":1}}],["52024",{"2":{"213":1}}],["52075",{"2":{"182":1}}],["527077759058019e",{"2":{"258":1}}],["52711f",{"2":{"256":1}}],["52717",{"2":{"136":1}}],["527559",{"2":{"213":2}}],["5221656",{"2":{"174":1}}],["522139",{"2":{"137":1}}],["5246957326643204e",{"2":{"258":1}}],["5242485585335465e",{"2":{"258":1}}],["524789f",{"2":{"256":1}}],["524781",{"2":{"137":1}}],["5243796f",{"2":{"256":1}}],["5245s\\ttraining",{"2":{"233":1}}],["524008",{"2":{"166":1}}],["574516952178291e",{"2":{"258":1}}],["5745494f",{"2":{"256":1}}],["57567456454027e",{"2":{"258":1}}],["575611f",{"2":{"256":1}}],["572312f",{"2":{"256":1}}],["5791521720241938e",{"2":{"258":1}}],["5798567f",{"2":{"256":1}}],["5792652f",{"2":{"256":1}}],["5794",{"2":{"249":1}}],["5701315f",{"2":{"256":1}}],["5718",{"2":{"254":2}}],["578374863",{"2":{"244":2}}],["57882756",{"2":{"213":1}}],["57",{"2":{"233":5}}],["57s",{"2":{"191":1}}],["577181",{"2":{"166":1}}],["5779067839062536",{"2":{"164":2}}],["5776367e",{"2":{"137":1}}],["5776052f0",{"2":{"53":1}}],["57671577",{"2":{"131":1}}],["568936985126093e",{"2":{"258":1}}],["5680085614623662",{"2":{"164":2}}],["569878618596989e",{"2":{"258":1}}],["56917f",{"2":{"256":1}}],["563256531943786e",{"2":{"258":1}}],["564313010158575e",{"2":{"258":1}}],["5647335f",{"2":{"256":1}}],["564096f",{"2":{"256":1}}],["56245056062654e",{"2":{"258":1}}],["562572f",{"2":{"256":1}}],["5627227",{"2":{"134":1}}],["5652245538656208e",{"2":{"258":1}}],["565299f",{"2":{"256":1}}],["5651116f",{"2":{"256":1}}],["56538",{"2":{"182":1}}],["5663",{"2":{"249":1}}],["56s",{"2":{"191":1}}],["56",{"2":{"191":2,"225":1,"233":1}}],["5673",{"2":{"249":1}}],["5675557670686644",{"2":{"169":1}}],["56779",{"2":{"166":1}}],["567794",{"2":{"166":2}}],["5609365773470854e",{"2":{"258":1}}],["5607555f",{"2":{"256":1}}],["560",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["56055",{"2":{"166":1}}],["56069607",{"2":{"106":1}}],["561137f",{"2":{"106":1}}],["5943811747557506e",{"2":{"258":1}}],["594261",{"2":{"137":1}}],["591863239591659e",{"2":{"258":1}}],["591823f",{"2":{"256":1}}],["5962072188025316e",{"2":{"258":1}}],["5967776009313548e",{"2":{"258":1}}],["596387548933425e",{"2":{"258":1}}],["5961747f",{"2":{"256":1}}],["596922f",{"2":{"256":1}}],["5964511f",{"2":{"256":1}}],["590648801920449e",{"2":{"258":1}}],["590853f",{"2":{"256":1}}],["5905511",{"2":{"213":2}}],["598751317803042e",{"2":{"258":1}}],["598519f",{"2":{"256":1}}],["598505080",{"2":{"244":2}}],["5983",{"2":{"249":1}}],["595907479349758e",{"2":{"258":1}}],["595533f",{"2":{"256":1}}],["5957945f",{"2":{"256":1}}],["5956",{"2":{"249":1}}],["595081787143777e",{"2":{"258":1}}],["5950818f",{"2":{"256":1}}],["5950",{"2":{"249":1}}],["59543324",{"2":{"216":1}}],["59259",{"2":{"235":1}}],["59s",{"2":{"191":2}}],["59",{"2":{"137":1,"191":2,"225":8}}],["5979974",{"2":{"134":1}}],["5975106",{"2":{"80":1}}],["5991102498432955e",{"2":{"258":1}}],["599139f",{"2":{"256":1}}],["5991",{"2":{"249":1}}],["5993s\\ttraining",{"2":{"233":1}}],["5999714",{"2":{"120":1}}],["5994072",{"2":{"106":1}}],["5994074",{"2":{"106":1}}],["535156190517071e",{"2":{"258":1}}],["534265824133726e",{"2":{"258":1}}],["5344176f",{"2":{"256":1}}],["5345716",{"2":{"80":1}}],["538919617630628e",{"2":{"258":1}}],["5386",{"2":{"249":1}}],["5381101016248151",{"2":{"164":2}}],["5302475749296935e",{"2":{"258":1}}],["5302s\\ttraining",{"2":{"233":1}}],["5301544f",{"2":{"256":1}}],["5309s\\ttraining",{"2":{"233":1}}],["53s",{"2":{"191":1}}],["53213",{"2":{"182":1}}],["532213im",{"2":{"162":1}}],["5315s\\ttraining",{"2":{"233":1}}],["5310851",{"2":{"174":1}}],["53144f",{"2":{"80":1}}],["5364523507647356e",{"2":{"258":1}}],["53642f",{"2":{"256":1}}],["536743e",{"2":{"141":1}}],["5367613",{"2":{"136":1}}],["5365157",{"2":{"137":1}}],["53",{"2":{"110":1,"225":1,"249":1}}],["5378918",{"2":{"80":1}}],["5d",{"2":{"73":1,"208":1,"244":2,"257":1}}],["510291900762605e",{"2":{"258":1}}],["5103287f",{"2":{"256":1}}],["51066",{"2":{"162":1}}],["5147839778305374e",{"2":{"258":1}}],["5149867f",{"2":{"256":1}}],["5144s\\ttraining",{"2":{"233":1}}],["5175691242038057e",{"2":{"258":1}}],["517727752818883e",{"2":{"258":1}}],["517981f",{"2":{"256":1}}],["5178",{"2":{"249":1}}],["518635550652207e",{"2":{"258":1}}],["518214753706244e",{"2":{"258":1}}],["5182915f",{"2":{"256":1}}],["518311f",{"2":{"256":1}}],["518063f",{"2":{"256":1}}],["5181s\\ttraining",{"2":{"235":1}}],["51852",{"2":{"233":1}}],["5199992f",{"2":{"256":1}}],["5193s\\ttraining",{"2":{"233":1}}],["51968503",{"2":{"213":2}}],["515377077039578e",{"2":{"258":1}}],["515377f",{"2":{"256":1}}],["5154s\\ttraining",{"2":{"233":1}}],["51524514",{"2":{"174":1}}],["511083048774223e",{"2":{"258":1}}],["5118s\\ttraining",{"2":{"233":1}}],["51165134",{"2":{"216":1}}],["5161818924709434e",{"2":{"258":1}}],["5162637734446107e",{"2":{"258":1}}],["516869255986725e",{"2":{"258":1}}],["516353f",{"2":{"256":1}}],["516548f",{"2":{"256":1}}],["5167208",{"2":{"216":1}}],["5164032",{"2":{"174":1}}],["51s",{"2":{"191":3}}],["51",{"2":{"72":3,"109":1,"191":1,"208":2,"217":1}}],["5120025f",{"2":{"256":1}}],["512",{"2":{"59":3,"243":1}}],["507221923617522e",{"2":{"258":1}}],["507182f",{"2":{"256":1}}],["5079s\\ttraining",{"2":{"233":1}}],["50798f",{"2":{"80":1}}],["5053",{"2":{"249":1}}],["501",{"2":{"244":1}}],["5016017",{"2":{"136":1}}],["5087s\\ttraining",{"2":{"233":1}}],["5022s\\ttraining",{"2":{"233":1}}],["5021s\\ttraining",{"2":{"233":1}}],["502312852219817",{"2":{"53":1}}],["50381",{"2":{"182":1}}],["503186",{"2":{"167":1}}],["5031421",{"2":{"106":1}}],["506485",{"2":{"167":1}}],["509345",{"2":{"162":1}}],["500368f",{"2":{"256":1}}],["500512351761415e",{"2":{"258":1}}],["5005",{"2":{"249":1}}],["5000×30×1",{"2":{"249":1}}],["50000",{"2":{"244":100}}],["5000",{"2":{"244":1,"249":5}}],["5009s\\ttraining",{"2":{"233":1}}],["5009556",{"2":{"216":1}}],["50074",{"2":{"182":1}}],["500132635249613e",{"2":{"258":1}}],["5001",{"2":{"174":1,"244":1}}],["500",{"2":{"107":1,"178":2,"244":1}}],["50",{"2":{"66":2,"72":3,"191":13,"217":1,"225":115,"249":1}}],["5∗δ",{"2":{"53":1}}],["5∗|y−y^|2if",{"2":{"53":1}}],["55861693391912e",{"2":{"258":1}}],["5584899f",{"2":{"256":1}}],["554694f",{"2":{"256":1}}],["55476f",{"2":{"80":1}}],["557863583954096e",{"2":{"258":1}}],["557661f",{"2":{"256":1}}],["5571054f",{"2":{"256":1}}],["550622234496614e",{"2":{"258":1}}],["550835744282355e",{"2":{"258":1}}],["550899f",{"2":{"256":1}}],["550440848984027e",{"2":{"258":1}}],["5501",{"2":{"244":1}}],["5567809000901426e",{"2":{"258":1}}],["556483098836061e",{"2":{"258":1}}],["5564316f",{"2":{"256":1}}],["5564",{"2":{"249":1}}],["55658394",{"2":{"213":1}}],["5511811",{"2":{"213":2}}],["55556",{"2":{"233":2}}],["55580",{"2":{"182":1}}],["5553797706980106",{"2":{"169":1}}],["559180457263342e",{"2":{"258":1}}],["5590439484819587e",{"2":{"258":1}}],["5590551",{"2":{"213":2}}],["559302f",{"2":{"256":1}}],["5595588f",{"2":{"256":1}}],["5595843665394066",{"2":{"169":1}}],["5594571",{"2":{"213":1}}],["55989707",{"2":{"137":1}}],["5532",{"2":{"166":1}}],["553631",{"2":{"136":1}}],["5530689",{"2":{"136":1}}],["55",{"2":{"53":1,"225":1,"235":1}}],["5where",{"2":{"53":1}}],["5",{"2":{"9":8,"24":9,"32":2,"46":4,"53":15,"59":11,"66":2,"72":6,"74":1,"80":16,"86":1,"102":1,"106":5,"115":5,"123":1,"124":1,"136":1,"137":2,"162":6,"166":29,"167":1,"168":3,"172":3,"174":6,"175":3,"182":14,"184":3,"188":14,"191":2,"205":1,"206":1,"210":3,"213":8,"217":2,"218":3,"225":2,"226":3,"233":5,"234":2,"235":1,"236":40,"237":3,"239":1,"246":3,"248":4,"249":19,"250":1,"254":4,"255":2,"256":772,"258":739,"259":3}}],["6e",{"2":{"208":1}}],["6d",{"2":{"174":1}}],["665271012258093e",{"2":{"258":1}}],["665241",{"2":{"53":10}}],["661572499622105e",{"2":{"258":1}}],["661277f",{"2":{"256":1}}],["66",{"2":{"256":1}}],["664669469933737e",{"2":{"258":1}}],["664525f",{"2":{"256":1}}],["6642563709240284",{"2":{"164":2}}],["66852736",{"2":{"216":1}}],["667572",{"2":{"213":1}}],["663826f",{"2":{"256":1}}],["663036",{"2":{"167":1}}],["6637242",{"2":{"136":1}}],["6661781481641573e",{"2":{"258":1}}],["6663824f",{"2":{"256":1}}],["6663244",{"2":{"136":1}}],["66667",{"2":{"233":15,"235":4}}],["666682",{"2":{"136":1}}],["666965",{"2":{"162":1}}],["682348957298774e",{"2":{"258":1}}],["6821777f",{"2":{"256":1}}],["688422825954933e",{"2":{"258":1}}],["68817804003514e",{"2":{"258":1}}],["688554319983476e",{"2":{"258":1}}],["688665",{"2":{"137":1}}],["6886741",{"2":{"120":1}}],["680361289786686e",{"2":{"258":1}}],["680182063003482e",{"2":{"258":1}}],["680748",{"2":{"120":1}}],["6860275f",{"2":{"256":1}}],["6867142",{"2":{"137":1}}],["684051595886362e",{"2":{"258":1}}],["684031f",{"2":{"256":1}}],["6849014f",{"2":{"256":1}}],["684605982778897e",{"2":{"258":1}}],["6846",{"2":{"249":1}}],["6894352f",{"2":{"256":1}}],["6897",{"2":{"249":1}}],["683416848824828e",{"2":{"258":1}}],["68343f",{"2":{"256":1}}],["683541977943642e",{"2":{"258":1}}],["6835862658147272e",{"2":{"258":1}}],["683316787181636e",{"2":{"258":1}}],["683339f",{"2":{"256":1}}],["6837905f",{"2":{"256":1}}],["6830945313415872",{"2":{"169":1}}],["68",{"2":{"225":8,"233":2,"235":2,"249":1}}],["681728",{"2":{"167":1}}],["6854445f",{"2":{"256":1}}],["6852",{"2":{"249":1}}],["6859683",{"2":{"213":1}}],["6850394",{"2":{"213":2}}],["685079",{"2":{"166":1}}],["6853896",{"2":{"137":1}}],["687769898238739e",{"2":{"258":1}}],["687749f",{"2":{"256":1}}],["6878565f",{"2":{"256":1}}],["687611",{"2":{"162":1}}],["687264",{"2":{"137":1}}],["687216",{"2":{"125":2}}],["6958564474267366e",{"2":{"258":1}}],["69589s\\ttraining",{"2":{"225":1}}],["693981873362311e",{"2":{"258":1}}],["693961f",{"2":{"256":1}}],["693282638069095e",{"2":{"258":1}}],["6933034f",{"2":{"256":1}}],["6933",{"2":{"249":1}}],["692567044719153e",{"2":{"258":1}}],["692474f",{"2":{"256":1}}],["6920981",{"2":{"213":1}}],["699498286513317e",{"2":{"258":1}}],["699267030059231e",{"2":{"258":1}}],["699296f",{"2":{"256":1}}],["699794f",{"2":{"256":1}}],["6965826080393775e",{"2":{"258":1}}],["696958f",{"2":{"256":1}}],["6964543f",{"2":{"256":1}}],["6960375f",{"2":{"256":1}}],["6960",{"2":{"249":1}}],["697279636463018e",{"2":{"258":1}}],["6970",{"2":{"249":1}}],["697709",{"2":{"162":1}}],["6943",{"2":{"249":1}}],["69472f",{"2":{"80":1}}],["6904922",{"2":{"213":1}}],["6909446526419574",{"2":{"164":2}}],["6983237",{"2":{"213":1}}],["69s",{"2":{"191":1}}],["6913705",{"2":{"134":1}}],["69",{"2":{"115":1,"191":1,"233":4}}],["6184361279588816e",{"2":{"258":1}}],["618323f",{"2":{"256":1}}],["6189",{"2":{"249":1}}],["6134",{"2":{"249":1}}],["6133",{"2":{"249":1}}],["6138451",{"2":{"106":1}}],["61384505",{"2":{"106":1}}],["611637545185293e",{"2":{"258":1}}],["611313f",{"2":{"256":1}}],["6117",{"2":{"249":1}}],["61155146",{"2":{"213":1}}],["611584190904272",{"2":{"164":2}}],["6150s\\ttraining",{"2":{"233":1}}],["61451813834961e",{"2":{"258":1}}],["614374f",{"2":{"256":1}}],["6143286",{"2":{"120":1}}],["6141",{"2":{"249":1}}],["61417323",{"2":{"213":2}}],["61",{"2":{"191":3,"225":1}}],["6124",{"2":{"249":1}}],["61271",{"2":{"182":1}}],["6120",{"2":{"9":1}}],["6101906778665383e",{"2":{"258":1}}],["6101341f",{"2":{"256":1}}],["610",{"2":{"137":1}}],["654624220598772e",{"2":{"258":1}}],["6541584",{"2":{"106":1}}],["6541586",{"2":{"106":1}}],["658730314649844e",{"2":{"258":1}}],["658559f",{"2":{"256":1}}],["656384109725839e",{"2":{"258":1}}],["656613f",{"2":{"106":1}}],["659961f",{"2":{"256":1}}],["6519148172840214e",{"2":{"258":1}}],["6517353396471644e",{"2":{"258":1}}],["6517703f",{"2":{"256":1}}],["65178",{"2":{"136":1}}],["6518059f",{"2":{"256":1}}],["65327495172892e",{"2":{"258":1}}],["653275f",{"2":{"256":1}}],["6535434",{"2":{"213":2}}],["6571946",{"2":{"216":1}}],["657670184695808",{"2":{"164":2}}],["65",{"2":{"191":1,"225":13}}],["6556360448565952",{"2":{"169":1}}],["655881",{"2":{"166":1}}],["655999",{"2":{"162":1}}],["650786293836533e",{"2":{"258":1}}],["650549979609595e",{"2":{"258":1}}],["6505835f",{"2":{"256":1}}],["6504766f",{"2":{"256":1}}],["650369f",{"2":{"256":1}}],["6501",{"2":{"244":1}}],["650",{"2":{"72":3}}],["636686184525886e",{"2":{"258":1}}],["63661867048385e",{"2":{"258":1}}],["636922941018939e",{"2":{"258":1}}],["636373650307874e",{"2":{"258":1}}],["636353f",{"2":{"256":1}}],["636308",{"2":{"106":1}}],["636593f",{"2":{"256":1}}],["6365785f",{"2":{"256":1}}],["633592858151902e",{"2":{"258":1}}],["633276218779253e",{"2":{"258":1}}],["6336444756196498e",{"2":{"258":1}}],["6334916f",{"2":{"256":1}}],["6334403f",{"2":{"256":1}}],["633148f",{"2":{"256":1}}],["63",{"2":{"225":1}}],["6389992",{"2":{"213":1}}],["63838756",{"2":{"136":1}}],["63034",{"2":{"182":1}}],["6349",{"2":{"249":1}}],["6340362477836592",{"2":{"169":1}}],["634549",{"2":{"162":1}}],["63738012963482e",{"2":{"258":1}}],["6371475f",{"2":{"256":1}}],["637024f",{"2":{"256":1}}],["6370531107315014",{"2":{"164":2}}],["6372185",{"2":{"137":1}}],["63795453",{"2":{"106":1}}],["6323408",{"2":{"120":1}}],["6232174f",{"2":{"256":1}}],["626717760612538e",{"2":{"258":1}}],["6265903f",{"2":{"256":1}}],["6269556f",{"2":{"256":1}}],["62625877100596",{"2":{"166":1}}],["621050923994785e",{"2":{"258":1}}],["621489496172741e",{"2":{"258":1}}],["6214",{"2":{"249":1}}],["621958",{"2":{"166":1}}],["62963",{"2":{"233":2}}],["62",{"2":{"225":23}}],["6202136",{"2":{"213":1}}],["622773f",{"2":{"256":1}}],["6220472",{"2":{"213":2}}],["62243",{"2":{"182":1}}],["622865",{"2":{"167":1}}],["627620252854646e",{"2":{"258":1}}],["627208479078838e",{"2":{"258":1}}],["62755f",{"2":{"256":1}}],["627",{"2":{"175":1}}],["6246659472613986e",{"2":{"258":1}}],["6247295f",{"2":{"256":1}}],["624383955429775",{"2":{"169":1}}],["624384",{"2":{"166":1}}],["62450767",{"2":{"106":1}}],["625",{"2":{"53":1}}],["603697f",{"2":{"256":1}}],["607247478735757e",{"2":{"258":1}}],["6072877f",{"2":{"256":1}}],["6076053f0",{"2":{"53":2}}],["605136379971683e",{"2":{"258":1}}],["605369f",{"2":{"256":1}}],["60553",{"2":{"182":1}}],["606974450729679e",{"2":{"258":1}}],["6069952f",{"2":{"256":1}}],["606376723386703e",{"2":{"258":1}}],["6063779055314245e",{"2":{"258":1}}],["6065078f",{"2":{"256":1}}],["6064619",{"2":{"106":1}}],["604004060733888e",{"2":{"258":1}}],["604292f",{"2":{"256":1}}],["6049256",{"2":{"137":1}}],["609561157579041e",{"2":{"258":1}}],["609181646762502e",{"2":{"258":1}}],["6093584f",{"2":{"256":1}}],["609236f",{"2":{"256":1}}],["609",{"2":{"218":1}}],["60",{"2":{"191":2,"225":2,"249":1}}],["608428im",{"2":{"162":1}}],["600705119780681e",{"2":{"258":1}}],["6007382227096824e",{"2":{"258":1}}],["6007382f",{"2":{"256":1}}],["600574f",{"2":{"256":1}}],["6003383",{"2":{"213":1}}],["6001",{"2":{"174":1,"244":1}}],["600",{"2":{"107":1}}],["676636065581032e",{"2":{"258":1}}],["6768888f",{"2":{"256":1}}],["674661913358162e",{"2":{"258":1}}],["674345222495249e",{"2":{"258":1}}],["6745403f",{"2":{"256":1}}],["674141f",{"2":{"256":1}}],["672856188378627e",{"2":{"258":1}}],["6720200353106822e",{"2":{"258":1}}],["6720745f",{"2":{"256":1}}],["6729300549302098e",{"2":{"258":1}}],["672725f",{"2":{"256":1}}],["6726513",{"2":{"134":1}}],["673466915070392e",{"2":{"258":1}}],["6735214f",{"2":{"256":1}}],["6731343f",{"2":{"256":1}}],["6712",{"2":{"249":1}}],["671544e",{"2":{"208":1}}],["6752",{"2":{"249":1}}],["67595f",{"2":{"80":1}}],["677398078600014e",{"2":{"258":1}}],["677246f",{"2":{"256":1}}],["6779146f",{"2":{"256":1}}],["6771653",{"2":{"213":2}}],["6776220912718907",{"2":{"169":1}}],["67",{"2":{"191":1,"225":2,"233":1}}],["67dffc4a8ae",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["6780157979814266e",{"2":{"258":1}}],["6789377f",{"2":{"256":1}}],["678879339289912e",{"2":{"258":1}}],["6788",{"2":{"249":1}}],["67861",{"2":{"166":1}}],["6784163",{"2":{"80":1}}],["679808275176214e",{"2":{"258":1}}],["6798847",{"2":{"136":1}}],["679154592881643e",{"2":{"258":1}}],["6791339f",{"2":{"256":1}}],["679175f",{"2":{"256":1}}],["679681f",{"2":{"256":1}}],["679097053056454e",{"2":{"258":1}}],["67909",{"2":{"166":1}}],["67908657",{"2":{"166":1}}],["679087",{"2":{"166":2}}],["67094",{"2":{"162":1}}],["6702257",{"2":{"134":1}}],["6481518856795e",{"2":{"258":1}}],["6480974f",{"2":{"256":1}}],["6477046f",{"2":{"256":1}}],["6477456",{"2":{"134":1}}],["64337575464998e",{"2":{"258":1}}],["643319f",{"2":{"256":1}}],["6432158188171453e",{"2":{"258":1}}],["6431522f",{"2":{"256":1}}],["6431484",{"2":{"216":1}}],["645968f",{"2":{"256":1}}],["6456693",{"2":{"213":2}}],["6456416023530093",{"2":{"169":1}}],["6445567655606466e",{"2":{"258":1}}],["6444779445863475e",{"2":{"258":1}}],["6444257f",{"2":{"256":1}}],["6443496f",{"2":{"256":1}}],["644821608723057e",{"2":{"258":1}}],["6448233",{"2":{"213":1}}],["644873f",{"2":{"256":1}}],["64495987",{"2":{"213":1}}],["642651968535577e",{"2":{"258":1}}],["642905f",{"2":{"256":1}}],["6423511984038721",{"2":{"164":2}}],["6424197",{"2":{"136":1}}],["646508766143015e",{"2":{"258":1}}],["646572f",{"2":{"256":1}}],["646065389912549e",{"2":{"258":1}}],["646095125571264e",{"2":{"258":1}}],["64605f",{"2":{"80":1}}],["6461784f",{"2":{"256":1}}],["64644",{"2":{"213":1}}],["6469914",{"2":{"120":1}}],["640898532429348e",{"2":{"258":1}}],["640835f",{"2":{"256":1}}],["6409345",{"2":{"136":1}}],["6409691",{"2":{"106":1}}],["64058f",{"2":{"80":1}}],["64",{"2":{"23":1,"72":18,"81":1,"175":2,"184":2,"191":2,"192":2,"210":2,"218":2,"223":2,"225":1,"226":2,"237":2,"246":2,"251":2,"256":1,"260":2}}],["6",{"2":{"9":1,"46":2,"65":3,"80":7,"106":1,"109":1,"116":3,"134":3,"136":5,"137":7,"141":1,"166":2,"168":3,"174":1,"175":5,"182":14,"184":5,"188":8,"191":2,"192":1,"210":5,"213":6,"218":5,"225":2,"226":5,"233":5,"235":1,"237":5,"244":4,"246":5,"249":11,"250":6,"251":1,"255":2,"256":164,"258":159,"260":1}}],["know",{"2":{"88":1,"109":1,"133":1,"256":1}}],["known",{"0":{"147":1},"2":{"9":2,"52":1,"88":1}}],["knet",{"2":{"75":1}}],["k",{"2":{"66":6,"70":4,"174":1,"230":2,"234":2}}],["kwarg",{"2":{"102":2,"103":1}}],["kwargs",{"2":{"10":24,"33":1,"37":2,"49":4,"59":3,"65":9,"71":2,"103":1,"162":2,"163":2,"190":1,"230":7,"233":2,"234":4,"236":4}}],["kw",{"2":{"59":2}}],["kullback",{"2":{"53":1}}],["kl",{"2":{"53":1}}],["kldivergenceloss",{"2":{"53":5}}],["klambauer",{"2":{"22":1}}],["ki−1",{"2":{"66":1,"68":3}}],["kind",{"0":{"120":1},"2":{"34":1,"46":1,"59":2,"65":1,"109":1,"120":1}}],["kinds",{"2":{"13":1}}],["kiros",{"2":{"24":1}}],["kaiming",{"2":{"9":4,"24":1,"66":2,"70":1,"162":6}}],["keith",{"2":{"252":1}}],["keep",{"2":{"196":1}}],["kept",{"2":{"193":1}}],["keys",{"2":{"102":1,"115":1,"132":2}}],["keypath=keypath",{"2":{"33":1}}],["keypath",{"2":{"32":7,"102":1,"103":1,"109":4,"110":10,"121":2,"122":4}}],["key",{"0":{"146":1},"2":{"15":2,"142":1}}],["keywords",{"2":{"59":1,"73":1,"163":1}}],["keyword",{"2":{"2":1,"15":1,"33":2,"44":1,"48":1,"49":2,"65":5,"66":2,"67":4,"68":3,"69":2,"70":4,"71":2,"72":4,"73":2,"92":1,"104":1,"105":1,"163":1}}],["kernelabstractions",{"2":{"25":1,"164":7}}],["kernels",{"0":{"164":1},"2":{"21":1,"164":3}}],["kernel",{"2":{"9":1,"21":1,"25":1,"66":5,"164":6}}],["+=",{"2":{"189":2,"224":2,"232":2,"244":1,"249":1}}],["+ϵ∗γ+βwhere",{"2":{"72":1}}],["+ϵ∗γ+βand",{"2":{"24":1}}],["+",{"2":{"9":3,"21":1,"23":1,"32":1,"44":1,"49":2,"52":1,"55":1,"59":4,"65":5,"66":4,"68":6,"70":4,"73":1,"93":1,"129":2,"134":1,"135":1,"152":2,"166":4,"174":3,"178":2,"206":1,"213":1,"242":4,"243":3,"245":1,"248":4,"249":1,"254":13,"255":1,"256":2}}],["2f",{"2":{"190":2,"225":4}}],["2fs",{"2":{"190":1}}],["2`",{"2":{"130":1}}],["2784514380599464e",{"2":{"258":1}}],["2781671971535676e",{"2":{"258":1}}],["278323f",{"2":{"256":1}}],["2782187f",{"2":{"256":1}}],["2785s\\ttraining",{"2":{"233":1}}],["2719075137710264e",{"2":{"258":1}}],["271978f",{"2":{"256":1}}],["2714",{"2":{"249":1}}],["2717",{"2":{"249":1}}],["27",{"2":{"225":3,"233":1,"249":1}}],["2741375330697258e",{"2":{"258":1}}],["274737470061827e",{"2":{"258":1}}],["27471745",{"2":{"216":1}}],["27461f",{"2":{"256":1}}],["27458",{"2":{"182":1}}],["276",{"2":{"208":1}}],["2732370968505513e",{"2":{"258":1}}],["2731097f",{"2":{"256":1}}],["273907e",{"2":{"208":1}}],["27373654",{"2":{"120":1}}],["275944098073367e",{"2":{"258":1}}],["2759441f",{"2":{"256":1}}],["27501",{"2":{"244":1}}],["2755905",{"2":{"213":2}}],["27531",{"2":{"182":1}}],["27545732",{"2":{"106":1}}],["2702900780703617e",{"2":{"258":1}}],["2709",{"2":{"249":1}}],["27001",{"2":{"244":1}}],["2701178",{"2":{"213":1}}],["27035674",{"2":{"174":1}}],["27051234",{"2":{"174":1}}],["27966",{"2":{"182":1}}],["27993",{"2":{"167":1}}],["2797048270773437",{"2":{"164":2}}],["2791573282471997",{"2":{"164":2}}],["27789",{"2":{"182":1}}],["2771789",{"2":{"174":1}}],["2771862617010155",{"2":{"164":2}}],["2774315",{"2":{"134":1}}],["27th",{"2":{"9":1}}],["2×4",{"2":{"168":1}}],["2×5",{"2":{"162":6}}],["2×32",{"2":{"106":2}}],["2×2",{"2":{"53":2,"166":1}}],["291803689560512e",{"2":{"258":1}}],["291109f",{"2":{"256":1}}],["2916592f",{"2":{"256":1}}],["2913357",{"2":{"106":1}}],["297352239608012e",{"2":{"258":1}}],["297179547248012e",{"2":{"258":1}}],["2974806f",{"2":{"256":1}}],["297231f",{"2":{"256":1}}],["297959481822617",{"2":{"164":2}}],["295884f",{"2":{"256":1}}],["2950",{"2":{"249":1}}],["29501",{"2":{"244":1}}],["29561827",{"2":{"174":1}}],["29001",{"2":{"244":1}}],["2904677",{"2":{"123":1}}],["29",{"2":{"225":3,"249":1}}],["293577222362373e",{"2":{"258":1}}],["2935447f",{"2":{"256":1}}],["29315922",{"2":{"216":1}}],["2938747",{"2":{"216":1}}],["2932029",{"2":{"216":1}}],["29328",{"2":{"182":1}}],["293211",{"2":{"106":1}}],["29206026",{"2":{"216":1}}],["292002",{"2":{"162":1}}],["2922556",{"2":{"213":1}}],["29828635",{"2":{"174":1}}],["298787",{"2":{"137":1}}],["29937f",{"2":{"256":1}}],["2991",{"2":{"249":1}}],["2992126",{"2":{"213":2}}],["29940",{"2":{"182":1}}],["29944375",{"2":{"174":1}}],["2990853",{"2":{"174":1}}],["29955",{"2":{"162":1}}],["296012563046942e",{"2":{"258":1}}],["2964196f",{"2":{"256":1}}],["296496",{"2":{"166":1}}],["29630",{"2":{"233":2,"235":1}}],["296372",{"2":{"162":1}}],["262539032231118e",{"2":{"258":1}}],["262446f",{"2":{"256":1}}],["269449967654701e",{"2":{"258":1}}],["2695945f",{"2":{"256":1}}],["2695",{"2":{"249":1}}],["263726f",{"2":{"256":1}}],["26311737",{"2":{"106":1}}],["260939208188153e",{"2":{"258":1}}],["26001",{"2":{"244":1}}],["260598",{"2":{"216":1}}],["2606924",{"2":{"137":1}}],["268187469414162e",{"2":{"258":1}}],["2681656",{"2":{"213":1}}],["2689521",{"2":{"106":2}}],["268941",{"2":{"53":2}}],["26",{"2":{"191":1,"208":2,"225":2,"249":1}}],["26452283326858e",{"2":{"258":1}}],["26451",{"2":{"182":1}}],["2643958f",{"2":{"256":1}}],["2647",{"2":{"249":1}}],["26477236",{"2":{"106":2}}],["2648",{"2":{"249":1}}],["266784f",{"2":{"256":1}}],["26623",{"2":{"182":1}}],["266121+0",{"2":{"162":1}}],["26657984",{"2":{"106":1}}],["2615215286920207e",{"2":{"258":1}}],["2615936",{"2":{"136":1}}],["261377f",{"2":{"256":1}}],["261273",{"2":{"125":2}}],["2674845335571465e",{"2":{"258":1}}],["2674477f",{"2":{"256":1}}],["26717530500157e",{"2":{"258":1}}],["267124f",{"2":{"256":1}}],["2679833f",{"2":{"256":1}}],["26793814",{"2":{"216":1}}],["2672",{"2":{"249":1}}],["26726",{"2":{"182":1}}],["26771653",{"2":{"213":2}}],["267644",{"2":{"136":1,"137":1}}],["26700416",{"2":{"106":1}}],["26700422",{"2":{"106":1}}],["26501",{"2":{"244":1}}],["265788",{"2":{"162":1}}],["265372",{"2":{"162":1}}],["265",{"2":{"81":1}}],["220337650267332e",{"2":{"258":1}}],["2209422355381365e",{"2":{"258":1}}],["220186f",{"2":{"256":1}}],["22001",{"2":{"244":1}}],["220",{"2":{"236":6}}],["2287450168839055e",{"2":{"258":1}}],["2287",{"2":{"249":1}}],["22807482",{"2":{"213":1}}],["22846f",{"2":{"80":1}}],["2260983704556828e",{"2":{"258":1}}],["2261",{"2":{"249":1}}],["226",{"2":{"208":1}}],["22635892",{"2":{"174":1}}],["226381",{"2":{"80":1}}],["22949",{"2":{"182":1}}],["2290224145974982",{"2":{"164":2}}],["2274967104751507e",{"2":{"258":1}}],["2275335f",{"2":{"256":1}}],["227513",{"2":{"162":1}}],["2276159f",{"2":{"256":1}}],["2271",{"2":{"249":1}}],["22705",{"2":{"182":1}}],["22156",{"2":{"182":1}}],["225348495884073e",{"2":{"258":1}}],["2255f",{"2":{"256":1}}],["22501",{"2":{"244":1}}],["225",{"2":{"179":1,"207":2,"230":1,"236":18,"241":1}}],["22543387",{"2":{"80":1}}],["2222539534702144e",{"2":{"258":1}}],["22222",{"2":{"233":2,"235":1}}],["2221094f",{"2":{"256":1}}],["2221465",{"2":{"137":1}}],["2228417f",{"2":{"256":1}}],["2228831",{"2":{"106":1}}],["2227837233928576",{"2":{"164":2}}],["222528",{"2":{"137":1}}],["22",{"2":{"134":1,"182":14,"191":4,"225":2,"233":1,"254":14}}],["2230763882255954e",{"2":{"258":1}}],["223163322609779e",{"2":{"258":1}}],["2232209f",{"2":{"256":1}}],["22336",{"2":{"182":1}}],["22363997",{"2":{"106":1}}],["2236399",{"2":{"106":1}}],["2237958",{"2":{"80":1}}],["224085f",{"2":{"256":1}}],["22406094",{"2":{"106":1}}],["224337639854286e",{"2":{"258":1}}],["2243",{"2":{"249":1}}],["22459084",{"2":{"80":1}}],["254450656895337e",{"2":{"258":1}}],["254786911885069e",{"2":{"258":1}}],["2547869f",{"2":{"256":1}}],["254155f",{"2":{"256":1}}],["2545935",{"2":{"80":1}}],["2557",{"2":{"249":1}}],["2555",{"2":{"249":1}}],["25501",{"2":{"244":1}}],["25587f",{"2":{"80":1}}],["25926",{"2":{"233":3,"235":1}}],["2508691195987064e",{"2":{"258":1}}],["250684651193247e",{"2":{"258":1}}],["2506847f",{"2":{"256":1}}],["25001",{"2":{"244":1}}],["2501",{"2":{"244":1}}],["250",{"2":{"217":2,"250":1,"255":1}}],["25055695",{"2":{"172":1,"173":1}}],["258934873928717e",{"2":{"258":1}}],["2588643f",{"2":{"256":1}}],["2581186",{"2":{"213":1}}],["2586653",{"2":{"106":1}}],["251",{"2":{"208":1}}],["25",{"2":{"166":1,"182":15,"208":2,"225":29,"249":1}}],["25250",{"2":{"182":1}}],["2525357728685884",{"2":{"164":2}}],["25278285",{"2":{"174":1}}],["252985",{"2":{"137":1}}],["252925",{"2":{"137":1}}],["2523673",{"2":{"137":1}}],["25378025",{"2":{"106":2}}],["25389904",{"2":{"106":1}}],["25385493",{"2":{"106":1}}],["257",{"2":{"80":1}}],["25662464",{"2":{"120":1}}],["256",{"2":{"46":1,"59":1,"80":6,"188":3,"223":2}}],["2476707575809744e",{"2":{"258":1}}],["247271009369957e",{"2":{"258":1}}],["247271f",{"2":{"256":1}}],["249270200262119e",{"2":{"258":1}}],["2494147f",{"2":{"256":1}}],["246708812823869e",{"2":{"258":1}}],["246047126437611e",{"2":{"258":1}}],["2468062521676514e",{"2":{"258":1}}],["2461438073596785e",{"2":{"258":1}}],["246506f",{"2":{"256":1}}],["2464001",{"2":{"106":1}}],["248457275797153e",{"2":{"258":1}}],["248019757621968e",{"2":{"258":1}}],["248588f",{"2":{"256":1}}],["248121f",{"2":{"256":1}}],["244777f",{"2":{"256":1}}],["244728",{"2":{"53":10}}],["2440945",{"2":{"213":2}}],["24s",{"2":{"191":1}}],["24220",{"2":{"182":1}}],["245467265503101e",{"2":{"258":1}}],["24501",{"2":{"244":1}}],["24585",{"2":{"182":1}}],["24550",{"2":{"182":1}}],["24304",{"2":{"162":1}}],["24",{"2":{"115":2,"175":3,"182":14,"184":3,"192":2,"210":3,"218":3,"225":3,"226":3,"233":1,"237":3,"246":3,"249":2}}],["241207f",{"2":{"256":1}}],["241937",{"2":{"213":1}}],["241512482727741e",{"2":{"258":1}}],["241533f",{"2":{"256":1}}],["241556e",{"2":{"208":1}}],["2415948",{"2":{"137":1}}],["24150778",{"2":{"107":1}}],["24169",{"2":{"106":1}}],["2405247771001035e",{"2":{"258":1}}],["2407417f",{"2":{"256":1}}],["24001",{"2":{"244":1}}],["24099025",{"2":{"80":1}}],["240",{"2":{"72":3}}],["238058228589095e",{"2":{"258":1}}],["2380037f",{"2":{"256":1}}],["237305f",{"2":{"256":1}}],["23710957",{"2":{"131":1}}],["234448306032229e",{"2":{"258":1}}],["234304f",{"2":{"256":1}}],["234872e",{"2":{"208":1}}],["23234403",{"2":{"216":1}}],["23242",{"2":{"136":1}}],["23501",{"2":{"244":1}}],["2354317",{"2":{"213":1}}],["2351742f",{"2":{"106":3}}],["233565486875458e",{"2":{"258":1}}],["23373",{"2":{"182":1}}],["23325463",{"2":{"120":1}}],["2390137336297228e",{"2":{"258":1}}],["2399238431729244e",{"2":{"258":1}}],["23992883",{"2":{"174":1}}],["23914115f",{"2":{"256":1}}],["239756",{"2":{"167":1}}],["23921265",{"2":{"136":1}}],["230415622234766e",{"2":{"258":1}}],["23056f",{"2":{"256":1}}],["23001",{"2":{"244":1}}],["23095",{"2":{"166":1}}],["230954",{"2":{"166":2}}],["23021114",{"2":{"106":1}}],["231913831615613e",{"2":{"258":1}}],["231336125211143e",{"2":{"258":1}}],["2313906f",{"2":{"256":1}}],["231862f",{"2":{"256":1}}],["231723",{"2":{"162":1}}],["23162955",{"2":{"106":1}}],["2360924f",{"2":{"256":1}}],["2361219621183299e",{"2":{"258":1}}],["2361765f",{"2":{"256":1}}],["2361472",{"2":{"120":1}}],["2366",{"2":{"249":1}}],["23622048",{"2":{"213":2}}],["23626f",{"2":{"80":1}}],["23",{"2":{"65":2,"175":1,"182":14,"184":1,"192":1,"210":1,"218":1,"225":3,"226":1,"233":1,"237":1,"246":1,"249":2,"251":1,"260":1}}],["2031875541867535e",{"2":{"258":1}}],["2031405532107285e",{"2":{"258":1}}],["2032244f",{"2":{"256":1}}],["20301",{"2":{"182":1}}],["203036",{"2":{"106":1}}],["2099673f",{"2":{"256":1}}],["20991",{"2":{"182":1}}],["2051929042638165e",{"2":{"258":1}}],["2052635f",{"2":{"256":1}}],["2050655f",{"2":{"256":1}}],["2050",{"2":{"249":1}}],["2050116f",{"2":{"256":1}}],["20501",{"2":{"244":1}}],["2058456",{"2":{"216":1}}],["20579764",{"2":{"216":1}}],["20548",{"2":{"162":1}}],["204743960464444e",{"2":{"258":1}}],["20472442",{"2":{"213":2}}],["204853f",{"2":{"256":1}}],["2042089",{"2":{"106":1}}],["207003371301013e",{"2":{"258":1}}],["2071551f",{"2":{"256":1}}],["2072048f",{"2":{"256":1}}],["20777",{"2":{"182":1}}],["20780",{"2":{"182":1}}],["2080838f",{"2":{"256":1}}],["20868",{"2":{"182":1}}],["20881107",{"2":{"120":1}}],["2021",{"2":{"252":1}}],["20257",{"2":{"182":1}}],["2024",{"2":{"175":2,"184":2,"192":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"260":1}}],["20277858",{"2":{"174":1}}],["2023",{"2":{"74":2}}],["206883208208888e",{"2":{"258":1}}],["2067",{"2":{"249":1}}],["206476",{"2":{"162":1}}],["20668754",{"2":{"136":1}}],["200385f",{"2":{"256":1}}],["20001",{"2":{"244":1}}],["20001543",{"2":{"174":1}}],["2000",{"2":{"187":1}}],["200155002588563e",{"2":{"258":1}}],["200195f",{"2":{"256":1}}],["20017",{"2":{"182":1}}],["2001",{"2":{"174":1,"244":1}}],["200",{"2":{"107":1,"236":18}}],["20053944",{"2":{"106":1}}],["2006",{"2":{"53":2}}],["20",{"2":{"65":1,"72":3,"80":1,"116":1,"136":1,"166":2,"174":3,"182":14,"225":2,"231":5,"233":2,"236":24,"249":9}}],["201",{"2":{"208":1,"217":1,"236":18}}],["20114",{"2":{"166":1}}],["201145",{"2":{"166":2}}],["2016",{"2":{"24":2,"53":2,"72":1}}],["20188306",{"2":{"106":1}}],["2018",{"2":{"24":1}}],["2017",{"2":{"22":1,"53":2}}],["2014",{"2":{"9":1,"22":1}}],["2015",{"2":{"9":2,"24":1}}],["2010",{"2":{"9":3}}],["214793468675314e",{"2":{"258":1}}],["2145766f",{"2":{"256":1}}],["218803148938489e",{"2":{"258":1}}],["21856f",{"2":{"80":1}}],["2157589486701711e",{"2":{"258":1}}],["2157841518188328e",{"2":{"258":1}}],["215804541550998e",{"2":{"258":1}}],["215671251196544e",{"2":{"258":1}}],["215347f",{"2":{"256":1}}],["21501",{"2":{"244":1}}],["212484226291877e",{"2":{"258":1}}],["212421f",{"2":{"256":1}}],["2125984",{"2":{"213":2}}],["21252f",{"2":{"80":1}}],["2176003524236732e",{"2":{"258":1}}],["217605709890307e",{"2":{"258":1}}],["2174575634941974e",{"2":{"258":1}}],["2175438f",{"2":{"256":1}}],["2173058f",{"2":{"256":1}}],["217373f",{"2":{"256":1}}],["2170127f",{"2":{"256":1}}],["2170",{"2":{"249":1}}],["2118468201797582e",{"2":{"258":1}}],["211707f",{"2":{"256":1}}],["2117198f",{"2":{"256":1}}],["2119",{"2":{"249":1}}],["211",{"2":{"236":6}}],["2111273",{"2":{"137":1}}],["210261868921642e",{"2":{"258":1}}],["210199952218848e",{"2":{"258":1}}],["21001",{"2":{"244":1}}],["210",{"2":{"236":18}}],["210484e",{"2":{"208":1}}],["2130",{"2":{"249":1}}],["21327",{"2":{"182":1}}],["21351",{"2":{"182":1}}],["2168856322686e",{"2":{"258":1}}],["216583947857849e",{"2":{"258":1}}],["216007f",{"2":{"256":1}}],["21690917",{"2":{"174":1}}],["21672015",{"2":{"120":1}}],["2197997f",{"2":{"256":1}}],["2197981041108443",{"2":{"166":1}}],["21905605f",{"2":{"256":1}}],["2191",{"2":{"249":1}}],["21940619",{"2":{"213":1}}],["2192001",{"2":{"137":1}}],["21950458",{"2":{"106":1}}],["21",{"2":{"48":1,"72":1,"182":14,"191":1,"225":5,"249":4}}],["288851661567153e",{"2":{"258":1}}],["288575150393261e",{"2":{"258":1}}],["288575f",{"2":{"256":1}}],["2875613513145147e",{"2":{"258":1}}],["287424452344741e",{"2":{"258":1}}],["287445f",{"2":{"256":1}}],["287433f",{"2":{"256":1}}],["2877",{"2":{"249":1}}],["28725442",{"2":{"106":1}}],["285495149137722e",{"2":{"258":1}}],["285105051816366e",{"2":{"258":1}}],["2852924f",{"2":{"256":1}}],["2852",{"2":{"249":1}}],["28501",{"2":{"244":1}}],["2891",{"2":{"249":1}}],["2899156",{"2":{"216":1}}],["28946856",{"2":{"216":1}}],["2896426",{"2":{"213":1}}],["2896142",{"2":{"136":1}}],["281272622814832e",{"2":{"258":1}}],["281252",{"2":{"213":1}}],["281707864879642e",{"2":{"258":1}}],["2817784f",{"2":{"256":1}}],["281313f",{"2":{"256":1}}],["281053",{"2":{"162":1}}],["28s",{"2":{"191":1}}],["28271104925644e",{"2":{"258":1}}],["282779f",{"2":{"256":1}}],["282657761888761e",{"2":{"258":1}}],["2828628f",{"2":{"256":1}}],["28289196",{"2":{"80":1}}],["2823362f",{"2":{"256":1}}],["28234",{"2":{"182":1}}],["28001",{"2":{"244":1}}],["28022",{"2":{"182":1}}],["28042415",{"2":{"106":1}}],["2837046954176047e",{"2":{"258":1}}],["283452f",{"2":{"256":1}}],["2834552f",{"2":{"256":1}}],["2833426980289933e",{"2":{"258":1}}],["283326866226591e",{"2":{"258":1}}],["2833",{"2":{"249":1}}],["2835",{"2":{"249":1}}],["28355134",{"2":{"106":1}}],["283859im",{"2":{"162":1}}],["286525f",{"2":{"256":1}}],["2869913410456831",{"2":{"164":2}}],["2864522",{"2":{"137":1}}],["28682f",{"2":{"80":1}}],["2849533f",{"2":{"256":1}}],["2849667",{"2":{"137":1}}],["2842",{"2":{"136":1}}],["28",{"2":{"46":4,"175":1,"184":1,"188":2,"190":2,"192":1,"210":1,"218":1,"221":4,"225":2,"226":1,"236":2,"237":1,"246":1,"249":1,"251":1,"260":1}}],["2=dense",{"2":{"32":1}}],["2nd",{"0":{"29":1},"2":{"86":1,"115":1,"240":2,"242":1}}],["2d",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"9":2,"66":3,"67":2,"68":3,"190":2,"238":1}}],["2",{"2":{"5":5,"9":5,"28":2,"31":5,"32":4,"34":8,"44":4,"46":4,"48":1,"49":1,"53":22,"59":13,"65":30,"66":9,"68":15,"69":11,"70":2,"71":15,"72":6,"73":3,"80":49,"81":5,"86":7,"106":32,"107":4,"109":21,"110":41,"115":1,"116":1,"117":2,"120":3,"121":2,"122":2,"123":4,"125":2,"126":1,"129":3,"130":12,"131":2,"134":7,"135":4,"136":13,"137":13,"141":1,"149":5,"162":8,"164":23,"166":20,"167":1,"168":4,"169":3,"171":1,"172":1,"175":4,"178":5,"179":1,"180":1,"182":15,"183":1,"184":4,"187":1,"188":18,"191":2,"192":2,"193":6,"194":6,"206":6,"208":9,"209":5,"210":4,"213":14,"214":1,"216":6,"217":3,"218":4,"223":1,"225":4,"226":5,"229":1,"230":1,"231":1,"233":5,"235":1,"236":67,"237":4,"242":4,"243":10,"245":4,"246":4,"248":2,"249":34,"254":50,"255":8,"256":108,"258":105}}],["v=v",{"2":{"242":1}}],["vtav",{"2":{"138":1}}],["vvt",{"2":{"138":1}}],["v∈rd",{"2":{"138":1}}],["vs",{"2":{"127":1,"208":1}}],["v0",{"2":{"74":1,"102":1,"124":1,"234":1}}],["voila",{"2":{"109":1}}],["vocabulary",{"2":{"70":2}}],["volterra",{"2":{"206":2}}],["vol",{"2":{"53":1}}],["volumetric",{"2":{"53":1}}],["vcat",{"2":{"69":1,"178":1,"254":1}}],["v",{"2":{"27":2,"53":1,"54":3,"72":1,"95":1,"139":5,"140":5,"141":10,"171":4,"172":2,"173":1,"241":4,"242":10,"254":21}}],["vjp",{"0":{"27":1},"2":{"27":1,"139":7,"141":16,"170":3,"173":4,"217":3}}],["vanilla",{"2":{"106":1}}],["various",{"2":{"77":1,"82":1,"85":1,"204":1}}],["variants",{"2":{"152":1}}],["variance",{"2":{"22":2,"24":8,"72":6,"249":1}}],["variationalhiddendropout",{"2":{"67":5}}],["variable",{"2":{"59":2,"102":1,"249":1}}],["variables",{"2":{"52":2,"59":1,"77":2,"80":1,"241":1,"249":1,"254":1}}],["var",{"2":{"24":5,"59":1,"72":7,"109":2,"110":2,"116":3,"120":1,"123":1,"134":2,"179":1,"207":2,"230":1,"236":24,"241":1,"244":4}}],["validate",{"2":{"182":1}}],["validated",{"2":{"1":1}}],["validation",{"2":{"178":1,"182":101}}],["valid",{"2":{"72":1,"129":1,"158":1}}],["val",{"2":{"15":2,"22":4,"24":2,"33":3,"46":1,"53":4,"54":5,"55":4,"65":7,"67":1,"109":1,"110":2,"120":2,"123":2,"178":4,"179":1,"180":1,"182":2,"207":1,"236":13,"243":8,"245":3}}],["valued",{"2":{"171":1}}],["value>",{"2":{"155":3}}],["value",{"2":{"3":1,"13":1,"15":2,"22":1,"24":4,"33":1,"39":2,"50":1,"53":6,"54":1,"55":3,"59":6,"65":1,"69":5,"72":6,"110":1,"135":2,"174":16,"178":1,"241":1,"244":4,"249":1,"250":4}}],["valuestorage",{"2":{"236":3}}],["values",{"2":{"3":2,"37":2,"52":4,"53":1,"55":1,"56":3,"73":1,"216":1}}],["v1",{"0":{"90":1},"1":{"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":1,"103":1,"104":1,"105":1},"2":{"12":1,"66":2,"79":4,"90":2,"93":1,"124":1}}],["vec",{"2":{"141":2,"179":1,"180":1,"222":1,"230":4,"243":1,"245":1}}],["vectors",{"2":{"69":1,"70":2,"72":3,"138":1,"169":6,"247":1}}],["vectorization",{"0":{"161":1},"2":{"25":1,"153":1,"161":1}}],["vectorize",{"2":{"18":1}}],["vector",{"0":{"139":1,"140":1,"172":1,"173":1},"2":{"9":2,"20":1,"23":1,"27":9,"34":2,"48":1,"53":3,"59":2,"69":16,"70":5,"77":1,"116":2,"133":2,"138":4,"139":1,"140":1,"164":3,"166":4,"169":2,"170":5,"171":1,"172":3,"173":1,"174":1,"183":1,"207":1,"230":3,"233":1,"249":4,"250":1,"254":1}}],["vendor",{"2":{"25":1}}],["vedaldi",{"2":{"24":1,"72":1}}],["verbatim",{"2":{"59":1}}],["verified",{"2":{"141":1}}],["verification",{"2":{"13":1}}],["verify",{"2":{"55":1,"134":1,"135":1,"136":1,"137":1,"141":1}}],["versioninfo",{"2":{"175":3,"184":3,"192":3,"210":3,"218":3,"226":3,"237":3,"246":3,"251":3,"260":3}}],["versioning",{"2":{"134":2}}],["versions",{"2":{"102":1,"153":1,"161":2}}],["version",{"2":{"31":1,"52":2,"55":1,"59":1,"61":3,"74":1,"79":1,"95":1,"114":1,"129":1,"139":1,"175":1,"184":1,"192":1,"203":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["very",{"2":{"12":1,"70":1,"77":1,"80":1,"106":2,"108":1,"109":1,"110":2,"165":1,"166":1,"193":1,"194":1,"254":1,"258":1}}],["visualization",{"0":{"250":1}}],["visualizing",{"0":{"245":1,"259":1}}],["visualize",{"2":{"213":1,"250":1,"259":1}}],["vision",{"2":{"9":2,"24":1,"53":4}}],["virtual",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["vi",{"2":{"138":1}}],["vitj",{"2":{"138":1}}],["viewaxis",{"2":{"236":68}}],["views",{"2":{"230":2,"242":1,"254":1}}],["view",{"2":{"71":4,"168":1,"249":1,"250":1}}],["victor",{"2":{"24":1,"72":1}}],["via",{"2":{"3":1,"5":2,"9":1,"11":1,"12":1,"23":1,"35":1,"52":2,"57":2,"77":2,"82":2,"102":1,"103":1,"105":1,"106":1,"143":1,"146":1,"164":1,"169":1,"171":4,"172":1,"249":1}}],["||",{"2":{"81":1,"107":1,"174":1,"217":1,"244":1}}],["|>",{"2":{"80":2,"81":3,"106":4,"107":2,"125":2,"126":3,"154":2,"164":2,"174":1,"182":3,"207":1,"208":2,"209":1,"216":1,"217":1,"222":2,"225":3,"231":2,"233":1,"244":3,"256":1}}],["|p|−di×",{"2":{"66":1,"68":3}}],["|y^−y|",{"2":{"53":1}}],["|y−y^|−0",{"2":{"53":1}}],["|y−y^|≤δδ∗",{"2":{"53":1}}],["|",{"2":{"3":4,"100":1,"109":5,"110":13,"245":1}}],["x~",{"2":{"250":1}}],["x~|θ",{"2":{"250":1}}],["x~|x",{"2":{"250":1}}],["x∈",{"2":{"243":1}}],["xyt",{"2":{"242":17,"243":18,"244":10}}],["xi∈rn",{"2":{"174":1}}],["xi",{"2":{"174":2}}],["xᵢ",{"2":{"107":4}}],["xdev",{"2":{"106":5,"107":3}}],["x3",{"2":{"65":1}}],["x3c",{"2":{"3":3,"4":2,"12":2,"21":1,"23":1,"37":3,"66":4,"69":2,"115":2,"117":2,"129":1,"130":1,"155":6,"171":1,"179":1,"194":1,"208":1,"230":2,"234":2,"236":2,"241":1,"244":2,"254":2}}],["x2s",{"2":{"248":8}}],["x2",{"2":{"65":3,"248":2,"250":18}}],["x26",{"2":{"34":6,"74":1,"121":2,"122":4,"175":4,"184":4,"192":4,"208":2,"210":4,"218":4,"226":4,"237":4,"244":4,"246":4,"251":4,"254":4,"260":4}}],["x1s",{"2":{"248":8}}],["x1",{"2":{"65":3,"248":2,"250":18}}],["xlogy",{"2":{"54":1}}],["xlogx",{"2":{"54":1}}],["xlabel=",{"2":{"213":1,"217":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["xladevice",{"2":{"2":2}}],["xla",{"0":{"63":1},"2":{"2":1,"63":3,"106":5,"107":1}}],["x=",{"2":{"49":1}}],["xs",{"2":{"25":2,"245":6,"248":1,"249":3}}],["xt0s",{"2":{"248":5}}],["xt1s",{"2":{"248":5}}],["xt",{"2":{"24":2,"117":1,"118":1}}],["x86",{"2":{"23":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["xoshiro",{"2":{"9":3,"34":1,"59":5,"81":1,"109":1,"120":2,"149":1,"164":1,"169":3,"182":1,"225":1}}],["xavier",{"2":{"9":4}}],["x",{"2":{"3":12,"5":12,"9":7,"12":3,"13":12,"16":2,"18":10,"19":2,"20":10,"21":3,"22":8,"23":3,"24":26,"25":1,"27":6,"28":10,"44":2,"46":2,"49":5,"53":2,"54":18,"55":16,"58":2,"59":31,"65":24,"66":23,"67":9,"68":27,"69":20,"70":15,"71":30,"72":13,"73":14,"80":4,"81":12,"106":14,"107":3,"109":4,"110":10,"113":3,"115":9,"116":2,"117":9,"120":3,"123":2,"125":4,"126":9,"129":6,"130":5,"131":7,"133":6,"134":10,"135":12,"137":8,"139":2,"140":2,"141":9,"144":3,"149":3,"152":3,"154":5,"164":3,"166":7,"167":2,"168":4,"171":7,"172":5,"173":1,"174":11,"178":6,"179":8,"180":8,"181":2,"182":4,"187":6,"189":2,"190":4,"193":6,"194":10,"206":4,"213":9,"217":8,"221":4,"222":2,"224":2,"225":2,"229":6,"230":10,"232":2,"233":2,"234":2,"236":13,"243":7,"245":1,"248":1,"249":5,"250":4,"254":6,"258":2}}],["3+0",{"2":{"175":2,"184":2,"210":2,"218":2,"226":2,"237":2,"246":2}}],["378444730332931e",{"2":{"258":1}}],["3783435f",{"2":{"256":1}}],["3755347840630702e",{"2":{"258":1}}],["3753903f",{"2":{"256":1}}],["37501",{"2":{"244":1}}],["37506",{"2":{"182":1}}],["3727",{"2":{"249":1}}],["37275374",{"2":{"213":1}}],["3729234",{"2":{"136":1}}],["370253f",{"2":{"256":1}}],["3705",{"2":{"249":1}}],["37001",{"2":{"244":1}}],["3700787",{"2":{"213":2}}],["37037",{"2":{"233":2}}],["3707023",{"2":{"134":1}}],["376",{"2":{"208":1}}],["376383",{"2":{"137":1}}],["37",{"2":{"191":1,"225":2,"233":5,"235":1}}],["37164552124023e",{"2":{"258":1}}],["37114",{"2":{"182":1}}],["3719285",{"2":{"137":1}}],["37779",{"2":{"182":1}}],["3742405e",{"2":{"174":1}}],["3rd",{"0":{"203":1},"2":{"136":1}}],["3418215555135889e",{"2":{"258":1}}],["34186175f",{"2":{"256":1}}],["3410565",{"2":{"130":1}}],["348321075826649e",{"2":{"258":1}}],["3483916f",{"2":{"256":1}}],["34855f",{"2":{"80":1}}],["3408782f",{"2":{"256":1}}],["3401",{"2":{"249":1}}],["34001",{"2":{"244":1}}],["34501",{"2":{"244":1}}],["345025",{"2":{"137":1}}],["34",{"2":{"225":3,"249":1}}],["34745",{"2":{"182":1}}],["34717593",{"2":{"174":1}}],["34254",{"2":{"166":1}}],["34253937",{"2":{"166":1}}],["342539",{"2":{"166":2}}],["343438806550787e",{"2":{"258":1}}],["343497",{"2":{"162":1}}],["3438114389553043e",{"2":{"258":1}}],["34383082",{"2":{"120":1}}],["34351700231383653",{"2":{"169":1}}],["3436353225087703",{"2":{"164":2}}],["349963017101542e",{"2":{"258":1}}],["3499065f",{"2":{"256":1}}],["349403471653878e",{"2":{"258":1}}],["3490",{"2":{"249":1}}],["34915",{"2":{"182":1}}],["349",{"2":{"125":2}}],["367387156383673e",{"2":{"258":1}}],["367131002493477e",{"2":{"258":1}}],["3669865f",{"2":{"256":1}}],["3669",{"2":{"249":1}}],["360763834265864e",{"2":{"258":1}}],["360724f",{"2":{"256":1}}],["3609398f",{"2":{"256":1}}],["36001",{"2":{"244":1}}],["3602331",{"2":{"136":1}}],["3619882760933495e",{"2":{"258":1}}],["3611788720594416e",{"2":{"258":1}}],["3611387f",{"2":{"256":1}}],["361084241307146e",{"2":{"258":1}}],["361895f",{"2":{"256":1}}],["3617s\\ttraining",{"2":{"233":1}}],["36132",{"2":{"182":1}}],["36",{"2":{"225":2}}],["363162025290119e",{"2":{"258":1}}],["3637495",{"2":{"213":1}}],["36395",{"2":{"182":1}}],["364928019190132e",{"2":{"258":1}}],["36497",{"2":{"182":1}}],["3646326f",{"2":{"256":1}}],["3645461",{"2":{"213":1}}],["36428708",{"2":{"134":1}}],["36829436",{"2":{"174":1}}],["369928412005192e",{"2":{"258":1}}],["3694",{"2":{"249":1}}],["3690",{"2":{"249":1}}],["3693867",{"2":{"213":1}}],["36938f",{"2":{"80":1}}],["3691778381831775",{"2":{"169":1}}],["369178",{"2":{"166":2}}],["36918",{"2":{"166":1}}],["3657451788470566e",{"2":{"258":1}}],["3656746f",{"2":{"256":1}}],["3653595f",{"2":{"256":1}}],["3653",{"2":{"249":1}}],["36501",{"2":{"244":1}}],["3650193059617517",{"2":{"164":2}}],["36542922",{"2":{"136":1}}],["3625263f",{"2":{"256":1}}],["362593",{"2":{"106":1}}],["3628",{"2":{"249":1}}],["36220473",{"2":{"213":2}}],["36222499022985155",{"2":{"164":2}}],["319868516109516e",{"2":{"258":1}}],["319666f",{"2":{"256":1}}],["3192",{"2":{"249":1}}],["317605074623253e",{"2":{"258":1}}],["3173524f",{"2":{"256":1}}],["3170582",{"2":{"213":1}}],["311363431997184e",{"2":{"258":1}}],["311596f",{"2":{"256":1}}],["31128f",{"2":{"80":1}}],["314829496200851e",{"2":{"258":1}}],["3141244f",{"2":{"256":1}}],["3146985f",{"2":{"256":1}}],["3147238",{"2":{"80":1}}],["3139531736822475e",{"2":{"258":1}}],["3137669768206132e",{"2":{"258":1}}],["3138953f",{"2":{"256":1}}],["313226f",{"2":{"106":3}}],["315934878893783e",{"2":{"258":1}}],["315727859711487e",{"2":{"258":1}}],["3156954f",{"2":{"256":1}}],["31501",{"2":{"244":1}}],["3155",{"2":{"249":1}}],["31559",{"2":{"182":1}}],["31550723",{"2":{"106":1}}],["31",{"2":{"225":5,"249":2}}],["312941696108749e",{"2":{"258":1}}],["31298",{"2":{"182":1}}],["3127606f",{"2":{"256":1}}],["31280",{"2":{"182":1}}],["3104570071210386e",{"2":{"258":1}}],["310483371894584e",{"2":{"258":1}}],["3109380607051135e",{"2":{"258":1}}],["310802f",{"2":{"256":1}}],["3108876",{"2":{"136":1}}],["310734f",{"2":{"256":1}}],["31001",{"2":{"244":1}}],["3102s\\ttraining",{"2":{"233":1}}],["31052",{"2":{"182":1}}],["31861955",{"2":{"134":1}}],["357343856216914e",{"2":{"258":1}}],["3573297f",{"2":{"256":1}}],["357148589894786e",{"2":{"258":1}}],["3537372519475696e",{"2":{"258":1}}],["3531034f",{"2":{"256":1}}],["353609f",{"2":{"256":1}}],["35395628",{"2":{"213":1}}],["3524",{"2":{"249":1}}],["3523193",{"2":{"174":1}}],["35501",{"2":{"244":1}}],["3558193508137715",{"2":{"164":2}}],["3500024256058437e",{"2":{"258":1}}],["35001",{"2":{"244":1}}],["3501",{"2":{"244":1}}],["3560644238060075e",{"2":{"258":1}}],["35653025",{"2":{"213":1}}],["35638642",{"2":{"136":1}}],["354396f",{"2":{"256":1}}],["35458",{"2":{"182":1}}],["354464",{"2":{"106":1}}],["35",{"2":{"175":2,"184":2,"210":2,"218":2,"225":2,"226":2,"237":2,"246":2,"249":1}}],["3583874f",{"2":{"256":1}}],["35836592",{"2":{"80":1}}],["3585914f",{"2":{"256":1}}],["35817",{"2":{"166":1}}],["351658263368668e",{"2":{"258":1}}],["351662",{"2":{"162":1}}],["351214f",{"2":{"256":1}}],["351",{"2":{"208":1}}],["35149138733595564",{"2":{"169":4}}],["351491",{"2":{"166":1}}],["35181466",{"2":{"120":1}}],["35188f",{"2":{"80":1}}],["35152f",{"2":{"80":1}}],["388097810490849e",{"2":{"258":1}}],["3886",{"2":{"249":1}}],["386319574376392e",{"2":{"258":1}}],["386982f",{"2":{"256":1}}],["3864326f",{"2":{"256":1}}],["3864",{"2":{"249":1}}],["384426238766085e",{"2":{"258":1}}],["3843974f",{"2":{"256":1}}],["3841858f",{"2":{"106":2}}],["38",{"2":{"225":27,"233":1}}],["3871129872274463e",{"2":{"258":1}}],["3871873",{"2":{"216":1}}],["38798f",{"2":{"80":1}}],["38374982045109e",{"2":{"258":1}}],["383033677535268e",{"2":{"258":1}}],["3838204f",{"2":{"256":1}}],["38313",{"2":{"182":1}}],["3834447782571341",{"2":{"164":1}}],["3834447782571344",{"2":{"164":1}}],["3892529706206e",{"2":{"258":1}}],["38929f",{"2":{"256":1}}],["389381f",{"2":{"256":1}}],["38999",{"2":{"182":1}}],["38911813",{"2":{"106":1}}],["3891182",{"2":{"106":1}}],["38501",{"2":{"244":1}}],["3850993",{"2":{"120":1}}],["38515",{"2":{"182":1}}],["382982f",{"2":{"256":1}}],["382971",{"2":{"162":1}}],["38200346",{"2":{"213":1}}],["38242024",{"2":{"137":1}}],["38001",{"2":{"244":1}}],["380777f0",{"2":{"135":1}}],["38068f",{"2":{"80":1}}],["38187847",{"2":{"120":1}}],["3225470459100275e",{"2":{"258":1}}],["3223301f",{"2":{"256":1}}],["324903786468119e",{"2":{"258":1}}],["32494",{"2":{"182":1}}],["3242259164174353e",{"2":{"258":1}}],["3240985f",{"2":{"256":1}}],["3250552913316052e",{"2":{"258":1}}],["32501",{"2":{"244":1}}],["3251116f",{"2":{"256":1}}],["3251189f",{"2":{"256":1}}],["323463064342426e",{"2":{"258":1}}],["3231676f",{"2":{"256":1}}],["32322538",{"2":{"120":1}}],["3270",{"2":{"249":1}}],["3279041356366077",{"2":{"164":2}}],["3217581420886376e",{"2":{"258":1}}],["3216366f",{"2":{"256":1}}],["321",{"2":{"236":6}}],["32159",{"2":{"182":1}}],["321506",{"2":{"162":1}}],["3206786863723692e",{"2":{"258":1}}],["3206787f",{"2":{"256":1}}],["32001",{"2":{"244":1}}],["320",{"2":{"236":6}}],["3294",{"2":{"249":1}}],["3291591",{"2":{"216":1}}],["32984197",{"2":{"80":1}}],["328542",{"2":{"213":1}}],["32839986131789933",{"2":{"164":2}}],["326512247765953e",{"2":{"258":1}}],["326",{"2":{"208":1}}],["32623518",{"2":{"120":1}}],["32",{"2":{"44":2,"59":2,"81":13,"106":9,"107":2,"109":1,"110":1,"188":2,"206":1,"208":4,"214":1,"223":2,"225":4,"244":1,"249":2,"251":1,"256":8,"260":1}}],["3=dense",{"2":{"32":1}}],["3dv",{"2":{"53":1}}],["3d",{"2":{"24":1,"28":1,"53":1,"135":1,"182":1,"217":1,"225":2}}],["303844f",{"2":{"256":1}}],["3035",{"2":{"249":1}}],["3027693792000462e",{"2":{"258":1}}],["3027369f",{"2":{"256":1}}],["302637171037346e",{"2":{"258":1}}],["302260603794181e",{"2":{"258":1}}],["302818f",{"2":{"256":1}}],["302315f",{"2":{"256":1}}],["30215",{"2":{"166":1}}],["3042796517059236e",{"2":{"258":1}}],["3043342f",{"2":{"256":1}}],["304025272798725e",{"2":{"258":1}}],["3040",{"2":{"249":1}}],["30406",{"2":{"182":1}}],["3048",{"2":{"249":1}}],["30501",{"2":{"244":1}}],["3050475",{"2":{"216":1}}],["30569053",{"2":{"216":1}}],["305844",{"2":{"136":1,"137":1}}],["3070866",{"2":{"213":2}}],["3071294",{"2":{"137":1}}],["30673835",{"2":{"213":1}}],["30674052",{"2":{"174":1}}],["306641",{"2":{"162":1}}],["306147",{"2":{"162":1}}],["309089061376478e",{"2":{"258":1}}],["3092",{"2":{"249":1}}],["3096",{"2":{"249":1}}],["3097294",{"2":{"174":1}}],["309",{"2":{"134":2}}],["3008",{"2":{"249":1}}],["30001",{"2":{"244":1}}],["3001",{"2":{"174":1,"244":1}}],["300",{"2":{"107":1}}],["301397636464194e",{"2":{"258":1}}],["3016809370038249e",{"2":{"258":1}}],["3018025f",{"2":{"256":1}}],["301",{"2":{"80":1,"208":1}}],["30",{"2":{"9":1,"22":1,"191":1,"225":2,"249":1}}],["3×2",{"2":{"59":2,"125":2}}],["3×5",{"2":{"53":4,"59":1}}],["3×3×1×1",{"2":{"9":1}}],["3×7",{"2":{"5":2}}],["3×13",{"2":{"5":4}}],["331143604459376e",{"2":{"258":1}}],["3383893365242713e",{"2":{"258":1}}],["338042821812204e",{"2":{"258":1}}],["3381364f",{"2":{"256":1}}],["3385826",{"2":{"213":2}}],["337227905111243e",{"2":{"258":1}}],["337237f",{"2":{"256":1}}],["337108799607962e",{"2":{"258":1}}],["337188f",{"2":{"256":1}}],["3378386f",{"2":{"256":1}}],["33758628",{"2":{"118":1}}],["3335",{"2":{"249":1}}],["3333333333333335",{"2":{"249":1}}],["33333",{"2":{"233":15,"235":2}}],["3303734100979687e",{"2":{"258":1}}],["3301923f",{"2":{"256":1}}],["330891f",{"2":{"256":1}}],["33001",{"2":{"244":1}}],["33070865",{"2":{"213":2}}],["334530358084579e",{"2":{"258":1}}],["3343754",{"2":{"213":1}}],["3344088f",{"2":{"256":1}}],["3344351",{"2":{"137":1}}],["33442986",{"2":{"106":1}}],["3356",{"2":{"249":1}}],["33501",{"2":{"244":1}}],["33544478",{"2":{"213":1}}],["335454",{"2":{"162":1}}],["33572",{"2":{"182":1}}],["339079",{"2":{"137":1}}],["3367505",{"2":{"120":1}}],["33",{"2":{"5":1,"80":2,"81":1,"191":2,"225":2}}],["3917499963287668e",{"2":{"258":1}}],["39192125f",{"2":{"256":1}}],["391988e",{"2":{"208":1}}],["397930227147059e",{"2":{"258":1}}],["397967f",{"2":{"256":1}}],["397325162368042e",{"2":{"258":1}}],["397365f",{"2":{"256":1}}],["3977319",{"2":{"137":1}}],["3928808630423847e",{"2":{"258":1}}],["3928175",{"2":{"134":1}}],["3927525f",{"2":{"256":1}}],["3956",{"2":{"249":1}}],["3952",{"2":{"249":1}}],["39501",{"2":{"244":1}}],["39588368",{"2":{"213":1}}],["395306",{"2":{"134":1}}],["398038",{"2":{"213":1}}],["39800483",{"2":{"137":1}}],["393138650556651e",{"2":{"258":1}}],["3935",{"2":{"249":1}}],["39300445",{"2":{"216":1}}],["39370078",{"2":{"213":2}}],["3939896",{"2":{"137":1}}],["39960",{"2":{"182":1}}],["39001",{"2":{"244":1}}],["390200114697191",{"2":{"164":2}}],["3903124",{"2":{"106":1}}],["3942307655311696",{"2":{"164":2}}],["3940224213371761",{"2":{"164":2}}],["3963474843987855e",{"2":{"258":1}}],["396399f",{"2":{"256":1}}],["396323",{"2":{"162":1}}],["3960",{"2":{"249":1}}],["396435e",{"2":{"208":1}}],["3969839f",{"2":{"106":1}}],["39",{"2":{"2":1,"4":2,"10":1,"13":1,"15":1,"18":2,"21":1,"23":1,"32":1,"34":1,"44":1,"47":1,"52":1,"53":1,"54":1,"57":3,"59":7,"65":1,"69":1,"71":1,"72":3,"73":1,"77":2,"80":2,"88":3,"89":1,"95":2,"98":1,"102":1,"106":2,"107":1,"109":1,"110":1,"111":1,"113":1,"114":1,"120":1,"129":5,"130":1,"131":3,"132":1,"133":4,"134":4,"135":1,"136":2,"137":1,"141":3,"144":2,"146":1,"147":1,"149":1,"164":2,"165":5,"166":8,"168":2,"171":1,"172":1,"174":1,"179":3,"181":1,"183":2,"191":1,"194":2,"196":1,"197":1,"204":2,"207":1,"213":1,"214":1,"217":1,"222":1,"225":2,"230":1,"233":1,"248":1,"249":4,"254":1,"255":1,"256":2}}],["3",{"2":{"2":1,"5":3,"9":5,"32":7,"44":2,"46":1,"48":1,"52":1,"53":30,"59":27,"65":17,"66":2,"67":2,"69":4,"71":4,"72":3,"80":10,"81":4,"86":2,"106":16,"107":1,"109":12,"110":7,"115":3,"116":1,"118":1,"120":4,"123":6,"125":1,"126":1,"129":1,"131":1,"134":4,"136":12,"137":7,"164":2,"166":23,"167":1,"168":3,"169":7,"175":6,"179":1,"180":1,"182":14,"184":6,"187":1,"188":9,"190":1,"191":3,"206":1,"208":2,"210":6,"213":7,"216":4,"218":6,"225":7,"226":6,"229":1,"231":2,"233":5,"235":1,"236":56,"237":7,"241":3,"242":2,"243":1,"244":2,"246":6,"249":17,"250":1,"254":12,"255":1,"256":78,"258":80}}],["4t",{"2":{"243":1}}],["4th",{"2":{"121":1}}],["4fs",{"2":{"233":1}}],["4x",{"2":{"191":1}}],["469945827307626e",{"2":{"258":1}}],["4667064688319504e",{"2":{"258":1}}],["4665254f",{"2":{"256":1}}],["4673",{"2":{"249":1}}],["464045504599007e",{"2":{"258":1}}],["464025f",{"2":{"256":1}}],["4644",{"2":{"249":1}}],["464567",{"2":{"213":2}}],["4656832046366923e",{"2":{"258":1}}],["46501",{"2":{"244":1}}],["46514034",{"2":{"174":1}}],["4626s\\ttraining",{"2":{"233":1}}],["46276",{"2":{"182":1}}],["46011095765146e",{"2":{"258":1}}],["4604740497695822e",{"2":{"258":1}}],["4604845",{"2":{"137":1}}],["460204f",{"2":{"256":1}}],["460617945800818e",{"2":{"258":1}}],["46061087",{"2":{"213":1}}],["4606725f",{"2":{"256":1}}],["46001",{"2":{"244":1}}],["46",{"2":{"191":1,"225":4,"249":1}}],["46806",{"2":{"182":1}}],["4687857",{"2":{"137":1}}],["463341f",{"2":{"256":1}}],["46300",{"2":{"182":1}}],["46327f",{"2":{"80":1}}],["461823480987718e",{"2":{"258":1}}],["4617574f",{"2":{"256":1}}],["461908",{"2":{"166":1}}],["4619778701480337",{"2":{"164":2}}],["4614",{"2":{"136":1}}],["487954028918456e",{"2":{"258":1}}],["487419562837569e",{"2":{"258":1}}],["487349f",{"2":{"256":1}}],["4899254f",{"2":{"256":1}}],["489854872851174e",{"2":{"258":1}}],["4898s\\ttraining",{"2":{"233":1}}],["48983693",{"2":{"213":1}}],["48301f",{"2":{"256":1}}],["48351598",{"2":{"174":1}}],["485830862011259e",{"2":{"258":1}}],["48501",{"2":{"244":1}}],["4851346",{"2":{"131":1}}],["48001",{"2":{"244":1}}],["480",{"2":{"236":3}}],["48027995",{"2":{"213":1}}],["484",{"2":{"210":1,"246":1}}],["484056",{"2":{"162":1}}],["48",{"2":{"175":3,"184":3,"192":3,"210":3,"218":3,"225":4,"226":3,"237":3,"246":3}}],["482258084849063e",{"2":{"258":1}}],["482908981910036e",{"2":{"258":1}}],["482401944411764e",{"2":{"258":1}}],["482439f",{"2":{"256":1}}],["482431f",{"2":{"256":1}}],["48263642",{"2":{"174":1}}],["48203f",{"2":{"80":1}}],["4861524f",{"2":{"256":1}}],["4866s\\ttraining",{"2":{"235":1}}],["4869s\\ttraining",{"2":{"233":1}}],["486550215883336",{"2":{"164":2}}],["486214",{"2":{"162":1}}],["488858615878919e",{"2":{"258":1}}],["488304392744379e",{"2":{"258":1}}],["4885195f",{"2":{"256":1}}],["488788f",{"2":{"256":1}}],["4880",{"2":{"249":1}}],["488028235902512",{"2":{"164":2}}],["4881f",{"2":{"256":1}}],["48818898",{"2":{"213":2}}],["488105",{"2":{"136":1,"137":1}}],["488623",{"2":{"137":1}}],["4816968912365366e",{"2":{"258":1}}],["481328499037518e",{"2":{"258":1}}],["48137343",{"2":{"120":1}}],["4819496f",{"2":{"256":1}}],["48193252",{"2":{"134":1}}],["4812273f",{"2":{"256":1}}],["48148",{"2":{"233":6,"235":1}}],["4810884",{"2":{"134":1}}],["429847808839796e",{"2":{"258":1}}],["429819f",{"2":{"256":1}}],["428054671737328e",{"2":{"258":1}}],["4280547f",{"2":{"256":1}}],["4273280612531504e",{"2":{"258":1}}],["4273281f",{"2":{"256":1}}],["4270489f",{"2":{"256":1}}],["427156e",{"2":{"208":1}}],["42501",{"2":{"244":1}}],["42519686",{"2":{"213":2}}],["42514",{"2":{"182":1}}],["426601716194696e",{"2":{"258":1}}],["42669478",{"2":{"106":1}}],["42645f",{"2":{"256":1}}],["426",{"2":{"208":1}}],["422075312677048e",{"2":{"258":1}}],["422043f",{"2":{"256":1}}],["42278607236792e",{"2":{"258":1}}],["422938f",{"2":{"256":1}}],["422918",{"2":{"166":1}}],["4225",{"2":{"249":1}}],["422",{"2":{"184":1}}],["421800804554029e",{"2":{"258":1}}],["42189494",{"2":{"174":1}}],["421945f",{"2":{"256":1}}],["4216236",{"2":{"120":1}}],["4242767",{"2":{"174":1}}],["4209998016843574e",{"2":{"258":1}}],["4208715f",{"2":{"256":1}}],["42001",{"2":{"244":1}}],["420058350197999",{"2":{"164":1}}],["42005835019799886",{"2":{"164":1}}],["420698",{"2":{"136":1,"137":1}}],["42",{"2":{"162":2,"225":2}}],["403067804987242e",{"2":{"258":1}}],["40354207",{"2":{"174":1}}],["4092722392875731e",{"2":{"258":1}}],["409741f",{"2":{"256":1}}],["4093011f",{"2":{"256":1}}],["4029447987912596e",{"2":{"258":1}}],["4029395f",{"2":{"256":1}}],["4020445f",{"2":{"256":1}}],["402712f",{"2":{"256":1}}],["40229",{"2":{"166":1}}],["4058173587873332e",{"2":{"258":1}}],["4059",{"2":{"249":1}}],["40501",{"2":{"244":1}}],["4051151",{"2":{"171":4}}],["40",{"2":{"225":2,"233":5,"235":2}}],["407238698258605e",{"2":{"258":1}}],["4072755f",{"2":{"256":1}}],["40727592",{"2":{"216":1}}],["40704671792656e",{"2":{"258":1}}],["4071912f",{"2":{"256":1}}],["40741",{"2":{"233":3}}],["40789893",{"2":{"174":1}}],["401900018784194e",{"2":{"258":1}}],["4017",{"2":{"249":1}}],["4018757",{"2":{"213":1}}],["4015749",{"2":{"213":2}}],["401",{"2":{"208":1}}],["401215",{"2":{"117":1}}],["4068297f",{"2":{"256":1}}],["40689",{"2":{"182":1}}],["4061",{"2":{"249":1}}],["40623155",{"2":{"216":1}}],["4066515",{"2":{"137":1}}],["40gb",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["40473",{"2":{"166":1}}],["404728",{"2":{"166":2}}],["40001",{"2":{"244":1}}],["4001",{"2":{"174":1,"244":1}}],["4007292",{"2":{"136":1}}],["4007905",{"2":{"134":1}}],["400",{"2":{"107":1}}],["456762205793661e",{"2":{"258":1}}],["4568328f",{"2":{"256":1}}],["4566929",{"2":{"213":2}}],["458327594393708e",{"2":{"258":1}}],["4583564f",{"2":{"256":1}}],["4585489",{"2":{"120":1}}],["454849221909746e",{"2":{"258":1}}],["4546465f",{"2":{"256":1}}],["454679",{"2":{"162":1}}],["4543",{"2":{"249":1}}],["45501",{"2":{"244":1}}],["45583528",{"2":{"213":1}}],["4552384158732863",{"2":{"169":4}}],["455238",{"2":{"166":1}}],["4517148058514547e",{"2":{"258":1}}],["4517335f",{"2":{"256":1}}],["451986397017856e",{"2":{"258":1}}],["4516136f",{"2":{"256":1}}],["451",{"2":{"208":1}}],["4510427",{"2":{"120":1}}],["45",{"2":{"191":2,"225":3}}],["4529158253777705e",{"2":{"258":1}}],["45298263",{"2":{"80":1}}],["4528523f",{"2":{"256":1}}],["4524014",{"2":{"213":1}}],["45247704",{"2":{"174":1}}],["459508855258904e",{"2":{"258":1}}],["4593376f",{"2":{"256":1}}],["4593922",{"2":{"171":4}}],["459019",{"2":{"166":1}}],["450538238276147e",{"2":{"258":1}}],["450581f",{"2":{"106":5}}],["4502088210235905e",{"2":{"258":1}}],["4504257f",{"2":{"256":1}}],["45001",{"2":{"244":1}}],["4500963619011972",{"2":{"164":2}}],["4501",{"2":{"244":1}}],["45016125",{"2":{"174":1}}],["45325348",{"2":{"129":1}}],["413137f",{"2":{"256":1}}],["41303",{"2":{"182":1}}],["4170578f",{"2":{"256":1}}],["4170046f",{"2":{"256":1}}],["41501",{"2":{"244":1}}],["4158693023143286",{"2":{"164":2}}],["415858",{"2":{"106":1}}],["41",{"2":{"225":4}}],["41s",{"2":{"191":1}}],["4169362640261138e",{"2":{"258":1}}],["41687352815843e",{"2":{"258":1}}],["4161023766989025e",{"2":{"258":1}}],["4160479f",{"2":{"256":1}}],["416",{"2":{"188":2}}],["41919118",{"2":{"174":1}}],["414646797963396e",{"2":{"258":1}}],["41474f",{"2":{"256":1}}],["414769",{"2":{"166":1}}],["4144732",{"2":{"137":1}}],["410387820355415e",{"2":{"258":1}}],["410399im",{"2":{"162":1}}],["4104203f",{"2":{"256":1}}],["41001",{"2":{"244":1}}],["4101033",{"2":{"136":1}}],["4122403897882985e",{"2":{"258":1}}],["412515931943289e",{"2":{"258":1}}],["412545f",{"2":{"256":1}}],["4121472f",{"2":{"256":1}}],["41207",{"2":{"182":1}}],["41238892",{"2":{"136":1}}],["41246277",{"2":{"120":1}}],["4112944",{"2":{"106":1}}],["4112945",{"2":{"106":1}}],["430622331914719e",{"2":{"258":1}}],["4306223f",{"2":{"256":1}}],["43001",{"2":{"244":1}}],["4305115e",{"2":{"141":1}}],["4305781",{"2":{"120":1}}],["43501",{"2":{"244":1}}],["4353992",{"2":{"213":1}}],["43551",{"2":{"182":1}}],["432619e",{"2":{"208":1}}],["4322f",{"2":{"80":1}}],["4371308",{"2":{"213":1}}],["43783",{"2":{"182":1}}],["4376572711003852",{"2":{"164":2}}],["43af",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["4330709",{"2":{"213":2}}],["4339262",{"2":{"174":1}}],["433656",{"2":{"162":1}}],["43350348",{"2":{"136":1}}],["431861561350837e",{"2":{"258":1}}],["4318977",{"2":{"120":1}}],["431629f",{"2":{"256":1}}],["4319346874291314",{"2":{"166":1}}],["4314566",{"2":{"137":1}}],["43676835",{"2":{"136":1}}],["439655395962534e",{"2":{"258":1}}],["4398596f",{"2":{"256":1}}],["4397957",{"2":{"134":1}}],["4391564",{"2":{"106":1}}],["43",{"2":{"95":1,"191":1,"225":4,"233":1}}],["4382721",{"2":{"213":1}}],["43846482",{"2":{"174":1}}],["438519",{"2":{"106":1}}],["43866f",{"2":{"80":1}}],["4389",{"2":{"71":1}}],["4749576273166298e",{"2":{"258":1}}],["4748131f",{"2":{"256":1}}],["4791",{"2":{"249":1}}],["4702704f",{"2":{"256":1}}],["47023",{"2":{"182":1}}],["47001",{"2":{"244":1}}],["4784s\\ttraining",{"2":{"235":1}}],["47872233",{"2":{"134":1}}],["477605501713446e",{"2":{"258":1}}],["477077686641228e",{"2":{"258":1}}],["4774925f",{"2":{"256":1}}],["4771065f",{"2":{"256":1}}],["4778283",{"2":{"213":1}}],["47777",{"2":{"182":1}}],["4761046f",{"2":{"256":1}}],["476",{"2":{"208":1}}],["4763277207638013",{"2":{"164":1}}],["4763277207638008",{"2":{"164":1}}],["47",{"2":{"188":2,"191":1,"225":4}}],["473215789617122e",{"2":{"258":1}}],["4731226f",{"2":{"256":1}}],["47391",{"2":{"182":1}}],["4730743722547668",{"2":{"164":2}}],["475809156010668e",{"2":{"258":1}}],["47501",{"2":{"244":1}}],["47505",{"2":{"182":1}}],["47526",{"2":{"182":1}}],["47551f",{"2":{"80":1}}],["472568102937236e",{"2":{"258":1}}],["472669f",{"2":{"256":1}}],["472",{"2":{"69":1}}],["4d",{"2":{"73":3,"107":2}}],["49152067781239e",{"2":{"258":1}}],["491649f",{"2":{"256":1}}],["497523485239959e",{"2":{"258":1}}],["497117481740846e",{"2":{"258":1}}],["497651f",{"2":{"256":1}}],["4972449f",{"2":{"256":1}}],["49787",{"2":{"182":1}}],["4937",{"2":{"249":1}}],["4936414",{"2":{"213":1}}],["49501",{"2":{"244":1}}],["49001",{"2":{"244":1}}],["4901161f",{"2":{"106":6}}],["49012f",{"2":{"106":2}}],["4962754597401675e",{"2":{"258":1}}],["4962962f",{"2":{"256":1}}],["4968",{"2":{"249":1}}],["4966s\\ttraining",{"2":{"235":1}}],["496063",{"2":{"213":2}}],["4922s\\ttraining",{"2":{"233":1}}],["4981s\\ttraining",{"2":{"235":1}}],["4981351",{"2":{"106":1}}],["49804282",{"2":{"213":1}}],["499201f",{"2":{"256":1}}],["4999s\\ttraining",{"2":{"235":1}}],["49998975",{"2":{"106":1}}],["4996s\\ttraining",{"2":{"233":1}}],["49913f",{"2":{"80":1}}],["49",{"2":{"59":1,"191":1,"214":1,"217":1,"225":2}}],["4490909958627636e",{"2":{"258":1}}],["44995f",{"2":{"256":1}}],["449272f",{"2":{"256":1}}],["4448305944960618e",{"2":{"258":1}}],["444591127976329e",{"2":{"258":1}}],["44449f",{"2":{"256":1}}],["44439f",{"2":{"80":1}}],["441404523245204e",{"2":{"258":1}}],["441893264403294e",{"2":{"258":1}}],["4418456",{"2":{"174":1}}],["4416214f",{"2":{"256":1}}],["4416404f",{"2":{"256":1}}],["4472",{"2":{"249":1}}],["44746",{"2":{"166":1}}],["4420",{"2":{"249":1}}],["4450914722628342e",{"2":{"258":1}}],["44501",{"2":{"244":1}}],["4456",{"2":{"249":1}}],["44571686",{"2":{"137":1}}],["440316279559432e",{"2":{"258":1}}],["440349f",{"2":{"256":1}}],["44001",{"2":{"244":1}}],["44096947",{"2":{"216":1}}],["44078717",{"2":{"213":1}}],["44048917",{"2":{"174":1}}],["4460484365718736e",{"2":{"258":1}}],["446950899084912e",{"2":{"258":1}}],["44691",{"2":{"182":1}}],["44617f",{"2":{"256":1}}],["4468226f",{"2":{"256":1}}],["446214",{"2":{"136":1}}],["44868",{"2":{"182":1}}],["44811",{"2":{"182":1}}],["4488101521328277",{"2":{"169":1}}],["44807646",{"2":{"107":1}}],["4439217677594005e",{"2":{"258":1}}],["44397",{"2":{"137":1}}],["4432705f",{"2":{"256":1}}],["4438652f",{"2":{"256":1}}],["4437156",{"2":{"136":1}}],["44",{"2":{"34":1,"225":2,"233":1,"249":1}}],["4",{"2":{"2":1,"9":1,"31":1,"32":1,"34":8,"52":2,"53":2,"59":2,"67":1,"69":4,"71":1,"72":5,"80":3,"81":1,"86":1,"106":2,"107":4,"115":8,"118":6,"120":5,"123":8,"125":1,"129":2,"130":2,"134":9,"135":2,"136":3,"137":11,"141":7,"149":2,"164":16,"166":15,"168":5,"171":2,"174":1,"175":3,"182":17,"184":3,"188":2,"190":1,"191":2,"193":8,"194":4,"210":3,"213":6,"216":1,"218":3,"221":1,"225":2,"226":2,"233":5,"235":1,"236":36,"237":2,"244":9,"246":3,"248":9,"249":20,"254":7,"255":1,"256":102,"258":100}}],["0f4",{"2":{"255":1}}],["0f",{"2":{"190":1,"231":2}}],["0f0",{"2":{"59":2,"81":2,"117":2,"118":1,"141":2,"178":2,"206":4,"213":2,"230":4,"234":2,"243":8,"245":6,"248":4,"255":1}}],["0+560",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["0x1911b814c02405e8",{"2":{"169":1}}],["0x12c522b8034ae186",{"2":{"120":1}}],["0x8c49bc52dc8a77ea",{"2":{"169":1}}],["0x8e0c3a65079041bb",{"2":{"120":1}}],["0x48d73dc42d195740",{"2":{"169":1}}],["0x4fa3403dd074e603",{"2":{"120":1}}],["0xdb2fa90498613fdf",{"2":{"169":1}}],["0x22a21880af5dc689",{"2":{"120":1,"169":1}}],["0x21617f7747d97206",{"2":{"120":1}}],["0e",{"2":{"80":7,"215":1,"217":1}}],["0810577802704876e",{"2":{"258":1}}],["0813",{"2":{"249":1}}],["082086",{"2":{"217":1}}],["0829282",{"2":{"134":1}}],["083291894757975e",{"2":{"258":1}}],["083253303418634e",{"2":{"258":1}}],["0831015f",{"2":{"256":1}}],["08342",{"2":{"182":1}}],["0836414",{"2":{"162":1}}],["0842",{"2":{"249":1}}],["0843",{"2":{"249":1}}],["08470",{"2":{"182":1}}],["08456781",{"2":{"80":1}}],["08",{"2":{"172":1}}],["085647262362489e",{"2":{"258":1}}],["0855837f",{"2":{"256":1}}],["08517",{"2":{"166":1}}],["085170805",{"2":{"166":1}}],["0851708",{"2":{"166":2}}],["08526564",{"2":{"80":1}}],["088932604765186e",{"2":{"258":1}}],["088176817632428e",{"2":{"258":1}}],["0881303f",{"2":{"256":1}}],["088046f",{"2":{"256":1}}],["0885",{"2":{"249":1}}],["0887881f",{"2":{"256":1}}],["0887",{"2":{"249":1}}],["08820387",{"2":{"213":1}}],["088205874",{"2":{"106":1}}],["0884834705765853",{"2":{"164":2}}],["0867",{"2":{"249":1}}],["0865",{"2":{"249":1}}],["0866141",{"2":{"213":2}}],["08667634",{"2":{"106":2}}],["08694684883050086",{"2":{"169":4}}],["0869468",{"2":{"166":1}}],["086429894",{"2":{"136":1}}],["0879133750331555e",{"2":{"258":1}}],["08791955",{"2":{"106":1}}],["087215296099596e",{"2":{"258":1}}],["087667f",{"2":{"256":1}}],["0876",{"2":{"249":1}}],["08760",{"2":{"182":1}}],["08713347",{"2":{"131":1}}],["08922641",{"2":{"213":1}}],["08922",{"2":{"182":1}}],["089530945",{"2":{"106":1}}],["08953094",{"2":{"106":1}}],["08993f",{"2":{"80":1}}],["080841f",{"2":{"256":1}}],["08085155",{"2":{"81":1}}],["080046f",{"2":{"256":1}}],["0803379",{"2":{"213":1}}],["08028",{"2":{"182":1}}],["08022",{"2":{"24":1,"72":1}}],["0801",{"2":{"81":1}}],["03f0",{"2":{"215":1}}],["03",{"2":{"208":4,"215":1,"217":1}}],["036351404834028e",{"2":{"258":1}}],["0365556f",{"2":{"256":1}}],["03666s\\ttraining",{"2":{"225":1}}],["03682s\\ttraining",{"2":{"225":1}}],["03686",{"2":{"182":1}}],["03641s\\ttraining",{"2":{"225":1}}],["03641",{"2":{"182":1}}],["034694161183524e",{"2":{"258":1}}],["034911f",{"2":{"256":1}}],["03479s\\ttraining",{"2":{"225":1}}],["03427s\\ttraining",{"2":{"225":1}}],["03428s\\ttraining",{"2":{"225":1}}],["034259234",{"2":{"174":1}}],["03405",{"2":{"182":1}}],["03407s\\ttraining",{"2":{"225":2}}],["03407",{"2":{"182":1}}],["03413",{"2":{"182":1}}],["03459",{"2":{"182":1}}],["03704",{"2":{"233":3}}],["03705",{"2":{"182":1}}],["03729",{"2":{"182":1}}],["03798",{"2":{"182":1}}],["03779",{"2":{"182":1}}],["037500158",{"2":{"106":1}}],["0351993168584786e",{"2":{"258":1}}],["035587200219031e",{"2":{"258":1}}],["035556547",{"2":{"244":1}}],["0355555",{"2":{"166":1}}],["0350863f",{"2":{"256":1}}],["0350433",{"2":{"80":1}}],["0357146f",{"2":{"256":1}}],["03524",{"2":{"182":1}}],["033320315",{"2":{"244":1}}],["03339",{"2":{"182":1}}],["03395s\\ttraining",{"2":{"225":1}}],["03322105",{"2":{"213":1}}],["03388",{"2":{"182":1}}],["0338569",{"2":{"136":1}}],["03342",{"2":{"182":1}}],["03367",{"2":{"182":1}}],["0330944",{"2":{"80":1}}],["03306928",{"2":{"80":1}}],["0312354753107738e",{"2":{"258":1}}],["0310642f",{"2":{"256":1}}],["031768262",{"2":{"244":1}}],["03179s\\ttraining",{"2":{"225":1}}],["03184s\\ttraining",{"2":{"225":1}}],["03196",{"2":{"182":1}}],["031414255389526885",{"2":{"164":2}}],["031403534",{"2":{"106":1}}],["031403527",{"2":{"106":1}}],["03154",{"2":{"182":1}}],["031503614",{"2":{"106":1}}],["0315817",{"2":{"80":1}}],["032705",{"2":{"213":1}}],["03278",{"2":{"182":1}}],["03254",{"2":{"182":1}}],["03258145",{"2":{"174":1}}],["03202",{"2":{"182":1}}],["0320963",{"2":{"106":1}}],["032846544",{"2":{"136":1}}],["03261588",{"2":{"106":1}}],["032615878",{"2":{"106":1}}],["032405667",{"2":{"106":1}}],["032405674",{"2":{"106":1}}],["0324213",{"2":{"106":1}}],["03242131",{"2":{"106":1}}],["03952",{"2":{"182":1}}],["03965",{"2":{"182":1}}],["03960",{"2":{"182":1}}],["039647065",{"2":{"106":2}}],["039132368",{"2":{"80":1}}],["0308782431336086e",{"2":{"258":1}}],["030881835",{"2":{"106":2}}],["0307575f",{"2":{"256":1}}],["03070261",{"2":{"107":1}}],["030331207",{"2":{"244":1}}],["03058s\\ttraining",{"2":{"225":1}}],["03050s\\ttraining",{"2":{"225":1}}],["03050",{"2":{"182":1}}],["030016",{"2":{"217":1}}],["03003",{"2":{"182":1}}],["03042",{"2":{"182":1}}],["030990155271338e",{"2":{"258":1}}],["0309327f",{"2":{"256":1}}],["03091",{"2":{"182":1}}],["030926049",{"2":{"106":1}}],["030926052",{"2":{"106":1}}],["03094203",{"2":{"80":1}}],["0301",{"2":{"81":1}}],["0302464",{"2":{"80":1}}],["03820",{"2":{"182":1}}],["03869",{"2":{"182":1}}],["0381317",{"2":{"137":1}}],["0381727",{"2":{"136":1}}],["03804328",{"2":{"80":1}}],["038",{"2":{"72":3}}],["02f0",{"2":{"245":2}}],["02",{"2":{"208":17,"225":2}}],["025718f",{"2":{"256":1}}],["0256",{"2":{"249":1}}],["025281426",{"2":{"244":1}}],["02524",{"2":{"182":1}}],["025970555854449e",{"2":{"258":1}}],["02598s\\ttraining",{"2":{"225":1}}],["02593s\\ttraining",{"2":{"225":1}}],["02557s\\ttraining",{"2":{"225":1}}],["025583776",{"2":{"244":1}}],["02558",{"2":{"182":1}}],["02580s\\ttraining",{"2":{"225":1}}],["02586",{"2":{"182":1}}],["02504s\\ttraining",{"2":{"225":1}}],["025140665",{"2":{"244":1}}],["02514s\\ttraining",{"2":{"225":1}}],["02517",{"2":{"182":1}}],["02516",{"2":{"182":1}}],["02510",{"2":{"182":1}}],["02535537",{"2":{"174":1}}],["02535769",{"2":{"136":1}}],["02142s\\ttraining",{"2":{"225":1}}],["02149s\\ttraining",{"2":{"225":1}}],["02143306",{"2":{"107":1}}],["02191s\\ttraining",{"2":{"225":2}}],["02199s\\ttraining",{"2":{"225":1}}],["021166507",{"2":{"244":1}}],["02114s\\ttraining",{"2":{"225":1}}],["02119s\\ttraining",{"2":{"225":1}}],["02118",{"2":{"182":1}}],["02179s\\ttraining",{"2":{"225":1}}],["02177s\\ttraining",{"2":{"225":1}}],["02173s\\ttraining",{"2":{"225":1}}],["02161s\\ttraining",{"2":{"225":2}}],["02120s\\ttraining",{"2":{"225":1}}],["02121",{"2":{"182":1}}],["02133s\\ttraining",{"2":{"225":1}}],["02136s\\ttraining",{"2":{"225":1}}],["02135",{"2":{"182":1}}],["02159",{"2":{"182":1}}],["02156",{"2":{"182":1}}],["02106s\\ttraining",{"2":{"225":1}}],["02101s\\ttraining",{"2":{"225":2}}],["02107",{"2":{"182":1}}],["02103",{"2":{"182":1}}],["02183",{"2":{"182":1}}],["029164422",{"2":{"244":1}}],["02938s\\ttraining",{"2":{"225":1}}],["02957s\\ttraining",{"2":{"225":1}}],["02941s\\ttraining",{"2":{"225":1}}],["02909s\\ttraining",{"2":{"225":1}}],["02963",{"2":{"182":1}}],["02964765308691042",{"2":{"169":4}}],["0296477",{"2":{"166":1}}],["02993",{"2":{"182":1}}],["02971",{"2":{"182":1}}],["029290024",{"2":{"80":1}}],["0288",{"2":{"249":1}}],["028305896",{"2":{"244":1}}],["028128654",{"2":{"244":1}}],["02816",{"2":{"182":1}}],["02858s\\ttraining",{"2":{"225":1}}],["02859s\\ttraining",{"2":{"225":1}}],["02859",{"2":{"182":1}}],["0285962",{"2":{"136":1}}],["02864s\\ttraining",{"2":{"225":1}}],["02860",{"2":{"182":1}}],["02840",{"2":{"182":1}}],["028461456",{"2":{"120":1}}],["02898",{"2":{"182":1}}],["02899",{"2":{"182":1}}],["02877s\\ttraining",{"2":{"225":1}}],["02875",{"2":{"182":1}}],["02878",{"2":{"182":2}}],["028738942",{"2":{"136":1}}],["02826",{"2":{"182":1}}],["02825",{"2":{"182":1}}],["028251555",{"2":{"80":1}}],["02827571",{"2":{"106":1}}],["028275706",{"2":{"106":1}}],["0279214580006563e",{"2":{"258":1}}],["0279729f",{"2":{"256":1}}],["02797s\\ttraining",{"2":{"225":1}}],["0276934697509827e",{"2":{"258":1}}],["0276937966449545e",{"2":{"258":1}}],["027615f",{"2":{"256":1}}],["027620461",{"2":{"244":1}}],["027273500",{"2":{"244":1}}],["027226416",{"2":{"106":1}}],["027226413",{"2":{"106":1}}],["02782s\\ttraining",{"2":{"225":1}}],["0273722f",{"2":{"256":1}}],["02732s\\ttraining",{"2":{"225":1}}],["02736s\\ttraining",{"2":{"225":1}}],["02745s\\ttraining",{"2":{"225":1}}],["027746114181075e",{"2":{"258":1}}],["0277897",{"2":{"213":1}}],["027702922",{"2":{"80":1}}],["02750",{"2":{"182":1}}],["0209",{"2":{"249":1}}],["02096s\\ttraining",{"2":{"225":1}}],["02098s\\ttraining",{"2":{"225":1}}],["020624701",{"2":{"244":1}}],["02066s\\ttraining",{"2":{"225":1}}],["02063s\\ttraining",{"2":{"225":1}}],["02069",{"2":{"182":1}}],["020095f",{"2":{"256":1}}],["02000s\\ttraining",{"2":{"225":1}}],["02003",{"2":{"182":1}}],["0204",{"2":{"249":1}}],["020493284",{"2":{"244":1}}],["02045s\\ttraining",{"2":{"225":1}}],["02041s\\ttraining",{"2":{"225":1}}],["02048s\\ttraining",{"2":{"225":1}}],["02044",{"2":{"182":1}}],["020389127",{"2":{"244":1}}],["02038s\\ttraining",{"2":{"225":1}}],["02034s\\ttraining",{"2":{"225":1}}],["02030s\\ttraining",{"2":{"225":1}}],["02036",{"2":{"182":1}}],["02057",{"2":{"182":1}}],["02077s\\ttraining",{"2":{"225":1}}],["0207279",{"2":{"129":1}}],["0207926",{"2":{"80":1}}],["0201",{"2":{"81":1}}],["02325s\\ttraining",{"2":{"225":1}}],["02321s\\ttraining",{"2":{"225":1}}],["02324",{"2":{"182":1}}],["02332s\\ttraining",{"2":{"225":1}}],["02359s\\ttraining",{"2":{"225":1}}],["02362s\\ttraining",{"2":{"225":1}}],["023622",{"2":{"213":2}}],["0236516",{"2":{"162":1}}],["02310",{"2":{"182":1}}],["0231775",{"2":{"80":1}}],["02303",{"2":{"182":1}}],["023031702",{"2":{"80":1}}],["023423782662333e",{"2":{"258":1}}],["02342",{"2":{"182":1}}],["02391",{"2":{"182":1}}],["02234s\\ttraining",{"2":{"225":1}}],["02233",{"2":{"182":1}}],["022739200",{"2":{"244":1}}],["022735020",{"2":{"244":1}}],["02270s\\ttraining",{"2":{"225":2}}],["0227737",{"2":{"80":1}}],["022150228",{"2":{"244":1}}],["02216s\\ttraining",{"2":{"225":1}}],["02218s\\ttraining",{"2":{"225":1}}],["02212s\\ttraining",{"2":{"225":1}}],["02212",{"2":{"182":1}}],["022293953",{"2":{"244":1}}],["02226s\\ttraining",{"2":{"225":1}}],["02225s\\ttraining",{"2":{"225":1}}],["02223s\\ttraining",{"2":{"225":1}}],["02224s\\ttraining",{"2":{"225":1}}],["022215",{"2":{"217":1}}],["0228583f",{"2":{"256":1}}],["02288s\\ttraining",{"2":{"225":1}}],["02282s\\ttraining",{"2":{"225":1}}],["0228109822209213",{"2":{"164":2}}],["02200s\\ttraining",{"2":{"225":1}}],["02201",{"2":{"182":1}}],["02296s\\ttraining",{"2":{"225":1}}],["02296",{"2":{"182":1}}],["02294",{"2":{"182":2}}],["02257s\\ttraining",{"2":{"225":1}}],["02259",{"2":{"182":1}}],["02250",{"2":{"182":1}}],["02246s\\ttraining",{"2":{"225":1}}],["02244",{"2":{"182":1}}],["022434518",{"2":{"80":1}}],["022618050",{"2":{"244":1}}],["02264",{"2":{"182":1}}],["02266",{"2":{"182":1}}],["026301135713982e",{"2":{"258":1}}],["026358f",{"2":{"256":1}}],["026170366",{"2":{"244":1}}],["02614s\\ttraining",{"2":{"225":1}}],["02601s\\ttraining",{"2":{"225":1}}],["02608",{"2":{"182":1}}],["02669s\\ttraining",{"2":{"225":1}}],["026282774",{"2":{"244":1}}],["02628",{"2":{"182":1}}],["02622s\\ttraining",{"2":{"225":1}}],["02622",{"2":{"182":1}}],["02671",{"2":{"182":1}}],["02694",{"2":{"182":1}}],["026851978",{"2":{"80":1}}],["026516732",{"2":{"80":1}}],["0241523762792873e",{"2":{"258":1}}],["0242",{"2":{"249":1}}],["02427s\\ttraining",{"2":{"225":1}}],["0240308f",{"2":{"256":1}}],["0240",{"2":{"249":1}}],["02499s\\ttraining",{"2":{"225":1}}],["0246",{"2":{"249":1}}],["02468",{"2":{"182":1}}],["02467",{"2":{"182":2}}],["02435",{"2":{"182":1}}],["02489",{"2":{"182":1}}],["02447",{"2":{"182":1}}],["02449",{"2":{"182":1}}],["0244548f",{"2":{"106":1}}],["0244214",{"2":{"80":1}}],["024",{"2":{"80":1}}],["09",{"2":{"225":3,"249":1}}],["09406",{"2":{"182":1}}],["09495",{"2":{"182":1}}],["0935077",{"2":{"254":2}}],["09351656",{"2":{"172":1,"173":1}}],["093925126",{"2":{"174":1}}],["091738287416977e",{"2":{"258":1}}],["0919208131320155e",{"2":{"258":1}}],["0916253f",{"2":{"256":1}}],["091892f",{"2":{"256":1}}],["0913593358445652",{"2":{"164":1}}],["091359335844565",{"2":{"164":1}}],["0914626",{"2":{"137":1}}],["0987890542689164e",{"2":{"258":1}}],["09875",{"2":{"182":1}}],["0987538",{"2":{"162":1}}],["098992f",{"2":{"256":1}}],["09852",{"2":{"166":1}}],["0985227",{"2":{"166":2}}],["098861866",{"2":{"80":1}}],["09276",{"2":{"182":1}}],["09244",{"2":{"182":1}}],["09299411",{"2":{"107":1}}],["09289699",{"2":{"106":1}}],["0965",{"2":{"249":1}}],["09693718",{"2":{"216":1}}],["09699093",{"2":{"136":1}}],["096988946",{"2":{"120":1}}],["09692798",{"2":{"80":1}}],["09643307",{"2":{"106":2}}],["0956541099726786e",{"2":{"258":1}}],["0955096f",{"2":{"256":1}}],["095568806",{"2":{"216":1}}],["0954186",{"2":{"106":1}}],["09599f",{"2":{"80":1}}],["099080602537575e",{"2":{"258":1}}],["0993232f",{"2":{"256":1}}],["099252652302867e",{"2":{"258":1}}],["0992076f",{"2":{"256":1}}],["09926938",{"2":{"106":1}}],["0991955",{"2":{"106":1}}],["09919551",{"2":{"106":1}}],["099906564",{"2":{"106":1}}],["097948219408506e",{"2":{"258":1}}],["0978037f",{"2":{"256":1}}],["09785451",{"2":{"80":1}}],["0973",{"2":{"249":1}}],["0971",{"2":{"249":1}}],["097182155",{"2":{"106":1}}],["09701932",{"2":{"174":1}}],["097253",{"2":{"120":1}}],["09724942",{"2":{"106":1}}],["09724939",{"2":{"106":1}}],["097701035",{"2":{"106":2}}],["09070",{"2":{"182":1}}],["0903271",{"2":{"106":2}}],["0901",{"2":{"81":1}}],["09098f",{"2":{"80":1}}],["0900306",{"2":{"53":10}}],["07",{"2":{"225":2}}],["073884671976903e",{"2":{"258":1}}],["0736262194625994e",{"2":{"258":1}}],["0736777f",{"2":{"256":1}}],["073589f",{"2":{"256":1}}],["0733",{"2":{"249":1}}],["07375115",{"2":{"216":1}}],["073442355",{"2":{"80":1}}],["078019918491195e",{"2":{"258":1}}],["0782255714890612e",{"2":{"258":1}}],["0782",{"2":{"249":1}}],["0787",{"2":{"249":1}}],["07874016",{"2":{"213":2}}],["07890",{"2":{"182":1}}],["07644",{"2":{"182":1}}],["076404385",{"2":{"106":1}}],["07640441",{"2":{"106":1}}],["077927f",{"2":{"256":1}}],["07787",{"2":{"182":1}}],["07702",{"2":{"166":1}}],["0770206",{"2":{"166":2}}],["07704306",{"2":{"80":1}}],["075340574175034e",{"2":{"258":1}}],["0752132f",{"2":{"256":1}}],["07565363",{"2":{"216":1}}],["07543",{"2":{"182":1}}],["07559",{"2":{"182":1}}],["07588",{"2":{"182":1}}],["075135306",{"2":{"136":1}}],["0795567754784686e",{"2":{"258":1}}],["07963410837309e",{"2":{"258":1}}],["079613f",{"2":{"256":1}}],["07964921",{"2":{"107":1}}],["07934f",{"2":{"256":1}}],["079208925",{"2":{"118":1}}],["0742047",{"2":{"213":1}}],["07425846",{"2":{"106":1}}],["07425845",{"2":{"106":1}}],["07404",{"2":{"182":1}}],["07412187",{"2":{"80":1}}],["070489332307503e",{"2":{"258":1}}],["070884735805013e",{"2":{"258":1}}],["0700496f",{"2":{"256":1}}],["0709",{"2":{"213":1}}],["070142779814479e",{"2":{"258":1}}],["0701",{"2":{"81":1}}],["07058401",{"2":{"80":1}}],["07057328",{"2":{"80":1}}],["0710118f",{"2":{"256":1}}],["071482725",{"2":{"174":1}}],["07133968",{"2":{"109":1}}],["07167f",{"2":{"80":1}}],["07125676",{"2":{"80":1}}],["0728675615927385",{"2":{"53":1}}],["04",{"2":{"225":1}}],["0444",{"2":{"249":1}}],["04436s\\ttraining",{"2":{"225":1}}],["04416",{"2":{"217":1}}],["041541956",{"2":{"244":1}}],["04171s\\ttraining",{"2":{"225":1}}],["04115",{"2":{"182":1}}],["04111",{"2":{"182":1}}],["04191",{"2":{"182":1}}],["04198",{"2":{"182":1}}],["04199602",{"2":{"174":1}}],["04143",{"2":{"182":1}}],["041031003",{"2":{"134":1}}],["04108f",{"2":{"80":1}}],["049448585437825e",{"2":{"258":1}}],["049417756",{"2":{"106":1}}],["049417753",{"2":{"106":1}}],["0492704155778817e",{"2":{"258":1}}],["049288750691079e",{"2":{"258":1}}],["049237f",{"2":{"256":1}}],["04903387",{"2":{"174":1}}],["0439388993198004e",{"2":{"258":1}}],["0437871f",{"2":{"256":1}}],["04370",{"2":{"182":1}}],["0435",{"2":{"249":1}}],["0435721",{"2":{"80":1}}],["043883",{"2":{"166":1}}],["046389500421066e",{"2":{"258":1}}],["0464263f",{"2":{"256":1}}],["04688s\\ttraining",{"2":{"225":1}}],["04686",{"2":{"182":1}}],["04691",{"2":{"182":1}}],["0469301",{"2":{"137":1}}],["04673",{"2":{"182":1}}],["0462954",{"2":{"137":1}}],["040918160",{"2":{"244":1}}],["04077s\\ttraining",{"2":{"225":1}}],["04000",{"2":{"182":1}}],["0402095",{"2":{"136":1}}],["0401",{"2":{"81":1}}],["04019f",{"2":{"80":1}}],["04557",{"2":{"182":1}}],["04551310",{"2":{"107":1}}],["04566476",{"2":{"106":1}}],["047244094",{"2":{"213":2}}],["047897488",{"2":{"106":2}}],["047356773",{"2":{"80":1}}],["048070636485676e",{"2":{"258":1}}],["0480501",{"2":{"80":1}}],["048662454782473e",{"2":{"258":1}}],["048608f",{"2":{"256":1}}],["048975f",{"2":{"256":1}}],["04846",{"2":{"182":1}}],["048198f",{"2":{"256":1}}],["0481652f",{"2":{"256":1}}],["04811s\\ttraining",{"2":{"225":1}}],["048109",{"2":{"137":1}}],["048133414",{"2":{"106":2}}],["048522998",{"2":{"123":1}}],["04853325",{"2":{"106":1}}],["048533253",{"2":{"106":1}}],["048811335",{"2":{"106":2}}],["04d",{"2":{"81":1}}],["042037274184826e",{"2":{"258":1}}],["0420834",{"2":{"80":1}}],["042499166",{"2":{"244":1}}],["0428307",{"2":{"213":1}}],["04284",{"2":{"182":1}}],["04253",{"2":{"182":1}}],["0425432",{"2":{"136":1}}],["04298",{"2":{"182":1}}],["04265762",{"2":{"174":1}}],["042254835",{"2":{"106":1}}],["04225484",{"2":{"106":1}}],["042256515",{"2":{"106":1}}],["04224006",{"2":{"80":1}}],["04226003",{"2":{"80":1}}],["0619583297677294e",{"2":{"258":1}}],["0616338f",{"2":{"256":1}}],["0618",{"2":{"249":1}}],["0613",{"2":{"249":1}}],["06123",{"2":{"182":1}}],["067580179181019e",{"2":{"258":1}}],["0677514f",{"2":{"256":1}}],["067828169553457e",{"2":{"258":1}}],["067828f",{"2":{"256":1}}],["067872316",{"2":{"80":1}}],["0672",{"2":{"249":1}}],["069179f",{"2":{"256":1}}],["0690",{"2":{"249":1}}],["0690823",{"2":{"216":1}}],["06942138",{"2":{"80":1}}],["0689f",{"2":{"256":1}}],["0686756",{"2":{"213":1}}],["06837",{"2":{"182":1}}],["064211361154162e",{"2":{"258":1}}],["0641746f",{"2":{"256":1}}],["06419",{"2":{"182":1}}],["0640855f",{"2":{"256":1}}],["0640122f",{"2":{"256":1}}],["0643809424456035e",{"2":{"258":1}}],["06433817",{"2":{"213":1}}],["06436",{"2":{"182":1}}],["06450",{"2":{"24":1}}],["062907",{"2":{"217":1}}],["06270",{"2":{"182":1}}],["062155924699545",{"2":{"164":1}}],["0621559246995447",{"2":{"164":1}}],["062079933",{"2":{"136":1}}],["066640621357982e",{"2":{"258":1}}],["06669",{"2":{"125":2}}],["066305f",{"2":{"256":1}}],["06657f",{"2":{"256":1}}],["06616s\\ttraining",{"2":{"225":1}}],["066128",{"2":{"137":1}}],["06612328",{"2":{"107":1}}],["06618893",{"2":{"106":1}}],["06675106",{"2":{"106":1}}],["066751055",{"2":{"106":1}}],["063462388366766e",{"2":{"258":1}}],["063480526",{"2":{"80":1}}],["0639607059931383e",{"2":{"258":1}}],["0633179f",{"2":{"256":1}}],["0633",{"2":{"249":1}}],["06301335",{"2":{"106":1}}],["06325f",{"2":{"80":1}}],["06097",{"2":{"182":1}}],["06027697",{"2":{"106":1}}],["06026962",{"2":{"80":1}}],["0601",{"2":{"81":1}}],["060364496",{"2":{"80":1}}],["06053336",{"2":{"80":1}}],["06519687",{"2":{"107":1}}],["06550143",{"2":{"80":1}}],["065317236",{"2":{"80":1}}],["06",{"2":{"53":1,"191":2,"225":1}}],["052273409778598e",{"2":{"258":1}}],["05253045212093e",{"2":{"258":1}}],["052526f",{"2":{"256":1}}],["052403f",{"2":{"256":1}}],["05212",{"2":{"182":1}}],["05218964",{"2":{"174":1}}],["059373882676194e",{"2":{"258":1}}],["05917f",{"2":{"256":1}}],["059518166",{"2":{"106":2}}],["05f0",{"2":{"244":2,"245":1}}],["0546",{"2":{"249":1}}],["05453s\\ttraining",{"2":{"225":1}}],["054718e",{"2":{"208":1}}],["0540",{"2":{"249":1}}],["05409",{"2":{"182":1}}],["0540537",{"2":{"80":1}}],["05s",{"2":{"191":1}}],["0584",{"2":{"249":1}}],["05821",{"2":{"182":1}}],["0585f",{"2":{"80":1}}],["0510",{"2":{"249":1}}],["051912e",{"2":{"208":1}}],["05115",{"2":{"182":1}}],["05125",{"2":{"182":1}}],["055989884115363e",{"2":{"258":1}}],["0551181",{"2":{"213":2}}],["05515",{"2":{"182":1}}],["05552",{"2":{"182":1}}],["05587",{"2":{"182":1}}],["05574",{"2":{"182":1}}],["05506",{"2":{"182":1}}],["0572646196095261e",{"2":{"258":1}}],["05729505",{"2":{"106":1}}],["057538548585531e",{"2":{"258":1}}],["057462009083322e",{"2":{"258":1}}],["0574115f",{"2":{"256":1}}],["057491f",{"2":{"256":1}}],["0578545",{"2":{"216":1}}],["05732",{"2":{"182":1}}],["057005208",{"2":{"172":1,"173":1}}],["05779423",{"2":{"120":1}}],["05359831175501e",{"2":{"258":1}}],["0535775f",{"2":{"256":1}}],["0534",{"2":{"249":1}}],["05341",{"2":{"182":1}}],["0536",{"2":{"249":1}}],["053627778",{"2":{"106":1}}],["05305s\\ttraining",{"2":{"225":1}}],["053747915",{"2":{"106":2}}],["056591985203844e",{"2":{"258":1}}],["05657739601024536",{"2":{"169":1}}],["056347985817206e",{"2":{"258":1}}],["0562965f",{"2":{"256":1}}],["0568",{"2":{"249":1}}],["0569401f",{"2":{"256":1}}],["0569",{"2":{"249":1}}],["0560605f",{"2":{"256":1}}],["05600s\\ttraining",{"2":{"225":1}}],["05601",{"2":{"182":1}}],["0564903986057956",{"2":{"164":2}}],["056",{"2":{"81":3,"256":1}}],["05099385134345e",{"2":{"258":1}}],["0509233f",{"2":{"256":1}}],["0501",{"2":{"81":1}}],["05058f",{"2":{"80":1}}],["050089",{"2":{"80":1}}],["05",{"2":{"53":1,"225":1,"249":1}}],["00",{"2":{"191":8,"225":42}}],["008724037",{"2":{"244":1}}],["008772802",{"2":{"244":1}}],["008732248",{"2":{"244":1}}],["00878",{"2":{"182":3}}],["008153505",{"2":{"244":1}}],["008185850",{"2":{"244":1}}],["00812",{"2":{"182":1}}],["008405063",{"2":{"244":1}}],["008404830",{"2":{"244":1}}],["00849",{"2":{"182":1}}],["008074892",{"2":{"244":1}}],["008014920",{"2":{"244":1}}],["00806",{"2":{"182":1}}],["008026988",{"2":{"244":1}}],["00802",{"2":{"182":1}}],["008556721",{"2":{"244":1}}],["00852",{"2":{"182":1}}],["00851",{"2":{"182":1}}],["00856",{"2":{"182":1}}],["00850",{"2":{"182":2}}],["008925664",{"2":{"244":1}}],["00895",{"2":{"182":1}}],["00893",{"2":{"182":1}}],["008808360",{"2":{"244":1}}],["00887",{"2":{"182":1}}],["008870317",{"2":{"80":1}}],["00883",{"2":{"182":1}}],["00833",{"2":{"182":1}}],["00829",{"2":{"182":1}}],["00821",{"2":{"182":1}}],["00828",{"2":{"182":1}}],["00825",{"2":{"182":1}}],["00824",{"2":{"182":1}}],["00823197",{"2":{"166":1}}],["00866",{"2":{"182":1}}],["007802235329321e",{"2":{"258":1}}],["007885863",{"2":{"244":1}}],["00788",{"2":{"182":1}}],["007478f",{"2":{"256":1}}],["00745",{"2":{"182":1}}],["007220342",{"2":{"244":1}}],["007273634",{"2":{"244":1}}],["007249862",{"2":{"106":1}}],["0072498624",{"2":{"106":1}}],["007325693",{"2":{"244":1}}],["007372384",{"2":{"244":1}}],["007333768",{"2":{"244":1}}],["007363341",{"2":{"244":1}}],["0073489705",{"2":{"106":2}}],["007187600",{"2":{"244":1}}],["00713",{"2":{"182":1}}],["007129070",{"2":{"244":1}}],["00712",{"2":{"182":1}}],["007947534",{"2":{"244":1}}],["007950513",{"2":{"244":1}}],["007905648",{"2":{"244":1}}],["00790",{"2":{"182":2}}],["007984845",{"2":{"136":1}}],["007676640",{"2":{"244":1}}],["007679918",{"2":{"244":1}}],["00766",{"2":{"182":1}}],["0076355f",{"2":{"256":1}}],["00763",{"2":{"182":1}}],["007501942",{"2":{"244":1}}],["007509941",{"2":{"244":1}}],["007593059",{"2":{"244":1}}],["007535893",{"2":{"244":1}}],["00755",{"2":{"182":1}}],["00754",{"2":{"182":1}}],["007075997",{"2":{"244":1}}],["00701",{"2":{"182":1}}],["00700",{"2":{"182":1}}],["00706411",{"2":{"80":1}}],["007736761530801e",{"2":{"258":1}}],["00773",{"2":{"182":1}}],["00775",{"2":{"182":3}}],["00777",{"2":{"182":1}}],["00771",{"2":{"182":1}}],["00776",{"2":{"182":2}}],["009083126",{"2":{"244":1}}],["009084041f0",{"2":{"53":1}}],["009895653",{"2":{"244":1}}],["009887908",{"2":{"244":1}}],["00981",{"2":{"182":1}}],["009196416",{"2":{"244":1}}],["00914141",{"2":{"174":1}}],["00937",{"2":{"182":1}}],["009313912",{"2":{"244":1}}],["00931",{"2":{"182":1}}],["00948",{"2":{"182":1}}],["00949",{"2":{"182":1}}],["009256126",{"2":{"244":1}}],["00929",{"2":{"182":1}}],["00924",{"2":{"182":2}}],["009766437",{"2":{"213":1}}],["00970",{"2":{"182":1}}],["00973899",{"2":{"125":2}}],["009693242",{"2":{"244":1}}],["00966",{"2":{"182":1}}],["00961",{"2":{"182":1}}],["00968",{"2":{"162":1}}],["009987052",{"2":{"244":1}}],["00997",{"2":{"182":1}}],["009963844931984",{"2":{"164":2}}],["009514628",{"2":{"244":1}}],["009516605",{"2":{"244":1}}],["00953",{"2":{"182":1}}],["00957",{"2":{"182":1}}],["009578831",{"2":{"80":1}}],["0055308723243565e",{"2":{"258":1}}],["0059614f",{"2":{"256":1}}],["005978104",{"2":{"244":1}}],["005315070",{"2":{"244":1}}],["005351806",{"2":{"244":1}}],["0053856913",{"2":{"80":1}}],["005200472",{"2":{"244":1}}],["005807751",{"2":{"244":1}}],["005835793",{"2":{"244":1}}],["005824474",{"2":{"244":1}}],["0054907f",{"2":{"256":1}}],["005463609",{"2":{"244":1}}],["005461216",{"2":{"244":1}}],["005433898",{"2":{"244":1}}],["005112350",{"2":{"244":1}}],["005628760",{"2":{"244":1}}],["005615299",{"2":{"244":1}}],["00564f",{"2":{"80":1}}],["005f0",{"2":{"244":1}}],["005",{"2":{"208":1}}],["005065940",{"2":{"244":1}}],["005044522",{"2":{"244":1}}],["005014097",{"2":{"80":1}}],["005000000000000009",{"2":{"53":1}}],["006313350468768e",{"2":{"258":1}}],["006305961",{"2":{"244":1}}],["0060319702182694e",{"2":{"258":1}}],["006616294",{"2":{"244":1}}],["006642802",{"2":{"244":1}}],["0066712513",{"2":{"174":1}}],["006221022",{"2":{"244":1}}],["006288857",{"2":{"106":1}}],["0062888456",{"2":{"106":1}}],["006521f",{"2":{"256":1}}],["006572613",{"2":{"244":1}}],["0065859724",{"2":{"80":1}}],["006942283",{"2":{"244":1}}],["006962581",{"2":{"244":1}}],["006906096",{"2":{"244":1}}],["0069942693",{"2":{"106":1}}],["006432486",{"2":{"80":1}}],["006105572",{"2":{"244":1}}],["0061099795",{"2":{"80":1}}],["006184267",{"2":{"80":1}}],["006736353",{"2":{"80":1}}],["0067729075",{"2":{"80":1}}],["004855067",{"2":{"244":1}}],["004880759",{"2":{"244":1}}],["00488f",{"2":{"80":1}}],["004657542",{"2":{"244":1}}],["004607514",{"2":{"244":1}}],["004591393831892e",{"2":{"258":1}}],["004590358",{"2":{"244":1}}],["004518208",{"2":{"244":1}}],["004542338",{"2":{"244":1}}],["004978007",{"2":{"244":1}}],["004947479",{"2":{"244":1}}],["004958895",{"2":{"244":1}}],["004958282",{"2":{"244":1}}],["004790282",{"2":{"244":1}}],["004253434",{"2":{"244":1}}],["004287243",{"2":{"106":2}}],["0044982f",{"2":{"256":1}}],["004491198",{"2":{"244":1}}],["004434146",{"2":{"244":1}}],["004466736",{"2":{"244":1}}],["0044189203",{"2":{"80":1}}],["004397592",{"2":{"244":1}}],["004335414",{"2":{"244":1}}],["004338491",{"2":{"244":1}}],["004365115",{"2":{"244":1}}],["004326937",{"2":{"244":1}}],["004329988",{"2":{"244":1}}],["0043s\\ttraining",{"2":{"233":1}}],["004043682",{"2":{"244":1}}],["004047669",{"2":{"80":1}}],["004061943",{"2":{"244":1}}],["00407581",{"2":{"174":1}}],["004119421",{"2":{"244":1}}],["004116789",{"2":{"244":1}}],["0041674743",{"2":{"106":1}}],["0041674753",{"2":{"106":1}}],["004169049",{"2":{"80":1}}],["004144425",{"2":{"80":1}}],["001398915",{"2":{"244":1}}],["00139126",{"2":{"80":1}}],["001329915",{"2":{"244":1}}],["001325711",{"2":{"244":1}}],["001325290",{"2":{"244":1}}],["001367094",{"2":{"244":1}}],["001362987",{"2":{"244":1}}],["001362466",{"2":{"244":1}}],["001364744",{"2":{"244":1}}],["001380907",{"2":{"244":1}}],["001386037",{"2":{"244":1}}],["001346401",{"2":{"244":1}}],["001356683",{"2":{"244":1}}],["001336156",{"2":{"244":1}}],["001307026",{"2":{"244":1}}],["001303701",{"2":{"244":1}}],["001302390",{"2":{"244":1}}],["001302087",{"2":{"244":1}}],["001300987",{"2":{"244":1}}],["001309370",{"2":{"244":1}}],["001305370",{"2":{"244":1}}],["001636085",{"2":{"244":1}}],["001639404",{"2":{"244":1}}],["001659834",{"2":{"244":1}}],["001658659",{"2":{"244":1}}],["001672024",{"2":{"244":1}}],["001671217",{"2":{"244":1}}],["001649067",{"2":{"244":1}}],["001608545",{"2":{"244":1}}],["001694928",{"2":{"244":1}}],["001690981",{"2":{"244":1}}],["001615586",{"2":{"244":1}}],["001611185",{"2":{"244":1}}],["001592739",{"2":{"244":1}}],["001594031",{"2":{"244":1}}],["001531350",{"2":{"244":1}}],["0015313211",{"2":{"80":1}}],["001541014",{"2":{"244":1}}],["001513747",{"2":{"244":1}}],["001510418",{"2":{"244":1}}],["001517879",{"2":{"244":1}}],["001560361",{"2":{"244":1}}],["001566568",{"2":{"244":1}}],["001564218",{"2":{"244":1}}],["001571818",{"2":{"244":1}}],["001",{"2":{"208":1}}],["0010325599774881834",{"2":{"258":1}}],["0010350421",{"2":{"213":1}}],["001093546",{"2":{"244":1}}],["001014936",{"2":{"244":1}}],["001011056",{"2":{"244":1}}],["00101147",{"2":{"174":1}}],["001068081",{"2":{"244":1}}],["001060384",{"2":{"244":1}}],["001060399",{"2":{"244":1}}],["00106574",{"2":{"80":1}}],["001085206",{"2":{"244":1}}],["001085730",{"2":{"244":1}}],["001085884",{"2":{"244":1}}],["001089040",{"2":{"244":1}}],["001008533",{"2":{"244":1}}],["001009728",{"2":{"244":1}}],["001049175",{"2":{"244":1}}],["001044580",{"2":{"244":1}}],["001041825",{"2":{"244":1}}],["001047856",{"2":{"244":1}}],["001053359",{"2":{"244":1}}],["001050717",{"2":{"244":1}}],["001058994",{"2":{"244":1}}],["001059842",{"2":{"244":1}}],["001f0",{"2":{"81":1,"107":1,"225":1,"233":1}}],["0012",{"2":{"249":1}}],["001202180",{"2":{"244":1}}],["001209691",{"2":{"244":1}}],["001203537",{"2":{"244":1}}],["001257007",{"2":{"244":1}}],["001258897",{"2":{"244":1}}],["001272172",{"2":{"244":1}}],["001279382",{"2":{"244":1}}],["001228882",{"2":{"244":1}}],["001224264",{"2":{"244":1}}],["001240770",{"2":{"244":1}}],["001243982",{"2":{"244":1}}],["001243673",{"2":{"244":1}}],["001241080",{"2":{"244":1}}],["001245214",{"2":{"244":1}}],["001218763",{"2":{"244":1}}],["001215764",{"2":{"244":1}}],["001292084",{"2":{"244":1}}],["00126273",{"2":{"80":1}}],["001289588",{"2":{"244":1}}],["00128162",{"2":{"80":1}}],["0012843215",{"2":{"80":1}}],["001886566",{"2":{"244":1}}],["001884541",{"2":{"244":1}}],["001864895",{"2":{"244":1}}],["001825325",{"2":{"244":1}}],["001837625",{"2":{"244":1}}],["001894138",{"2":{"244":1}}],["001845157",{"2":{"244":1}}],["001849154",{"2":{"244":1}}],["001848093",{"2":{"244":1}}],["001879478",{"2":{"244":1}}],["001871931",{"2":{"244":1}}],["00187748",{"2":{"80":1}}],["0018058",{"2":{"80":1}}],["0018510937",{"2":{"80":1}}],["001448896",{"2":{"244":1}}],["001448493",{"2":{"244":1}}],["001439294",{"2":{"244":1}}],["001479216",{"2":{"244":1}}],["001498525",{"2":{"244":1}}],["001494603",{"2":{"244":1}}],["001492234",{"2":{"244":1}}],["001485110",{"2":{"244":1}}],["001454655",{"2":{"244":1}}],["00145586",{"2":{"80":1}}],["001404967",{"2":{"244":1}}],["001409405",{"2":{"244":1}}],["001402911",{"2":{"244":1}}],["00140763051",{"2":{"81":1}}],["0014175134",{"2":{"80":1}}],["001185708",{"2":{"244":1}}],["001182971",{"2":{"244":1}}],["001163361902599e",{"2":{"258":1}}],["001168622",{"2":{"244":1}}],["001160439",{"2":{"244":1}}],["001101400",{"2":{"244":1}}],["001103305",{"2":{"244":1}}],["0011130512125973e",{"2":{"258":1}}],["001113559",{"2":{"244":1}}],["001110448",{"2":{"244":1}}],["001141232",{"2":{"244":1}}],["001143823",{"2":{"244":1}}],["001148389",{"2":{"244":1}}],["001171167",{"2":{"244":1}}],["001179150",{"2":{"244":1}}],["00117228",{"2":{"80":1}}],["001126162",{"2":{"244":1}}],["001191893",{"2":{"244":1}}],["001197650",{"2":{"244":1}}],["001136703",{"2":{"244":1}}],["001135292",{"2":{"244":1}}],["001135202",{"2":{"244":1}}],["001138862",{"2":{"244":1}}],["001153235",{"2":{"244":1}}],["001154157",{"2":{"244":1}}],["00115296",{"2":{"80":1}}],["001720798",{"2":{"244":1}}],["001728887",{"2":{"244":1}}],["001789054",{"2":{"244":1}}],["001770784",{"2":{"244":1}}],["001747428",{"2":{"244":1}}],["001717428",{"2":{"244":1}}],["001714717",{"2":{"244":1}}],["001715113",{"2":{"244":1}}],["001764537",{"2":{"244":1}}],["001731674",{"2":{"244":1}}],["001730468",{"2":{"244":1}}],["0017369908",{"2":{"80":1}}],["001798752",{"2":{"244":1}}],["001798573",{"2":{"244":1}}],["001792779",{"2":{"244":1}}],["0017900232",{"2":{"80":1}}],["00175152",{"2":{"80":1}}],["00170922",{"2":{"80":1}}],["001930058",{"2":{"244":1}}],["001994215",{"2":{"244":1}}],["001959185",{"2":{"244":1}}],["0019504696",{"2":{"80":1}}],["001972227",{"2":{"244":1}}],["001975382",{"2":{"244":1}}],["001970405",{"2":{"244":1}}],["001973281",{"2":{"80":1}}],["001919697",{"2":{"244":1}}],["001913586",{"2":{"244":1}}],["0019153744",{"2":{"80":1}}],["001940841",{"2":{"244":1}}],["001982885",{"2":{"244":1}}],["00198415",{"2":{"174":1}}],["00198767",{"2":{"80":1}}],["0019262917",{"2":{"80":1}}],["0029402060874113e",{"2":{"258":1}}],["002989073",{"2":{"244":1}}],["002972272",{"2":{"244":1}}],["002935605",{"2":{"244":1}}],["002511334",{"2":{"244":1}}],["002511769",{"2":{"244":1}}],["002553018",{"2":{"244":1}}],["002556076",{"2":{"244":1}}],["002371583",{"2":{"244":1}}],["002343247",{"2":{"244":1}}],["002318957",{"2":{"244":1}}],["00230445f",{"2":{"256":1}}],["002304791",{"2":{"244":1}}],["002301548",{"2":{"244":1}}],["002303589",{"2":{"244":1}}],["002395016",{"2":{"244":1}}],["002358936",{"2":{"244":1}}],["0028186f",{"2":{"256":1}}],["002834306",{"2":{"244":1}}],["002806861",{"2":{"244":1}}],["002853032",{"2":{"244":1}}],["0028297466",{"2":{"106":1}}],["0028297475",{"2":{"106":1}}],["002670718",{"2":{"244":1}}],["002677995",{"2":{"244":1}}],["002668741",{"2":{"244":1}}],["002668936",{"2":{"244":1}}],["002606506",{"2":{"244":1}}],["002613829",{"2":{"244":1}}],["002696904",{"2":{"244":1}}],["002695386",{"2":{"244":1}}],["0026271285",{"2":{"110":1}}],["0022676488003671e",{"2":{"258":1}}],["002262217",{"2":{"244":1}}],["002236427",{"2":{"244":1}}],["002234935",{"2":{"244":1}}],["0022308934",{"2":{"80":1}}],["002276814",{"2":{"244":1}}],["002210422",{"2":{"244":1}}],["002218451",{"2":{"244":1}}],["002247394",{"2":{"244":1}}],["0022497",{"2":{"213":1}}],["002201537",{"2":{"244":1}}],["002068657",{"2":{"244":1}}],["002089650",{"2":{"244":1}}],["002081697",{"2":{"244":1}}],["002059042",{"2":{"244":1}}],["0020579377",{"2":{"106":1}}],["002057937",{"2":{"106":1}}],["002023818",{"2":{"244":1}}],["002015869",{"2":{"244":1}}],["002019164",{"2":{"244":1}}],["002037087",{"2":{"244":1}}],["00209504",{"2":{"80":1}}],["0027607225820363e",{"2":{"258":1}}],["002752088",{"2":{"244":1}}],["00270753",{"2":{"80":1}}],["0027386106",{"2":{"80":1}}],["0027955552",{"2":{"80":1}}],["002155375",{"2":{"244":1}}],["002129085",{"2":{"244":1}}],["002125610",{"2":{"244":1}}],["002123826",{"2":{"244":1}}],["0021203319",{"2":{"80":1}}],["00219681",{"2":{"80":1}}],["00214946",{"2":{"80":1}}],["002106787",{"2":{"80":1}}],["002417301",{"2":{"244":1}}],["002489866",{"2":{"244":1}}],["002474443",{"2":{"244":1}}],["002448601",{"2":{"244":1}}],["0024971329",{"2":{"80":1}}],["0024323382",{"2":{"80":1}}],["0032770909973542e",{"2":{"258":1}}],["003299790",{"2":{"244":1}}],["003374260108281e",{"2":{"258":1}}],["0033375f",{"2":{"256":1}}],["003329928",{"2":{"244":1}}],["003847855",{"2":{"244":1}}],["003813077",{"2":{"244":1}}],["0038745",{"2":{"80":1}}],["0034582f",{"2":{"256":1}}],["003463188",{"2":{"244":1}}],["003488734",{"2":{"244":1}}],["0034101289",{"2":{"80":1}}],["003180855",{"2":{"244":1}}],["003136263",{"2":{"244":1}}],["003132161",{"2":{"244":1}}],["0031376407",{"2":{"80":1}}],["003162773",{"2":{"244":1}}],["003125851",{"2":{"244":1}}],["003911218",{"2":{"244":1}}],["003920683",{"2":{"244":1}}],["00390526722",{"2":{"81":1}}],["003504427",{"2":{"244":1}}],["003519468",{"2":{"244":1}}],["00351666",{"2":{"80":1}}],["003553676",{"2":{"244":1}}],["003585588",{"2":{"244":1}}],["0036923f",{"2":{"256":1}}],["003658900",{"2":{"244":1}}],["003630795",{"2":{"244":1}}],["0036810287",{"2":{"106":1}}],["0036810301",{"2":{"106":1}}],["00364828",{"2":{"80":1}}],["003095849",{"2":{"244":1}}],["003031955",{"2":{"244":1}}],["003061282",{"2":{"244":1}}],["00306169",{"2":{"80":1}}],["003071612",{"2":{"244":1}}],["003027094",{"2":{"80":1}}],["0037725805",{"2":{"80":1}}],["003750000000000005",{"2":{"53":1}}],["000",{"2":{"250":1}}],["000846881",{"2":{"244":1}}],["000847362",{"2":{"244":1}}],["000826713",{"2":{"244":1}}],["000826687",{"2":{"244":1}}],["000824789",{"2":{"244":1}}],["000868335",{"2":{"244":1}}],["000866074",{"2":{"244":1}}],["000864926",{"2":{"244":1}}],["0008341402321897578",{"2":{"258":1}}],["000835373",{"2":{"244":1}}],["000835131",{"2":{"244":1}}],["000831873",{"2":{"244":1}}],["000887047",{"2":{"244":1}}],["000883626",{"2":{"244":1}}],["000881248",{"2":{"244":1}}],["000873007",{"2":{"244":1}}],["000871024327",{"2":{"81":1}}],["000892342",{"2":{"244":1}}],["000892691",{"2":{"244":1}}],["000890729",{"2":{"244":1}}],["000893347",{"2":{"244":1}}],["000894285",{"2":{"244":1}}],["000857553",{"2":{"244":1}}],["000809421",{"2":{"244":1}}],["000805100",{"2":{"244":1}}],["000807529",{"2":{"244":1}}],["000816396",{"2":{"244":1}}],["000814481",{"2":{"244":1}}],["000043311",{"2":{"244":1}}],["000046194",{"2":{"244":1}}],["000044412",{"2":{"244":1}}],["000098905",{"2":{"244":1}}],["000098134",{"2":{"244":1}}],["000092742",{"2":{"244":1}}],["000094398",{"2":{"244":1}}],["000097348",{"2":{"244":1}}],["000097578",{"2":{"244":1}}],["000097049",{"2":{"244":1}}],["000095254",{"2":{"244":1}}],["000058640",{"2":{"244":1}}],["000056836",{"2":{"244":1}}],["000059547",{"2":{"244":1}}],["000052356",{"2":{"244":1}}],["000067247",{"2":{"244":1}}],["000088090",{"2":{"244":1}}],["000086152",{"2":{"244":1}}],["000089840",{"2":{"244":1}}],["000083818",{"2":{"244":1}}],["000083883",{"2":{"244":1}}],["000085040",{"2":{"244":1}}],["000081115",{"2":{"244":1}}],["000087234",{"2":{"244":1}}],["000079111",{"2":{"244":1}}],["000070152",{"2":{"244":1}}],["000075086",{"2":{"244":1}}],["000077215",{"2":{"244":1}}],["000071670",{"2":{"244":1}}],["00000",{"2":{"182":100,"233":18,"235":3}}],["0002494370336098561",{"2":{"258":1}}],["00024919270251556733",{"2":{"258":1}}],["00024919503",{"2":{"256":1}}],["00024182199209542147",{"2":{"258":1}}],["00024355072355621558",{"2":{"258":1}}],["00024355052",{"2":{"256":1}}],["00024232536751971285",{"2":{"258":1}}],["00024232482",{"2":{"256":1}}],["000242183",{"2":{"244":1}}],["000242905",{"2":{"244":1}}],["000242692",{"2":{"244":1}}],["00022974120193819142",{"2":{"258":1}}],["000229884",{"2":{"244":1}}],["00022661089732072592",{"2":{"258":1}}],["00022660938",{"2":{"256":1}}],["0002225035034313972",{"2":{"258":1}}],["00022250146",{"2":{"256":1}}],["00022302984",{"2":{"256":1}}],["00022355978",{"2":{"256":1}}],["00026806443487209774",{"2":{"258":1}}],["00026848768970112405",{"2":{"258":1}}],["00026848647",{"2":{"256":1}}],["00026173845471244757",{"2":{"258":1}}],["00026173593",{"2":{"256":1}}],["00026297763722425135",{"2":{"258":1}}],["00026297724",{"2":{"256":1}}],["00026345439717731614",{"2":{"258":1}}],["00026345407",{"2":{"256":1}}],["000263541",{"2":{"244":1}}],["00026994064683101955",{"2":{"258":1}}],["00026994065",{"2":{"256":1}}],["000269295",{"2":{"244":1}}],["000264404",{"2":{"244":1}}],["00027277934166079255",{"2":{"258":1}}],["00027277894",{"2":{"256":1}}],["00027880083687079906",{"2":{"258":1}}],["00027880043",{"2":{"256":1}}],["00027308802",{"2":{"256":1}}],["000273393",{"2":{"244":1}}],["0002799995755460038",{"2":{"258":1}}],["00027999925",{"2":{"256":1}}],["00027948244448684073",{"2":{"258":1}}],["00027948042",{"2":{"256":1}}],["0002794219808517045",{"2":{"258":1}}],["00027942262",{"2":{"256":1}}],["000275985",{"2":{"244":1}}],["000271556",{"2":{"244":1}}],["000277635",{"2":{"244":1}}],["000270049",{"2":{"244":1}}],["000270824",{"2":{"244":1}}],["00020444168978222695",{"2":{"258":1}}],["000204869",{"2":{"244":1}}],["00020707335391228292",{"2":{"258":1}}],["00020298259875836352",{"2":{"258":1}}],["00020298584",{"2":{"256":1}}],["00020252994441945248",{"2":{"258":1}}],["00020252966",{"2":{"256":1}}],["00020181267676989644",{"2":{"258":1}}],["00020181146",{"2":{"256":1}}],["000201183",{"2":{"244":1}}],["00020374188086040435",{"2":{"258":1}}],["00020374315",{"2":{"256":1}}],["00020351446985593347",{"2":{"258":1}}],["00020351214",{"2":{"256":1}}],["0002033883647527833",{"2":{"258":1}}],["00020338836",{"2":{"256":1}}],["00020560982886453667",{"2":{"258":1}}],["00020579882702922013",{"2":{"258":1}}],["00020592711193319547",{"2":{"258":1}}],["0002059266",{"2":{"256":1}}],["0002053108669065728",{"2":{"258":1}}],["00020533117336322745",{"2":{"258":1}}],["00020530884",{"2":{"256":1}}],["00020532864",{"2":{"256":1}}],["0002058001",{"2":{"256":1}}],["000205284",{"2":{"244":1}}],["000206055",{"2":{"244":1}}],["000206944",{"2":{"244":1}}],["000209925",{"2":{"244":1}}],["00020849705",{"2":{"135":1}}],["00023786085032865976",{"2":{"258":1}}],["00023375447245039922",{"2":{"258":1}}],["00023375152",{"2":{"256":1}}],["0002366002740962891",{"2":{"258":1}}],["00023659732",{"2":{"256":1}}],["0002389099909964718",{"2":{"258":1}}],["00023890898",{"2":{"256":1}}],["00023000075770749568",{"2":{"258":1}}],["00023000097",{"2":{"256":1}}],["000230181",{"2":{"244":1}}],["0002350791619390836",{"2":{"258":1}}],["00023507916",{"2":{"256":1}}],["00023913896259444115",{"2":{"258":1}}],["00023913936",{"2":{"256":1}}],["000239130808",{"2":{"81":1}}],["000239237",{"2":{"244":1}}],["0002559733568663764",{"2":{"258":1}}],["0002587695883460528",{"2":{"258":1}}],["00025877086",{"2":{"256":1}}],["00025738433799142086",{"2":{"258":1}}],["0002573811",{"2":{"256":1}}],["000257633",{"2":{"244":1}}],["000250679",{"2":{"244":1}}],["00025311636093398925",{"2":{"258":1}}],["00025311403",{"2":{"256":1}}],["000253915",{"2":{"244":1}}],["00025326014",{"2":{"135":1}}],["00028043017727226373",{"2":{"258":1}}],["000280093",{"2":{"244":1}}],["000288360",{"2":{"244":1}}],["000287677",{"2":{"244":1}}],["000289123",{"2":{"244":1}}],["000286281",{"2":{"244":1}}],["00021940120941039688",{"2":{"258":1}}],["00021939976",{"2":{"256":1}}],["000216686357272e",{"2":{"258":1}}],["000216420",{"2":{"244":1}}],["0002123351",{"2":{"256":1}}],["00021089415570622416",{"2":{"258":1}}],["00021089091",{"2":{"256":1}}],["0002107104",{"2":{"256":1}}],["00021762433389771325",{"2":{"258":1}}],["00021762686",{"2":{"256":1}}],["000217459",{"2":{"244":1}}],["00021550229536935868",{"2":{"258":1}}],["00021550554",{"2":{"256":1}}],["000215771",{"2":{"244":1}}],["00021152006023158516",{"2":{"258":1}}],["0002115195",{"2":{"256":1}}],["0002117683668545441",{"2":{"258":1}}],["0002117695",{"2":{"256":1}}],["000211123",{"2":{"244":1}}],["000213564",{"2":{"244":1}}],["000213855",{"2":{"244":1}}],["000218087",{"2":{"244":1}}],["000214095",{"2":{"244":1}}],["000214067404",{"2":{"81":1}}],["0002953885231182095",{"2":{"258":1}}],["00029528814104235834",{"2":{"258":1}}],["0002992532673827499",{"2":{"258":1}}],["00029925458",{"2":{"256":1}}],["00029179127530466097",{"2":{"258":1}}],["00029170960103101106",{"2":{"258":1}}],["00029171017",{"2":{"256":1}}],["000291289",{"2":{"244":1}}],["00029680230859930646",{"2":{"258":1}}],["0002966056968806676",{"2":{"258":1}}],["00029660418",{"2":{"256":1}}],["0002961341187959484",{"2":{"258":1}}],["00029613348",{"2":{"256":1}}],["000296087",{"2":{"244":1}}],["000298148",{"2":{"244":1}}],["00029203",{"2":{"174":1}}],["0004130149265874651",{"2":{"258":1}}],["000410463",{"2":{"244":1}}],["000412207",{"2":{"244":1}}],["000412151",{"2":{"244":1}}],["000412629",{"2":{"244":1}}],["000454265",{"2":{"244":1}}],["000454922556",{"2":{"81":1}}],["000455096",{"2":{"244":1}}],["000405329",{"2":{"244":1}}],["000401171",{"2":{"244":1}}],["000408040",{"2":{"244":1}}],["000490567",{"2":{"244":1}}],["000485339",{"2":{"244":1}}],["000489585",{"2":{"244":1}}],["000486189",{"2":{"244":1}}],["000482833",{"2":{"244":1}}],["000482373",{"2":{"244":1}}],["000484853",{"2":{"244":1}}],["000421408",{"2":{"244":1}}],["000421422",{"2":{"244":1}}],["000426714",{"2":{"244":1}}],["000426463",{"2":{"244":1}}],["000425117",{"2":{"244":1}}],["000422994",{"2":{"244":1}}],["000420685",{"2":{"244":1}}],["0004625431346698359",{"2":{"258":1}}],["000461797",{"2":{"244":1}}],["000461412",{"2":{"244":1}}],["000460401",{"2":{"244":1}}],["00046014786",{"2":{"134":1}}],["000477057",{"2":{"244":1}}],["000474958",{"2":{"244":1}}],["000473843",{"2":{"244":1}}],["000435325",{"2":{"244":1}}],["000432069",{"2":{"244":1}}],["000431564",{"2":{"244":1}}],["000438132",{"2":{"80":1}}],["00044045159335233126",{"2":{"258":1}}],["000449657",{"2":{"244":1}}],["000445312",{"2":{"244":1}}],["000444564",{"2":{"244":1}}],["000448536",{"2":{"244":1}}],["000447986",{"2":{"244":1}}],["000442930",{"2":{"244":1}}],["000441076729",{"2":{"81":1}}],["0006246384804114001",{"2":{"258":1}}],["000624455",{"2":{"244":1}}],["000620959",{"2":{"244":1}}],["000622923",{"2":{"244":1}}],["000600817",{"2":{"244":1}}],["000609521",{"2":{"80":1}}],["0006752262663322216",{"2":{"258":1}}],["000675569",{"2":{"244":1}}],["0006701476440012",{"2":{"258":1}}],["000673423",{"2":{"244":1}}],["000691815",{"2":{"244":1}}],["000694873",{"2":{"244":1}}],["000699265",{"2":{"80":1}}],["000635906",{"2":{"244":1}}],["000637475",{"2":{"244":1}}],["000641108",{"2":{"244":1}}],["000645338",{"2":{"244":1}}],["000649345",{"2":{"244":1}}],["000640270",{"2":{"244":1}}],["000648518",{"2":{"244":1}}],["000652327",{"2":{"244":1}}],["000652603",{"2":{"244":1}}],["000650020",{"2":{"244":1}}],["000659882",{"2":{"244":1}}],["0006631387855498853",{"2":{"258":1}}],["000664698",{"2":{"244":1}}],["000667115",{"2":{"244":1}}],["000667028",{"2":{"80":1}}],["000666251",{"2":{"244":1}}],["000666125",{"2":{"244":1}}],["0006134657177658567",{"2":{"258":1}}],["000613936",{"2":{"244":1}}],["000612554",{"2":{"244":1}}],["000612726",{"2":{"80":1}}],["000619220",{"2":{"244":1}}],["000618138",{"2":{"244":1}}],["0006861029847291114",{"2":{"258":1}}],["0006887888845526638",{"2":{"258":1}}],["000688489",{"2":{"244":1}}],["0006837621003302308",{"2":{"258":1}}],["000687853",{"2":{"244":1}}],["000685339",{"2":{"244":1}}],["000685782",{"2":{"244":1}}],["00068473816",{"2":{"134":1}}],["000689046",{"2":{"80":1}}],["000382126",{"2":{"244":1}}],["000386392",{"2":{"244":1}}],["000381213",{"2":{"244":1}}],["000389668",{"2":{"244":1}}],["000387762",{"2":{"244":1}}],["000314013",{"2":{"244":1}}],["000314716",{"2":{"244":1}}],["000312205",{"2":{"244":1}}],["000317277",{"2":{"244":1}}],["000317301",{"2":{"244":1}}],["000313576",{"2":{"244":1}}],["000311768",{"2":{"244":1}}],["000310122",{"2":{"244":1}}],["0003683949033299282",{"2":{"258":1}}],["000368751",{"2":{"244":1}}],["0003617533259373307",{"2":{"258":1}}],["000362759",{"2":{"244":1}}],["000367492",{"2":{"244":1}}],["000367264",{"2":{"244":1}}],["000369106",{"2":{"244":1}}],["000363892",{"2":{"244":1}}],["000366831",{"2":{"244":1}}],["000366380",{"2":{"244":1}}],["00036629336",{"2":{"81":1}}],["000322635",{"2":{"244":1}}],["000324281",{"2":{"244":1}}],["000321921",{"2":{"244":1}}],["00032624596727140253",{"2":{"258":1}}],["000326662",{"2":{"244":1}}],["000326938",{"2":{"244":1}}],["000323999",{"2":{"244":1}}],["000376061",{"2":{"244":1}}],["000379668",{"2":{"244":1}}],["000379169",{"2":{"244":1}}],["000378937",{"2":{"244":1}}],["000378884",{"2":{"244":1}}],["000374305",{"2":{"244":1}}],["000374875",{"2":{"80":1}}],["000377450",{"2":{"244":1}}],["000372513",{"2":{"244":1}}],["000350503",{"2":{"244":1}}],["000350560",{"2":{"244":1}}],["000353724",{"2":{"244":1}}],["000354412",{"2":{"244":1}}],["000351864",{"2":{"244":1}}],["000359855",{"2":{"244":1}}],["0003026522687613105",{"2":{"258":1}}],["000301691",{"2":{"244":1}}],["000306839",{"2":{"244":1}}],["000306047",{"2":{"244":1}}],["000305183",{"2":{"244":1}}],["000300986",{"2":{"244":1}}],["000300984",{"2":{"244":1}}],["000308547",{"2":{"244":1}}],["000307154",{"2":{"244":2}}],["00033405789094706863",{"2":{"258":1}}],["000334467",{"2":{"244":1}}],["000335865",{"2":{"244":1}}],["000330688",{"2":{"244":1}}],["000330550",{"2":{"244":1}}],["000333685",{"2":{"244":1}}],["00034963475290870455",{"2":{"258":1}}],["00034645712",{"2":{"256":1}}],["000346362",{"2":{"244":1}}],["000348117",{"2":{"244":1}}],["000344165",{"2":{"244":1}}],["000344849",{"2":{"244":1}}],["000345872",{"2":{"244":1}}],["000347842",{"2":{"244":1}}],["000392804",{"2":{"244":1}}],["0003920444",{"2":{"80":1}}],["000395994",{"2":{"244":1}}],["0003917327014566102",{"2":{"258":1}}],["000391877",{"2":{"244":1}}],["000391076",{"2":{"80":1}}],["000559556",{"2":{"244":1}}],["000554417",{"2":{"244":1}}],["000554349",{"2":{"244":1}}],["000554918",{"2":{"80":1}}],["000506370",{"2":{"244":1}}],["000503808",{"2":{"244":1}}],["0005443204642705102",{"2":{"258":1}}],["000545968",{"2":{"244":1}}],["000545245",{"2":{"244":1}}],["000549527",{"2":{"244":1}}],["000540564",{"2":{"244":1}}],["000547043",{"2":{"244":1}}],["000547293",{"2":{"244":1}}],["000543516",{"2":{"244":1}}],["0005963366240466802",{"2":{"258":1}}],["0005925590173431629",{"2":{"258":1}}],["0005920055483634651",{"2":{"258":1}}],["0005922087",{"2":{"106":1}}],["00059220614",{"2":{"106":1}}],["000590442",{"2":{"244":1}}],["000595116",{"2":{"244":1}}],["000595709",{"2":{"244":1}}],["000599332",{"2":{"244":1}}],["000583868",{"2":{"244":1}}],["000583094",{"2":{"244":1}}],["000580873",{"2":{"244":1}}],["000580056",{"2":{"244":1}}],["000580705",{"2":{"80":1}}],["000525872",{"2":{"244":1}}],["000525994",{"2":{"244":1}}],["000522715",{"2":{"244":1}}],["000521560",{"2":{"244":1}}],["000512687",{"2":{"244":1}}],["000511645",{"2":{"244":1}}],["000516831",{"2":{"244":1}}],["000516864",{"2":{"244":1}}],["0005106042139339528",{"2":{"258":1}}],["000510494",{"2":{"244":1}}],["000510198",{"2":{"80":1}}],["0005707143385862188",{"2":{"258":1}}],["000575209",{"2":{"244":1}}],["000579479",{"2":{"244":1}}],["000577084",{"2":{"244":1}}],["000574894",{"2":{"80":1}}],["000560744",{"2":{"244":1}}],["000566683",{"2":{"244":1}}],["000561465",{"2":{"244":1}}],["000564703",{"2":{"244":1}}],["000563284",{"2":{"80":1}}],["000539345",{"2":{"244":1}}],["000530431",{"2":{"244":1}}],["000537486",{"2":{"244":1}}],["000536315",{"2":{"244":1}}],["000535839",{"2":{"244":1}}],["00053332",{"2":{"174":1}}],["0005f0",{"2":{"244":1}}],["0007606024799357885",{"2":{"258":1}}],["0007681066177949802",{"2":{"258":1}}],["000764329",{"2":{"244":1}}],["0007263039337167419",{"2":{"258":1}}],["000726597",{"2":{"244":1}}],["0007289307705100168",{"2":{"258":1}}],["0007242005769272898",{"2":{"258":1}}],["000723986",{"2":{"244":1}}],["000723805",{"2":{"244":1}}],["000710499",{"2":{"244":1}}],["000714949",{"2":{"244":1}}],["000740137",{"2":{"244":1}}],["000744279",{"2":{"244":1}}],["000746255",{"2":{"244":1}}],["0007831547945311147",{"2":{"258":1}}],["0007827024388413978",{"2":{"258":1}}],["000788119",{"2":{"244":1}}],["000780730",{"2":{"244":1}}],["000784697",{"2":{"244":1}}],["000781053",{"2":{"244":1}}],["000734937",{"2":{"244":1}}],["000732894",{"2":{"244":1}}],["000731462",{"2":{"80":1}}],["0007973507560408622",{"2":{"258":1}}],["0007904217429736376",{"2":{"258":1}}],["000792289",{"2":{"244":1}}],["000792221",{"2":{"244":1}}],["000798627",{"2":{"244":1}}],["000798481",{"2":{"244":1}}],["000798861",{"2":{"244":1}}],["000796866",{"2":{"244":1}}],["000756769",{"2":{"244":1}}],["000754971",{"2":{"244":1}}],["000752965",{"2":{"244":1}}],["000753467",{"2":{"244":1}}],["0007025799817840762",{"2":{"258":1}}],["0007073437719725264",{"2":{"258":1}}],["000707158",{"2":{"244":1}}],["0007003734721873451",{"2":{"258":1}}],["0007098617733391764",{"2":{"257":1}}],["000709375",{"2":{"244":1}}],["000708074",{"2":{"244":1}}],["000705982",{"2":{"244":1}}],["0007774837715133924",{"2":{"258":1}}],["0007784759925414032",{"2":{"258":1}}],["000778712",{"2":{"244":1}}],["000778960",{"2":{"244":1}}],["000776893",{"2":{"244":1}}],["00077649",{"2":{"80":1}}],["0007736237",{"2":{"174":1}}],["00077338",{"2":{"80":1}}],["000959f",{"2":{"256":1}}],["000952036",{"2":{"244":1}}],["000925605",{"2":{"244":1}}],["000924496",{"2":{"244":1}}],["000924424",{"2":{"244":1}}],["000963809",{"2":{"244":1}}],["000964463",{"2":{"244":1}}],["0009418f",{"2":{"256":1}}],["000944194",{"2":{"244":1}}],["000942026",{"2":{"80":1}}],["0009091325645501539",{"2":{"258":1}}],["000907963",{"2":{"244":1}}],["000908168",{"2":{"244":1}}],["000901216",{"2":{"244":1}}],["000906830",{"2":{"244":1}}],["000939374",{"2":{"244":1}}],["000938448",{"2":{"244":1}}],["000936727",{"2":{"80":1}}],["000994050",{"2":{"244":1}}],["000992391663909964",{"2":{"53":2}}],["000972152",{"2":{"244":1}}],["000971555",{"2":{"244":1}}],["000975201",{"2":{"244":1}}],["000983244",{"2":{"244":1}}],["000987124",{"2":{"244":1}}],["000910802",{"2":{"80":1}}],["00018321261859757007",{"2":{"258":1}}],["00018321241",{"2":{"256":1}}],["00018953242907405934",{"2":{"258":1}}],["00018953496",{"2":{"256":1}}],["0001852610738412444",{"2":{"258":1}}],["00018525976",{"2":{"256":1}}],["00018027052",{"2":{"256":1}}],["00018429903345122147",{"2":{"258":1}}],["0001842996",{"2":{"256":1}}],["00018420626113517394",{"2":{"258":1}}],["00018420393",{"2":{"256":1}}],["0001822852957661807",{"2":{"258":1}}],["00018228586",{"2":{"256":1}}],["0001821270206428582",{"2":{"258":1}}],["00018212575",{"2":{"256":1}}],["000182339",{"2":{"244":1}}],["0001872625625789189",{"2":{"258":1}}],["0001872632",{"2":{"256":1}}],["00018710389803904177",{"2":{"258":1}}],["00018710685",{"2":{"256":1}}],["00018664228310068032",{"2":{"258":1}}],["00018663933",{"2":{"256":1}}],["000186362",{"2":{"244":1}}],["00014911272809737047",{"2":{"258":1}}],["0001491102",{"2":{"256":1}}],["000145716190315948",{"2":{"258":1}}],["00014571872",{"2":{"256":1}}],["00014575480089883838",{"2":{"258":1}}],["00014575227",{"2":{"256":1}}],["0001401495784240279",{"2":{"258":1}}],["00014015102",{"2":{"256":1}}],["00014059855589059123",{"2":{"258":1}}],["00014059727",{"2":{"256":1}}],["00014433913323955196",{"2":{"258":1}}],["0001443395",{"2":{"256":1}}],["00014449343860221752",{"2":{"258":1}}],["00014449273",{"2":{"256":1}}],["00014419519071699367",{"2":{"258":1}}],["00014419388",{"2":{"256":1}}],["000144504",{"2":{"244":1}}],["0001471923977349845",{"2":{"258":1}}],["00014719421",{"2":{"256":1}}],["00014760663829045643",{"2":{"258":1}}],["00014760795",{"2":{"256":1}}],["00014759533302159713",{"2":{"258":1}}],["00014759402",{"2":{"256":1}}],["00014628277937180577",{"2":{"258":1}}],["00014628147",{"2":{"256":1}}],["00014616011222416605",{"2":{"258":1}}],["00014616112",{"2":{"256":1}}],["000146f",{"2":{"256":1}}],["00014202934531568027",{"2":{"258":1}}],["00014202898",{"2":{"256":1}}],["00014283853440478876",{"2":{"258":1}}],["0001428365",{"2":{"256":1}}],["0001429795125732551",{"2":{"258":1}}],["00014297807",{"2":{"256":1}}],["00014223980588562476",{"2":{"258":1}}],["00014223685",{"2":{"256":1}}],["00014241667845919624",{"2":{"258":1}}],["0001424172",{"2":{"256":1}}],["0001426945472804816",{"2":{"258":1}}],["00014269506",{"2":{"256":1}}],["00014180776903302748",{"2":{"258":1}}],["00014180606",{"2":{"256":1}}],["0001414263898545934",{"2":{"258":1}}],["0001414271",{"2":{"256":1}}],["00014163293539446397",{"2":{"258":1}}],["00014163546",{"2":{"256":1}}],["00014178251",{"2":{"256":1}}],["00014105983003900941",{"2":{"258":1}}],["000141080946066501",{"2":{"258":1}}],["0001410602",{"2":{"256":1}}],["00014107842",{"2":{"256":1}}],["00014366027145554176",{"2":{"258":1}}],["0001436606",{"2":{"256":1}}],["00014309589441114764",{"2":{"258":1}}],["00014309268",{"2":{"256":1}}],["00014315193879495047",{"2":{"258":1}}],["0001431499",{"2":{"256":1}}],["00014377594925456472",{"2":{"258":1}}],["00014377595",{"2":{"256":1}}],["00014338266124368565",{"2":{"258":1}}],["00014338388",{"2":{"256":1}}],["00014335215382743272",{"2":{"258":1}}],["00014335215",{"2":{"256":1}}],["00014349088761731043",{"2":{"258":1}}],["0001434924",{"2":{"256":1}}],["0001434153630737371",{"2":{"258":1}}],["00014341663",{"2":{"256":1}}],["00014840522114126738",{"2":{"258":1}}],["00014840579",{"2":{"256":1}}],["00014826165988538294",{"2":{"258":1}}],["00014826102",{"2":{"256":1}}],["00014835137745090697",{"2":{"258":1}}],["00014834842",{"2":{"256":1}}],["00014872338035935024",{"2":{"258":1}}],["00014872105",{"2":{"256":1}}],["00014803729",{"2":{"256":1}}],["000148092",{"2":{"244":1}}],["000148993",{"2":{"244":1}}],["000148591",{"2":{"244":1}}],["000148517",{"2":{"244":1}}],["00011110823067702369",{"2":{"258":1}}],["000111111476",{"2":{"256":1}}],["00011172731115191943",{"2":{"258":1}}],["00011172844",{"2":{"256":1}}],["00011103335729340141",{"2":{"258":1}}],["000111034875",{"2":{"256":1}}],["00011121149668878578",{"2":{"258":1}}],["00011121263",{"2":{"256":1}}],["00011124783",{"2":{"256":1}}],["00011199531312668745",{"2":{"258":1}}],["000111997026",{"2":{"256":1}}],["00011196417396575523",{"2":{"258":1}}],["00011196634",{"2":{"256":1}}],["00011880085636293478",{"2":{"258":1}}],["00011888554104759725",{"2":{"258":1}}],["00011888758",{"2":{"256":1}}],["00011808607541993425",{"2":{"258":1}}],["00011808671",{"2":{"256":1}}],["000118799144",{"2":{"256":1}}],["0001132779438457816",{"2":{"258":1}}],["0001132747",{"2":{"256":1}}],["00011397922162008577",{"2":{"258":1}}],["000113982176",{"2":{"256":1}}],["00011313802979320028",{"2":{"258":1}}],["0001131393",{"2":{"256":1}}],["00011690346376099852",{"2":{"258":1}}],["0001169029",{"2":{"256":1}}],["000116517733656427",{"2":{"258":1}}],["0001165171",{"2":{"256":1}}],["00011685057767197896",{"2":{"258":1}}],["00011684927",{"2":{"256":1}}],["000116407",{"2":{"244":1}}],["00011774689528815062",{"2":{"258":1}}],["00011774394",{"2":{"256":1}}],["00011798592886290081",{"2":{"258":1}}],["00011798466",{"2":{"256":1}}],["00011735034019785079",{"2":{"258":1}}],["0001173525713045014",{"2":{"258":1}}],["000117353586",{"2":{"256":1}}],["0001173513",{"2":{"256":1}}],["000117897",{"2":{"244":1}}],["00011250213447297528",{"2":{"258":1}}],["00011250277",{"2":{"256":1}}],["00011297130538923156",{"2":{"258":1}}],["0001129706",{"2":{"256":1}}],["00011273330592185735",{"2":{"258":1}}],["000112732036",{"2":{"256":1}}],["00011218212503765081",{"2":{"258":1}}],["00011218314",{"2":{"256":1}}],["00011238765958065533",{"2":{"258":1}}],["00011238969",{"2":{"256":1}}],["00011234512648559416",{"2":{"258":1}}],["00011234368",{"2":{"256":1}}],["000112325",{"2":{"244":1}}],["00011445079170911656",{"2":{"258":1}}],["00011459839673306628",{"2":{"258":1}}],["00011459783",{"2":{"256":1}}],["00011431477508174869",{"2":{"258":1}}],["00011431153",{"2":{"256":1}}],["0001143078671567958",{"2":{"258":1}}],["00011430754",{"2":{"256":1}}],["00011433706833441294",{"2":{"258":1}}],["000114337774",{"2":{"256":1}}],["00011434576741704361",{"2":{"258":1}}],["00011434548",{"2":{"256":1}}],["00011400749862556023",{"2":{"258":1}}],["00011400497",{"2":{"256":1}}],["00011404573602886837",{"2":{"258":1}}],["00011404545",{"2":{"256":1}}],["00011406373518001133",{"2":{"258":1}}],["00011406402",{"2":{"256":1}}],["00011426929268051497",{"2":{"258":1}}],["00011426929",{"2":{"256":1}}],["00011950996672101251",{"2":{"258":1}}],["00011950945",{"2":{"256":1}}],["00011966117680732812",{"2":{"258":1}}],["00011966063",{"2":{"256":1}}],["000119893564",{"2":{"256":1}}],["00011516978243961795",{"2":{"258":1}}],["00011516807",{"2":{"256":1}}],["00011559072367263004",{"2":{"258":1}}],["00011558951",{"2":{"256":1}}],["00011588641128308456",{"2":{"258":1}}],["000115886956",{"2":{"256":1}}],["0001152873232322168",{"2":{"258":1}}],["00011528408",{"2":{"256":1}}],["00011529041",{"2":{"256":1}}],["0001153885",{"2":{"256":1}}],["00011573251865513483",{"2":{"258":1}}],["00011573212",{"2":{"256":1}}],["000115742",{"2":{"244":1}}],["00011037792903970437",{"2":{"258":1}}],["00011037468",{"2":{"256":1}}],["00011074483540898065",{"2":{"258":1}}],["000110743706",{"2":{"256":1}}],["00011026335489973004",{"2":{"258":1}}],["000110265166",{"2":{"256":1}}],["00011052547360301221",{"2":{"258":1}}],["00011052692",{"2":{"256":1}}],["00011054835591155347",{"2":{"258":1}}],["0001105504",{"2":{"256":1}}],["00011081036791418321",{"2":{"258":1}}],["00011081037",{"2":{"256":1}}],["00011086220183643978",{"2":{"258":1}}],["0001108622",{"2":{"256":1}}],["00017462112606557147",{"2":{"258":1}}],["0001797804577128186",{"2":{"258":1}}],["00017978139",{"2":{"256":1}}],["0001716806770309658",{"2":{"258":1}}],["00017168031",{"2":{"256":1}}],["000178552077711665",{"2":{"258":1}}],["0001785546",{"2":{"256":1}}],["00017555949296410188",{"2":{"258":1}}],["00017555879",{"2":{"256":1}}],["00017549854",{"2":{"256":1}}],["00017615732457630542",{"2":{"258":1}}],["00017683811686993795",{"2":{"258":1}}],["00017683755",{"2":{"256":1}}],["00017630673",{"2":{"256":1}}],["000176415",{"2":{"244":1}}],["0001721674278358903",{"2":{"258":1}}],["00017216924",{"2":{"256":1}}],["00017213980072834412",{"2":{"258":1}}],["00017213685",{"2":{"256":1}}],["00017308623197095634",{"2":{"258":1}}],["00017308586",{"2":{"256":1}}],["0001733976871434768",{"2":{"258":1}}],["00017339565",{"2":{"256":1}}],["000173648",{"2":{"244":1}}],["00017074520085998894",{"2":{"258":1}}],["00017074464",{"2":{"256":1}}],["000170466",{"2":{"244":1}}],["000170909",{"2":{"244":1}}],["00015681993171932752",{"2":{"258":1}}],["0001568174",{"2":{"256":1}}],["00015261088000378096",{"2":{"258":1}}],["00015231897585652043",{"2":{"258":1}}],["00015231865",{"2":{"256":1}}],["00015212992267590782",{"2":{"258":1}}],["00015213025",{"2":{"256":1}}],["000152438",{"2":{"244":1}}],["0001532562971452538",{"2":{"258":1}}],["00015325508",{"2":{"256":1}}],["0001537796242725119",{"2":{"258":1}}],["00015378026",{"2":{"256":1}}],["00015004433877961992",{"2":{"258":1}}],["00015004253",{"2":{"256":1}}],["00015082376639372836",{"2":{"258":1}}],["00015082232",{"2":{"256":1}}],["0001506850488949556",{"2":{"258":1}}],["0001506865",{"2":{"256":1}}],["00015090158835866352",{"2":{"258":1}}],["0001509003",{"2":{"256":1}}],["00015902992621632663",{"2":{"258":1}}],["00015903164",{"2":{"256":1}}],["00015922139527554951",{"2":{"258":1}}],["00015922435",{"2":{"256":1}}],["000159178",{"2":{"244":1}}],["00015735103003876595",{"2":{"258":1}}],["00015734922",{"2":{"256":1}}],["00015716880462417928",{"2":{"258":1}}],["00015717202",{"2":{"256":1}}],["00015721960019390505",{"2":{"258":1}}],["00015726659406058402",{"2":{"258":1}}],["00015726443",{"2":{"256":1}}],["00015722129746674127",{"2":{"258":1}}],["00015722163",{"2":{"256":1}}],["00015722003",{"2":{"256":1}}],["00015406645233457655",{"2":{"258":1}}],["00015406324",{"2":{"256":1}}],["0001540251396362168",{"2":{"258":1}}],["00015402514",{"2":{"256":1}}],["00015571431549989628",{"2":{"258":1}}],["00015571411",{"2":{"256":1}}],["00015583055870494473",{"2":{"258":1}}],["00015583177",{"2":{"256":1}}],["00015557776633170152",{"2":{"258":1}}],["00015557921",{"2":{"256":1}}],["00015508627006391803",{"2":{"258":1}}],["00015508423",{"2":{"256":1}}],["00015532227477720406",{"2":{"258":1}}],["00015532046",{"2":{"256":1}}],["000155360",{"2":{"244":1}}],["00015894688377995764",{"2":{"258":1}}],["00015894721",{"2":{"256":1}}],["00015811274779362782",{"2":{"258":1}}],["00015811148",{"2":{"256":1}}],["00015873539372295175",{"2":{"258":1}}],["00015873568",{"2":{"256":1}}],["00015808186933328908",{"2":{"258":1}}],["00015807984",{"2":{"256":1}}],["000158011",{"2":{"244":1}}],["000158824",{"2":{"244":1}}],["00015121389420599422",{"2":{"258":1}}],["00015121218",{"2":{"256":1}}],["00015168579624614677",{"2":{"258":1}}],["00015168458",{"2":{"256":1}}],["00015139882498611479",{"2":{"258":1}}],["00015139587",{"2":{"256":1}}],["000151947",{"2":{"244":1}}],["00015113852",{"2":{"81":1}}],["00012123663631427866",{"2":{"258":1}}],["000121237004",{"2":{"256":1}}],["0001210999806626343",{"2":{"258":1}}],["00012109969",{"2":{"256":1}}],["00012913095487643582",{"2":{"258":1}}],["00012912843",{"2":{"256":1}}],["000129240",{"2":{"244":1}}],["0001237098920958899",{"2":{"258":1}}],["00012370772",{"2":{"256":1}}],["0001235838214971321",{"2":{"258":1}}],["0001235826",{"2":{"256":1}}],["0001234499840346639",{"2":{"258":1}}],["00012344854",{"2":{"256":1}}],["00012801252075493572",{"2":{"258":1}}],["0001280138",{"2":{"256":1}}],["0001282768203701277",{"2":{"258":1}}],["00012826038352701946",{"2":{"258":1}}],["00012826166",{"2":{"256":1}}],["00012828007",{"2":{"256":1}}],["00012897993574836596",{"2":{"258":1}}],["0001289141673375904",{"2":{"258":1}}],["0001289165",{"2":{"256":1}}],["00012898138",{"2":{"256":1}}],["0001228108338075376",{"2":{"258":1}}],["00012267875690861888",{"2":{"258":1}}],["00012267969",{"2":{"256":1}}],["0001220554519095803",{"2":{"258":1}}],["00012205401",{"2":{"256":1}}],["00012270893235392902",{"2":{"258":1}}],["00012270945",{"2":{"256":1}}],["000122752",{"2":{"244":1}}],["0001247968009306606",{"2":{"258":1}}],["00012479624",{"2":{"256":1}}],["00012480812477309142",{"2":{"258":1}}],["00012480488",{"2":{"256":1}}],["00012430046181646445",{"2":{"258":1}}],["0001243019",{"2":{"256":1}}],["00012402023541183133",{"2":{"258":1}}],["00012402056",{"2":{"256":1}}],["000124030307108128",{"2":{"258":1}}],["00012403002",{"2":{"256":1}}],["000124517",{"2":{"244":1}}],["00012045182638951863",{"2":{"258":1}}],["00012045276",{"2":{"256":1}}],["0001209863399444701",{"2":{"258":1}}],["000120989585",{"2":{"256":1}}],["00012063635157848382",{"2":{"258":1}}],["00012063581",{"2":{"256":1}}],["0001208191922671903",{"2":{"258":1}}],["000120821234",{"2":{"256":1}}],["0001203544299370227",{"2":{"258":1}}],["00012035262",{"2":{"256":1}}],["00012036865218643604",{"2":{"258":1}}],["00012036738",{"2":{"256":1}}],["000120209",{"2":{"244":1}}],["00012684979143159005",{"2":{"258":1}}],["00012684942",{"2":{"256":1}}],["000126796724628076",{"2":{"258":1}}],["00012679571",{"2":{"256":1}}],["00012603782361842033",{"2":{"258":1}}],["00012603638",{"2":{"256":1}}],["000126034",{"2":{"244":1}}],["000126193",{"2":{"244":1}}],["0001258476231212468",{"2":{"258":1}}],["0001258489",{"2":{"256":1}}],["00012558122379420223",{"2":{"258":1}}],["00012557801",{"2":{"256":1}}],["000125504",{"2":{"244":1}}],["000125214",{"2":{"244":1}}],["0001272431503275569",{"2":{"258":1}}],["00012724352",{"2":{"256":1}}],["00012722292462947907",{"2":{"258":1}}],["00012722421",{"2":{"256":1}}],["00012702088363535395",{"2":{"258":1}}],["00012702291",{"2":{"256":1}}],["000127040",{"2":{"244":1}}],["000127500",{"2":{"244":1}}],["0001963581228291365",{"2":{"258":1}}],["00019635934",{"2":{"256":1}}],["00019965971844328957",{"2":{"258":1}}],["00019966012",{"2":{"256":1}}],["00019008694642297169",{"2":{"258":1}}],["00019008624",{"2":{"256":1}}],["00019447271848448938",{"2":{"258":1}}],["00019447393",{"2":{"256":1}}],["00019466919603295402",{"2":{"258":1}}],["00019466775",{"2":{"256":1}}],["00019530848486119698",{"2":{"258":1}}],["0001954865410423597",{"2":{"258":1}}],["00019548782",{"2":{"256":1}}],["000195276",{"2":{"244":1}}],["00019282329044938032",{"2":{"258":1}}],["0001928238",{"2":{"256":1}}],["00019216477262955044",{"2":{"258":1}}],["00019216529",{"2":{"256":1}}],["00019241103573688025",{"2":{"258":1}}],["00019241248",{"2":{"256":1}}],["000192454",{"2":{"244":1}}],["00019113839834670673",{"2":{"258":1}}],["0001911386",{"2":{"256":1}}],["000191858",{"2":{"244":1}}],["000191290",{"2":{"244":1}}],["000193357",{"2":{"244":1}}],["00013717771825929085",{"2":{"258":1}}],["00013717679",{"2":{"256":1}}],["0001377560881325826",{"2":{"258":1}}],["00013775482",{"2":{"256":1}}],["00013907814899625118",{"2":{"258":1}}],["00013979616078165446",{"2":{"258":1}}],["00013979523",{"2":{"256":1}}],["0001395339012695561",{"2":{"258":1}}],["00013953423",{"2":{"256":1}}],["00013379563051483503",{"2":{"258":1}}],["00013379584",{"2":{"256":1}}],["00013323667689899772",{"2":{"258":1}}],["00013323688",{"2":{"256":1}}],["00013365241924403315",{"2":{"258":1}}],["00013365413",{"2":{"256":1}}],["00013340454011105866",{"2":{"258":1}}],["00013340326",{"2":{"256":1}}],["00013020407143581424",{"2":{"258":1}}],["0001302061",{"2":{"256":1}}],["00013073334205956205",{"2":{"258":1}}],["00013073278",{"2":{"256":1}}],["00013009375007415393",{"2":{"258":1}}],["00013009476",{"2":{"256":1}}],["0001354948329188775",{"2":{"258":1}}],["0001354939",{"2":{"256":1}}],["00013538516313245464",{"2":{"258":1}}],["00013538415",{"2":{"256":1}}],["0001351110883297239",{"2":{"258":1}}],["00013511072",{"2":{"256":1}}],["00013517490672939649",{"2":{"258":1}}],["00013517434",{"2":{"256":1}}],["00013526252129011354",{"2":{"258":1}}],["00013526081",{"2":{"256":1}}],["00013523812270157737",{"2":{"258":1}}],["00013523488",{"2":{"256":1}}],["0001343505565907006",{"2":{"258":1}}],["00013435273",{"2":{"256":1}}],["00013415243329097305",{"2":{"258":1}}],["0001341515",{"2":{"256":1}}],["00013412299947280758",{"2":{"258":1}}],["00013412097",{"2":{"256":1}}],["00013444802149092396",{"2":{"258":1}}],["00013444631",{"2":{"256":1}}],["00013447786195217666",{"2":{"258":1}}],["00013447642",{"2":{"256":1}}],["00013499990222799242",{"2":{"258":1}}],["00013499877",{"2":{"256":1}}],["000134948",{"2":{"244":1}}],["00013116674929449456",{"2":{"258":1}}],["00013116715",{"2":{"256":1}}],["00013143228735129704",{"2":{"258":1}}],["0001314355",{"2":{"256":1}}],["00013121",{"2":{"80":1}}],["00013656724062385112",{"2":{"258":1}}],["00013656895",{"2":{"256":1}}],["0001364494985568802",{"2":{"258":1}}],["00013645006",{"2":{"256":1}}],["00013627896808145697",{"2":{"258":1}}],["00013627937",{"2":{"256":1}}],["00013608629809640216",{"2":{"258":1}}],["00013608659",{"2":{"256":1}}],["0001361124807708773",{"2":{"258":1}}],["00013611194",{"2":{"256":1}}],["000136768",{"2":{"244":1}}],["0001320288436821541",{"2":{"258":1}}],["00013202667",{"2":{"256":1}}],["00013201217",{"2":{"256":1}}],["00013298625281212424",{"2":{"258":1}}],["00013298752",{"2":{"256":1}}],["000132906",{"2":{"244":1}}],["000132676",{"2":{"244":1}}],["00013809933867378477",{"2":{"258":1}}],["00013809897",{"2":{"256":1}}],["00013876824755396457",{"2":{"258":1}}],["00013876938",{"2":{"256":1}}],["00013859734444125957",{"2":{"258":1}}],["00013859413",{"2":{"256":1}}],["0001389406416367069",{"2":{"258":1}}],["00013893936",{"2":{"256":1}}],["00013860250597716513",{"2":{"258":1}}],["0001386008",{"2":{"256":1}}],["00013865373",{"2":{"256":1}}],["0001382626168931522",{"2":{"258":1}}],["0001382609",{"2":{"256":1}}],["000138281",{"2":{"244":1}}],["000138344",{"2":{"244":1}}],["00010499090931947774",{"2":{"258":1}}],["000104993436",{"2":{"256":1}}],["00010479109999235196",{"2":{"258":1}}],["000104789826",{"2":{"256":1}}],["0001043167",{"2":{"256":1}}],["00010431886",{"2":{"256":1}}],["00010197222506544903",{"2":{"258":1}}],["000101974394",{"2":{"256":1}}],["00010101308202894625",{"2":{"258":1}}],["00010101195",{"2":{"256":1}}],["00010112778750072644",{"2":{"258":1}}],["00010112666",{"2":{"256":1}}],["00010169376454077221",{"2":{"258":1}}],["00010169225",{"2":{"256":1}}],["00010132866428469075",{"2":{"258":1}}],["00010132722",{"2":{"256":1}}],["00010900864016727386",{"2":{"258":1}}],["000109010085",{"2":{"256":1}}],["0001093042593488524",{"2":{"258":1}}],["000109304965",{"2":{"256":1}}],["00010973868961057376",{"2":{"258":1}}],["00010973665",{"2":{"256":1}}],["0001021550660856051",{"2":{"258":1}}],["00010215443",{"2":{"256":1}}],["0001025940219910863",{"2":{"258":1}}],["00010259473",{"2":{"256":1}}],["00010268462164837113",{"2":{"258":1}}],["00010268483375237967",{"2":{"258":1}}],["000102687576",{"2":{"256":1}}],["00010268538",{"2":{"256":1}}],["0001030455395530517",{"2":{"258":1}}],["000103044265",{"2":{"256":1}}],["0001035132185206405",{"2":{"258":1}}],["00010351466",{"2":{"256":1}}],["0001033573636389419",{"2":{"258":1}}],["000103358674",{"2":{"256":1}}],["00010370548716550833",{"2":{"258":1}}],["00010370612",{"2":{"256":1}}],["000103729",{"2":{"244":1}}],["00010725007375199672",{"2":{"258":1}}],["00010724826",{"2":{"256":1}}],["00010705464201430764",{"2":{"258":1}}],["00010705557",{"2":{"256":1}}],["00010703093297911067",{"2":{"258":1}}],["00010703346",{"2":{"256":1}}],["0001079858057823773",{"2":{"258":1}}],["00010798479",{"2":{"256":1}}],["00010794513317930726",{"2":{"258":1}}],["00010794412",{"2":{"256":1}}],["00010093201137306672",{"2":{"258":1}}],["00010093272",{"2":{"256":1}}],["00010087260288134235",{"2":{"258":1}}],["00010086939",{"2":{"256":1}}],["00010022550925927883",{"2":{"258":1}}],["00010022519278392329",{"2":{"258":1}}],["00010022334",{"2":{"256":1}}],["000100224905",{"2":{"256":1}}],["000100323169058197",{"2":{"258":1}}],["000100323494",{"2":{"256":1}}],["00010036986271851036",{"2":{"258":1}}],["00010037117",{"2":{"256":1}}],["00010818517792829506",{"2":{"258":1}}],["00010818485",{"2":{"256":1}}],["00010837116337672065",{"2":{"258":1}}],["00010837003",{"2":{"256":1}}],["00010846958012556371",{"2":{"258":1}}],["000108468135",{"2":{"256":1}}],["0001082424915192628",{"2":{"258":1}}],["00010824198",{"2":{"256":1}}],["00010894272722202147",{"2":{"258":1}}],["000108945256",{"2":{"256":1}}],["00010862327061220267",{"2":{"258":1}}],["000108622946",{"2":{"256":1}}],["000108696694842814",{"2":{"258":1}}],["00010869599",{"2":{"256":1}}],["000108091",{"2":{"244":1}}],["00010675962430272944",{"2":{"258":1}}],["00010675869",{"2":{"256":1}}],["00010621809191251024",{"2":{"258":1}}],["00010621838",{"2":{"256":1}}],["00010653282801102781",{"2":{"258":1}}],["000106533116",{"2":{"256":1}}],["00010693036",{"2":{"256":1}}],["000106987",{"2":{"244":1}}],["00010666078921815155",{"2":{"258":1}}],["00010666039",{"2":{"256":1}}],["00010669987104381709",{"2":{"258":1}}],["000106698426",{"2":{"256":1}}],["00010664191908733837",{"2":{"258":1}}],["000106641404",{"2":{"256":1}}],["00010635621035611067",{"2":{"258":1}}],["00010635854",{"2":{"256":1}}],["000106323",{"2":{"244":1}}],["000106139860816243",{"2":{"258":1}}],["00010614015",{"2":{"256":1}}],["000106173",{"2":{"244":1}}],["00010566372735658319",{"2":{"258":1}}],["00010566242",{"2":{"256":1}}],["00010572515478926155",{"2":{"258":1}}],["00010572152025847508",{"2":{"258":1}}],["00010572687",{"2":{"256":1}}],["00010571935",{"2":{"256":1}}],["00010591180829678763",{"2":{"258":1}}],["00010591059",{"2":{"256":1}}],["00010595445483443115",{"2":{"258":1}}],["0001059559",{"2":{"256":1}}],["00010517491403246128",{"2":{"258":1}}],["00010517364",{"2":{"256":1}}],["00010511739374003677",{"2":{"258":1}}],["00010511612",{"2":{"256":1}}],["00010522137699458063",{"2":{"258":1}}],["00010522433",{"2":{"256":1}}],["000105296",{"2":{"80":1}}],["00010554614057612856",{"2":{"258":1}}],["00010554397",{"2":{"256":1}}],["00010550347282085964",{"2":{"258":1}}],["00010550478",{"2":{"256":1}}],["00010551",{"2":{"174":1}}],["000105879",{"2":{"244":1}}],["000105440",{"2":{"244":1}}],["00010546855",{"2":{"174":1}}],["00016719957263148342",{"2":{"258":1}}],["00016782591960225285",{"2":{"258":1}}],["0001678254",{"2":{"256":1}}],["00016720014",{"2":{"256":1}}],["00016948325176315674",{"2":{"258":1}}],["00016948365",{"2":{"256":1}}],["00016995428892370348",{"2":{"258":1}}],["000169954",{"2":{"256":1}}],["000164341023239769",{"2":{"258":1}}],["00016434335",{"2":{"256":1}}],["00016438964",{"2":{"256":1}}],["000164069",{"2":{"244":1}}],["00016857952345919098",{"2":{"258":1}}],["0001685784",{"2":{"256":1}}],["00016843380659584684",{"2":{"258":1}}],["0001684325",{"2":{"256":1}}],["00016878",{"2":{"174":1}}],["0001629717",{"2":{"256":1}}],["00016297076918101878",{"2":{"258":1}}],["000162970",{"2":{"244":1}}],["000162085",{"2":{"244":1}}],["000166804",{"2":{"244":1}}],["000166462",{"2":{"244":1}}],["00016174985866423866",{"2":{"258":1}}],["00016174733",{"2":{"256":1}}],["00016190548270092207",{"2":{"258":1}}],["00016190331",{"2":{"256":1}}],["00016199157",{"2":{"256":1}}],["000161607",{"2":{"244":1}}],["000161545",{"2":{"244":1}}],["000160591",{"2":{"244":1}}],["000160725",{"2":{"244":1}}],["0001638659131113016",{"2":{"258":1}}],["00016386562",{"2":{"256":1}}],["000163823",{"2":{"244":1}}],["000163182",{"2":{"244":1}}],["000165743",{"2":{"244":1}}],["000165069",{"2":{"244":1}}],["000165657",{"2":{"80":1}}],["0001",{"2":{"80":7,"81":1}}],["0001f0",{"2":{"80":1}}],["013499700",{"2":{"244":1}}],["01348",{"2":{"182":1}}],["01339",{"2":{"182":1}}],["01335",{"2":{"182":1}}],["01319",{"2":{"182":1}}],["01311",{"2":{"182":1}}],["01316",{"2":{"182":1}}],["013011228",{"2":{"244":1}}],["01301",{"2":{"182":1}}],["01370",{"2":{"182":1}}],["01321",{"2":{"182":1}}],["01328",{"2":{"182":1}}],["01383",{"2":{"182":1}}],["01364",{"2":{"182":1}}],["01352",{"2":{"182":1}}],["01398",{"2":{"182":1}}],["01394",{"2":{"182":1}}],["01390",{"2":{"182":1}}],["013992591",{"2":{"106":1}}],["013992601",{"2":{"106":1}}],["017799046",{"2":{"244":1}}],["017750096",{"2":{"244":1}}],["01777",{"2":{"182":1}}],["017614847",{"2":{"244":1}}],["017605182",{"2":{"244":1}}],["017645847",{"2":{"244":1}}],["01700",{"2":{"182":1}}],["0172675f",{"2":{"256":1}}],["017215939",{"2":{"244":1}}],["01723",{"2":{"182":1}}],["01722",{"2":{"182":1}}],["0175547228302636e",{"2":{"258":1}}],["01757",{"2":{"182":1}}],["017520388922748e",{"2":{"258":1}}],["01752",{"2":{"182":1}}],["0178074f",{"2":{"256":1}}],["01780",{"2":{"182":1}}],["017197266",{"2":{"244":1}}],["01719",{"2":{"182":1}}],["0171745",{"2":{"162":1}}],["01741",{"2":{"182":1}}],["017432783",{"2":{"106":2}}],["01795",{"2":{"182":1}}],["01732",{"2":{"182":1}}],["016287093322083e",{"2":{"258":1}}],["016228525",{"2":{"244":1}}],["0162294",{"2":{"162":1}}],["016987354",{"2":{"244":1}}],["0163416f",{"2":{"256":1}}],["0163",{"2":{"249":1}}],["016386209",{"2":{"244":1}}],["01638",{"2":{"182":1}}],["01639",{"2":{"182":1}}],["01667",{"2":{"182":1}}],["01602",{"2":{"182":1}}],["01608",{"2":{"182":1}}],["01687",{"2":{"182":1}}],["01682",{"2":{"182":1}}],["0168695",{"2":{"134":1}}],["016118063",{"2":{"244":1}}],["01619",{"2":{"182":1}}],["01614",{"2":{"182":1}}],["016490150",{"2":{"244":1}}],["01642",{"2":{"182":1}}],["01646",{"2":{"182":1}}],["016479947",{"2":{"106":2}}],["01f0",{"2":{"134":1,"135":1,"174":2,"182":1}}],["019427970",{"2":{"244":1}}],["019484218",{"2":{"244":1}}],["01948",{"2":{"182":1}}],["019859709",{"2":{"244":1}}],["019567011",{"2":{"244":1}}],["01957",{"2":{"182":1}}],["01999",{"2":{"182":1}}],["01905",{"2":{"182":1}}],["019325165",{"2":{"244":1}}],["01931",{"2":{"182":1}}],["01935",{"2":{"182":1}}],["01967",{"2":{"182":1}}],["01962",{"2":{"182":1}}],["01965834",{"2":{"120":1}}],["0191156f",{"2":{"256":1}}],["01911576",{"2":{"106":1}}],["019115759",{"2":{"106":1}}],["019155467",{"2":{"244":1}}],["01919",{"2":{"182":1}}],["019722143932669e",{"2":{"258":1}}],["01977s\\ttraining",{"2":{"225":1}}],["01973",{"2":{"182":1}}],["01970",{"2":{"182":1}}],["019791335",{"2":{"106":1}}],["019791333",{"2":{"106":1}}],["019236464",{"2":{"80":1}}],["018655118416289e",{"2":{"258":1}}],["01866",{"2":{"182":1}}],["018008159",{"2":{"244":1}}],["018073166",{"2":{"244":1}}],["01805",{"2":{"182":1}}],["018483378",{"2":{"244":1}}],["0189805",{"2":{"213":1}}],["01859",{"2":{"182":2}}],["01883",{"2":{"182":1}}],["018884756",{"2":{"106":1}}],["018884761",{"2":{"106":1}}],["01836",{"2":{"182":1}}],["01831",{"2":{"182":1}}],["018338893",{"2":{"106":1}}],["018338889",{"2":{"106":1}}],["0187681f",{"2":{"256":1}}],["0187668",{"2":{"106":1}}],["018706074",{"2":{"244":1}}],["018701272",{"2":{"80":1}}],["018723227",{"2":{"244":1}}],["01872db4",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["0187476022524926e",{"2":{"258":1}}],["01874",{"2":{"182":1}}],["018743665453639813",{"2":{"169":1}}],["01873",{"2":{"182":1}}],["015280124",{"2":{"244":1}}],["015216017",{"2":{"244":1}}],["015340659",{"2":{"244":1}}],["015494239",{"2":{"244":1}}],["015478307",{"2":{"216":1}}],["01542492",{"2":{"107":1}}],["01562",{"2":{"182":1}}],["015808068084576e",{"2":{"258":1}}],["01588",{"2":{"182":1}}],["0158726",{"2":{"80":1}}],["01503",{"2":{"182":1}}],["015748031",{"2":{"213":2}}],["01577",{"2":{"182":1}}],["01578",{"2":{"182":1}}],["01571",{"2":{"182":1}}],["01519",{"2":{"182":1}}],["01510",{"2":{"182":1}}],["015555f",{"2":{"256":1}}],["01556",{"2":{"182":2}}],["01554",{"2":{"182":1}}],["0155126",{"2":{"106":1}}],["015937408",{"2":{"120":1,"123":1}}],["010920972",{"2":{"244":1}}],["010936455",{"2":{"244":1}}],["010004668",{"2":{"244":1}}],["01000",{"2":{"182":1}}],["01003",{"2":{"182":1}}],["0100971125",{"2":{"80":1}}],["010452257",{"2":{"244":1}}],["01047",{"2":{"182":1}}],["01040",{"2":{"182":1}}],["010404",{"2":{"134":1}}],["010898398",{"2":{"244":1}}],["010876314",{"2":{"244":1}}],["01080",{"2":{"182":1}}],["01086",{"2":{"182":1}}],["0108668577150213",{"2":{"166":1}}],["010629211",{"2":{"244":1}}],["01062",{"2":{"182":1}}],["01061",{"2":{"182":1}}],["010637204",{"2":{"106":1}}],["010637202",{"2":{"106":1}}],["010336538",{"2":{"244":1}}],["01039",{"2":{"182":1}}],["01032",{"2":{"182":1}}],["010755754",{"2":{"244":1}}],["01073",{"2":{"182":1}}],["01072",{"2":{"182":1}}],["01070",{"2":{"182":1}}],["01079",{"2":{"182":1}}],["01076",{"2":{"182":1}}],["010518193",{"2":{"244":1}}],["010586374",{"2":{"244":1}}],["01057",{"2":{"182":2}}],["01054",{"2":{"182":1}}],["0105465",{"2":{"80":1}}],["0102189295",{"2":{"106":1}}],["010218928",{"2":{"106":1}}],["01012",{"2":{"182":1}}],["0101",{"2":{"81":1}}],["0122",{"2":{"249":1}}],["01221",{"2":{"182":1}}],["012140292",{"2":{"244":1}}],["012183795",{"2":{"106":1}}],["012183794",{"2":{"106":1}}],["012508835",{"2":{"244":1}}],["012522168",{"2":{"80":1}}],["012002973",{"2":{"244":1}}],["01203",{"2":{"182":1}}],["01245",{"2":{"182":1}}],["01246",{"2":{"182":1}}],["012775686",{"2":{"244":1}}],["01272",{"2":{"182":1}}],["01271",{"2":{"182":1}}],["01260",{"2":{"182":1}}],["01268",{"2":{"182":1}}],["0123517173553542e",{"2":{"258":1}}],["012323526",{"2":{"244":1}}],["01233",{"2":{"182":1}}],["0123139",{"2":{"80":1}}],["01288",{"2":{"182":1}}],["01281",{"2":{"182":1}}],["014098606",{"2":{"244":1}}],["01408",{"2":{"182":2}}],["014340003",{"2":{"244":1}}],["014307282",{"2":{"244":1}}],["01438",{"2":{"182":1}}],["014530762",{"2":{"244":1}}],["014555071",{"2":{"244":1}}],["01459",{"2":{"182":1}}],["014241929",{"2":{"244":1}}],["014243508",{"2":{"244":1}}],["01425",{"2":{"182":1}}],["01428",{"2":{"182":1}}],["0142814",{"2":{"80":1}}],["014400685",{"2":{"244":1}}],["01447",{"2":{"182":1}}],["01442",{"2":{"182":1}}],["014708381",{"2":{"244":1}}],["01476",{"2":{"182":1}}],["01479",{"2":{"182":1}}],["01474",{"2":{"182":1}}],["0149",{"2":{"249":1}}],["01495",{"2":{"182":1}}],["01499",{"2":{"182":1}}],["014815190",{"2":{"244":1}}],["01484",{"2":{"182":2}}],["014877922",{"2":{"106":1}}],["01460",{"2":{"182":1}}],["0146587",{"2":{"80":1}}],["01418",{"2":{"182":1}}],["01413",{"2":{"182":1}}],["01415047",{"2":{"106":1}}],["014150472",{"2":{"106":1}}],["014100174",{"2":{"80":1}}],["011247103",{"2":{"244":1}}],["011284535",{"2":{"244":1}}],["011457845",{"2":{"244":1}}],["01148",{"2":{"182":1}}],["0110833f",{"2":{"256":1}}],["011087346",{"2":{"244":1}}],["011037273",{"2":{"244":1}}],["011062279",{"2":{"244":1}}],["011073254",{"2":{"244":1}}],["011374202",{"2":{"244":1}}],["011342090",{"2":{"244":1}}],["01134",{"2":{"182":1}}],["01187",{"2":{"182":1}}],["01189",{"2":{"182":1}}],["01185",{"2":{"182":2}}],["011598296",{"2":{"244":1}}],["01159",{"2":{"182":1}}],["01155",{"2":{"182":1}}],["01151",{"2":{"182":1}}],["011505678",{"2":{"244":1}}],["01150",{"2":{"182":1}}],["011605300341022e",{"2":{"258":1}}],["01167",{"2":{"182":1}}],["011611394",{"2":{"106":1}}],["011611392",{"2":{"106":1}}],["011134821404839e",{"2":{"258":1}}],["01114",{"2":{"182":1}}],["011100831f0",{"2":{"53":1}}],["01193f",{"2":{"256":1}}],["011937321",{"2":{"174":1}}],["011906523",{"2":{"244":1}}],["011915382",{"2":{"244":1}}],["01194",{"2":{"182":1}}],["01199",{"2":{"182":1}}],["01177",{"2":{"182":2}}],["01172",{"2":{"182":1}}],["011734666",{"2":{"80":1}}],["011739168",{"2":{"80":1}}],["01",{"2":{"9":3,"53":2,"174":1,"208":4,"258":1}}],["0",{"2":{"2":1,"4":1,"9":44,"12":3,"16":1,"24":3,"34":2,"49":2,"53":122,"54":4,"59":16,"65":3,"67":3,"70":2,"71":10,"72":1,"74":1,"80":204,"81":14,"106":292,"107":11,"109":5,"110":37,"115":1,"116":1,"118":2,"120":54,"123":25,"125":8,"129":6,"130":1,"131":7,"134":18,"135":5,"136":35,"137":23,"141":7,"143":1,"149":1,"162":54,"164":358,"165":1,"166":155,"167":15,"168":3,"169":62,"171":12,"172":9,"173":4,"174":72,"175":11,"178":1,"181":1,"182":452,"184":11,"187":1,"188":2,"189":2,"190":7,"192":3,"206":4,"208":2,"210":11,"213":139,"214":1,"215":4,"216":40,"217":11,"218":11,"221":2,"224":2,"225":105,"226":11,"229":1,"230":2,"231":1,"232":2,"233":26,"234":1,"235":8,"236":19,"237":11,"243":7,"244":800,"245":6,"246":11,"248":4,"249":65,"251":3,"254":4,"255":9,"256":479,"257":1,"258":386,"260":3}}],["1e",{"2":{"208":1,"254":1}}],["1g",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["1×16",{"2":{"136":1,"137":1}}],["1×32",{"2":{"59":1,"81":1}}],["1`",{"2":{"130":3}}],["1st",{"2":{"121":1}}],["1196495f",{"2":{"256":1}}],["119439f",{"2":{"256":1}}],["11984776",{"2":{"80":1}}],["114657620551897e",{"2":{"258":1}}],["114087861200215e",{"2":{"258":1}}],["11474f",{"2":{"256":1}}],["1147792f",{"2":{"256":1}}],["1143",{"2":{"249":1}}],["114975",{"2":{"136":1}}],["1155971969154164e",{"2":{"258":1}}],["1155407f",{"2":{"256":1}}],["1159557864773197e",{"2":{"258":1}}],["11501",{"2":{"244":1}}],["115832e",{"2":{"208":1}}],["11515346",{"2":{"106":1}}],["11515345",{"2":{"106":1}}],["110966875056439e",{"2":{"258":1}}],["11001",{"2":{"244":1}}],["11007148",{"2":{"213":1}}],["110",{"2":{"236":6}}],["11023622",{"2":{"213":2}}],["11023762",{"2":{"174":1}}],["110121e",{"2":{"208":1}}],["1112743602468684e",{"2":{"258":1}}],["1112419f",{"2":{"256":1}}],["1111696f",{"2":{"256":1}}],["11111",{"2":{"235":1}}],["1117284f",{"2":{"256":1}}],["11109",{"2":{"182":1}}],["111552626",{"2":{"106":1}}],["116598977995619e",{"2":{"258":1}}],["11662",{"2":{"182":1}}],["116989",{"2":{"166":1}}],["116796836",{"2":{"106":1}}],["1184050506944375e",{"2":{"258":1}}],["118481496391263e",{"2":{"258":1}}],["118613f",{"2":{"256":1}}],["118777f",{"2":{"256":1}}],["1187928",{"2":{"106":2}}],["1181102",{"2":{"213":2}}],["1188200246206075",{"2":{"164":2}}],["112052984669926e",{"2":{"258":1}}],["1120003114129957e",{"2":{"258":1}}],["1120639f",{"2":{"256":1}}],["1126604439399654e",{"2":{"258":1}}],["1126356",{"2":{"137":1}}],["112339f",{"2":{"256":1}}],["1121",{"2":{"249":1}}],["11242",{"2":{"182":1}}],["1124609",{"2":{"106":1}}],["11293",{"2":{"182":1}}],["1137304",{"2":{"136":1}}],["11365668",{"2":{"106":2}}],["117873f",{"2":{"256":1}}],["117236",{"2":{"213":1}}],["1173706",{"2":{"136":1}}],["1175871f",{"2":{"106":1}}],["11",{"2":{"134":2,"135":2,"136":2,"137":1,"175":2,"182":14,"184":2,"191":2,"210":2,"217":1,"218":2,"225":2,"226":2,"237":2,"246":2,"249":3}}],["14697204341642e",{"2":{"258":1}}],["1460652596785891e",{"2":{"258":1}}],["146441f",{"2":{"256":1}}],["1468446f",{"2":{"256":1}}],["1406",{"2":{"249":2}}],["1405",{"2":{"249":1}}],["14001",{"2":{"244":1}}],["14006",{"2":{"182":1}}],["1490868",{"2":{"216":1}}],["1496063",{"2":{"213":2}}],["14914",{"2":{"182":1}}],["14941669",{"2":{"80":1}}],["1439",{"2":{"249":1}}],["1437841",{"2":{"174":1}}],["14340512",{"2":{"107":1}}],["147705370924899",{"2":{"164":1}}],["1477053709248992",{"2":{"164":1}}],["142470881033475",{"2":{"164":2}}],["142251",{"2":{"106":1}}],["1457437f",{"2":{"256":1}}],["1458",{"2":{"249":1}}],["14501",{"2":{"244":1}}],["145296",{"2":{"162":1}}],["14568081",{"2":{"106":1}}],["148148177753555e",{"2":{"258":1}}],["14815",{"2":{"233":2}}],["1489395634045756e",{"2":{"258":1}}],["1489007",{"2":{"136":1}}],["148812f",{"2":{"256":1}}],["1488175",{"2":{"117":1}}],["14825",{"2":{"166":1}}],["148248",{"2":{"166":2}}],["148313+0",{"2":{"162":1}}],["14",{"2":{"134":1,"182":14,"225":2,"233":2,"249":7}}],["1441331857990563e",{"2":{"258":1}}],["1442849f",{"2":{"256":1}}],["14491",{"2":{"182":1}}],["14479",{"2":{"182":1}}],["14445141",{"2":{"106":1}}],["14488737",{"2":{"106":1}}],["141598635647141e",{"2":{"258":1}}],["1415927",{"2":{"53":1}}],["14132261",{"2":{"213":1}}],["14173229",{"2":{"213":2}}],["14118214",{"2":{"106":1}}],["1773439615602058e",{"2":{"258":1}}],["1770206f",{"2":{"256":1}}],["17706361",{"2":{"174":1}}],["1775482f",{"2":{"256":1}}],["1797763714314024e",{"2":{"258":1}}],["179797f",{"2":{"256":1}}],["179245272839102e",{"2":{"258":1}}],["1791008f",{"2":{"256":1}}],["1791",{"2":{"249":1}}],["179584495291061",{"2":{"164":2}}],["1751989581145076e",{"2":{"258":1}}],["17501",{"2":{"244":1}}],["17559",{"2":{"182":1}}],["171248505085587e",{"2":{"258":1}}],["171517631804133e",{"2":{"258":1}}],["17154759238133e",{"2":{"258":1}}],["171591f",{"2":{"256":1}}],["171654513068318e",{"2":{"258":1}}],["171692f",{"2":{"256":1}}],["1717204f",{"2":{"256":1}}],["1717s\\ttraining",{"2":{"233":1}}],["1710674f",{"2":{"256":1}}],["17111921",{"2":{"216":1}}],["171498e",{"2":{"208":1}}],["17816184",{"2":{"213":1}}],["17322835",{"2":{"213":2}}],["1720859346913115e",{"2":{"258":1}}],["17268",{"2":{"182":1}}],["1723186f",{"2":{"256":1}}],["172325e",{"2":{"137":1}}],["17239982",{"2":{"80":1}}],["1701625825915297e",{"2":{"258":1}}],["1706",{"2":{"249":1}}],["17001",{"2":{"244":1}}],["170",{"2":{"226":1}}],["170342",{"2":{"166":1}}],["17048413",{"2":{"106":1}}],["1766666167990785e",{"2":{"258":1}}],["176919413089905e",{"2":{"258":1}}],["1769385",{"2":{"136":1}}],["176811f",{"2":{"256":1}}],["1763",{"2":{"249":1}}],["176",{"2":{"208":1}}],["1764085",{"2":{"120":1}}],["17",{"2":{"110":1,"137":1,"182":14,"191":11,"214":1,"225":2,"249":4,"258":20}}],["1749663f",{"2":{"256":1}}],["17441",{"2":{"166":1}}],["17449912",{"2":{"80":1}}],["17435724",{"2":{"106":2}}],["1746128",{"2":{"80":1}}],["1th",{"2":{"72":3}}],["134748f",{"2":{"256":1}}],["13475561",{"2":{"213":1}}],["13292",{"2":{"182":1}}],["13",{"2":{"182":14,"225":3,"233":1,"249":3}}],["1361170338108406e",{"2":{"258":1}}],["136664291381576e",{"2":{"258":1}}],["136334f",{"2":{"256":1}}],["136356",{"2":{"162":1}}],["1365198f",{"2":{"256":1}}],["136",{"2":{"233":1}}],["13681",{"2":{"182":1}}],["1301954900639649e",{"2":{"258":1}}],["1303766f",{"2":{"256":1}}],["1303",{"2":{"249":1}}],["13001",{"2":{"244":1}}],["13073668",{"2":{"174":1}}],["1306755",{"2":{"137":1}}],["1302",{"2":{"71":1}}],["13813455076535e",{"2":{"258":1}}],["13819042",{"2":{"120":1}}],["138279f",{"2":{"256":1}}],["138082f",{"2":{"256":1}}],["13808",{"2":{"182":1}}],["13864",{"2":{"106":1}}],["137206633454982e",{"2":{"258":1}}],["1372066f",{"2":{"256":1}}],["13725637",{"2":{"106":1}}],["13725635",{"2":{"106":1}}],["13760868",{"2":{"106":1}}],["13760866",{"2":{"106":1}}],["1377298",{"2":{"106":2}}],["1359",{"2":{"249":1}}],["13567",{"2":{"182":1}}],["13559",{"2":{"166":1}}],["13500070491646e",{"2":{"258":1}}],["13501",{"2":{"244":1}}],["1350521",{"2":{"136":1}}],["13504265",{"2":{"106":1}}],["13518013",{"2":{"106":1}}],["13518015",{"2":{"106":1}}],["1395766",{"2":{"213":1}}],["139433",{"2":{"137":1}}],["13913931",{"2":{"106":1}}],["13988f",{"2":{"80":1}}],["13343152",{"2":{"106":1}}],["131717312186528e",{"2":{"258":1}}],["131726",{"2":{"136":1,"137":1}}],["1318886958613766e",{"2":{"258":1}}],["1318984f",{"2":{"256":1}}],["1316358f",{"2":{"256":1}}],["13168f",{"2":{"80":1}}],["1310",{"2":{"249":1}}],["131583631",{"2":{"81":1}}],["131247633675491e",{"2":{"258":1}}],["131211f",{"2":{"256":1}}],["1312",{"2":{"9":1}}],["1c",{"2":{"69":1}}],["1b",{"2":{"69":3}}],["1a",{"2":{"69":3}}],["1826329658209398e",{"2":{"258":1}}],["182458901909231e",{"2":{"258":1}}],["1825928f",{"2":{"256":1}}],["182712f",{"2":{"256":1}}],["1828",{"2":{"249":1}}],["1828456",{"2":{"213":1}}],["189840866665336e",{"2":{"258":1}}],["1895052668672963e",{"2":{"258":1}}],["1895053f",{"2":{"256":1}}],["1895193f",{"2":{"256":1}}],["18941",{"2":{"162":1}}],["18501",{"2":{"244":1}}],["18519",{"2":{"233":2}}],["185124",{"2":{"136":1}}],["18535946",{"2":{"216":1}}],["18564",{"2":{"213":1}}],["1846",{"2":{"249":1}}],["1846486482317626",{"2":{"164":2}}],["18481395",{"2":{"216":1}}],["18432",{"2":{"182":1}}],["18001",{"2":{"244":1}}],["18019",{"2":{"182":1}}],["18056087",{"2":{"174":1}}],["18089005",{"2":{"106":1}}],["187440346751602e",{"2":{"258":1}}],["1875416f",{"2":{"256":1}}],["1871622",{"2":{"171":4}}],["18794",{"2":{"162":1}}],["187646",{"2":{"162":1}}],["186676655139642e",{"2":{"258":1}}],["18664f",{"2":{"256":1}}],["186821237169626e",{"2":{"258":1}}],["186",{"2":{"256":1}}],["18679f",{"2":{"256":1}}],["18695377",{"2":{"174":1}}],["18652338",{"2":{"137":1}}],["186451",{"2":{"137":1}}],["1832",{"2":{"249":1}}],["1836515766351254",{"2":{"164":2}}],["18365",{"2":{"162":1}}],["183307",{"2":{"136":1}}],["18308182",{"2":{"120":1}}],["1887817",{"2":{"213":1}}],["18846",{"2":{"182":1}}],["18841726",{"2":{"174":1}}],["18860114",{"2":{"120":1}}],["18807876",{"2":{"106":2}}],["18811704",{"2":{"80":1}}],["1811024",{"2":{"213":2}}],["18150",{"2":{"182":1}}],["18102339",{"2":{"80":1}}],["181208",{"2":{"80":1}}],["18",{"2":{"59":1,"182":14,"191":1,"225":2,"233":1,"249":5,"258":6}}],["1−z",{"2":{"69":1}}],["1−2∑yy^+α∑y2+∑y^2+α",{"2":{"53":1}}],["1−α",{"2":{"53":2}}],["1−y",{"2":{"53":1}}],["1−yy^",{"2":{"53":2}}],["1−y^+ϵ",{"2":{"53":1}}],["1−y~",{"2":{"53":2}}],["163944652651308e",{"2":{"258":1}}],["163823f",{"2":{"256":1}}],["16758846695159e",{"2":{"258":1}}],["167267f",{"2":{"256":1}}],["16724",{"2":{"182":1}}],["1689905677970475e",{"2":{"258":1}}],["1681205f",{"2":{"256":1}}],["16845",{"2":{"182":1}}],["1652",{"2":{"249":1}}],["16501",{"2":{"244":1}}],["16569im",{"2":{"162":1}}],["16566901",{"2":{"134":1}}],["16241",{"2":{"236":12}}],["16240",{"2":{"236":12}}],["162500e",{"2":{"208":1}}],["1617228180412864",{"2":{"164":2}}],["16187939",{"2":{"106":1}}],["16450",{"2":{"236":6}}],["16454",{"2":{"182":1}}],["164786",{"2":{"166":1}}],["164379",{"2":{"162":1}}],["1642133",{"2":{"136":1}}],["1641532f",{"2":{"106":1}}],["16407",{"2":{"106":1}}],["16409892",{"2":{"106":1}}],["169767069547579e",{"2":{"258":1}}],["169984f",{"2":{"256":1}}],["16946f",{"2":{"80":1}}],["16909766",{"2":{"80":1}}],["1600781f",{"2":{"256":1}}],["16001",{"2":{"244":1}}],["160876",{"2":{"162":1}}],["16083185f0",{"2":{"53":1}}],["160832f0",{"2":{"53":1}}],["16094846",{"2":{"134":1}}],["1607",{"2":{"24":2,"72":1}}],["16",{"2":{"46":1,"59":3,"109":13,"110":14,"115":1,"136":1,"137":1,"166":1,"182":14,"188":3,"191":1,"214":4,"217":2,"225":3,"243":1,"244":8,"249":4,"251":4,"258":5,"260":4}}],["10f",{"2":{"257":1}}],["1049523f",{"2":{"256":1}}],["10458f",{"2":{"80":1}}],["108145",{"2":{"213":1}}],["10884",{"2":{"182":1}}],["1011765244248956e",{"2":{"258":1}}],["101",{"2":{"208":1,"217":1,"233":1,"236":6}}],["10140238162746013",{"2":{"164":2}}],["1051333940625235e",{"2":{"258":1}}],["1058",{"2":{"249":1}}],["10501",{"2":{"244":1}}],["10596",{"2":{"182":1}}],["10552457",{"2":{"106":1}}],["105524555",{"2":{"106":1}}],["10328",{"2":{"182":1}}],["1032465",{"2":{"106":1}}],["1071643160232267e",{"2":{"258":1}}],["107888",{"2":{"213":1}}],["1079326",{"2":{"136":1}}],["10754459",{"2":{"106":1}}],["1024",{"2":{"164":7}}],["102",{"2":{"109":1}}],["1097126348202013e",{"2":{"258":1}}],["10976",{"2":{"182":1}}],["109656f",{"2":{"256":1}}],["1090",{"2":{"254":2}}],["109261915",{"2":{"120":1}}],["109203495",{"2":{"106":1}}],["1095082",{"2":{"106":1}}],["10950818",{"2":{"106":1}}],["106602f",{"2":{"256":1}}],["1066495f",{"2":{"256":1}}],["106069",{"2":{"134":1}}],["10688411",{"2":{"106":1}}],["106884085",{"2":{"106":1}}],["106923225050862e",{"2":{"258":1}}],["1069",{"2":{"106":1}}],["10",{"2":{"34":2,"46":2,"49":1,"65":1,"72":12,"74":2,"79":1,"80":5,"81":1,"106":5,"117":2,"118":1,"134":2,"135":2,"137":2,"166":4,"169":2,"174":3,"175":5,"182":14,"184":5,"188":5,"190":1,"191":22,"192":2,"210":5,"218":5,"223":1,"225":2,"226":5,"231":5,"233":2,"236":30,"237":5,"246":5,"249":4,"250":1,"251":2,"254":2,"258":11,"260":2}}],["1001",{"2":{"174":1,"244":1}}],["1003911",{"2":{"131":1}}],["10001",{"2":{"244":1}}],["10000",{"2":{"174":2,"244":1}}],["1000",{"2":{"59":1,"81":3,"107":14,"174":1}}],["100",{"2":{"9":1,"49":2,"66":6,"81":1,"107":2,"175":2,"184":2,"190":2,"192":1,"210":2,"218":2,"225":35,"226":2,"233":2,"236":6,"237":2,"246":2,"251":1,"255":2,"260":1}}],["121608362079694e",{"2":{"258":1}}],["12103",{"2":{"182":1}}],["122214318020622e",{"2":{"258":1}}],["122345f",{"2":{"256":1}}],["12287",{"2":{"182":1}}],["12s",{"2":{"225":2}}],["1265",{"2":{"249":1}}],["126",{"2":{"208":1}}],["1261990389976131",{"2":{"169":1}}],["12726",{"2":{"182":1}}],["1277556f0",{"2":{"53":1}}],["120785590647832e",{"2":{"258":1}}],["120849f",{"2":{"256":1}}],["12001",{"2":{"244":1}}],["12096125",{"2":{"216":1}}],["12029",{"2":{"182":1}}],["1204217935313214",{"2":{"164":2}}],["12abac4f24f6",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["125566405206153e",{"2":{"258":1}}],["12556516",{"2":{"106":2}}],["1253566340090383e",{"2":{"258":1}}],["125385f",{"2":{"256":1}}],["12530515f",{"2":{"256":1}}],["12501",{"2":{"244":1}}],["12589",{"2":{"174":1}}],["1240825f",{"2":{"256":1}}],["1240385f",{"2":{"256":1}}],["12443",{"2":{"182":1}}],["12442",{"2":{"162":1}}],["12477568",{"2":{"174":1}}],["1245008",{"2":{"136":1}}],["1298182630716548e",{"2":{"258":1}}],["129581992879848e",{"2":{"258":1}}],["1295s\\ttraining",{"2":{"233":1}}],["129709f",{"2":{"256":1}}],["12978",{"2":{"182":1}}],["12902719",{"2":{"106":2}}],["1293236",{"2":{"106":1}}],["12992674",{"2":{"106":1}}],["129",{"2":{"59":1,"72":1}}],["1286439",{"2":{"213":1}}],["12827216",{"2":{"106":1}}],["128",{"2":{"46":2,"59":12,"72":3,"80":4,"188":6,"190":1,"213":4,"233":1,"251":1,"260":1}}],["12",{"2":{"34":2,"129":1,"136":6,"137":7,"141":9,"166":2,"175":6,"182":14,"184":6,"192":1,"210":6,"218":6,"225":6,"226":6,"233":1,"237":6,"246":6,"249":5,"251":1,"254":1,"260":1}}],["12378557970847e",{"2":{"258":1}}],["1234",{"2":{"248":1}}],["12345",{"2":{"213":1}}],["1239749674027375",{"2":{"164":2}}],["123",{"2":{"9":3,"65":1,"164":1}}],["1=dense",{"2":{"32":1}}],["1f0",{"2":{"24":3,"53":2,"59":1,"72":2,"81":1,"213":1}}],["1977191367793196e",{"2":{"258":1}}],["197029266272924e",{"2":{"258":1}}],["197045",{"2":{"166":1}}],["1976626f",{"2":{"256":1}}],["1968495f",{"2":{"256":1}}],["1969163f",{"2":{"256":1}}],["196703",{"2":{"125":2}}],["19449491509571e",{"2":{"258":1}}],["19400917966395e",{"2":{"258":1}}],["1941704f",{"2":{"256":1}}],["194185f",{"2":{"256":1}}],["1943",{"2":{"249":1}}],["1985100098565496e",{"2":{"258":1}}],["198623f",{"2":{"256":1}}],["1988",{"2":{"254":2}}],["1984668",{"2":{"134":1}}],["19501",{"2":{"244":1}}],["195746",{"2":{"213":1}}],["1953009",{"2":{"172":1,"173":1}}],["19597391412112541",{"2":{"169":1}}],["1958431",{"2":{"174":1}}],["1958",{"2":{"22":1,"249":1}}],["191083113625497e",{"2":{"258":1}}],["1910195f",{"2":{"256":1}}],["19167534314287e",{"2":{"258":1}}],["1917766f",{"2":{"256":1}}],["1914",{"2":{"249":1}}],["191956",{"2":{"167":1}}],["19124654",{"2":{"134":1}}],["19362906873599e",{"2":{"258":1}}],["1936812",{"2":{"134":1}}],["193804707326044e",{"2":{"258":1}}],["1933075f",{"2":{"256":1}}],["193483f",{"2":{"256":1}}],["19395226906025e",{"2":{"258":1}}],["1939577f",{"2":{"256":1}}],["19397983",{"2":{"174":1}}],["193291",{"2":{"166":1}}],["190370242358394e",{"2":{"258":1}}],["190226f",{"2":{"256":1}}],["19001",{"2":{"244":1}}],["1904755",{"2":{"137":1}}],["19084926",{"2":{"106":1}}],["1993764759212034e",{"2":{"258":1}}],["199387",{"2":{"134":1}}],["1990",{"2":{"138":1}}],["19975941",{"2":{"80":1}}],["19",{"2":{"116":1,"182":14,"225":3,"233":1,"249":5,"258":1}}],["19203",{"2":{"182":1}}],["19209f",{"2":{"106":2}}],["192836",{"2":{"162":1}}],["1921258",{"2":{"136":1}}],["1929",{"2":{"22":1}}],["1594392035681557e",{"2":{"258":1}}],["159468f",{"2":{"256":1}}],["1599497767766602e",{"2":{"258":1}}],["159599244983697e",{"2":{"258":1}}],["1595423f",{"2":{"256":1}}],["1593381391560795e",{"2":{"258":1}}],["1598034f",{"2":{"256":1}}],["159042358",{"2":{"244":2}}],["156047561982724e",{"2":{"258":1}}],["156671952072757e",{"2":{"258":1}}],["156455f",{"2":{"256":1}}],["1565",{"2":{"249":1}}],["15681",{"2":{"236":6}}],["15680",{"2":{"236":6}}],["15686738",{"2":{"106":1}}],["156",{"2":{"188":2}}],["1523379756114767e",{"2":{"258":1}}],["1521302f",{"2":{"256":1}}],["15258",{"2":{"182":1}}],["15293",{"2":{"182":1}}],["15276335",{"2":{"129":1}}],["1556674774525573e",{"2":{"258":1}}],["155613f",{"2":{"256":1}}],["1558397f",{"2":{"256":1}}],["1550",{"2":{"249":2}}],["15501",{"2":{"244":1}}],["15591",{"2":{"182":1}}],["1557574",{"2":{"137":1}}],["15544322",{"2":{"106":1}}],["15542f",{"2":{"80":1}}],["15701",{"2":{"236":6}}],["15700",{"2":{"236":12}}],["15707001",{"2":{"136":1}}],["1577715",{"2":{"80":1}}],["1515796586256786e",{"2":{"258":1}}],["1515656069153655e",{"2":{"258":1}}],["1517824f",{"2":{"256":1}}],["151333f",{"2":{"256":1}}],["15133555",{"2":{"106":1}}],["1511",{"2":{"249":1}}],["151",{"2":{"208":1,"217":1}}],["15107603",{"2":{"106":1}}],["15107606",{"2":{"106":1}}],["15827",{"2":{"182":1}}],["1586047939092094",{"2":{"164":2}}],["1585735",{"2":{"106":1}}],["15802455",{"2":{"106":1}}],["154041",{"2":{"166":1}}],["1545162",{"2":{"137":1}}],["15451166",{"2":{"80":1}}],["154",{"2":{"116":1,"188":2}}],["15497",{"2":{"182":1}}],["15493166",{"2":{"106":1}}],["1549292",{"2":{"106":1}}],["150272418380917e",{"2":{"258":1}}],["1501",{"2":{"244":1}}],["1500195f",{"2":{"256":1}}],["15001",{"2":{"244":1}}],["1500",{"2":{"229":2,"250":1}}],["150",{"2":{"72":1}}],["15",{"2":{"22":1,"53":1,"136":1,"137":3,"166":4,"175":3,"182":14,"184":3,"192":1,"210":3,"218":3,"225":2,"226":3,"237":3,"246":3,"249":5,"251":1,"260":1}}],["1d",{"2":{"9":1,"69":1}}],["1",{"2":{"2":1,"4":1,"5":2,"9":48,"12":1,"16":4,"22":8,"24":10,"32":6,"46":5,"48":6,"49":1,"52":1,"53":110,"54":1,"59":24,"65":27,"66":9,"67":1,"68":24,"70":7,"71":12,"72":13,"73":3,"80":57,"81":13,"86":2,"93":1,"102":1,"106":36,"107":5,"109":35,"110":47,"115":6,"117":1,"118":2,"120":4,"121":2,"122":2,"123":9,"125":4,"129":3,"130":11,"131":1,"134":13,"136":23,"137":19,"139":3,"140":3,"141":3,"162":6,"164":88,"166":45,"168":4,"169":5,"171":4,"172":6,"173":1,"174":6,"175":4,"178":11,"180":1,"181":1,"182":116,"184":4,"187":5,"188":8,"189":1,"190":5,"191":2,"192":1,"194":2,"206":12,"208":18,"209":7,"210":4,"213":90,"214":5,"215":1,"216":4,"217":17,"218":4,"221":4,"222":1,"223":1,"224":1,"225":7,"226":4,"229":5,"230":3,"232":1,"233":6,"234":1,"235":1,"236":118,"237":4,"241":1,"242":6,"243":3,"244":6,"245":5,"246":4,"248":6,"249":69,"250":9,"251":1,"254":37,"255":7,"256":130,"257":2,"258":138,"259":5,"260":1}}],["e^2",{"2":{"255":1}}],["err",{"2":{"245":3}}],["errs",{"2":{"245":3}}],["errored",{"2":{"108":1}}],["errors",{"2":{"33":3,"111":1,"208":1}}],["error",{"2":{"2":3,"3":1,"31":1,"33":6,"50":2,"52":1,"53":3,"55":1,"57":2,"60":1,"72":1,"95":2,"102":1,"109":3,"134":1,"174":1,"245":1,"249":2}}],["ess",{"2":{"249":3}}],["essentially",{"2":{"59":1,"65":1,"130":1,"160":1,"217":1}}],["estimated",{"2":{"138":1}}],["estimate",{"2":{"138":1,"141":1}}],["estimation",{"0":{"138":1},"1":{"139":1,"140":1,"141":1},"2":{"138":1,"208":4}}],["established",{"2":{"75":1,"169":1}}],["especially",{"2":{"88":1,"108":1}}],["epyc",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["epochs",{"2":{"208":2,"217":3}}],["epoch",{"2":{"59":1,"182":353,"190":2,"217":11,"225":2,"233":2}}],["eps",{"2":{"24":8,"53":1}}],["epsilon=1f",{"2":{"72":4}}],["epsilon=0",{"2":{"53":2}}],["epsilon=nothing",{"2":{"53":1}}],["epsilon",{"2":{"24":9,"53":9,"72":4}}],["emp",{"2":{"134":2,"135":2}}],["empting",{"2":{"66":1}}],["empty",{"2":{"12":1,"13":2,"52":2,"53":1,"66":2,"68":9,"70":4,"71":5,"72":5,"73":1,"129":1,"174":1,"256":1}}],["embeddings",{"2":{"70":2}}],["embedding",{"2":{"70":2,"104":1,"164":1,"223":1}}],["embed",{"2":{"59":8,"81":8,"164":1}}],["either",{"2":{"53":1,"66":4,"68":6,"254":1}}],["eagerly",{"2":{"154":1}}],["easily",{"2":{"76":1,"134":1,"151":1}}],["easier",{"2":{"52":1,"71":1,"250":1}}],["easy",{"2":{"61":1,"65":1,"77":1,"108":1,"149":1}}],["eachindex",{"2":{"249":1}}],["eachslice",{"2":{"54":2,"179":1,"180":1}}],["each",{"2":{"5":1,"9":1,"24":2,"34":4,"65":15,"66":8,"68":12,"69":4,"70":2,"71":4,"72":2,"135":3,"166":2,"207":1,"249":1,"255":1}}],["evolved",{"2":{"250":1}}],["eval=32",{"2":{"221":1}}],["evalpoly",{"2":{"213":2,"217":1}}],["eval",{"2":{"164":4,"221":5}}],["evaluate",{"2":{"50":1}}],["eventually",{"2":{"164":1}}],["even",{"2":{"3":1,"12":1,"13":1,"55":1,"76":1,"77":1,"84":1,"108":1,"110":1,"150":1,"166":1,"170":1,"191":1}}],["everystep",{"2":{"236":13}}],["everystep=false",{"2":{"231":1}}],["everything",{"2":{"77":2,"194":1}}],["every",{"2":{"48":1,"67":1,"113":1,"114":1,"154":1,"166":1}}],["ever",{"2":{"3":1,"77":1}}],["ecosystem",{"2":{"44":1,"81":1,"85":1}}],["eccv",{"2":{"24":1}}],["edges",{"2":{"133":1}}],["edge",{"2":{"30":1,"77":1,"88":1,"89":1,"111":1,"164":1}}],["effort",{"2":{"106":1}}],["efficient",{"2":{"53":1,"55":1,"74":1,"105":1,"132":1,"134":1,"135":1,"170":2,"240":1}}],["efficiently",{"2":{"27":2,"53":1}}],["effectively",{"2":{"208":2}}],["effect",{"2":{"1":1,"60":1,"73":1}}],["european",{"2":{"24":1}}],["equilibrium",{"2":{"77":1}}],["equivalent",{"2":{"13":1,"46":1,"53":2,"55":4,"59":1,"71":1,"96":1,"166":1}}],["equations",{"2":{"74":1,"257":1}}],["equals",{"2":{"71":1}}],["equally",{"2":{"24":1,"54":1}}],["equal",{"2":{"9":1,"34":1,"53":1,"66":2,"68":3}}],["elem",{"2":{"243":2,"245":2}}],["elementwise",{"2":{"20":1,"24":1,"71":2,"72":5}}],["element",{"0":{"57":1},"2":{"9":1,"22":2,"34":1,"53":1,"55":1,"57":10,"59":2,"65":1,"69":6,"72":1,"164":4,"166":4,"171":1,"172":1,"174":1,"179":1,"183":1}}],["elements",{"2":{"9":9,"13":1,"32":1,"66":2,"68":3,"70":1,"75":1,"166":6,"179":1}}],["elman",{"2":{"69":1}}],["eltypes",{"2":{"149":1,"159":1}}],["eltype",{"0":{"159":1},"2":{"24":9,"55":6,"56":3,"57":3,"115":1,"149":5,"159":2,"168":1}}],["elseif",{"2":{"126":1,"131":1,"254":1}}],["else",{"2":{"13":1,"22":1,"28":1,"44":1,"50":1,"52":1,"54":1,"55":1,"65":1,"67":3,"69":1,"71":1,"126":1,"254":3}}],["eg",{"2":{"13":1,"66":2,"68":3}}],["et",{"2":{"9":2,"22":2,"53":2,"252":1}}],["etc",{"2":{"3":2,"12":1,"25":1,"77":1,"129":1,"132":1,"134":2,"169":1,"193":1,"194":1,"196":1,"204":1}}],["energy",{"2":{"249":2}}],["environment",{"2":{"175":2,"184":2,"192":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"260":1}}],["engine",{"2":{"168":1}}],["enforce",{"2":{"168":1}}],["enough",{"2":{"154":1,"193":1,"208":1}}],["encountered",{"2":{"110":1}}],["encounter",{"2":{"89":1,"106":1,"133":1}}],["enhance",{"2":{"88":1}}],["entries",{"2":{"135":1}}],["entry",{"2":{"119":1}}],["entropy",{"2":{"53":3}}],["enter",{"2":{"79":1}}],["entire",{"2":{"67":2,"69":5,"72":1,"114":1,"208":1}}],["entirely",{"2":{"11":1,"66":2,"70":3,"234":1,"236":1}}],["enzymemlir",{"2":{"106":1}}],["enzyme",{"2":{"49":3,"52":1,"77":1,"86":1,"87":4,"89":1,"93":2,"106":10,"133":1,"164":1,"168":1,"170":1}}],["enable",{"2":{"151":1}}],["enables",{"2":{"23":1,"77":1,"247":1}}],["enabled",{"2":{"12":1,"33":2,"102":1,"247":1}}],["ensuring",{"2":{"22":1}}],["ensures",{"2":{"143":1}}],["ensure",{"2":{"2":1,"9":1,"13":1,"77":1,"89":1,"138":1,"143":1,"179":1,"216":1}}],["endpoints",{"2":{"254":2}}],["end",{"2":{"5":2,"9":2,"16":1,"32":2,"59":14,"65":4,"66":2,"68":3,"80":1,"81":7,"106":2,"107":4,"109":2,"110":4,"115":3,"116":1,"117":3,"120":1,"121":2,"122":2,"126":1,"129":4,"130":2,"131":3,"134":1,"135":1,"136":1,"137":1,"139":1,"140":1,"141":1,"154":2,"164":1,"167":1,"169":2,"174":3,"175":3,"176":1,"178":3,"179":4,"180":3,"181":1,"182":4,"184":3,"187":1,"189":2,"190":3,"192":3,"194":3,"206":2,"207":1,"208":4,"209":1,"210":3,"213":2,"217":4,"218":3,"221":2,"222":3,"223":1,"224":2,"225":4,"226":3,"229":1,"230":7,"231":1,"232":2,"233":3,"234":3,"237":3,"241":3,"242":3,"243":1,"244":3,"245":2,"246":3,"248":1,"249":4,"250":3,"251":3,"254":26,"255":2,"256":2,"257":2,"259":2,"260":3}}],["enumerate",{"2":{"5":2}}],["execution",{"2":{"169":1}}],["executables",{"2":{"106":1}}],["exhaustive",{"2":{"155":1}}],["existent",{"2":{"146":1}}],["exists",{"2":{"145":1,"230":1}}],["exist",{"2":{"59":2}}],["excellent",{"2":{"185":1}}],["except",{"2":{"12":1,"69":2,"71":1,"150":1}}],["exceptions",{"2":{"9":1}}],["exception",{"2":{"3":2,"50":1}}],["exciting",{"2":{"90":1}}],["exclusively",{"2":{"32":1,"47":1}}],["excluding",{"2":{"31":1,"46":1,"71":1,"72":1}}],["exclude",{"2":{"15":2,"92":1}}],["exclude=internal",{"2":{"15":1}}],["external",{"2":{"249":1}}],["extensively",{"2":{"77":1,"88":2,"89":1,"170":1}}],["extensive",{"2":{"76":1,"77":1}}],["extensible",{"2":{"76":1}}],["extensions",{"0":{"50":1}}],["extending",{"2":{"168":1,"179":1}}],["extended",{"2":{"31":1,"32":1,"33":1,"53":2,"57":1,"59":1,"65":6,"66":2,"68":3,"69":2,"71":1,"72":4,"73":1}}],["extend",{"2":{"12":1,"31":1,"77":1}}],["ext",{"2":{"116":2}}],["extrema",{"2":{"243":2}}],["extremely",{"2":{"33":1,"52":1,"124":1,"194":1}}],["extract",{"2":{"249":2,"250":1}}],["extra",{"2":{"9":1}}],["exactly",{"2":{"21":1,"54":1,"73":1,"96":1,"129":1,"130":1,"196":1}}],["exact",{"2":{"9":1,"130":1,"182":1}}],["examples",{"0":{"163":1},"2":{"5":1,"9":2,"53":2,"59":4,"84":1,"163":1,"202":1}}],["example",{"0":{"114":1},"1":{"115":1,"116":1,"117":1,"118":1},"2":{"2":1,"3":2,"4":1,"9":1,"13":1,"24":1,"28":1,"31":3,"32":2,"34":3,"44":1,"46":1,"48":1,"49":1,"52":1,"53":14,"58":1,"65":3,"71":5,"72":3,"84":1,"96":1,"99":1,"108":1,"120":1,"121":1,"131":1,"132":1,"135":4,"136":1,"137":1,"142":2,"149":1,"160":1,"171":1,"172":1,"193":1,"194":1,"256":1}}],["exp",{"2":{"243":1}}],["exposing",{"2":{"236":1}}],["exported",{"2":{"3":1,"11":1,"30":1,"167":1}}],["exploiting",{"2":{"154":1}}],["explore",{"2":{"133":2}}],["explicit",{"2":{"74":1,"77":1,"162":2,"247":1}}],["explicitly",{"2":{"12":1,"59":2}}],["express",{"2":{"249":1}}],["expressed",{"2":{"107":1}}],["expression",{"2":{"48":1,"59":1}}],["expr",{"2":{"50":4,"59":1}}],["expected",{"2":{"53":4,"65":1,"66":2,"70":1,"77":1,"110":1,"146":1,"169":1}}],["expect",{"2":{"30":1,"77":1,"106":1,"146":1,"164":1,"247":1}}],["expects",{"2":{"28":1,"46":1,"48":1,"66":1,"68":3,"73":1,"131":1}}],["experimental",{"0":{"30":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"2":1,"4":2,"12":1,"30":2,"31":10,"32":2,"33":6,"34":2,"102":5,"103":5,"106":2,"108":1,"109":2,"110":2,"111":1,"119":1,"120":3,"121":2,"122":2,"123":1,"124":1,"147":1,"149":1,"164":2}}],["e",{"2":{"2":1,"9":1,"16":1,"24":1,"31":1,"44":1,"46":1,"53":3,"57":1,"58":1,"66":2,"67":2,"68":9,"72":3,"95":2,"106":1,"109":4,"110":4,"116":2,"132":1,"138":1,"143":1,"254":4,"255":6,"256":3}}],["gw",{"2":{"255":1}}],["gc",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["gnu",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["guess",{"2":{"258":1}}],["guide",{"0":{"143":1,"144":1},"1":{"145":1,"146":1},"2":{"129":1,"144":1}}],["guarantee",{"2":{"103":1,"160":1}}],["guarantees",{"2":{"30":1}}],["guaranteed",{"2":{"21":1,"23":1,"134":2}}],["gdev",{"2":{"100":1,"125":1,"126":4,"154":6,"164":2,"205":1,"207":3,"208":3,"209":1,"239":1,"244":3}}],["g=tanh",{"2":{"69":1}}],["gif",{"2":{"245":1,"250":2}}],["gib",{"2":{"175":2,"184":2,"210":2,"218":2,"226":2,"237":2,"246":2}}],["gigantic",{"2":{"108":1}}],["github",{"2":{"61":2,"69":1,"84":2,"111":1,"133":1,"164":1}}],["give",{"2":{"178":1,"258":2}}],["gives",{"2":{"54":1,"59":1,"129":1}}],["given",{"2":{"2":1,"4":2,"9":6,"10":24,"24":1,"25":1,"37":6,"38":2,"39":5,"44":1,"53":3,"65":2,"70":3,"72":5,"76":1,"77":1,"208":1}}],["gs",{"2":{"59":2,"80":3,"190":1}}],["globals",{"2":{"256":1}}],["globally",{"2":{"247":1}}],["globallppool",{"2":{"68":1,"105":1}}],["globalmeanpool",{"2":{"68":1}}],["globalmaxpool",{"2":{"68":1}}],["global",{"2":{"48":1,"68":3,"149":1,"164":1,"207":1,"247":1}}],["glorot",{"2":{"9":4,"129":2}}],["grid",{"2":{"243":5,"245":8}}],["green",{"2":{"217":1}}],["great",{"2":{"204":1}}],["ground",{"2":{"174":3}}],["group",{"2":{"24":3,"72":1,"249":1}}],["groups=1",{"2":{"66":2}}],["groups",{"2":{"24":4,"66":8,"72":4}}],["groupnorm",{"2":{"24":1,"72":7,"196":1}}],["gravitational",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["gravitate",{"2":{"55":1}}],["graph",{"2":{"248":1,"250":1}}],["grads",{"2":{"52":11,"107":1,"174":1}}],["gradient|jacobian",{"2":{"133":2}}],["gradient",{"0":{"49":1,"136":1},"2":{"27":2,"47":1,"49":1,"52":1,"106":8,"110":4,"134":6,"135":3,"136":6,"137":6,"141":3,"171":9,"174":1,"193":2,"194":2,"204":1,"242":3}}],["gradients",{"0":{"171":1},"2":{"27":2,"40":1,"49":10,"52":29,"53":2,"54":1,"65":1,"80":6,"133":1,"134":1,"141":1,"145":1,"171":1,"190":1,"240":1}}],["grucell",{"2":{"69":6,"104":1}}],["gru",{"2":{"12":1,"69":1}}],["günter",{"2":{"22":1}}],["gaussian",{"2":{"249":1}}],["gaussadjoint",{"2":{"233":1}}],["gated",{"2":{"69":1}}],["gamma=0",{"2":{"53":1}}],["gamma",{"2":{"53":2}}],["ganguli",{"2":{"9":1}}],["gain=1",{"2":{"9":1,"162":2}}],["gain",{"2":{"9":15,"66":4,"70":2,"104":2}}],["gt",{"2":{"5":1,"9":1,"16":1,"53":1,"54":1,"59":2,"66":2,"68":3,"71":2,"72":2,"73":3,"82":1,"95":2,"100":1,"125":1,"130":2,"133":12,"179":1}}],["geared",{"2":{"204":1}}],["gelu",{"2":{"106":6,"107":2,"149":1}}],["geoffrey",{"2":{"24":1}}],["genericlossfunction",{"2":{"53":2}}],["genericbroadcastop",{"2":{"25":1}}],["generic",{"2":{"18":1,"21":2,"23":2,"81":1,"102":1,"106":1,"110":1,"125":1,"139":1,"140":1,"141":1,"164":1,"178":1,"180":1,"181":1,"187":1,"189":1,"190":1,"205":1,"213":1,"221":1,"222":1,"223":1,"224":1,"229":1,"230":2,"231":1,"232":1,"239":1,"242":2,"249":2,"250":1,"254":5,"256":1,"257":2}}],["generating",{"0":{"248":1},"2":{"73":1}}],["generation",{"2":{"69":3}}],["generator=lux",{"2":{"222":1}}],["generator",{"2":{"9":3,"22":4,"67":3,"129":1,"165":1,"169":2,"213":1,"222":4,"223":2}}],["generates",{"2":{"248":1}}],["generated",{"2":{"9":1,"22":1,"59":1,"67":1,"133":1,"175":1,"184":1,"192":1,"210":1,"211":1,"218":1,"226":1,"237":1,"246":1,"250":1,"251":1,"260":1}}],["generate",{"0":{"206":1,"243":1},"2":{"3":1,"14":1,"15":1,"59":1,"69":1,"132":1,"162":2,"174":2,"178":1,"213":4,"222":1,"243":1,"248":2}}],["generalization",{"2":{"100":1}}],["generalized",{"2":{"5":1}}],["general",{"0":{"13":1},"2":{"61":1,"65":1,"82":1,"148":1,"256":1}}],["generally",{"2":{"5":1}}],["getaxes",{"2":{"222":1}}],["getindex",{"2":{"207":1}}],["getproperty",{"2":{"54":3,"131":2,"236":3}}],["getting",{"0":{"78":1},"1":{"79":1,"80":1,"81":1,"82":1,"83":1},"2":{"13":1}}],["getfield",{"2":{"12":1,"115":3,"131":3}}],["get",{"0":{"84":1},"2":{"3":5,"18":1,"19":1,"25":1,"36":2,"37":2,"38":2,"54":1,"80":1,"131":1,"138":2,"143":2,"159":1,"163":1,"164":2,"166":2,"169":2,"178":3,"179":1,"182":2,"193":1,"203":1,"249":2,"254":1}}],["g",{"2":{"2":1,"65":4,"67":2,"72":4,"106":1,"254":1}}],["goal",{"2":{"248":1}}],["goodies",{"2":{"172":1}}],["good",{"2":{"127":1,"134":1,"208":1,"249":1,"258":2}}],["goodfellow",{"2":{"71":1}}],["going",{"2":{"44":1,"136":1}}],["go",{"2":{"2":1,"84":2,"110":1,"114":1,"119":1,"130":1,"131":1,"148":1,"176":1,"238":1}}],["gpuci",{"2":{"109":1,"116":3,"134":2,"236":3,"244":4}}],["gpuarray",{"2":{"54":1}}],["gpuarrayscore",{"2":{"150":2}}],["gpuarrays",{"2":{"7":7,"54":1}}],["gpubroadcastop",{"2":{"25":1}}],["gpusintel",{"2":{"62":2}}],["gpusmetal",{"2":{"62":2}}],["gpusamd",{"2":{"62":2}}],["gpus",{"2":{"18":1,"37":1,"62":2,"124":1,"149":1,"150":1,"154":1}}],["gpu",{"0":{"4":1,"62":1,"83":1,"124":1,"150":1,"157":1,"158":1,"164":1},"1":{"125":1,"126":1},"2":{"1":8,"2":7,"3":10,"5":2,"21":1,"25":1,"49":1,"62":4,"63":1,"68":3,"80":2,"81":2,"83":1,"86":1,"87":3,"103":2,"106":2,"115":1,"124":4,"125":9,"126":5,"146":4,"154":4,"158":3,"162":1,"164":15,"167":3,"182":1,"194":1,"205":1,"207":3,"216":3,"217":2,"225":1,"233":3,"236":1,"239":1}}],["nparameters",{"2":{"249":3}}],["nvml",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["nvidia",{"2":{"3":1,"5":1,"62":2,"175":4,"184":4,"192":2,"210":4,"218":4,"226":4,"237":4,"246":4}}],["ndrange=length",{"2":{"164":2}}],["ndims",{"2":{"16":1,"28":4,"66":4,"68":9,"71":4,"72":2,"73":2,"115":2,"139":2,"140":2}}],["nicely",{"2":{"134":1}}],["nilarray",{"2":{"95":1}}],["nitish",{"2":{"22":1}}],["n=tanh⁡",{"2":{"69":1}}],["nn",{"2":{"66":2,"70":1,"164":7,"249":12,"250":11,"256":14,"257":1,"259":9}}],["nnlib",{"2":{"18":2,"19":2,"66":1,"73":6,"102":3,"152":3,"193":2,"194":2,"217":1}}],["ntuple",{"2":{"65":1,"66":2,"72":2,"115":1,"236":13,"244":1}}],["nlayers",{"2":{"59":2,"81":2}}],["nccl",{"2":{"36":1,"37":4,"145":1}}],["ncclbackend",{"2":{"36":2,"37":2,"143":4}}],["native",{"2":{"197":1,"204":1}}],["nabla",{"2":{"171":1}}],["naming",{"2":{"65":9,"71":2}}],["name>=",{"2":{"155":3}}],["namefreezing",{"2":{"121":1}}],["name=",{"2":{"59":1,"65":1}}],["name=nothing",{"2":{"59":1,"65":11}}],["names",{"2":{"31":1,"59":1,"72":1}}],["named",{"2":{"24":2,"31":2,"53":1,"131":1,"231":2,"233":1,"249":1}}],["namedtuples",{"2":{"131":1}}],["namedtuple",{"2":{"12":4,"13":1,"15":3,"31":5,"32":1,"52":1,"54":1,"55":1,"59":3,"65":10,"66":2,"68":9,"69":2,"70":4,"71":9,"72":5,"73":1,"80":24,"81":1,"95":1,"106":7,"109":3,"110":6,"113":3,"115":3,"116":7,"117":3,"120":3,"123":4,"129":4,"130":3,"132":1,"164":2,"179":2,"216":2,"217":1,"236":147,"249":1,"256":6}}],["name",{"0":{"121":1},"2":{"12":2,"13":5,"31":1,"32":1,"59":3,"71":1,"82":1,"102":1,"115":5,"121":5,"122":6,"125":1,"131":1,"225":4}}],["navab",{"2":{"53":1}}],["nassir",{"2":{"53":1}}],["nans",{"0":{"110":1},"2":{"33":3,"110":7,"111":1}}],["nan",{"2":{"33":5,"110":31,"244":1}}],["nₙ",{"2":{"28":2}}],["n₂",{"2":{"28":2}}],["n₁",{"2":{"28":2}}],["n",{"2":{"24":5,"54":2,"59":5,"65":23,"66":27,"68":54,"70":2,"71":11,"72":11,"73":8,"81":5,"107":1,"174":3,"182":2,"187":3,"190":1,"208":1,"217":1,"221":12,"225":2,"229":3,"230":5,"233":1,"234":5,"244":1,"248":2,"249":3,"250":2,"257":1}}],["numer",{"2":{"255":3,"256":3}}],["numerically",{"2":{"181":1}}],["numerical",{"2":{"24":4,"53":1,"72":4,"149":1,"249":1}}],["numeric",{"2":{"9":2}}],["num",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"250":2,"251":1,"260":1}}],["number=0",{"2":{"9":2}}],["number=1",{"2":{"9":2}}],["number",{"2":{"9":8,"14":1,"15":1,"22":4,"24":1,"38":1,"52":1,"53":4,"54":3,"55":1,"59":3,"65":3,"66":10,"67":3,"68":3,"70":7,"71":2,"72":3,"73":3,"129":1,"165":1,"166":1,"169":2,"170":1,"213":1,"248":1,"249":4}}],["numbers",{"2":{"9":6,"10":12,"162":1,"166":3}}],["nepochs",{"2":{"174":3,"190":3,"225":3,"233":3}}],["nest",{"2":{"134":1}}],["nested",{"0":{"29":1,"133":1,"156":1},"1":{"134":1,"135":1,"136":1,"137":1,"138":1,"139":1,"140":1,"141":1},"2":{"3":1,"29":2,"34":2,"55":5,"58":2,"86":1,"133":1,"136":2,"156":3,"238":1,"240":1,"245":1}}],["next",{"2":{"129":1,"143":1,"249":1,"250":1,"254":1,"256":1,"257":1}}],["nearest",{"2":{"73":7}}],["necessary",{"2":{"68":3,"129":1}}],["negative",{"2":{"53":1,"254":1}}],["net",{"2":{"53":1,"242":11,"249":1,"256":1,"259":2}}],["network",{"0":{"214":1,"247":1,"249":1,"256":1,"257":1,"258":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"9":1,"24":1,"75":2,"77":1,"81":1,"114":1,"134":1,"135":2,"164":2,"165":1,"176":2,"214":1,"222":5,"223":3,"248":1,"249":6,"250":4,"256":6}}],["networks",{"0":{"164":1,"241":1},"2":{"9":3,"22":3,"46":1,"53":2,"71":1,"74":1,"77":3,"141":1,"164":1,"185":1,"204":2,"241":2,"249":1}}],["never",{"2":{"30":1,"44":1,"71":1,"256":1}}],["needed",{"2":{"18":2,"27":2,"28":1,"52":1,"102":1,"133":1,"145":1,"162":2,"194":1}}],["need",{"2":{"11":1,"13":1,"31":1,"34":1,"44":1,"59":2,"71":1,"80":1,"90":1,"102":1,"106":3,"107":1,"109":1,"113":1,"129":3,"130":2,"133":1,"143":1,"144":2,"160":1,"163":1,"164":1,"166":1,"178":1,"179":1,"188":2,"194":1,"208":1,"216":1,"223":1,"230":1,"242":1,"254":1,"256":1}}],["needs",{"2":{"3":1,"12":1,"15":1,"18":2,"66":1,"67":1,"115":1,"137":1,"144":1,"158":1}}],["neuralpde",{"2":{"238":2}}],["neuralode",{"2":{"230":7,"233":4,"236":5}}],["neuralodecompact",{"2":{"230":2,"233":1,"236":1}}],["neural",{"0":{"164":1,"214":1,"227":1,"230":1,"231":1,"235":1,"241":1,"247":1,"249":1,"252":1,"256":1,"257":1,"258":1},"1":{"228":1,"229":1,"230":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"248":1,"249":1,"250":1,"251":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"9":4,"22":4,"46":1,"53":1,"58":1,"74":2,"75":2,"77":4,"81":1,"114":3,"134":3,"135":2,"141":1,"164":3,"165":1,"176":2,"185":1,"204":5,"214":1,"227":2,"230":1,"231":1,"248":1,"249":7,"250":2,"256":7,"257":1,"259":2}}],["newtonian",{"2":{"254":2,"256":1,"258":1}}],["new",{"0":{"93":1,"96":1,"100":1,"105":1,"235":1},"2":{"1":1,"32":9,"34":5,"52":1,"59":1,"67":1,"69":4,"71":5,"90":1,"92":1,"103":1,"106":1,"113":1,"124":1,"127":1,"129":1,"130":1,"133":1,"144":1,"164":1,"222":2,"249":3}}],["nom",{"2":{"249":1}}],["noisy",{"2":{"178":2}}],["noise",{"2":{"174":1}}],["now",{"2":{"93":1,"100":1,"103":1,"104":4,"105":2,"106":4,"107":1,"109":2,"110":2,"129":1,"130":1,"134":1,"135":2,"136":1,"137":1,"141":2,"144":1,"146":3,"149":1,"164":2,"165":2,"170":1,"174":1,"179":2,"181":1,"188":1,"191":1,"194":1,"208":1,"217":1,"247":1,"249":2,"250":1,"254":1,"256":2,"257":1,"259":1}}],["norm",{"2":{"72":1,"134":7,"135":7,"136":6,"137":6,"141":12}}],["normally",{"2":{"106":1}}],["normalises",{"2":{"24":2,"72":2}}],["normalize",{"2":{"72":2}}],["normalized",{"2":{"24":8,"53":3,"72":7,"245":2}}],["normalizing",{"2":{"22":1}}],["normalization",{"0":{"24":1,"72":1},"2":{"9":1,"24":12,"28":1,"72":11,"135":1}}],["normal",{"2":{"7":2,"9":12,"10":6,"162":6,"249":1,"256":3}}],["nooplayer",{"2":{"65":6,"67":3,"71":3}}],["nonlinear",{"2":{"9":1}}],["non",{"0":{"118":1},"2":{"9":2,"21":1,"23":1,"31":1,"33":1,"37":2,"52":2,"53":1,"59":9,"70":1,"95":1,"98":1,"108":1,"146":1,"194":2,"203":1,"230":2,"254":1}}],["nonetheless",{"2":{"2":1}}],["none",{"2":{"2":1,"30":1,"31":1,"33":1,"57":1,"110":1}}],["node",{"2":{"3":1}}],["notice",{"2":{"129":1,"130":1,"135":2,"233":1}}],["notion",{"2":{"55":1}}],["notangent",{"2":{"49":1,"110":1}}],["not",{"2":{"2":1,"3":4,"4":2,"9":2,"11":1,"12":1,"13":1,"20":1,"21":1,"22":1,"24":3,"28":1,"33":1,"34":1,"36":2,"46":3,"47":1,"50":1,"52":1,"54":1,"57":1,"58":1,"59":1,"65":1,"69":11,"71":1,"72":1,"77":1,"80":1,"85":1,"88":1,"89":2,"95":2,"102":1,"106":1,"109":1,"110":1,"125":1,"127":1,"133":1,"134":5,"138":1,"141":1,"143":1,"146":1,"154":1,"161":1,"163":1,"164":1,"191":2,"194":3,"203":1,"208":1,"214":1,"230":1,"233":2,"235":1,"236":2,"249":1,"250":1,"254":1}}],["notes",{"2":{"21":1,"23":1,"49":1,"172":1,"227":1}}],["note",{"2":{"2":1,"3":4,"9":1,"18":2,"31":1,"35":1,"52":1,"53":1,"55":2,"58":1,"59":1,"71":1,"74":1,"89":1,"106":1,"107":1,"109":1,"129":1,"130":1,"133":1,"136":1,"143":1,"150":1,"168":1,"171":1,"183":1,"197":1,"230":1,"236":1,"247":1}}],["nothing",{"2":{"2":4,"3":5,"4":1,"13":2,"20":1,"21":1,"22":2,"23":1,"24":15,"31":5,"36":3,"37":1,"49":1,"53":12,"54":1,"55":2,"58":1,"65":4,"66":4,"67":1,"69":11,"70":4,"71":10,"72":5,"80":27,"81":1,"106":9,"110":1,"116":4,"134":2,"136":2,"137":2,"206":1,"207":4,"208":2,"217":6,"236":117,"249":1,"256":1}}],["no",{"2":{"2":2,"4":2,"5":1,"30":1,"31":3,"46":1,"48":1,"52":1,"53":2,"57":1,"60":1,"71":1,"77":1,"80":1,"97":1,"100":1,"102":1,"109":1,"110":1,"116":1,"125":2,"126":1,"129":1,"145":3}}],["u=u",{"2":{"242":1}}],["u0=res",{"2":{"208":1}}],["u0",{"2":{"206":2,"208":2,"209":1,"255":2,"256":2,"257":1,"259":2}}],["utc",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["utility",{"0":{"224":1,"232":1,"254":1}}],["utilities",{"0":{"51":1},"1":{"52":1,"53":1,"54":1,"55":1,"56":1,"57":1,"58":1,"59":1,"60":1},"2":{"47":1,"194":1}}],["utils",{"0":{"35":1},"1":{"36":1,"37":1,"38":1,"39":1,"40":1,"41":1},"2":{"31":2,"55":1,"134":4}}],["url",{"2":{"74":1}}],["url=",{"2":{"61":1}}],["u",{"2":{"27":7,"206":4,"208":11,"209":6,"230":7,"234":2,"241":4,"242":11,"244":4,"245":7,"255":3,"256":4,"258":1,"259":2}}],["ulyanov",{"2":{"24":1,"72":1}}],["upto",{"2":{"240":1}}],["upsampled",{"2":{"73":1}}],["upsample",{"2":{"73":5,"104":1}}],["upsampling",{"0":{"73":1},"2":{"73":2}}],["upscaling",{"2":{"73":1}}],["upscale",{"2":{"73":2}}],["up",{"0":{"257":1},"2":{"54":1,"138":1,"141":1,"203":2,"227":1}}],["updating",{"0":{"56":1,"90":1},"1":{"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":1,"103":1,"104":1,"105":1},"2":{"24":2,"40":1,"79":1,"174":1}}],["updates",{"2":{"34":1,"52":3,"72":1,"98":1,"143":1}}],["updated",{"2":{"20":1,"22":2,"24":2,"34":1,"52":5,"53":1,"65":6,"67":3,"69":7,"71":1,"72":1,"76":1,"77":1,"81":1,"104":1,"129":1,"146":1,"179":1,"203":1,"216":1}}],["update",{"2":{"15":2,"20":1,"22":2,"52":2,"56":1,"59":1,"67":3,"69":5,"72":3,"79":2,"90":1,"168":1}}],["upon",{"2":{"5":1}}],["untrained",{"2":{"256":2,"259":1}}],["until",{"2":{"67":1}}],["uncertain",{"2":{"250":1}}],["unchanged",{"2":{"65":1}}],["unable",{"2":{"168":1}}],["unnecessary",{"2":{"135":1}}],["unnormalized",{"2":{"72":1}}],["un",{"2":{"102":1}}],["unexpected",{"2":{"77":1}}],["unreasonably",{"2":{"65":1}}],["unreleased",{"2":{"61":1}}],["unrolls",{"2":{"65":1}}],["unroll",{"2":{"18":1}}],["unwrapped",{"2":{"55":1}}],["unwrap",{"2":{"55":2}}],["unwraps",{"2":{"31":1}}],["undone",{"2":{"34":1}}],["underlying",{"2":{"230":1}}],["under",{"2":{"194":1}}],["understand",{"2":{"59":1,"127":1,"179":1,"227":1,"230":1}}],["understanding",{"2":{"9":2,"222":1}}],["undef",{"2":{"169":1}}],["undefined",{"2":{"13":1,"69":1}}],["undesirable",{"2":{"16":1}}],["unfreezes",{"2":{"31":1}}],["unfreeze",{"2":{"31":2}}],["unpack",{"2":{"13":1}}],["unlike",{"2":{"10":1,"59":1}}],["unless",{"2":{"2":1,"33":1,"222":1}}],["unified",{"2":{"204":1}}],["uniformly",{"2":{"143":1}}],["uniform",{"2":{"9":6,"10":6,"66":5,"69":6,"70":4,"80":1,"129":3}}],["uninitiated",{"0":{"165":1},"1":{"166":1,"167":1,"168":1,"169":1,"170":1,"171":1,"172":1,"173":1,"174":1,"175":1}}],["universal",{"2":{"71":1}}],["unity",{"2":{"254":1}}],["unitrange",{"2":{"168":1}}],["unit",{"2":{"69":1}}],["union",{"2":{"2":3,"9":2,"13":1,"22":1,"33":2,"46":1,"53":3,"69":2,"72":1}}],["unsupported",{"2":{"7":1,"13":1}}],["unsafe",{"2":{"5":1}}],["unknown",{"2":{"3":2}}],["unknowndevice",{"2":{"3":2}}],["usr",{"2":{"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2}}],["usacases",{"2":{"87":2}}],["usage",{"2":{"31":1,"84":1}}],["usual",{"2":{"72":3,"137":1,"164":1}}],["usually",{"2":{"53":2,"59":1,"73":1,"166":1,"208":1}}],["us",{"2":{"27":2,"30":1,"108":1,"109":1,"110":6,"130":2,"131":3,"133":1,"165":1,"169":1,"172":1,"173":1,"174":3,"178":1,"194":1,"235":1,"256":1,"257":1,"259":2}}],["usecases",{"2":{"55":1,"58":1,"164":1,"238":1}}],["uses",{"2":{"21":1,"23":2,"44":1,"55":4,"56":1,"58":1,"72":1,"95":1,"133":1,"136":1,"149":1,"152":2,"155":1,"169":1,"204":1,"254":2,"255":1,"256":1,"258":1}}],["userbase",{"2":{"44":1}}],["user",{"2":{"12":2,"13":2,"18":2,"58":1,"65":1,"75":1,"113":1,"136":1,"194":1,"196":2}}],["users",{"2":{"11":1,"12":2,"20":1,"31":1,"36":2,"37":1,"52":1,"55":1,"75":1,"77":1,"95":1,"98":1,"149":1,"168":1,"227":2,"230":1,"236":1}}],["useful",{"2":{"3":1,"9":3,"25":1,"33":2,"53":1,"54":1,"57":1,"59":2,"74":1,"108":1,"109":1}}],["use",{"0":{"77":1,"197":1},"2":{"2":1,"4":2,"5":1,"9":1,"18":3,"20":1,"22":1,"23":2,"25":1,"27":2,"28":1,"30":1,"31":2,"33":1,"36":4,"47":2,"52":4,"59":1,"61":1,"66":11,"69":18,"70":16,"72":3,"74":1,"77":1,"80":2,"81":1,"84":2,"87":3,"89":1,"98":1,"102":2,"103":4,"104":1,"105":1,"106":3,"107":2,"108":1,"109":1,"120":1,"125":1,"133":3,"134":1,"135":1,"136":2,"138":2,"143":2,"147":1,"149":2,"152":1,"153":1,"157":1,"158":2,"161":2,"163":1,"164":4,"165":1,"166":1,"169":1,"170":1,"171":1,"172":2,"174":3,"178":2,"179":3,"181":2,"185":2,"187":1,"197":1,"215":1,"216":1,"217":1,"227":1,"229":1,"230":1,"231":2,"233":2,"238":1,"242":1,"244":4,"247":1,"248":1,"249":5,"250":1,"254":1,"256":5}}],["used",{"2":{"2":2,"3":6,"9":1,"13":2,"22":3,"33":2,"34":2,"41":1,"44":1,"46":2,"52":2,"53":5,"55":1,"58":1,"59":11,"66":3,"67":3,"69":1,"70":1,"71":1,"72":7,"73":1,"95":1,"99":1,"102":1,"103":1,"105":1,"106":1,"124":1,"125":2,"133":1,"134":2,"146":1,"149":1,"160":1,"161":1,"163":1,"166":1,"234":1,"252":1}}],["using",{"0":{"45":1,"48":1,"106":1,"107":1,"118":1,"135":1,"139":1,"140":1,"141":1,"180":1,"204":1,"211":1,"227":1,"234":1},"1":{"46":1,"107":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"228":1,"229":1,"230":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"2":1,"3":1,"4":2,"5":2,"9":2,"12":2,"25":3,"27":2,"31":1,"32":1,"39":1,"40":1,"41":1,"44":3,"46":4,"48":2,"49":1,"52":7,"53":4,"55":1,"59":9,"61":2,"65":11,"69":6,"70":1,"71":2,"72":1,"75":1,"76":1,"77":2,"79":1,"80":3,"81":4,"95":1,"100":1,"105":1,"106":8,"107":2,"109":1,"124":3,"126":2,"127":2,"131":2,"133":4,"134":6,"135":1,"136":1,"137":2,"138":5,"141":14,"142":1,"146":1,"149":1,"150":1,"157":1,"164":3,"165":1,"166":2,"169":1,"172":2,"173":1,"174":1,"175":1,"176":2,"178":1,"180":1,"183":1,"184":1,"185":1,"186":2,"191":1,"192":1,"193":2,"196":1,"197":1,"204":1,"205":1,"208":1,"210":1,"212":1,"214":1,"218":1,"226":1,"228":2,"230":1,"234":1,"236":2,"237":1,"238":3,"239":1,"240":1,"241":1,"242":1,"246":1,"247":2,"249":1,"250":1,"251":1,"253":1,"255":1,"256":1,"260":1}}],["w=w",{"2":{"242":1}}],["w=gv∥v∥weight",{"2":{"72":1}}],["w=",{"2":{"59":1}}],["w=rand",{"2":{"59":1}}],["w=ones",{"2":{"59":3}}],["w3",{"2":{"59":3,"81":3}}],["w3=dense",{"2":{"59":1,"81":1}}],["w2",{"2":{"59":3,"81":3}}],["w2=",{"2":{"59":1,"81":1}}],["w1",{"2":{"59":3,"81":3}}],["w1=dense",{"2":{"59":1,"81":1}}],["w",{"2":{"59":21,"68":6,"70":1,"73":4,"81":4,"131":2,"152":1,"166":1,"174":5,"187":1,"229":1,"241":4,"242":10}}],["write",{"2":{"166":2,"168":1,"174":1,"222":1}}],["writing",{"0":{"117":1},"2":{"102":1,"118":1,"127":1}}],["written",{"2":{"44":1}}],["wrote",{"0":{"75":1},"1":{"76":1,"77":1},"2":{"203":1}}],["wrapping",{"2":{"95":1}}],["wrappers",{"0":{"27":1}}],["wrapper",{"2":{"19":1,"27":2,"33":1,"58":1,"69":1,"130":2,"137":1,"217":1,"241":1}}],["wrappedlayer",{"2":{"67":1}}],["wrappedfunction",{"2":{"66":1,"71":2,"110":3,"236":9,"256":1}}],["wrapped",{"2":{"12":3,"33":1,"65":8,"71":2,"130":1}}],["wrap",{"2":{"40":1,"108":1,"130":1,"143":1,"207":1}}],["wraps",{"2":{"37":2,"46":1,"69":2,"71":1}}],["wrt",{"2":{"27":2,"49":1,"52":3,"53":2,"54":1,"137":2,"144":1,"242":1}}],["wu",{"2":{"24":1}}],["wide",{"2":{"170":1}}],["wider",{"2":{"77":1,"93":1,"97":1,"172":1}}],["width",{"2":{"66":1}}],["wio",{"2":{"69":1}}],["wio×x+who×hprev+bo",{"2":{"69":1}}],["wig",{"2":{"69":1}}],["wig×x+whg×hprev+bg",{"2":{"69":1}}],["wif",{"2":{"69":1}}],["wif×x+whf×hprev+bf",{"2":{"69":1}}],["wii",{"2":{"69":1}}],["wii×x+whi×hprev+bi",{"2":{"69":1}}],["wiz",{"2":{"69":1}}],["wiz×x+biz+whz×hprev+bhz",{"2":{"69":1}}],["wir",{"2":{"69":1}}],["wir×x+bir+whr×hprev+bhr",{"2":{"69":1}}],["win",{"2":{"69":1}}],["win×x+bin+r⋅",{"2":{"69":1}}],["window",{"2":{"68":18}}],["will",{"2":{"3":2,"5":1,"9":3,"16":1,"24":1,"28":4,"30":1,"33":2,"34":3,"39":2,"44":4,"46":1,"48":2,"49":1,"50":4,"52":1,"55":4,"57":4,"58":1,"59":8,"61":1,"65":5,"67":2,"69":3,"71":1,"72":3,"80":1,"81":1,"95":1,"106":5,"108":1,"109":1,"110":2,"114":1,"119":1,"124":1,"127":1,"130":1,"133":2,"134":3,"136":1,"138":2,"139":1,"149":2,"151":1,"161":1,"163":1,"164":8,"166":1,"167":1,"168":1,"169":2,"170":4,"171":1,"172":1,"174":2,"176":2,"178":3,"179":6,"181":1,"185":2,"191":2,"193":1,"194":2,"203":1,"204":2,"207":4,"208":1,"211":1,"215":1,"216":1,"217":2,"230":1,"233":1,"238":3,"240":1,"241":1,"242":2,"243":3,"247":2,"249":4,"256":1}}],["withgradient",{"2":{"59":1}}],["within",{"2":{"22":2,"24":2,"59":1,"134":2}}],["without",{"2":{"2":1,"11":1,"59":1,"77":2,"95":2,"106":1,"107":1,"127":1,"134":2,"168":1,"179":1,"208":2}}],["with",{"0":{"185":1},"1":{"186":1,"187":1,"188":1,"189":1,"190":1,"191":1,"192":1},"2":{"1":1,"2":1,"3":1,"5":2,"9":5,"11":1,"12":2,"13":1,"15":2,"16":1,"18":7,"20":1,"21":2,"23":1,"25":1,"28":2,"31":5,"32":2,"33":3,"34":1,"37":2,"40":1,"41":2,"44":1,"46":1,"48":2,"49":1,"52":2,"53":6,"54":2,"55":2,"56":3,"57":1,"59":3,"65":14,"66":11,"67":6,"68":12,"69":10,"70":12,"71":5,"72":3,"73":5,"75":1,"77":7,"81":3,"85":2,"87":1,"88":2,"89":2,"92":1,"93":1,"95":2,"102":3,"104":1,"105":1,"106":2,"107":1,"108":3,"109":1,"110":1,"118":2,"125":2,"127":1,"129":1,"130":1,"131":6,"133":3,"134":1,"135":4,"136":1,"139":1,"140":1,"141":1,"143":2,"145":2,"146":1,"147":1,"149":1,"151":1,"152":6,"157":1,"161":2,"162":2,"163":2,"164":5,"165":1,"166":5,"167":2,"168":1,"169":2,"170":1,"172":2,"174":4,"176":1,"178":1,"180":1,"181":1,"182":1,"187":1,"189":1,"190":1,"193":1,"203":1,"204":2,"205":1,"208":1,"213":1,"221":1,"222":2,"223":1,"224":1,"229":1,"230":3,"231":1,"232":1,"239":1,"240":1,"242":2,"247":2,"248":1,"249":6,"250":2,"254":5,"256":2,"257":2}}],["what",{"2":{"109":1,"110":1,"129":1,"131":1,"137":1,"159":1,"166":2,"196":1}}],["whatever",{"2":{"52":1,"71":1}}],["why",{"0":{"75":1,"77":1},"1":{"76":1,"77":1},"2":{"129":1}}],["whole",{"2":{"72":2}}],["whose",{"2":{"70":1,"72":1}}],["who",{"2":{"69":1}}],["whg",{"2":{"69":1}}],["whf",{"2":{"69":1}}],["whn",{"2":{"69":1}}],["whn×hprev+bhn",{"2":{"69":1}}],["whz",{"2":{"69":1}}],["whr",{"2":{"69":1}}],["whcn",{"2":{"66":1,"67":2,"72":3}}],["whi",{"2":{"69":1}}],["while",{"2":{"3":1,"32":1,"46":1,"58":1,"65":1,"73":1,"77":1,"107":1,"122":1,"127":1,"129":1,"149":1,"172":1}}],["which",{"0":{"113":1},"2":{"2":1,"3":1,"9":1,"12":1,"24":1,"30":1,"31":14,"34":1,"52":1,"53":5,"58":1,"59":5,"65":3,"66":1,"68":3,"69":1,"71":3,"72":4,"77":2,"84":1,"104":1,"105":1,"108":2,"127":1,"129":1,"130":2,"131":1,"138":1,"149":1,"165":1,"166":2,"168":1,"174":1,"180":1,"196":1,"208":1,"217":1,"230":1,"234":1,"247":1,"249":1,"252":1,"255":1,"256":1}}],["whether",{"2":{"33":2,"46":1,"67":1,"178":1}}],["whereas",{"2":{"66":1}}],["where",{"2":{"3":1,"9":7,"18":2,"21":1,"23":1,"24":1,"25":1,"32":1,"33":1,"53":4,"58":1,"66":5,"68":6,"69":6,"70":4,"71":3,"72":3,"73":1,"89":1,"110":2,"153":1,"171":1,"179":1,"180":1,"194":1,"221":1,"249":2,"250":2,"254":3}}],["when",{"2":{"3":2,"9":2,"21":1,"46":2,"48":1,"53":2,"54":1,"59":2,"66":1,"70":1,"71":1,"73":1,"88":1,"121":1,"129":2,"133":1,"145":1,"150":1,"159":1,"170":2,"179":1,"191":1,"230":1,"254":1,"257":1}}],["world",{"2":{"208":1}}],["worthwhile",{"2":{"77":1}}],["word",{"2":{"70":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["words",{"2":{"66":1}}],["workaround",{"2":{"164":1}}],["worker",{"2":{"154":1}}],["workers",{"2":{"38":3,"39":5}}],["worked",{"2":{"93":1,"136":1}}],["working",{"2":{"33":1,"85":1,"87":2,"89":1,"106":1,"125":1,"146":1,"147":1,"163":1,"248":1}}],["work",{"2":{"2":1,"5":1,"31":1,"71":1,"73":1,"74":1,"88":1,"89":1,"111":1,"113":1,"127":1,"133":1,"143":1,"146":1,"161":1,"166":2,"171":1,"194":1}}],["works",{"2":{"2":1,"33":1,"44":1,"55":1,"58":1,"73":2,"77":1,"89":1,"100":1,"118":1,"129":2,"131":1,"133":1,"134":1,"145":1,"154":1,"164":1,"166":1,"230":1}}],["would",{"2":{"12":2,"13":1,"66":2,"71":3,"75":1,"77":1,"95":1,"106":2,"109":1,"121":1,"194":1}}],["wondered",{"2":{"77":1}}],["won",{"2":{"2":1,"13":1,"59":1,"98":1,"114":1,"179":1,"204":1}}],["waveform",{"2":{"254":5,"255":7,"256":9,"257":4,"259":12}}],["waveforms",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["wall",{"2":{"249":1}}],["wan",{"2":{"133":1}}],["wants",{"2":{"196":1}}],["wanted",{"2":{"77":1}}],["want",{"0":{"62":1,"63":1},"2":{"13":2,"52":2,"61":1,"81":1,"107":1,"114":1,"121":1,"127":1,"134":1,"137":1,"166":1,"172":1,"204":1,"207":1,"243":1}}],["warmup",{"2":{"190":1,"257":1}}],["warde",{"2":{"71":1}}],["warntype",{"2":{"236":3}}],["warn",{"2":{"57":3,"60":1}}],["warning",{"2":{"2":3,"3":1,"4":2,"13":1,"21":1,"27":2,"44":3,"46":2,"47":1,"52":1,"53":1,"57":5,"58":1,"66":1,"72":1,"119":1,"125":1,"133":1,"134":2,"164":1,"202":1,"203":1,"244":4}}],["way",{"2":{"22":2,"44":3,"46":2,"65":1,"77":1,"108":1,"127":2,"139":1,"155":1,"180":2,"249":1}}],["wasteful",{"2":{"129":1}}],["was",{"2":{"2":1,"5":3,"9":1,"12":1,"16":1,"34":1,"44":1,"66":1,"95":3,"97":1,"98":1,"104":1,"106":1,"145":2,"149":2,"175":1,"184":1,"192":1,"208":1,"210":1,"218":1,"226":1,"230":1,"237":1,"246":1,"251":1,"260":1}}],["welcome",{"2":{"133":1}}],["well",{"2":{"46":1,"53":1,"75":1,"85":1,"87":1,"106":1,"134":1,"135":1,"165":1}}],["weird",{"2":{"129":1,"131":2}}],["weightnorm",{"2":{"72":4,"77":1,"129":1}}],["weightih×x+biasih+weighthh×hprev+biashh",{"2":{"69":1}}],["weighting",{"2":{"53":1}}],["weightinitializers",{"0":{"6":1,"97":1},"1":{"7":1,"8":1,"9":1,"10":1},"2":{"3":1,"9":8,"10":24,"129":3,"162":2,"256":1}}],["weight=truncated",{"2":{"256":3}}],["weight=l",{"2":{"129":1}}],["weight=randn32",{"2":{"104":1}}],["weight=rand32",{"2":{"70":1,"104":1}}],["weight=ones32",{"2":{"70":1}}],["weight=glorot",{"2":{"66":1,"129":1}}],["weight=nothing",{"2":{"66":1,"69":3,"70":3}}],["weight=ps",{"2":{"59":1}}],["weight=zero",{"2":{"32":1}}],["weights",{"0":{"162":1},"1":{"163":1},"2":{"53":1,"69":4,"162":10,"163":5,"222":1,"249":2,"250":3}}],["weight",{"2":{"6":1,"9":3,"21":4,"23":4,"31":3,"32":4,"34":4,"66":6,"69":12,"70":24,"72":5,"80":15,"106":12,"110":4,"116":2,"120":4,"121":2,"122":4,"129":6,"131":4,"134":2,"136":4,"137":4,"162":1,"174":1,"216":2,"222":5,"223":2,"236":30,"249":1,"256":3,"258":3}}],["weak",{"2":{"47":1}}],["weren",{"2":{"102":1}}],["were",{"2":{"16":1,"90":1,"97":1,"102":2,"133":1}}],["we",{"0":{"75":1,"197":1},"1":{"76":1,"77":1},"2":{"2":4,"3":3,"12":2,"13":4,"18":2,"21":2,"23":3,"30":2,"37":1,"44":1,"47":1,"52":2,"53":2,"55":5,"57":1,"59":3,"65":3,"66":4,"69":6,"70":4,"77":4,"80":4,"81":2,"85":1,"87":2,"88":3,"89":3,"90":2,"95":1,"97":1,"98":2,"102":1,"103":1,"104":1,"106":13,"107":2,"109":3,"110":6,"111":2,"113":1,"114":3,"118":1,"119":1,"120":1,"122":1,"127":1,"129":5,"130":4,"131":4,"132":1,"133":3,"134":2,"135":3,"136":3,"137":3,"138":3,"141":2,"144":1,"146":3,"147":2,"149":3,"153":1,"154":1,"164":11,"165":1,"166":11,"168":4,"169":9,"170":6,"172":2,"174":3,"176":1,"178":3,"179":11,"180":2,"181":1,"182":1,"183":3,"185":2,"188":2,"191":3,"193":2,"194":2,"197":1,"203":1,"204":4,"207":4,"208":5,"211":1,"215":1,"216":2,"217":2,"222":2,"227":1,"230":3,"233":1,"235":1,"236":2,"238":3,"240":2,"241":2,"242":3,"243":3,"247":3,"248":1,"249":8,"250":7,"254":3,"256":9,"257":1}}],[">randn32",{"2":{"59":2}}],[">",{"2":{"2":3,"3":8,"9":8,"10":24,"13":2,"53":3,"59":2,"69":1,"106":1,"110":2,"115":1,"134":2,"135":2,"136":2,"137":2,"141":1,"181":1,"193":2,"194":4,"213":1,"217":1,"244":2,"254":1,"258":1}}],["cycle",{"2":{"134":2,"244":2}}],["cdev",{"2":{"125":1,"126":1,"205":1,"209":1,"239":1,"244":2}}],["cdims",{"2":{"21":3}}],["circle",{"2":{"255":1,"256":2,"259":4}}],["circumvented",{"2":{"126":1}}],["circular",{"2":{"9":1,"66":1}}],["ci",{"2":{"77":1,"202":1}}],["cite",{"2":{"74":2}}],["citation",{"0":{"74":1}}],["cnew",{"2":{"69":2}}],["cnew=f⋅cprev+i⋅ghnew=o⋅tanh",{"2":{"69":1}}],["cvpr",{"2":{"53":1}}],["cluster",{"2":{"250":1}}],["clockwise",{"2":{"176":1,"178":4}}],["closest",{"2":{"116":1}}],["closures",{"2":{"59":1}}],["cl",{"2":{"130":3,"162":4}}],["classifying",{"2":{"250":1}}],["classify",{"2":{"176":1,"248":1}}],["classifier=st",{"2":{"179":1}}],["classifier",{"0":{"179":1},"2":{"179":11,"180":3,"249":1}}],["classifiers",{"2":{"53":2}}],["classified",{"2":{"53":1}}],["classification",{"0":{"185":1,"227":1},"1":{"186":1,"187":1,"188":1,"189":1,"190":1,"191":1,"192":1,"228":1,"229":1,"230":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"9":2,"53":2,"247":1}}],["classes",{"2":{"53":1}}],["class",{"0":{"88":1},"2":{"53":1,"86":1,"89":1,"189":5,"224":5,"232":5,"247":1}}],["clear",{"2":{"34":1,"110":1}}],["c",{"2":{"32":3,"66":4,"68":21,"69":1,"73":5,"179":2,"187":1,"229":1,"236":9,"250":2,"254":4}}],["cell=st",{"2":{"179":1}}],["cell`",{"2":{"179":1}}],["cells",{"2":{"69":1,"104":1}}],["cell",{"2":{"69":45,"179":10,"180":4}}],["certain",{"0":{"195":1},"1":{"196":1},"2":{"12":1,"13":2,"18":1,"54":1,"67":2,"75":1,"95":1,"102":1,"106":1,"119":1,"143":1,"147":1,"149":1,"161":1,"230":1}}],["central",{"2":{"9":1,"236":13}}],["ca",{"2":{"208":2,"222":3}}],["cairomakie",{"2":{"205":1,"206":1,"209":1,"212":1,"213":1,"217":1,"239":1,"245":2,"247":1,"248":1,"253":1,"255":1,"256":1,"259":2}}],["capabilities",{"2":{"167":1,"238":1}}],["capture",{"2":{"133":1}}],["causing",{"2":{"149":1}}],["causes",{"2":{"147":1}}],["cause",{"2":{"13":1,"18":1,"34":1,"149":1}}],["cat",{"2":{"115":1,"178":1}}],["catch",{"0":{"148":1},"1":{"149":1,"150":1,"151":1,"152":1,"153":1,"154":1},"2":{"52":1,"109":2,"110":2,"116":1,"151":2}}],["care",{"2":{"169":1}}],["cartesian",{"2":{"70":1}}],["carry",{"2":{"69":4,"179":3,"180":3}}],["caching",{"2":{"52":1}}],["cached",{"2":{"52":2}}],["cache",{"2":{"52":2,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["case",{"2":{"31":1,"57":1,"69":19,"129":1,"134":1,"145":1,"174":1,"197":1,"208":1,"222":1,"250":1,"256":1}}],["cases",{"2":{"9":1,"13":3,"30":1,"44":1,"52":2,"55":6,"57":1,"58":1,"59":1,"61":1,"77":1,"88":1,"89":1,"111":2,"135":1,"153":1,"164":1,"168":1,"193":1}}],["calculation",{"2":{"134":1,"135":3}}],["calculations",{"2":{"46":1}}],["calculates",{"2":{"68":3}}],["calculate",{"2":{"66":4,"68":3,"70":1}}],["calculated",{"2":{"12":2,"22":1,"53":7}}],["caller",{"2":{"244":4}}],["called",{"2":{"3":1,"22":2,"24":2,"37":1,"65":2,"67":3,"121":1,"129":1,"163":1,"166":2,"179":1,"194":2,"196":1}}],["callback",{"2":{"208":4,"257":3,"258":1}}],["callable",{"2":{"65":2}}],["calls",{"2":{"12":1,"13":2,"133":2}}],["call",{"2":{"12":1,"13":1,"22":1,"32":1,"48":6,"58":1,"67":5,"72":3,"77":1,"80":1,"133":1,"134":2,"162":4,"164":1,"169":2,"179":1}}],["calling",{"2":{"5":1,"12":1,"13":1,"37":1,"48":1,"107":1,"143":1,"146":1,"249":1}}],["caveats",{"2":{"9":1}}],["candidates",{"2":{"116":1}}],["cannot",{"2":{"9":1,"59":1,"76":1,"129":1,"130":1,"164":1,"194":1,"202":1}}],["can",{"0":{"197":1},"2":{"2":2,"3":4,"4":2,"12":3,"13":3,"16":1,"18":1,"20":1,"21":1,"22":2,"23":1,"24":14,"25":1,"31":1,"33":1,"37":3,"48":1,"49":1,"52":1,"53":2,"55":1,"58":1,"59":5,"60":1,"61":2,"65":6,"66":6,"68":3,"69":7,"70":3,"71":3,"79":2,"80":3,"81":1,"82":1,"103":1,"106":2,"108":2,"109":1,"110":2,"118":1,"120":1,"122":1,"124":1,"126":1,"130":1,"131":3,"133":2,"134":2,"135":3,"137":1,"138":2,"143":1,"149":2,"150":1,"151":1,"154":1,"157":1,"160":1,"161":1,"162":1,"163":1,"164":1,"166":11,"167":2,"169":1,"171":1,"174":1,"179":1,"180":1,"182":1,"183":1,"194":1,"196":1,"197":1,"208":1,"223":1,"230":2,"233":1,"234":1,"240":1,"241":1,"249":2,"250":3,"254":1,"256":1}}],["coordinates",{"2":{"250":1}}],["cos",{"2":{"136":1,"243":1,"254":2,"255":3,"256":8}}],["cover",{"2":{"193":1}}],["covered",{"2":{"111":1}}],["covariate",{"2":{"24":1}}],["could",{"2":{"129":1,"130":1,"249":1}}],["course",{"2":{"118":1,"134":1}}],["courville",{"2":{"71":1}}],["coupled",{"2":{"75":1,"161":1}}],["counterpart",{"2":{"197":1}}],["count",{"2":{"59":1}}],["colormap=",{"2":{"250":3}}],["colorbar",{"2":{"245":1}}],["color=",{"2":{"206":2,"209":4,"213":2,"217":2,"248":2}}],["columns",{"2":{"166":1}}],["column",{"2":{"70":1,"166":4}}],["collects",{"2":{"65":1}}],["collect",{"2":{"59":1,"81":1,"178":2,"187":2,"213":1,"229":2,"243":4,"245":1,"250":4}}],["cols",{"2":{"9":1}}],["coefficient",{"2":{"53":2}}],["corner",{"2":{"73":1}}],["corners",{"2":{"73":1,"104":1}}],["corners=false",{"2":{"73":1}}],["cores",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["core",{"2":{"44":1,"48":3,"77":1,"168":1,"175":1,"184":1,"192":1,"193":1,"210":1,"218":1,"222":5,"223":3,"226":1,"236":2,"237":1,"244":4,"246":1,"251":1,"260":1}}],["correlation=true",{"2":{"66":2,"103":1,"105":1}}],["correlation=false",{"2":{"66":2}}],["correlation",{"2":{"66":7}}],["corresponding",{"2":{"4":3,"32":2,"34":2,"53":3,"70":1,"73":2,"99":1,"102":1,"125":1,"135":1}}],["corresponds",{"2":{"2":1,"208":1}}],["corrections",{"2":{"118":1}}],["correctness",{"0":{"49":1},"2":{"47":1,"55":1}}],["correct",{"0":{"117":1},"2":{"3":2,"49":1,"149":2,"189":3,"224":3,"232":3}}],["correctly",{"2":{"2":1,"33":1,"111":1,"143":1}}],["codebases",{"2":{"58":1}}],["code",{"0":{"143":1},"2":{"30":2,"44":1,"46":1,"55":4,"59":4,"81":1,"87":1,"90":1,"106":1,"124":1,"132":1,"134":2,"143":1,"149":1,"150":1,"151":1,"155":1,"165":1,"180":1,"182":1,"196":1,"197":1,"236":3,"248":1,"252":2}}],["copying",{"2":{"179":1}}],["copyto",{"2":{"55":3}}],["copy",{"2":{"13":1,"52":2,"55":1,"168":5,"194":2}}],["copied",{"2":{"5":1,"69":4}}],["combination",{"2":{"133":1}}],["combined",{"2":{"69":2,"80":1}}],["combines",{"2":{"65":1}}],["come",{"2":{"109":1,"149":1,"172":1}}],["comes",{"2":{"65":1,"75":1,"77":2,"134":1,"136":1}}],["coming",{"2":{"81":1,"129":1,"149":1}}],["com",{"2":{"61":1,"69":1}}],["community",{"2":{"203":1}}],["communication",{"0":{"39":1},"2":{"145":1,"146":1}}],["communications",{"2":{"37":2}}],["commit",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["command",{"2":{"61":2,"155":2}}],["comm",{"2":{"36":2}}],["common",{"2":{"6":1,"34":1,"55":4,"130":1,"154":1,"160":1,"162":1}}],["comprises",{"2":{"130":1}}],["compilable",{"2":{"106":1}}],["compilation",{"2":{"106":1}}],["compiling",{"0":{"106":1},"1":{"107":1},"2":{"77":1}}],["compiled",{"2":{"87":1,"106":7}}],["compiler",{"2":{"75":2,"106":1}}],["compile=true",{"2":{"52":1}}],["compile",{"2":{"3":1,"48":1,"52":1,"65":1,"106":8}}],["completeness",{"2":{"141":1,"233":1}}],["completely",{"2":{"69":1,"75":1,"102":1}}],["complete",{"2":{"68":3,"69":1,"162":1}}],["complexity",{"2":{"84":1,"170":1}}],["complexf64",{"2":{"10":8}}],["complexf32",{"2":{"10":8,"162":2}}],["complexf16",{"2":{"10":8}}],["complex",{"2":{"5":1,"52":1,"59":1,"162":1}}],["compare",{"2":{"141":1,"144":1}}],["comparison",{"2":{"65":1}}],["compactmacroimpl",{"2":{"236":3}}],["compactluxlayer",{"2":{"59":2,"222":2,"236":3}}],["compact",{"0":{"59":1,"180":1},"2":{"58":1,"59":20,"81":3,"103":2,"127":2,"133":2,"180":2,"182":1,"222":1,"230":4,"234":1,"236":7}}],["compatibility",{"2":{"44":1,"89":1,"127":1}}],["compatible",{"2":{"11":1,"21":1,"23":1,"33":2,"40":1,"41":2,"53":2,"127":1,"133":1,"164":2,"233":1}}],["components",{"2":{"254":4}}],["component",{"2":{"254":7}}],["componentvector",{"2":{"174":1,"236":6,"258":1}}],["componentarray",{"2":{"34":1,"134":2,"135":2,"136":2,"137":3,"141":4,"174":1,"208":2,"222":2,"231":1,"233":1,"256":1}}],["componentarrays",{"2":{"34":2,"131":1,"133":1,"170":1,"205":1,"220":1,"228":1,"236":12,"249":1,"253":1}}],["composability",{"2":{"247":1}}],["composes",{"2":{"85":1}}],["composedlinear",{"2":{"130":3}}],["composed",{"2":{"52":1,"69":1}}],["compose",{"2":{"46":1,"249":1}}],["composition",{"2":{"24":1,"130":1}}],["computing",{"0":{"137":1,"139":1,"140":1,"141":1},"2":{"27":2,"28":1,"135":1,"139":1,"171":1}}],["computed",{"2":{"22":1,"24":1,"52":3,"53":3,"65":4,"66":2,"70":1,"71":1,"72":2,"171":4,"216":1}}],["compute",{"2":{"18":2,"23":1,"27":6,"28":1,"52":10,"53":1,"80":3,"95":1,"104":2,"106":1,"110":1,"137":3,"138":5,"141":1,"170":1,"171":1,"172":1,"173":1,"181":1,"190":1,"208":1,"240":1,"242":2,"249":1,"254":4,"255":1,"256":1,"257":1,"259":1}}],["computes",{"2":{"9":1,"19":1,"21":1,"24":3,"28":1,"52":2,"72":4,"138":1,"194":1}}],["computer",{"2":{"5":1,"9":2,"24":1,"53":4}}],["computationally",{"2":{"170":1}}],["computational",{"2":{"170":1}}],["computation",{"0":{"134":1,"136":1},"1":{"135":1},"2":{"3":1,"18":1,"49":1,"53":1,"65":2,"72":2,"95":1,"154":2}}],["concise",{"2":{"127":1,"180":1}}],["concatenate",{"2":{"115":1}}],["concatenated",{"2":{"69":7}}],["conclusion",{"0":{"111":1}}],["concrete",{"0":{"114":1},"1":{"115":1,"116":1,"117":1,"118":1},"2":{"90":1,"131":1,"194":1}}],["conjunction",{"2":{"77":1}}],["connection",{"2":{"65":26,"69":4}}],["connected",{"0":{"23":1},"2":{"9":1,"70":3}}],["confusing",{"2":{"208":1}}],["confusion",{"2":{"95":1}}],["conform",{"2":{"65":1}}],["conference",{"2":{"9":6,"24":2,"53":4}}],["cond",{"2":{"13":3}}],["conditions",{"2":{"191":1,"242":1,"255":1,"256":1}}],["condition",{"2":{"9":1,"13":2}}],["convtranspose",{"2":{"66":1,"104":1,"105":2}}],["convdims",{"2":{"21":2}}],["conv",{"2":{"21":3,"46":2,"66":4,"103":1,"104":1,"152":2,"188":6,"193":1}}],["convention",{"2":{"168":1}}],["conveniently",{"2":{"166":1}}],["convenience",{"0":{"10":1},"2":{"12":1,"58":1,"85":1,"217":1}}],["conversely",{"2":{"170":1}}],["converse",{"2":{"66":1}}],["conversions",{"2":{"57":2}}],["conversion",{"0":{"159":1},"2":{"5":1}}],["converts",{"2":{"56":3,"73":1}}],["converted",{"2":{"46":2,"57":1}}],["converting",{"2":{"34":1,"46":3}}],["convert",{"2":{"5":1,"13":1,"44":2,"46":4,"57":3,"164":1,"188":1,"194":1,"197":1,"208":1,"209":1,"230":1}}],["convolutions",{"2":{"66":5}}],["convolution",{"2":{"9":1,"66":9}}],["convolutional",{"0":{"21":1,"66":1},"2":{"9":2,"53":1,"66":4}}],["continua",{"2":{"254":1}}],["continue",{"2":{"146":1}}],["contour",{"2":{"245":1,"250":6}}],["context",{"2":{"22":1,"24":2,"54":1}}],["context`",{"2":{"22":1}}],["contained",{"2":{"65":1}}],["containerlayer",{"2":{"230":1}}],["containers",{"0":{"65":1},"2":{"95":2,"236":3}}],["container",{"0":{"130":1},"2":{"12":2,"13":1,"33":1,"41":1,"96":1,"130":2,"143":1,"179":2}}],["contains",{"0":{"136":1},"2":{"13":1,"28":1,"54":2,"67":1,"71":1}}],["contain",{"2":{"12":1,"57":1,"59":1,"77":1,"129":2,"179":1}}],["containing",{"0":{"134":1},"1":{"135":1},"2":{"3":1,"9":4,"10":24,"12":3,"24":2,"52":2,"53":2,"69":14,"70":1,"76":1}}],["contrib",{"2":{"109":1}}],["control",{"2":{"73":2,"106":2,"149":1,"165":1}}],["controlled",{"2":{"57":1,"76":1,"105":1}}],["controlling",{"2":{"57":1,"159":1,"160":1}}],["controls",{"2":{"9":1,"53":1,"66":4,"69":3,"72":8}}],["contrastive",{"2":{"53":1}}],["contrast",{"2":{"2":1}}],["consensus",{"2":{"136":1}}],["consecutive",{"2":{"65":1}}],["consequence",{"2":{"32":1}}],["consult",{"2":{"29":1}}],["considering",{"2":{"170":1}}],["consider",{"2":{"114":1,"129":1,"131":1,"149":1,"171":1,"174":1}}],["considered",{"2":{"3":1,"4":2,"49":1,"52":1,"119":1,"124":1}}],["consistent",{"2":{"69":1}}],["consistency",{"2":{"9":1}}],["consists",{"2":{"65":1,"135":1}}],["constrained",{"2":{"132":1}}],["constructured",{"2":{"194":1}}],["constructing",{"2":{"52":1,"143":1}}],["construction",{"2":{"12":1,"73":1,"102":1}}],["constructor",{"2":{"31":1}}],["constructed",{"2":{"22":1,"59":1,"129":1}}],["construct",{"2":{"9":1,"31":1,"33":2,"59":2,"76":1,"80":2,"107":1,"108":1,"130":2,"208":1,"231":1,"249":2}}],["constructs",{"2":{"9":2,"12":1,"31":1}}],["const",{"2":{"62":4,"63":3,"106":5,"110":1,"205":2,"206":2,"216":2,"236":2,"239":2,"254":6,"255":1,"256":2}}],["constants",{"2":{"22":1,"255":1,"256":1}}],["constant",{"2":{"3":1}}],["cupti",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["cusparse",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["cusolver",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["customparamtype",{"2":{"31":2}}],["customabstractluxlayer",{"2":{"12":4}}],["customize",{"2":{"3":1}}],["custom",{"0":{"81":1,"112":1,"194":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1},"2":{"3":1,"12":2,"13":1,"16":1,"25":1,"31":1,"59":1,"77":1,"81":1,"110":2,"117":1,"127":1,"131":1,"133":1,"157":1,"168":1,"176":1,"179":1,"180":1,"230":1,"238":1,"242":1}}],["cufft",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["curand",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["currently",{"2":{"4":4,"23":2,"25":1,"33":1,"68":3,"73":1,"88":2,"89":1,"106":1,"107":1,"124":1,"133":1,"147":1,"164":2,"172":1}}],["current",{"2":{"2":1,"5":1,"12":1,"31":2,"52":1,"67":1,"85":1,"114":1,"168":1,"249":1,"250":1}}],["cublas",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["cublaslt",{"2":{"23":1}}],["cu",{"2":{"167":4}}],["cuarray",{"2":{"5":3,"7":3,"80":18,"125":1,"162":2,"164":2,"167":1,"236":18}}],["cuiterator",{"2":{"5":2,"100":1,"154":1}}],["cudevice",{"2":{"4":1}}],["cudnn",{"2":{"2":2,"3":1,"21":1,"62":1}}],["cudadevice",{"2":{"2":2,"4":2,"5":1,"81":1,"99":1,"126":2,"146":1,"175":1,"184":1,"192":1,"207":2,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["cuda",{"0":{"167":1},"2":{"2":6,"3":2,"4":1,"5":5,"7":1,"21":1,"23":1,"37":7,"62":1,"80":19,"81":1,"83":1,"93":2,"100":1,"124":1,"125":1,"145":1,"146":1,"147":1,"154":1,"157":3,"158":1,"162":5,"164":2,"167":7,"175":10,"184":10,"192":3,"210":10,"218":10,"220":1,"226":10,"228":1,"236":36,"237":10,"239":1,"246":10,"251":3,"260":3}}],["ch",{"2":{"249":2,"250":1}}],["cheat",{"2":{"136":1}}],["checking",{"2":{"129":1,"236":1}}],["check=",{"2":{"110":2}}],["check",{"2":{"13":4,"33":11,"37":1,"49":1,"59":1,"67":3,"72":3,"106":1,"110":3,"141":1,"151":2,"160":3}}],["checked",{"2":{"3":1}}],["checks",{"2":{"3":2,"31":2,"33":1,"59":2}}],["chs",{"2":{"66":17,"72":19}}],["chunks",{"2":{"54":1}}],["choice",{"2":{"183":1}}],["chosen",{"2":{"163":1}}],["chopra",{"2":{"53":1}}],["choose",{"2":{"2":1}}],["christian",{"2":{"24":1}}],["chains",{"0":{"46":1},"2":{"46":4,"188":1,"191":1,"249":2}}],["chain=chain",{"2":{"32":1}}],["chain",{"0":{"115":1,"123":1},"2":{"32":10,"34":3,"44":1,"46":2,"59":1,"65":7,"69":2,"71":4,"72":6,"80":9,"102":1,"106":2,"107":1,"109":8,"110":6,"114":1,"115":2,"118":3,"120":2,"121":1,"123":2,"134":1,"135":1,"136":1,"137":1,"141":1,"164":1,"188":6,"193":2,"208":1,"214":2,"217":1,"223":2,"231":2,"236":18,"241":1,"249":5,"250":2,"256":2}}],["chainrulescore",{"2":{"110":2}}],["chainrules",{"2":{"21":1,"23":1,"33":2,"53":1,"86":1,"89":1,"133":1}}],["channel",{"2":{"68":3,"72":12,"73":3}}],["channels",{"2":{"24":1,"66":6,"67":2,"72":1,"73":1}}],["changed",{"2":{"92":1,"144":1,"196":1}}],["changes",{"0":{"92":1,"95":1,"99":1,"102":1,"103":1,"104":2},"2":{"12":1,"16":1,"60":1,"65":9,"71":2,"75":1,"90":1,"95":1,"97":1,"109":1,"133":1,"144":1,"193":1}}],["change",{"2":{"1":1,"2":1,"58":1,"95":1,"98":1,"129":1,"143":1,"233":1,"254":1}}],["crude",{"2":{"254":1}}],["crc",{"2":{"110":2}}],["critical",{"2":{"88":1,"124":1}}],["criteria",{"2":{"2":1,"33":1}}],["creating",{"0":{"179":1},"2":{"59":1,"230":1}}],["created",{"2":{"59":1}}],["create",{"0":{"223":1,"231":1},"2":{"2":1,"5":1,"36":2,"44":1,"52":1,"55":2,"59":1,"65":2,"70":3,"106":2,"107":1,"110":1,"117":1,"166":1,"169":1,"176":1,"178":3,"179":2,"182":1,"217":1,"223":2,"225":1,"231":2,"233":1,"236":3,"241":4,"249":2}}],["creates",{"2":{"1":1,"9":1,"13":1,"59":1,"69":5,"169":1,"249":1}}],["crosscor",{"2":{"66":1,"103":1}}],["crossentropyloss",{"2":{"53":6,"189":1,"224":1,"232":1}}],["cross",{"2":{"53":3,"66":11,"103":1,"105":1,"127":1}}],["cpu=true",{"2":{"233":2}}],["cpus",{"2":{"18":1,"19":2,"46":1,"149":1,"153":1}}],["cpudevice",{"2":{"2":4,"4":2,"125":3,"126":1,"205":1,"239":1}}],["cpu",{"2":{"2":4,"5":1,"23":1,"25":1,"49":1,"63":2,"68":3,"77":1,"86":1,"87":3,"89":1,"103":2,"106":1,"124":1,"125":6,"126":7,"161":1,"164":3,"175":2,"182":1,"183":1,"184":2,"191":1,"192":2,"194":1,"205":1,"210":2,"216":2,"217":1,"218":2,"226":2,"233":4,"237":2,"239":1,"246":2,"251":2,"260":2}}],["rk4",{"2":{"255":1,"256":1,"257":1,"259":1}}],["r₂",{"2":{"254":2}}],["r₁",{"2":{"254":2}}],["r2",{"2":{"254":1}}],["r1",{"2":{"254":1}}],["r=r1−r2",{"2":{"254":1}}],["r=σ",{"2":{"69":1}}],["rhat",{"2":{"249":1}}],["risk",{"2":{"147":1}}],["right",{"2":{"71":1}}],["rrule",{"2":{"110":1}}],["rrules",{"2":{"89":1}}],["rᴰ",{"2":{"73":1}}],["r²",{"2":{"73":1}}],["rgb",{"2":{"66":1}}],["r",{"2":{"59":9,"73":9,"254":3}}],["rnns",{"2":{"69":1}}],["rnncell",{"2":{"69":9,"104":1,"105":1}}],["rnn",{"2":{"54":1,"69":3}}],["rng=xoshiro",{"2":{"190":1}}],["rngs",{"2":{"76":1}}],["rng",{"0":{"7":1},"2":{"3":3,"7":10,"9":11,"10":24,"12":2,"13":6,"14":1,"15":1,"16":2,"22":6,"32":3,"44":1,"46":1,"59":5,"65":4,"67":6,"69":3,"70":4,"71":18,"80":7,"81":3,"97":2,"102":1,"106":1,"107":1,"109":3,"110":2,"116":2,"118":1,"120":4,"123":6,"129":9,"130":2,"131":5,"149":4,"162":13,"163":6,"165":2,"166":1,"169":5,"171":1,"172":1,"174":8,"182":2,"190":2,"193":7,"194":9,"207":1,"208":1,"213":4,"216":1,"222":3,"225":2,"231":4,"244":4,"248":11,"249":1,"256":1}}],["ryan",{"2":{"24":1}}],["row",{"2":{"166":3,"250":1}}],["rows",{"2":{"9":1,"166":1}}],["routines",{"2":{"133":1}}],["route",{"2":{"130":1}}],["rough",{"2":{"133":1}}],["rounded",{"2":{"66":2,"68":3}}],["round",{"2":{"37":1,"225":4,"248":1}}],["root",{"2":{"39":7,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["robin",{"2":{"37":1}}],["rocarray",{"2":{"7":4}}],["rocrand",{"2":{"7":1}}],["rocm",{"2":{"3":1,"62":2,"146":1,"147":1,"157":3,"158":1}}],["raw",{"2":{"187":2,"229":2}}],["rademacher",{"2":{"141":1}}],["ra",{"2":{"106":18,"107":9}}],["ratio≥0",{"2":{"254":1}}],["ratio≤1",{"2":{"254":1}}],["ratio",{"2":{"254":6,"255":2,"256":1,"257":1,"259":1}}],["rationale",{"2":{"95":1}}],["rate",{"2":{"174":1,"249":1}}],["rather",{"2":{"3":1,"21":1,"69":1,"76":1,"179":1}}],["raia",{"2":{"53":1}}],["ran",{"2":{"109":1}}],["randc64",{"2":{"10":1}}],["randc32",{"2":{"10":1}}],["randc16",{"2":{"10":1}}],["rand64",{"2":{"10":1}}],["rand32",{"2":{"10":1}}],["rand16",{"2":{"10":1}}],["randnc64",{"2":{"10":1}}],["randnc32",{"2":{"10":1}}],["randnc16",{"2":{"10":1}}],["randn64",{"2":{"10":1}}],["randn32",{"2":{"10":1}}],["randn16",{"2":{"10":1}}],["randn",{"2":{"9":1,"44":1,"46":1,"59":1,"65":2,"71":2,"106":1,"107":1,"109":1,"116":1,"120":1,"123":1,"125":1,"126":1,"129":1,"130":1,"131":1,"134":2,"135":2,"166":2,"169":1,"171":1,"172":1,"174":4,"190":1,"193":2,"194":6,"213":1}}],["randomness",{"0":{"169":1},"2":{"69":3,"76":1,"80":1,"138":1,"165":1,"169":1}}],["randomly",{"2":{"67":2}}],["random",{"2":{"7":1,"9":8,"10":12,"22":4,"32":2,"44":2,"46":2,"59":3,"65":2,"67":3,"71":8,"80":4,"81":3,"106":3,"107":1,"109":1,"115":1,"120":2,"123":4,"129":4,"131":3,"133":1,"138":1,"149":2,"162":3,"164":1,"165":5,"166":1,"169":8,"174":4,"177":1,"186":1,"193":4,"194":4,"205":1,"207":1,"208":1,"212":1,"213":2,"220":1,"222":1,"228":1,"231":2,"239":1,"243":1,"244":2,"247":1,"248":2,"253":1,"256":1}}],["rand",{"2":{"5":1,"59":1,"80":3,"81":1,"136":1,"137":1,"141":2,"149":2,"164":1,"166":5,"167":1,"169":3,"248":8}}],["rank",{"2":{"4":3,"38":3,"143":1,"144":1}}],["ranges",{"2":{"65":1}}],["range",{"2":{"3":2,"97":1,"172":1,"206":1,"213":1,"243":4,"250":23,"255":1}}],["rule",{"2":{"73":1,"217":2}}],["rules",{"0":{"117":1},"2":{"0":1,"55":1,"88":1}}],["runtime",{"2":{"49":2,"175":2,"184":2,"210":2,"218":2,"226":2,"237":2,"246":2}}],["running",{"0":{"116":1},"2":{"22":2,"24":18,"72":16,"95":1,"109":5,"110":11,"120":2,"123":2,"150":1,"179":1,"196":1}}],["run",{"2":{"2":1,"3":1,"5":1,"48":1,"61":2,"62":1,"63":1,"67":1,"80":1,"102":1,"106":3,"110":3,"117":1,"129":1,"147":1,"149":1,"155":3,"164":3,"169":1,"179":1,"194":1,"202":1,"250":1}}],["red",{"2":{"206":1,"209":2,"248":1}}],["reduction",{"2":{"53":1,"68":3}}],["reduce",{"2":{"39":3,"47":1,"71":1,"249":1}}],["reducing",{"2":{"24":1,"65":1}}],["reiterate",{"2":{"194":1}}],["refresher",{"2":{"129":1}}],["referred",{"2":{"166":1}}],["refer",{"2":{"81":1,"203":1,"204":1,"238":1}}],["references",{"2":{"9":6,"22":2,"24":4,"53":4,"71":1,"72":1}}],["reference",{"0":{"8":1},"1":{"9":1,"10":1},"2":{"185":1}}],["reexport",{"2":{"102":2}}],["renamed",{"2":{"95":2,"98":1,"99":2}}],["renormalize",{"2":{"72":2}}],["request",{"2":{"84":1,"95":1}}],["requisites",{"2":{"80":1}}],["requirements",{"2":{"65":1,"217":1}}],["require",{"2":{"24":1,"44":1,"46":1,"59":1,"146":1,"208":1}}],["requires",{"2":{"13":1,"15":1,"37":4,"54":1,"83":1,"113":1,"208":1,"222":1}}],["rev=true",{"2":{"244":4}}],["revising",{"2":{"77":1}}],["reverses",{"2":{"71":1}}],["reversesequence",{"2":{"71":3}}],["reversed",{"2":{"71":1}}],["reversediffadjoint",{"2":{"230":1}}],["reversediff",{"2":{"13":2,"21":1,"23":1,"25":1,"49":1,"52":1,"55":1,"86":1,"87":1,"116":1,"233":1}}],["reverse",{"2":{"27":2,"33":2,"49":1,"59":1,"71":2,"86":6,"106":1,"110":1,"170":2,"171":2}}],["retcode",{"2":{"258":1}}],["retrieve",{"2":{"70":1}}],["retained",{"2":{"67":1}}],["retuened",{"2":{"3":2}}],["returning",{"2":{"13":1,"129":1}}],["returned",{"2":{"7":1,"9":1,"22":1,"24":1,"34":1,"41":1,"44":1,"46":1,"52":6,"54":1,"67":4,"69":3,"70":1,"71":1}}],["returns",{"2":{"3":3,"9":6,"12":2,"13":3,"18":2,"22":2,"24":4,"25":1,"27":2,"28":1,"34":1,"50":1,"52":2,"53":6,"54":2,"57":1,"65":7,"66":2,"67":3,"68":9,"69":8,"70":5,"71":6,"72":5,"73":2,"100":1,"125":3,"135":1,"136":1,"137":1,"208":1,"236":6,"256":1}}],["return",{"2":{"2":3,"3":5,"9":6,"10":24,"12":1,"14":1,"15":1,"16":2,"31":1,"32":2,"52":4,"53":9,"54":2,"55":2,"59":19,"65":3,"69":4,"71":1,"76":1,"81":3,"95":1,"106":2,"107":1,"110":2,"115":1,"117":2,"120":1,"121":3,"122":4,"129":5,"130":2,"131":3,"134":1,"135":1,"136":1,"137":1,"139":1,"140":1,"141":1,"163":1,"174":1,"178":1,"179":2,"180":2,"181":1,"182":1,"187":1,"189":1,"190":1,"194":1,"206":1,"208":3,"213":1,"216":1,"217":1,"221":2,"222":3,"223":1,"224":1,"225":1,"229":1,"230":5,"231":1,"232":1,"234":2,"236":3,"241":2,"242":3,"244":1,"245":1,"248":1,"249":2,"250":2,"254":11,"255":1,"256":1,"257":2}}],["remove",{"2":{"169":1}}],["removed",{"0":{"102":1,"145":1},"2":{"66":1,"92":1,"95":2,"99":1,"102":10,"103":4,"106":1,"124":1,"134":2,"135":1,"145":1}}],["remake",{"2":{"208":1}}],["remark",{"2":{"135":1}}],["remains",{"2":{"22":1}}],["remember",{"2":{"107":1,"108":1,"111":1,"129":1,"133":1,"164":1,"179":1,"233":1}}],["regions",{"2":{"250":1}}],["registry",{"2":{"61":1,"82":1}}],["registered",{"2":{"59":1,"61":1}}],["regarding",{"2":{"170":1,"194":1}}],["regression",{"0":{"174":1},"2":{"149":1,"174":1}}],["regressions",{"2":{"13":1,"124":1,"191":1}}],["reg",{"2":{"134":2,"135":2}}],["regularization",{"2":{"134":1,"141":1,"249":1}}],["regular",{"2":{"46":2,"59":2,"164":1}}],["reuse",{"2":{"21":1,"23":1}}],["reusing",{"2":{"21":1,"23":1}}],["reltol",{"2":{"236":13}}],["reltol=1",{"2":{"231":1}}],["relies",{"2":{"155":1,"169":1,"194":2}}],["reliance",{"2":{"134":2}}],["reliable",{"2":{"87":2}}],["reliability",{"2":{"77":1}}],["relevant",{"2":{"106":1,"247":1,"254":1}}],["release",{"2":{"90":2,"97":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["released",{"2":{"61":1}}],["relativisticorbitmodel",{"2":{"255":3}}],["relatively",{"2":{"133":1,"164":1}}],["relation",{"2":{"69":1}}],["related",{"2":{"84":1}}],["relu",{"2":{"21":1,"44":1,"46":4,"59":3,"65":4,"69":1,"71":3,"72":8,"81":1,"109":5,"110":5,"164":1,"188":12,"193":2,"214":2,"217":2,"223":2}}],["rely",{"2":{"20":1,"59":1,"95":1,"169":1}}],["rewrite",{"2":{"18":1,"44":1,"240":1}}],["readme",{"2":{"106":1}}],["read",{"2":{"84":1,"168":1}}],["reason",{"2":{"87":1,"149":1}}],["reasonable",{"2":{"65":1}}],["reasons",{"2":{"13":1,"230":1}}],["reactant",{"0":{"106":1},"1":{"107":1},"2":{"63":7,"77":1,"106":9,"107":1}}],["really",{"2":{"46":1,"59":1,"89":1,"129":1,"166":1}}],["reallocations",{"2":{"21":1,"23":1}}],["real=eps",{"2":{"24":1}}],["real",{"2":{"9":1,"53":4,"67":1,"117":1,"245":2}}],["reproducer",{"2":{"147":1}}],["represents",{"2":{"249":1}}],["representing",{"2":{"73":1}}],["representation",{"2":{"59":1}}],["represent",{"2":{"9":1}}],["repository",{"2":{"133":1}}],["report",{"2":{"30":1,"84":1,"89":1}}],["reparameterized",{"2":{"72":2}}],["reparameterization",{"2":{"72":1}}],["repack",{"2":{"13":1}}],["repeat",{"2":{"178":2}}],["repeatedly",{"2":{"65":1}}],["repeatedlayer",{"2":{"65":1}}],["repeats",{"2":{"65":6,"69":5}}],["replacing",{"2":{"166":1,"193":1}}],["replacement",{"2":{"95":1}}],["replaces",{"2":{"33":1,"68":3}}],["replaced",{"2":{"18":1}}],["replace",{"2":{"18":2,"118":1,"152":5}}],["repl",{"2":{"48":1,"61":1,"79":1,"150":2,"155":1}}],["replicated",{"2":{"169":1}}],["replicate",{"2":{"12":1,"13":1,"65":1,"169":3}}],["receives",{"2":{"71":1}}],["reconstruction",{"2":{"129":1}}],["recognition",{"2":{"53":1}}],["record",{"2":{"50":1,"245":1,"250":1}}],["recorded",{"2":{"49":1,"50":4}}],["recomputing",{"2":{"22":1}}],["recommendations",{"0":{"87":1},"2":{"148":1,"149":1}}],["recommendation",{"2":{"77":1}}],["recommend",{"2":{"12":1,"77":1,"127":1,"129":1,"146":1,"168":1,"172":1,"180":1,"183":1,"222":1,"227":1,"230":1,"236":1}}],["recommended",{"0":{"125":1},"2":{"3":1,"12":2,"13":1,"20":1,"31":1,"44":1,"47":1,"55":1,"59":2,"77":1,"127":1,"129":2,"133":1,"134":2,"139":1,"141":1,"150":1,"158":1,"164":1,"181":1,"194":1,"256":1}}],["recvbuf",{"2":{"39":6}}],["recur",{"2":{"69":3}}],["recurrence",{"2":{"69":4,"179":1}}],["recurrent",{"0":{"69":1},"2":{"9":1,"69":10,"104":1,"176":2}}],["recurse",{"2":{"55":3}}],["recurses",{"2":{"33":1,"56":3}}],["recursion",{"2":{"55":1}}],["recursive",{"0":{"55":1},"2":{"55":9,"149":1,"164":2}}],["recursively",{"2":{"15":1,"55":5}}],["rectifiers",{"2":{"9":2}}],["resolve",{"2":{"137":1}}],["resolution",{"2":{"73":1}}],["resources",{"0":{"84":1}}],["rescale",{"2":{"72":4}}],["resnet",{"2":{"65":1}}],["res",{"2":{"59":2,"65":4,"208":5,"258":1,"259":1}}],["respective",{"2":{"75":1,"143":2,"168":1,"194":1}}],["respect",{"2":{"28":2,"49":1,"135":2}}],["responsibility",{"2":{"13":1,"131":1}}],["reserved",{"2":{"59":1}}],["research",{"2":{"22":1}}],["resets",{"2":{"3":1}}],["reset",{"2":{"3":1}}],["reshapes",{"2":{"71":1}}],["reshapelayer",{"2":{"71":3}}],["reshape",{"2":{"53":4,"81":1,"139":2,"140":2,"168":2,"178":2,"187":1,"213":1,"221":2,"229":1,"230":4,"243":2,"245":2,"249":1}}],["reshaped",{"2":{"21":1,"71":1}}],["reshaping",{"2":{"9":1}}],["results",{"0":{"209":1,"245":1,"259":1},"2":{"106":1,"141":2,"217":1,"250":2,"255":1,"256":1,"258":1,"259":1}}],["result",{"2":{"9":1,"20":1,"28":2,"39":4,"50":4,"72":1,"76":1,"131":4}}],["restricted",{"2":{"55":1}}],["rest",{"2":{"9":1,"179":2,"180":2}}],["restarted",{"2":{"1":1,"60":1}}],["mtensor",{"2":{"254":4}}],["mtlarray",{"2":{"7":3}}],["mcse",{"2":{"249":1}}],["mcmcchains",{"2":{"249":1}}],["mcmc",{"2":{"249":1,"250":1}}],["mcclelland",{"2":{"9":1}}],["mvnormal",{"2":{"249":1}}],["md",{"2":{"244":8}}],["mnist\\ttraining",{"2":{"225":1}}],["mnist\\ttime",{"2":{"225":50}}],["mnist",{"0":{"185":1,"187":1,"219":1,"227":1,"229":1},"1":{"186":1,"187":1,"188":1,"189":1,"190":1,"191":1,"192":1,"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1,"228":1,"229":1,"230":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"186":1,"187":2,"221":1,"225":2,"228":1,"229":2}}],["my",{"2":{"191":1}}],["mybias",{"2":{"131":3}}],["myinputtype",{"2":{"113":3}}],["myfancychain",{"2":{"65":2}}],["myweight",{"2":{"31":1,"131":4}}],["mse",{"2":{"242":3}}],["mseloss",{"2":{"53":4,"80":2,"81":1,"106":1,"107":1,"174":1,"208":1,"216":1,"242":1,"257":3}}],["msleloss",{"2":{"53":2}}],["m2",{"2":{"44":3}}],["mpi",{"0":{"157":1},"2":{"36":2,"37":8,"146":2,"147":1,"157":5}}],["mpibackend",{"2":{"36":2,"37":2}}],["mₘ",{"2":{"28":2}}],["m₂",{"2":{"28":2,"254":7}}],["m₁",{"2":{"28":2,"254":6}}],["m",{"2":{"24":2,"44":3,"56":6,"62":2,"65":2,"230":2,"234":2,"248":15,"254":4,"255":5,"256":3}}],["mkl",{"2":{"23":1}}],["mldatasets",{"2":{"186":1,"220":1,"228":1}}],["mldatadevices",{"0":{"0":1,"98":1},"1":{"1":1,"2":1,"3":1,"4":1,"5":1,"99":1,"100":1},"2":{"0":1,"1":1,"2":3,"3":12,"4":1,"5":2,"98":2,"126":2,"158":2,"175":3,"184":3,"192":3,"205":1,"207":3,"210":3,"218":3,"226":3,"237":3,"239":1,"246":3,"251":3,"260":3}}],["ml",{"2":{"166":1,"208":2}}],["mlir",{"2":{"106":1}}],["mlp",{"0":{"211":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1},"2":{"59":1,"81":1,"106":1,"211":1,"223":1,"241":4}}],["mlutils",{"0":{"41":1},"2":{"5":3,"41":4,"100":1,"143":1,"154":2,"177":1,"178":3,"186":1,"205":1,"207":1,"220":1,"228":1,"239":1}}],["motion",{"2":{"254":1,"255":1,"256":1}}],["motivating",{"2":{"13":1,"58":1}}],["moment",{"2":{"136":1,"254":1}}],["momentum=0",{"2":{"72":2}}],["momentum",{"2":{"24":6,"72":2}}],["mooncake",{"2":{"86":1}}],["monolithic",{"2":{"77":1}}],["month",{"2":{"74":1}}],["mouthful",{"2":{"34":1}}],["moved",{"0":{"103":1},"2":{"95":1,"103":2,"106":1}}],["move",{"2":{"30":1,"106":1,"107":2,"167":1,"168":1,"207":3}}],["movement",{"2":{"3":2}}],["modivations",{"2":{"102":1}}],["modify",{"2":{"72":1}}],["modified",{"2":{"55":2,"115":1}}],["modulating",{"2":{"53":1}}],["modules=",{"2":{"48":2}}],["modules",{"2":{"48":8}}],["module",{"0":{"54":1},"2":{"30":1,"35":1,"48":1,"54":1,"102":1,"103":1}}],["modes",{"2":{"25":1,"60":1,"73":1}}],["mode",{"0":{"196":1},"2":{"22":2,"24":2,"25":5,"27":2,"33":7,"49":1,"60":2,"67":6,"69":2,"72":3,"73":5,"86":1,"87":1,"103":1,"108":3,"109":2,"110":2,"134":2,"149":1,"150":1,"155":1,"170":1,"171":4,"172":2,"196":4,"254":1}}],["model`",{"2":{"230":1}}],["model",{"0":{"109":1,"119":1,"182":1,"183":1,"188":1,"191":1,"208":1,"252":1,"255":1,"256":1},"1":{"120":1,"121":1,"122":1,"123":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"13":8,"32":1,"33":2,"34":2,"44":6,"46":11,"52":5,"53":26,"58":3,"59":14,"65":16,"69":3,"71":12,"72":4,"75":1,"80":6,"81":10,"106":18,"107":8,"108":3,"109":17,"110":25,"111":1,"114":1,"115":1,"116":2,"117":2,"118":2,"119":1,"120":6,"123":7,"129":1,"130":5,"134":10,"135":7,"136":7,"137":7,"139":2,"140":2,"141":9,"143":2,"149":6,"164":4,"174":11,"179":2,"180":1,"181":2,"182":10,"183":6,"185":1,"188":4,"189":2,"190":7,"191":4,"193":9,"194":11,"196":1,"197":1,"202":1,"208":9,"209":2,"216":2,"217":3,"223":4,"224":2,"225":9,"230":11,"231":8,"232":2,"233":10,"234":8,"236":17,"242":4,"243":1,"244":7,"249":7,"250":1,"254":7,"255":4,"256":11,"257":1,"258":1,"259":2}}],["models",{"0":{"44":2,"46":1,"106":1,"108":1,"118":1,"202":1,"204":1},"1":{"107":1,"109":1,"110":1,"111":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"6":1,"13":1,"44":2,"46":2,"47":1,"52":2,"59":2,"77":6,"80":1,"106":1,"108":2,"111":1,"114":1,"162":1,"164":1,"166":1,"172":1,"176":1,"180":1,"202":2,"204":1}}],["mostly",{"2":{"19":1,"77":1,"90":1,"102":1,"147":1,"230":1}}],["most",{"2":{"9":1,"12":1,"13":1,"25":1,"33":1,"52":2,"55":1,"61":1,"77":1,"85":1,"87":2,"98":1,"107":1,"127":1,"137":1,"139":1,"164":1,"193":1,"203":1,"256":1}}],["more",{"2":{"5":1,"9":2,"12":1,"13":1,"16":1,"19":1,"44":1,"46":2,"49":1,"54":1,"55":1,"58":1,"59":3,"65":1,"73":1,"77":1,"84":1,"92":1,"95":1,"97":1,"100":1,"102":1,"104":2,"134":1,"135":1,"146":1,"149":1,"151":1,"156":1,"159":1,"160":1,"166":3,"170":1,"179":1,"180":1,"181":1,"204":1,"250":1}}],["mechanics",{"2":{"254":1}}],["mechanism",{"2":{"77":1}}],["message",{"2":{"133":1}}],["mersennetwister",{"2":{"116":1,"162":2,"213":1}}],["merged",{"2":{"69":1}}],["merge",{"2":{"31":1,"32":1,"69":2,"115":1,"179":1}}],["mentioned",{"2":{"82":1,"129":1}}],["measure",{"2":{"53":1}}],["meanpool",{"2":{"68":1}}],["meant",{"2":{"33":1,"52":1,"58":1,"163":1}}],["meaningless",{"2":{"22":1}}],["mean",{"2":{"9":2,"22":2,"24":13,"53":17,"68":7,"72":13,"109":1,"110":2,"120":1,"123":1,"131":1,"174":1,"242":3,"244":12,"249":1,"250":1}}],["means",{"2":{"9":1,"30":1,"59":1,"118":1,"129":1,"131":1,"149":1,"166":1,"168":1,"208":2}}],["medical",{"2":{"53":1}}],["medium",{"2":{"23":1}}],["met",{"2":{"9":1,"33":1}}],["methodinstance",{"2":{"236":3}}],["methoderror",{"2":{"116":1}}],["methods",{"2":{"73":1,"81":1,"125":1,"138":1,"163":2,"164":1,"204":2,"205":1,"208":1,"221":1,"230":1,"231":1,"239":1,"249":1,"254":2}}],["method",{"2":{"9":3,"106":1,"110":1,"113":1,"116":1,"139":1,"140":1,"141":1,"178":1,"180":1,"181":1,"187":1,"189":1,"190":1,"213":1,"222":1,"223":1,"224":1,"229":1,"230":1,"232":1,"242":2,"249":1,"250":2,"254":3,"256":1,"257":2}}],["metaldevice",{"2":{"4":2}}],["metal",{"2":{"2":1,"3":2,"7":1,"62":2,"80":1,"81":1,"83":2,"124":4,"158":1}}],["memory=false",{"2":{"69":1}}],["memory=zeros32",{"2":{"69":1}}],["memory",{"2":{"5":1,"21":1,"23":1,"69":20,"175":2,"179":1,"184":2,"192":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"260":1}}],["mutability",{"0":{"168":1}}],["mutables",{"2":{"194":1}}],["mutable",{"2":{"129":1,"194":1}}],["mutations",{"2":{"75":1,"77":2,"87":1}}],["mutation",{"2":{"21":1,"23":1,"168":2}}],["mutating",{"2":{"21":1,"23":1}}],["mutate",{"2":{"168":1}}],["mutated",{"2":{"20":1,"55":1,"168":2,"169":1}}],["mutates",{"2":{"13":1}}],["much",{"2":{"53":2,"77":1,"144":1,"250":1}}],["mul",{"2":{"19":2,"152":1}}],["multilayer",{"2":{"211":1}}],["multigate",{"2":{"54":1}}],["multiply",{"2":{"166":1}}],["multiplied",{"2":{"22":1}}],["multiplication",{"2":{"19":3}}],["multiple",{"0":{"135":1},"2":{"21":1,"23":1,"65":1,"69":1,"72":1,"124":1,"164":2,"250":1}}],["multi",{"0":{"4":1},"2":{"53":1,"70":1}}],["must",{"2":{"1":1,"3":2,"4":1,"9":4,"12":1,"24":1,"28":2,"32":2,"34":2,"37":1,"41":2,"44":1,"46":2,"52":3,"60":1,"65":3,"66":2,"67":3,"69":7,"70":4,"72":1,"73":1,"76":4,"77":1,"129":1,"132":1,"143":1,"216":1,"254":4}}],["major",{"0":{"93":1,"96":1,"100":1},"2":{"90":1,"97":1}}],["masses",{"2":{"254":1}}],["mass2",{"2":{"254":5}}],["mass1",{"2":{"254":6}}],["mass",{"2":{"254":14,"255":2,"256":1,"257":1,"259":1}}],["mass=1",{"2":{"254":3}}],["massachusetts",{"2":{"74":1}}],["master",{"2":{"143":1}}],["mask",{"2":{"22":9,"67":10}}],["magnitude",{"2":{"72":2}}],["maeloss",{"2":{"53":2}}],["made",{"2":{"52":1,"90":1,"95":1,"98":1,"129":1,"133":1}}],["making",{"2":{"52":1,"75":1,"77":1,"98":1,"106":1,"164":1}}],["makes",{"2":{"32":1,"53":1,"69":1,"77":1,"105":1,"127":1}}],["make",{"2":{"15":2,"34":1,"52":1,"55":6,"69":1,"75":2,"77":1,"109":1,"129":1,"131":1,"134":1,"135":1,"144":1,"155":1,"164":1,"166":1,"170":1,"178":1,"193":1,"249":1,"256":2}}],["macro",{"2":{"48":1,"49":1,"59":4,"81":2,"106":1,"127":2,"180":1,"230":2}}],["machines",{"2":{"53":2}}],["machine",{"2":{"9":1,"22":1,"24":1,"138":1,"166":1,"191":1,"254":1}}],["maxiters",{"2":{"244":4}}],["maxiters=1000",{"2":{"258":1}}],["maxiters=500",{"2":{"208":1}}],["maxiters=epochs",{"2":{"208":1}}],["maximum",{"2":{"21":1,"23":1,"71":1,"243":2,"245":1}}],["maxout",{"2":{"71":5}}],["max",{"2":{"53":2,"68":4,"71":1,"243":8,"245":1,"250":1}}],["maxpool",{"2":{"46":2,"68":1,"188":6}}],["matches",{"2":{"181":2}}],["matched",{"2":{"116":5}}],["matching",{"0":{"57":1},"2":{"116":1}}],["match",{"2":{"32":1,"57":4,"69":5,"132":2,"149":1,"159":1}}],["matrices",{"2":{"23":1,"70":1}}],["matrix",{"2":{"5":3,"9":13,"19":3,"20":1,"23":2,"53":6,"59":4,"70":6,"71":1,"81":1,"106":2,"109":5,"110":13,"116":3,"117":1,"118":1,"125":1,"135":1,"136":1,"137":1,"138":5,"162":4,"164":1,"166":13,"170":3,"207":2}}],["matmul",{"2":{"19":2,"139":1,"140":1,"152":1}}],["managing",{"0":{"169":1},"2":{"169":1}}],["management",{"0":{"124":1,"125":1,"126":1},"1":{"125":1,"126":1},"2":{"98":1,"124":1,"125":1}}],["manager",{"2":{"79":1}}],["manage",{"2":{"59":1}}],["many",{"2":{"135":1,"166":1,"170":1}}],["mandatorily",{"2":{"129":1}}],["manual",{"0":{"126":1},"2":{"13":1,"29":1,"31":1,"58":1,"111":1,"119":1,"133":1,"156":1,"167":1}}],["manually",{"2":{"2":1,"44":1,"46":1,"127":1,"143":1,"167":1}}],["manipulation",{"2":{"12":2,"77":1}}],["margin=2",{"2":{"53":1}}],["margin−y^",{"2":{"53":1}}],["margin",{"2":{"53":2}}],["marker=",{"2":{"255":1,"256":2,"259":4}}],["markersize=16",{"2":{"248":2}}],["markersize=12",{"2":{"213":1,"217":2,"255":1,"256":2,"259":4}}],["marked",{"2":{"5":1,"56":3,"59":1}}],["mark",{"2":{"59":2,"256":1}}],["marks",{"2":{"48":2,"98":1}}],["martens",{"2":{"9":1}}],["map",{"0":{"32":1},"2":{"12":1,"32":4,"55":2,"69":4,"102":2,"103":1,"120":2,"121":1,"164":2,"250":1}}],["maps",{"2":{"9":2,"53":1,"68":3,"69":2}}],["mapping",{"2":{"9":7,"53":1}}],["maintaining",{"2":{"24":1}}],["main",{"0":{"9":1},"2":{"47":1,"117":1,"118":1,"129":1,"154":1,"179":1,"182":3,"207":2,"217":2,"230":1,"236":18,"241":1}}],["maybe",{"2":{"57":1,"170":1}}],["may",{"2":{"3":1,"59":3,"73":2,"191":1,"203":1}}],["min",{"2":{"243":10,"245":2}}],["minibatching",{"2":{"204":1,"208":1}}],["minibatch",{"2":{"178":1,"187":1,"229":1}}],["mini",{"2":{"172":1}}],["minimum",{"2":{"243":4,"245":2}}],["minimally",{"2":{"252":1}}],["minimal",{"2":{"147":1}}],["minimized",{"2":{"257":1}}],["minimize",{"2":{"23":1}}],["minimizes",{"2":{"21":1,"174":1}}],["minor",{"2":{"133":1,"164":1}}],["migrating",{"0":{"193":1},"1":{"194":1,"195":1,"196":1,"197":1}}],["migration",{"0":{"144":1},"1":{"145":1,"146":1},"2":{"129":1}}],["mig",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["might",{"2":{"2":1,"13":1,"18":1,"20":1,"44":1,"46":1,"52":3,"58":2,"59":1,"65":1,"71":1,"89":1,"106":1,"107":1,"111":1,"124":1,"129":2,"133":1,"134":2,"149":1,"153":1,"161":2,"166":1,"168":1,"191":1,"208":1,"235":1}}],["miopen",{"2":{"105":1}}],["mirza",{"2":{"71":1}}],["milletari",{"2":{"53":1}}],["mixing",{"2":{"28":1}}],["mixed",{"2":{"21":1,"27":2}}],["mistakes",{"2":{"77":1}}],["misc",{"0":{"71":1}}],["miscellaneous",{"0":{"3":1,"60":1},"2":{"65":1}}],["mismatch",{"0":{"109":1},"2":{"57":3,"146":1,"149":1,"159":1}}],["missing",{"2":{"24":1,"37":1,"72":1}}],["s3",{"2":{"259":2}}],["s0025",{"2":{"254":2}}],["s2",{"2":{"217":2,"256":2,"259":2}}],["s1",{"2":{"217":2,"256":2,"259":2}}],["sgd",{"2":{"174":1}}],["sz",{"2":{"115":2}}],["szegedy",{"2":{"24":1}}],["src",{"2":{"109":1,"116":1,"134":2,"236":3,"244":4}}],["srivastava",{"2":{"22":1}}],["switch",{"2":{"67":3,"98":1,"133":1,"170":1}}],["switching",{"0":{"43":1},"1":{"44":1},"2":{"133":4,"156":2}}],["square",{"2":{"166":2,"243":1}}],["squaredhingeloss",{"2":{"53":2}}],["squared",{"2":{"53":3,"174":1}}],["sqrt",{"2":{"9":4,"66":2,"69":6,"70":3,"249":1,"255":2}}],["sm",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["smarter",{"2":{"243":1}}],["smatrix",{"2":{"164":4}}],["smaller",{"2":{"77":1}}],["small",{"2":{"23":1,"46":1,"77":1,"106":1,"149":1,"166":1,"185":1,"208":1}}],["smodel",{"2":{"134":3,"135":4,"136":2,"137":2,"139":2,"140":2,"141":2,"208":3}}],["smooth",{"2":{"53":2}}],["smoothing=0",{"2":{"53":3}}],["smoothing",{"2":{"53":16}}],["skipped",{"2":{"65":1,"254":1}}],["skipconnection",{"2":{"65":4}}],["skip",{"2":{"49":3,"65":1,"106":1,"243":1}}],["slow",{"2":{"134":2}}],["slower",{"2":{"108":1}}],["slightly",{"2":{"65":1}}],["slight",{"2":{"18":1,"170":1}}],["slices",{"2":{"54":1}}],["slice",{"2":{"9":1,"24":2,"69":2,"72":2}}],["sleefpirates",{"2":{"18":2}}],["s",{"2":{"15":1,"25":1,"57":3,"58":1,"59":1,"62":1,"65":1,"67":2,"70":1,"72":3,"73":1,"77":1,"80":1,"106":2,"107":1,"109":1,"110":1,"120":1,"129":1,"130":1,"133":2,"134":3,"136":1,"137":1,"138":1,"141":3,"144":2,"149":1,"164":1,"165":4,"166":5,"171":1,"174":2,"179":5,"181":1,"183":1,"194":1,"204":1,"208":2,"213":3,"214":1,"217":1,"233":1,"249":2,"255":3}}],["syntax",{"2":{"59":1,"65":1,"166":2,"167":1}}],["synchronized",{"2":{"143":2}}],["synchronize",{"2":{"39":2,"143":6,"144":2,"146":2,"164":2}}],["symbol",{"2":{"15":1,"33":1,"59":1,"72":1,"106":1,"131":1,"183":1,"236":29,"249":2}}],["systems",{"2":{"22":1,"77":2,"80":1,"161":1}}],["system",{"2":{"2":1,"3":1,"106":1,"124":2,"131":1,"170":1,"238":1,"255":1,"256":1}}],["sorted",{"2":{"84":1}}],["soln",{"2":{"254":13,"255":2,"256":2,"259":2}}],["soln2orbit",{"2":{"254":3}}],["solution",{"2":{"243":5,"245":1}}],["solutions",{"2":{"9":1}}],["sol",{"2":{"209":2}}],["solver",{"2":{"230":6,"234":3,"236":7}}],["solver=tsit5",{"2":{"230":2,"234":1}}],["solvers",{"2":{"77":1}}],["solves",{"2":{"208":1}}],["solve",{"2":{"206":1,"208":4,"209":1,"230":2,"234":1,"238":1,"255":1,"256":1,"257":1,"258":1,"259":1}}],["society",{"2":{"53":1}}],["software",{"2":{"74":2}}],["softmax",{"2":{"53":5}}],["softfail",{"2":{"50":2}}],["soft",{"2":{"49":4}}],["sooner",{"2":{"30":1}}],["so",{"2":{"18":1,"80":1,"133":3,"134":1,"135":1,"194":1,"207":1,"216":1,"230":2,"234":2,"240":1,"241":1}}],["sometimes",{"2":{"166":2}}],["somewhere",{"2":{"44":1,"46":1}}],["some",{"0":{"206":1,"254":1},"2":{"13":1,"33":1,"44":2,"59":1,"69":1,"71":1,"77":1,"85":1,"87":1,"88":1,"89":1,"90":1,"102":1,"108":1,"110":1,"122":1,"130":1,"133":7,"134":1,"142":1,"146":1,"149":1,"152":1,"154":2,"166":1,"194":1,"222":1,"233":1,"243":1}}],["source",{"2":{"1":1,"2":3,"3":8,"4":2,"5":1,"9":8,"10":24,"12":3,"13":7,"14":2,"15":5,"16":1,"18":2,"19":1,"20":2,"21":1,"22":2,"23":1,"24":4,"25":1,"27":2,"28":1,"31":5,"32":1,"33":2,"34":1,"36":2,"37":3,"38":2,"39":4,"40":1,"41":1,"44":3,"46":3,"48":2,"49":2,"50":1,"52":6,"53":15,"54":8,"55":6,"56":3,"57":1,"58":1,"59":3,"60":1,"65":6,"66":2,"67":3,"68":9,"69":6,"70":4,"71":7,"72":5,"73":2,"95":1,"165":2}}],["safely",{"2":{"256":1}}],["saveat=tsteps",{"2":{"255":1,"256":1,"257":1,"259":1}}],["saveat=t",{"2":{"206":1,"208":1,"209":1}}],["save",{"2":{"183":4,"231":2,"236":26}}],["saving",{"0":{"183":1},"2":{"183":1}}],["sake",{"2":{"179":1,"181":1,"243":1}}],["sarray",{"2":{"164":7}}],["sarrays",{"2":{"164":1}}],["sanity",{"2":{"141":1}}],["sampling",{"2":{"243":1,"247":1}}],["sampler",{"2":{"249":1}}],["sampled",{"2":{"249":2}}],["samples",{"2":{"174":15,"249":2,"250":2}}],["sample",{"2":{"138":3,"141":1,"249":3,"250":1}}],["samepad",{"2":{"66":2,"68":3}}],["same",{"0":{"118":1},"2":{"3":1,"5":2,"9":1,"12":3,"18":2,"20":1,"22":1,"24":3,"27":2,"32":1,"33":1,"34":1,"52":2,"54":2,"57":2,"61":1,"65":1,"66":2,"68":3,"69":3,"71":2,"72":1,"76":2,"81":1,"106":1,"130":1,"135":1,"141":2,"143":1,"164":1,"167":1,"169":1,"173":1,"182":1,"185":1,"230":1}}],["saw",{"2":{"107":1}}],["say",{"2":{"77":1,"120":1}}],["satisfactory",{"2":{"250":1}}],["satisfying",{"2":{"66":2,"68":6,"129":1}}],["satisfies",{"2":{"46":2,"71":1}}],["satisfied",{"2":{"13":1,"216":1}}],["said",{"2":{"30":1}}],["saxe",{"2":{"9":1}}],["speedup",{"2":{"191":2}}],["spectralnorm",{"2":{"77":1}}],["specify",{"2":{"53":1,"59":1,"67":2,"73":1,"122":1,"249":1}}],["specifying",{"2":{"9":1,"48":1,"59":1,"66":2,"68":3,"72":2}}],["specifies",{"2":{"66":4,"68":3,"71":1,"172":1}}],["specified",{"2":{"2":1,"9":2,"49":1,"65":8,"70":1,"71":3,"73":1,"125":1,"163":1}}],["specifically",{"2":{"166":1,"196":1}}],["specifications",{"2":{"111":1}}],["specification",{"0":{"109":1},"2":{"249":1}}],["specific",{"2":{"4":2,"13":1,"55":1,"70":1,"88":1,"158":1}}],["specialized",{"2":{"18":1,"25":1,"59":1,"164":1,"168":1}}],["special",{"2":{"3":2,"23":2,"39":2,"53":1,"59":1,"69":1,"88":1,"236":3}}],["spiralclassifiercompact",{"2":{"180":2,"182":1}}],["spiralclassifier",{"2":{"179":5,"182":1}}],["spiral",{"2":{"178":1}}],["spirals",{"2":{"176":1,"178":6}}],["spiritual",{"2":{"142":1}}],["spurious",{"0":{"149":1}}],["space",{"2":{"69":6}}],["spatial",{"2":{"66":5,"68":6,"243":1}}],["sparsity=0",{"2":{"9":1}}],["sparsity",{"2":{"9":4}}],["sparsely",{"2":{"9":2,"70":1}}],["sparse",{"2":{"9":2}}],["spliced",{"2":{"69":2}}],["split=",{"2":{"187":1,"229":1}}],["splitobs",{"2":{"178":1,"187":1,"228":1,"229":1}}],["split",{"2":{"54":1,"143":2,"178":1,"179":1,"187":2,"229":2}}],["splatted",{"2":{"59":1}}],["splatting",{"2":{"59":1}}],["situations",{"2":{"147":1}}],["si+1",{"2":{"66":1,"68":3}}],["siamese",{"2":{"53":1}}],["siamesecontrastiveloss",{"2":{"53":3}}],["sig",{"2":{"249":2}}],["sigmoid",{"2":{"53":3,"179":1,"180":1,"249":2}}],["signify",{"2":{"90":1,"97":1}}],["significantly",{"2":{"127":1,"233":1}}],["significant",{"2":{"13":1,"191":1,"235":1}}],["signature",{"2":{"22":1,"32":1}}],["silently",{"2":{"34":1}}],["sided",{"2":{"254":2}}],["side",{"2":{"21":1,"59":1}}],["sin",{"2":{"254":1}}],["singular",{"0":{"129":1},"2":{"129":1,"130":2}}],["singleton",{"2":{"12":3,"66":2,"68":3,"95":2,"96":1}}],["single",{"2":{"12":2,"13":1,"20":1,"21":1,"52":5,"53":1,"59":1,"66":6,"68":9,"69":13,"80":2,"81":1,"95":1,"107":3,"130":1,"135":1,"138":1,"164":4,"166":1,"174":1,"182":1,"190":1,"194":1,"217":1,"225":1,"233":1,"244":1,"249":1}}],["since",{"2":{"9":1,"13":1,"32":1,"59":1,"61":1,"77":1,"95":1,"107":1,"113":1,"131":1,"143":1,"172":1,"179":1,"181":1,"208":2,"240":1}}],["simultaneously",{"2":{"131":1}}],["simulate",{"2":{"110":1,"254":1,"255":1,"256":2}}],["simulating",{"0":{"255":1},"2":{"9":1}}],["simd",{"2":{"25":1}}],["simplicity",{"2":{"138":1,"181":1,"243":1}}],["simplified",{"2":{"129":1}}],["simplest",{"2":{"65":1}}],["simplechainslayer",{"2":{"46":2,"188":1}}],["simplechains",{"0":{"185":1},"1":{"186":1,"187":1,"188":1,"189":1,"190":1,"191":1,"192":1},"2":{"46":14,"77":1,"185":3,"186":2,"188":1,"191":2}}],["simple",{"0":{"46":1,"176":1},"1":{"177":1,"178":1,"179":1,"180":1,"181":1,"182":1,"183":1,"184":1},"2":{"22":2,"25":1,"46":4,"52":1,"59":1,"69":1,"106":1,"125":2,"165":1,"171":1,"188":1,"191":1,"194":1,"196":1,"204":1}}],["simply",{"2":{"13":1,"18":1,"52":1,"56":1,"57":1,"59":1,"61":1,"107":1,"108":1,"118":1,"122":1,"131":1,"222":1,"249":1}}],["similarly",{"2":{"130":1}}],["similarity",{"2":{"5":1}}],["similar",{"2":{"2":1,"3":1,"12":1,"24":1,"53":1,"54":1,"55":1,"69":1,"70":1,"106":1,"107":1,"131":1,"154":1,"170":1,"193":1,"256":1}}],["size=1000",{"2":{"178":1}}],["sized",{"2":{"23":1,"24":1,"54":1}}],["sizes",{"2":{"9":2,"23":1,"95":1}}],["size",{"0":{"16":1},"2":{"9":24,"10":72,"16":2,"18":2,"24":3,"28":2,"44":1,"46":1,"59":6,"65":1,"66":16,"68":48,"69":11,"70":16,"71":8,"72":7,"73":8,"105":1,"115":1,"139":2,"140":2,"164":2,"166":1,"174":2,"175":1,"178":7,"184":1,"187":3,"192":1,"210":1,"218":1,"226":1,"229":3,"230":2,"233":1,"237":1,"246":1,"249":4,"250":1,"251":1,"254":4,"260":1}}],["scientific",{"2":{"254":1}}],["scientificmachinelearning",{"2":{"252":1}}],["scimlsensitivity",{"2":{"205":1,"228":1,"236":26,"253":1}}],["sciml",{"2":{"58":1,"59":1,"77":2,"81":1,"172":1}}],["scatter",{"2":{"213":1,"217":2,"248":2,"255":1,"256":2,"259":4}}],["scalable",{"2":{"139":1}}],["scalar",{"0":{"150":1},"2":{"3":2,"25":1,"107":1,"150":1,"171":1,"241":1}}],["scale=ones32",{"2":{"72":4}}],["scale",{"2":{"24":15,"70":2,"72":20,"73":6,"110":1,"123":1,"134":1}}],["scaled",{"2":{"9":3}}],["scaling",{"2":{"9":2,"22":2,"254":6}}],["scratch",{"2":{"174":1,"179":1,"227":1}}],["schwarzschild",{"2":{"255":1}}],["school",{"2":{"74":1}}],["schemes",{"2":{"6":1,"162":1}}],["score",{"2":{"53":1}}],["ship",{"2":{"108":1}}],["shifted",{"2":{"9":1}}],["shift",{"2":{"9":4,"24":1,"72":4}}],["shuffle=true",{"2":{"178":2,"187":1,"221":1,"229":1,"244":2}}],["shuffle=false",{"2":{"5":1,"178":1,"187":1,"221":1,"229":1}}],["shuffle",{"2":{"73":1,"178":2,"187":2,"229":2}}],["shuffling",{"2":{"73":1}}],["shapedaxis",{"2":{"236":30}}],["shaped",{"2":{"68":9}}],["shape",{"2":{"66":1,"69":16,"71":1,"72":17,"109":4,"166":1,"174":4}}],["shate",{"2":{"34":1}}],["sharing",{"2":{"34":9}}],["share",{"2":{"34":4}}],["shared",{"2":{"24":2,"34":1,"98":1}}],["shooting",{"2":{"208":1}}],["shortcomings",{"0":{"147":1}}],["shortcut",{"2":{"13":1,"65":1}}],["short",{"2":{"69":1,"170":1}}],["shorthand",{"2":{"13":1}}],["shown",{"2":{"249":1}}],["showcasing",{"2":{"203":1}}],["showing",{"2":{"141":1}}],["shows",{"2":{"131":1,"137":1,"138":1,"141":1,"250":1}}],["showerror",{"2":{"116":1}}],["show",{"2":{"5":2,"80":1,"134":1,"167":1,"204":1,"247":1}}],["shouldn",{"2":{"52":1,"95":1,"131":1,"136":1}}],["should",{"0":{"113":1},"2":{"4":2,"20":1,"25":1,"31":1,"34":1,"36":2,"46":1,"52":2,"53":1,"55":1,"59":1,"66":5,"68":6,"69":1,"72":1,"73":1,"98":1,"99":1,"119":1,"124":1,"129":2,"130":2,"131":1,"149":1,"168":1,"169":1,"193":1,"194":2,"214":1,"227":1,"256":1}}],["stencils",{"2":{"254":2}}],["stepnorm=0",{"2":{"258":1}}],["steprangelen",{"2":{"207":2}}],["steps",{"2":{"52":1,"80":1,"107":1,"249":1}}],["step",{"2":{"52":10,"80":1,"81":1,"107":3,"174":1,"182":1,"190":1,"217":2,"225":1,"233":1,"244":1,"249":3}}],["stime",{"2":{"190":2,"225":2,"233":2}}],["stick",{"2":{"170":1}}],["still",{"0":{"197":1},"2":{"2":1,"13":1,"46":1,"109":1,"118":1,"124":1,"129":1,"130":1,"179":1,"214":1,"236":1,"250":1}}],["stop=6",{"2":{"250":4}}],["stochastic",{"2":{"76":1,"77":1,"174":1}}],["stores",{"2":{"58":1,"67":1,"70":1,"146":1,"166":1,"194":1,"230":1}}],["stored",{"2":{"52":3,"66":1}}],["store",{"2":{"39":2,"70":1,"76":2,"194":1,"208":1,"248":1,"256":1,"257":1}}],["storing",{"2":{"24":1,"117":1,"194":1}}],["stylization",{"2":{"24":1,"72":1}}],["st",{"2":{"12":2,"13":4,"15":6,"31":2,"32":6,"34":1,"44":4,"46":2,"52":1,"53":2,"54":2,"57":1,"58":4,"59":19,"65":6,"69":1,"71":14,"80":4,"81":6,"103":2,"106":14,"107":6,"109":5,"110":7,"113":3,"115":7,"116":2,"117":10,"118":2,"120":9,"121":7,"122":6,"123":2,"129":7,"130":11,"131":3,"132":2,"134":7,"135":7,"136":6,"137":6,"139":2,"140":2,"141":9,"143":2,"149":5,"164":8,"174":6,"179":10,"181":4,"182":7,"183":3,"189":4,"190":2,"193":3,"194":6,"208":5,"216":1,"217":1,"224":4,"225":2,"230":4,"231":4,"232":4,"233":2,"234":7,"236":16,"242":7,"244":3,"249":2,"256":2}}],["std=1e",{"2":{"256":3}}],["std=0",{"2":{"9":1}}],["stdlib",{"2":{"169":1}}],["stdout",{"2":{"116":1}}],["std",{"2":{"9":5,"24":1,"249":1}}],["stage",{"2":{"133":1}}],["start=false",{"2":{"231":1}}],["start",{"2":{"106":1,"165":1,"236":13,"247":1}}],["starting",{"2":{"93":1,"124":1,"166":1,"234":1}}],["started",{"0":{"78":1,"84":1},"1":{"79":1,"80":1,"81":1,"82":1,"83":1},"2":{"193":1}}],["stacktraces",{"2":{"108":1}}],["stacktrace",{"2":{"102":2,"109":1}}],["stack",{"2":{"69":2,"243":6,"245":2}}],["stackedrnncells",{"2":{"69":1}}],["stabilities",{"2":{"235":1}}],["stability",{"0":{"236":1},"2":{"13":1,"24":4,"53":1,"59":1,"72":4,"90":1,"97":1,"160":1}}],["stablerng",{"2":{"134":3,"135":3,"136":2,"137":2,"141":3}}],["stablerngs",{"2":{"7":1,"133":1}}],["stable",{"2":{"52":1,"55":5,"59":2,"160":1,"181":1,"230":1,"236":1}}],["stats=false",{"2":{"72":5}}],["stats=true",{"2":{"65":2,"72":7,"109":1,"110":2}}],["stats",{"2":{"52":2,"80":2,"107":1,"244":7}}],["staticint",{"2":{"188":2}}],["staticarrayscore",{"2":{"164":4}}],["staticarrays",{"2":{"164":1}}],["staticsymbol",{"2":{"33":1,"54":2}}],["static",{"2":{"25":1,"46":1,"54":1,"58":1,"80":8,"106":3,"116":2,"164":7,"188":4,"217":2,"236":58}}],["staticbool",{"2":{"22":1,"33":1,"54":1}}],["statistics",{"2":{"9":2,"24":2,"52":2,"72":7,"105":1,"177":1,"186":1,"212":1,"216":1,"220":1,"228":1,"239":1,"249":1}}],["state=false",{"2":{"69":4}}],["state=zeros32",{"2":{"69":3}}],["statements",{"2":{"59":1}}],["statefulneuralode",{"2":{"234":4,"235":1,"236":5}}],["statefulrecurrentcell",{"2":{"69":2}}],["statefulluxlayers",{"2":{"133":1}}],["statefulluxlayer",{"2":{"58":1,"103":4,"133":7,"134":1,"135":1,"136":1,"137":1,"139":1,"140":1,"141":1,"208":2,"234":2,"242":7,"244":2,"249":2,"256":2}}],["stateful",{"0":{"58":1,"234":1,"235":1},"2":{"69":2,"134":1,"135":1,"137":2,"236":6}}],["stateless",{"2":{"13":1,"71":1}}],["statelength",{"2":{"12":1,"15":1,"129":4,"130":1}}],["state",{"0":{"132":1},"2":{"12":1,"13":3,"15":5,"22":2,"24":1,"31":1,"44":1,"52":7,"58":4,"59":7,"65":6,"67":4,"69":64,"71":1,"72":4,"76":3,"77":3,"80":8,"81":4,"107":4,"129":5,"130":3,"132":4,"143":5,"179":3,"182":7,"190":8,"194":1,"208":6,"216":1,"225":11,"244":6,"249":2,"256":1}}],["states",{"0":{"15":1},"2":{"3":1,"12":4,"13":1,"15":2,"31":4,"32":6,"33":1,"34":1,"44":1,"46":1,"52":2,"53":2,"56":2,"57":4,"58":1,"59":5,"65":16,"67":3,"69":5,"71":2,"72":7,"80":2,"81":2,"95":2,"107":1,"109":2,"110":1,"115":1,"120":3,"123":1,"127":1,"129":5,"130":3,"132":1,"143":2,"174":2,"179":2,"182":2,"183":1,"188":2,"190":2,"194":3,"214":1,"216":1,"217":4,"225":4,"230":1,"233":2,"244":1,"249":1,"256":1}}],["standard",{"2":{"9":3,"10":6,"66":2,"72":1,"80":1,"134":1,"256":1}}],["strain",{"2":{"254":5}}],["strokewidth=2",{"2":{"213":1,"217":2,"248":2,"256":2,"259":4}}],["strokecolor=",{"2":{"213":1,"217":2,"248":2}}],["strongly",{"2":{"168":1}}],["struct",{"2":{"115":1,"127":1,"129":1,"183":1,"194":3,"207":1}}],["structs",{"2":{"56":3}}],["structured",{"2":{"69":3,"131":1,"222":1}}],["structures",{"2":{"55":2,"77":1,"149":1}}],["structure",{"2":{"3":5,"12":2,"13":3,"27":2,"32":4,"34":1,"39":1,"55":3,"70":1,"109":5,"110":13,"129":2,"130":2,"132":1,"194":1,"233":1}}],["strength",{"2":{"53":1}}],["stride=window",{"2":{"68":3}}],["stride=1",{"2":{"66":2}}],["stride",{"2":{"66":5,"68":6}}],["stridearray",{"2":{"46":1}}],["stridearrayscore",{"2":{"46":1}}],["strings",{"2":{"103":1}}],["string=",{"2":{"60":2}}],["string",{"2":{"1":2,"3":1,"48":1,"60":1}}],["surprise",{"2":{"149":1}}],["surpassing",{"2":{"9":2}}],["sure",{"2":{"131":1,"256":1}}],["super",{"2":{"254":1}}],["supertype",{"2":{"130":1}}],["suppose",{"2":{"250":1}}],["supposed",{"2":{"77":1,"102":1,"160":1}}],["supporting",{"2":{"87":1}}],["support",{"0":{"4":1,"62":1,"63":1,"83":1,"88":1,"157":1},"2":{"3":4,"4":4,"12":2,"21":2,"23":2,"31":1,"53":2,"55":1,"58":1,"68":3,"77":3,"80":2,"81":2,"83":5,"85":1,"86":1,"87":1,"89":1,"93":2,"97":1,"124":2,"131":1,"147":1,"196":1}}],["supports",{"2":{"3":1,"59":1,"105":1,"157":1,"166":1,"167":1,"196":1,"240":1}}],["supported",{"0":{"7":1},"2":{"2":1,"3":2,"25":1,"27":2,"28":1,"44":1,"52":6,"57":1,"68":3,"73":1,"76":2,"80":1,"88":2,"89":1,"97":1,"124":2,"154":1,"162":2}}],["supplied",{"2":{"37":1,"65":1}}],["supply",{"2":{"37":1}}],["suggests",{"2":{"71":1}}],["subtracts",{"2":{"166":1}}],["subtract",{"2":{"166":1}}],["subtyping",{"2":{"130":1}}],["subtype",{"2":{"129":1,"130":2}}],["subarray",{"2":{"54":1}}],["suboptimal",{"2":{"52":1,"164":1}}],["sumit",{"2":{"53":1}}],["sum",{"2":{"48":3,"49":2,"59":5,"110":2,"134":1,"135":1,"136":2,"137":1,"139":1,"140":1,"181":1,"189":1,"193":2,"194":2,"224":1,"232":1,"242":3,"254":1}}],["summary",{"2":{"5":8,"194":1,"249":1}}],["success",{"2":{"258":1}}],["successor",{"2":{"142":1}}],["successfully",{"2":{"1":1}}],["such",{"2":{"4":2,"11":1,"13":1,"59":1,"66":2,"68":6,"77":1,"106":1,"111":1,"163":1,"180":1,"222":1}}],["seaborn",{"2":{"250":3}}],["seamlessly",{"2":{"52":1}}],["seq",{"2":{"178":1}}],["sequentially",{"2":{"65":2,"69":5}}],["sequences",{"2":{"178":1}}],["sequence=true",{"2":{"69":2}}],["sequence",{"2":{"65":1,"69":9,"178":7,"179":3}}],["separately",{"2":{"167":1}}],["separation",{"2":{"77":1}}],["separating",{"2":{"77":1}}],["several",{"2":{"155":1,"170":1}}],["sensealg=reversediffadjoint",{"2":{"233":1}}],["sensealg=gaussadjoint",{"2":{"233":1}}],["sensealg=interpolatingadjoint",{"2":{"231":1,"233":1}}],["sensealg",{"2":{"231":1,"233":1,"236":13}}],["sensitivities",{"2":{"230":1,"233":1}}],["sensible",{"2":{"77":1}}],["sensibly",{"2":{"54":1}}],["sensical",{"2":{"59":1}}],["send",{"2":{"84":1}}],["sendbuf",{"2":{"39":6}}],["sendrecvbuf",{"2":{"39":6}}],["seyed",{"2":{"53":1}}],["segmentation",{"2":{"53":2}}],["sec",{"2":{"249":1}}],["section",{"2":{"46":1,"65":2,"79":1,"111":1,"135":1,"156":1,"167":1,"254":1}}],["seconds",{"2":{"249":2}}],["second",{"2":{"22":1,"65":1,"166":1,"172":1,"254":2}}],["server",{"2":{"175":1,"184":1,"191":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["serves",{"2":{"44":1}}],["serious",{"2":{"238":1}}],["seriously",{"2":{"80":1}}],["serializes",{"2":{"143":1}}],["serialization",{"2":{"143":1,"183":1}}],["series",{"2":{"62":2}}],["sergey",{"2":{"24":1}}],["self",{"2":{"22":1}}],["selu",{"2":{"22":1}}],["selecting",{"2":{"148":1}}],["selection",{"0":{"158":1},"2":{"2":1,"3":1,"125":1,"126":1,"158":1}}],["selectdim",{"2":{"71":1}}],["select",{"2":{"2":4}}],["selected",{"0":{"203":1},"2":{"2":3,"3":1,"102":1}}],["selects",{"2":{"2":1}}],["semvar",{"2":{"103":1}}],["semver",{"2":{"30":1}}],["semantic",{"2":{"134":2}}],["semantically",{"2":{"65":1}}],["semantics",{"2":{"5":2,"149":1}}],["semi",{"2":{"9":2}}],["seems",{"2":{"258":2}}],["seeding",{"2":{"80":1}}],["seed",{"2":{"65":1,"71":4,"77":1,"80":1,"81":1,"123":1,"129":1,"131":1,"165":1,"169":2,"174":1,"213":1,"231":1,"244":3,"248":1}}],["see",{"2":{"3":1,"12":3,"13":3,"16":1,"19":1,"20":2,"22":2,"24":4,"31":1,"33":2,"34":1,"44":1,"46":2,"49":1,"52":1,"53":1,"55":2,"57":1,"58":2,"59":2,"60":1,"65":4,"67":3,"69":5,"71":2,"72":4,"73":1,"77":2,"79":1,"92":1,"95":3,"97":1,"102":2,"104":2,"106":1,"107":1,"108":1,"109":3,"110":2,"130":1,"142":1,"146":1,"149":2,"153":1,"156":1,"159":1,"160":1,"162":1,"166":2,"167":1,"179":1,"191":1,"235":1,"250":2,"254":2}}],["session",{"2":{"1":1,"60":1}}],["setprogress",{"2":{"247":1}}],["sets",{"2":{"48":1,"160":2}}],["setindexing",{"2":{"21":1,"23":1}}],["setups",{"2":{"137":1}}],["setup",{"2":{"13":1,"32":1,"34":1,"44":1,"46":1,"59":6,"65":1,"71":4,"80":1,"81":1,"106":1,"107":1,"109":2,"110":2,"116":1,"118":1,"120":1,"123":1,"129":2,"130":1,"131":1,"134":1,"135":1,"136":1,"137":1,"141":1,"143":1,"149":1,"164":1,"174":1,"182":1,"190":1,"193":1,"194":1,"208":1,"216":1,"225":1,"231":1,"244":1,"249":1,"256":1}}],["setfield",{"2":{"12":1,"95":1,"220":1}}],["setting",{"0":{"257":1},"2":{"4":2,"66":2,"69":4,"70":3,"110":1,"133":1,"151":1,"160":2,"161":1}}],["set",{"2":{"1":2,"2":1,"4":6,"9":1,"22":4,"24":4,"25":1,"31":2,"34":1,"37":2,"44":2,"46":1,"48":1,"49":1,"53":2,"55":1,"58":1,"59":3,"60":5,"63":3,"66":2,"67":1,"69":18,"73":2,"90":1,"110":2,"125":1,"133":1,"134":4,"146":1,"151":2,"155":2,"156":1,"157":3,"158":3,"160":2,"174":3,"247":1}}],["os",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["oops",{"2":{"149":1}}],["ok",{"2":{"136":1}}],["own",{"2":{"131":1,"147":1,"166":1}}],["oh",{"2":{"110":1}}],["our",{"2":{"77":1,"80":1,"88":1,"110":3,"134":1,"138":1,"149":2,"165":1,"166":1,"168":1,"169":1,"171":1,"174":2,"178":1,"179":1,"196":1,"204":1,"208":3,"216":1,"217":1,"238":1,"248":1,"249":3,"250":4}}],["outperforms",{"2":{"89":1}}],["outpad",{"2":{"66":3,"105":1}}],["outpad=0",{"2":{"66":1}}],["outputs",{"2":{"16":1,"33":1,"46":1,"58":1,"65":2,"69":6,"71":2,"76":1,"107":1,"135":1,"136":1,"169":1,"171":1,"256":1}}],["outputsize",{"2":{"16":3,"95":1}}],["output",{"2":{"9":6,"16":1,"18":2,"21":1,"22":3,"23":1,"33":1,"46":2,"53":6,"58":1,"59":2,"65":7,"66":8,"68":33,"69":13,"70":7,"71":4,"72":1,"73":4,"81":1,"105":1,"106":2,"109":4,"110":14,"129":1,"132":2,"149":1,"164":13,"169":1,"179":1,"230":1,"241":1,"249":2}}],["outside",{"2":{"55":1,"102":2}}],["out",{"2":{"9":2,"22":2,"30":1,"55":1,"59":10,"65":1,"66":8,"67":2,"68":3,"69":15,"70":19,"81":6,"90":2,"108":1,"110":1,"127":1,"129":8,"132":1,"151":1,"179":3,"180":2,"233":1,"247":1}}],["o=σ",{"2":{"69":1}}],["oi=",{"2":{"66":1,"68":3}}],["o",{"2":{"66":4,"68":6}}],["odefunction",{"2":{"230":2,"234":1}}],["odeproblem",{"2":{"206":1,"208":1,"209":1,"230":2,"234":1,"255":1,"256":1,"259":1}}],["odesolution",{"2":{"230":1}}],["odes",{"0":{"227":1},"1":{"228":1,"229":1,"230":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"77":1,"114":1,"227":2,"255":1,"256":1}}],["ode",{"0":{"230":1,"231":1,"235":1,"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"58":1,"114":1,"204":3,"206":3,"207":1,"208":2,"209":2,"230":2,"231":1,"255":3,"256":6,"257":1,"259":2}}],["odd",{"2":{"9":1}}],["old",{"2":{"52":1,"92":1,"96":2,"124":2}}],["older",{"0":{"43":1},"1":{"44":1},"2":{"102":1,"146":1}}],["observe",{"2":{"249":1}}],["observations",{"2":{"66":1}}],["obtain",{"2":{"143":1}}],["obtained",{"2":{"25":1}}],["obviously",{"2":{"108":1}}],["obj",{"2":{"52":2,"130":2,"131":1}}],["objects",{"2":{"53":1,"99":3,"106":1,"126":1,"149":1,"207":1}}],["objective",{"2":{"52":8,"257":1}}],["object",{"2":{"2":3,"3":4,"21":1,"27":2,"52":6,"53":2,"125":3,"143":1}}],["occurs",{"2":{"110":1}}],["occurred",{"2":{"110":3}}],["occurrences",{"2":{"15":3}}],["octavian",{"0":{"161":1},"2":{"23":2,"153":1,"161":1}}],["other",{"0":{"10":1,"42":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"5":1,"9":1,"10":1,"13":1,"21":1,"39":2,"46":1,"49":1,"53":1,"55":2,"65":1,"66":1,"71":2,"73":1,"80":1,"81":1,"89":1,"93":1,"102":1,"107":1,"127":1,"129":1,"130":1,"166":1,"170":1,"183":1,"204":1}}],["otherwisewhere",{"2":{"53":1}}],["otherwise",{"2":{"1":1,"2":2,"3":1,"60":1,"70":1}}],["opening",{"2":{"133":1}}],["open",{"2":{"87":1,"106":1,"111":1,"147":1,"164":1,"202":1,"203":2}}],["operating",{"2":{"69":1}}],["operation",{"2":{"21":2,"23":1,"25":4,"39":4,"68":3,"73":1,"170":1}}],["operations",{"0":{"19":1,"55":1},"2":{"2":1,"21":3,"23":1,"28":1,"46":1,"54":2,"55":1,"93":1,"133":2,"161":1,"164":2,"167":1}}],["operates",{"2":{"106":1}}],["operate",{"2":{"69":1,"194":1}}],["operators",{"2":{"71":1}}],["operator",{"2":{"55":1}}],["optprob",{"2":{"258":2}}],["optf",{"2":{"258":2}}],["opt",{"2":{"48":6,"143":5,"174":2,"208":10,"217":1}}],["optimal",{"2":{"208":1}}],["optim",{"2":{"59":2}}],["optimiser",{"0":{"181":1}}],["optimisers",{"2":{"40":1,"44":2,"52":2,"59":3,"80":10,"81":1,"106":1,"143":1,"174":2,"176":1,"177":1,"186":1,"194":2,"197":1,"204":1,"208":2,"212":1,"215":1,"220":1,"228":1,"239":1,"244":1}}],["optimizer",{"0":{"215":1},"2":{"40":4,"52":4,"143":3,"217":2}}],["optimizers",{"0":{"40":1},"2":{"258":1}}],["optimized",{"2":{"25":1,"153":1,"161":1,"208":1}}],["optimizationproblem",{"2":{"208":2,"258":1}}],["optimizationfunction",{"2":{"208":1,"258":1}}],["optimizationoptimjl",{"2":{"205":1,"253":1}}],["optimizationoptimisers",{"2":{"205":1}}],["optimizations",{"2":{"102":1,"106":1}}],["optimization",{"0":{"204":1},"1":{"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"9":1,"80":1,"81":3,"106":1,"204":4,"205":1,"208":8,"253":1,"258":4}}],["options",{"2":{"57":1,"158":1}}],["option",{"2":{"25":1,"57":2,"73":2,"87":2}}],["optionally",{"2":{"9":1,"12":1,"72":1,"129":1,"163":1}}],["optional",{"0":{"153":1},"2":{"9":1,"21":1,"23":1,"31":3,"53":1,"58":1,"59":2,"69":1,"95":1,"161":1,"163":1}}],["op",{"2":{"4":2,"5":1,"24":1,"39":6,"48":1,"54":3,"57":1,"100":1}}],["orbit₂",{"2":{"254":2}}],["orbit₁",{"2":{"254":2}}],["orbits",{"2":{"254":1}}],["orbit2",{"2":{"254":4}}],["orbit2tensor",{"2":{"254":2}}],["orbit1",{"2":{"254":4}}],["orbit",{"2":{"254":17}}],["orange",{"2":{"213":1,"217":1}}],["ordinarydiffeqloworderrk",{"2":{"253":1}}],["ordinarydiffeqcore",{"2":{"236":26}}],["ordinarydiffeq",{"2":{"230":1,"255":1}}],["ordinarydiffeqtsit5",{"2":{"205":1,"228":1,"236":13}}],["ordering",{"2":{"69":6}}],["order",{"0":{"29":1},"2":{"2":1,"31":1,"66":1,"86":1,"134":1,"136":1,"172":1,"240":3,"242":1,"254":2}}],["orcjit",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["originates",{"2":{"33":1}}],["originally",{"2":{"252":1}}],["original",{"2":{"9":1,"31":1,"34":1,"44":1,"106":1,"109":1,"168":2}}],["org",{"2":{"9":1,"71":1,"74":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"254":2,"260":1}}],["orthogonal",{"2":{"7":4,"9":6}}],["or",{"2":{"2":3,"3":2,"4":2,"9":2,"13":1,"21":2,"22":3,"23":1,"24":2,"25":2,"31":1,"33":3,"39":3,"44":1,"46":2,"52":1,"53":5,"54":2,"59":3,"60":1,"62":1,"65":7,"66":6,"68":9,"69":3,"70":7,"73":4,"77":2,"79":1,"80":1,"84":1,"87":1,"107":1,"110":2,"129":2,"130":1,"133":1,"136":1,"146":2,"147":1,"149":1,"152":2,"157":1,"163":1,"166":4,"167":1,"178":1,"194":1,"196":1,"202":1,"203":2,"254":1}}],["overloaded",{"2":{"102":1}}],["overloading",{"2":{"55":1}}],["overview",{"0":{"86":1}}],["overcome",{"2":{"44":1}}],["overridden",{"2":{"23":1}}],["overrides",{"2":{"49":1}}],["override",{"2":{"16":1,"44":1}}],["overfitting",{"2":{"22":2}}],["over",{"0":{"32":1,"77":1},"2":{"2":1,"5":2,"32":1,"33":1,"58":1,"69":1,"71":1,"72":3,"80":1,"119":1,"176":1,"207":1,"217":1,"238":1,"241":1,"257":1,"259":1}}],["onlinestats",{"2":{"239":1,"244":4}}],["online",{"2":{"203":1}}],["only",{"2":{"2":1,"13":1,"33":2,"49":3,"52":1,"53":3,"57":2,"59":1,"66":1,"67":1,"68":3,"69":9,"70":1,"73":1,"106":1,"122":1,"130":1,"131":1,"133":2,"136":1,"137":1,"143":1,"164":2,"171":2,"183":1,"229":1,"242":3}}],["once",{"2":{"16":1,"57":1,"69":2,"110":1,"129":1,"167":1,"168":1,"170":1}}],["onwards",{"2":{"12":1,"102":1}}],["on",{"0":{"112":1,"150":1,"219":1,"238":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1,"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"2":2,"3":6,"5":1,"9":11,"11":2,"12":1,"13":2,"18":2,"19":3,"20":1,"21":3,"23":3,"24":4,"29":2,"30":1,"31":1,"32":2,"39":2,"46":1,"48":1,"53":5,"55":3,"58":2,"59":2,"60":1,"61":1,"65":2,"66":2,"67":2,"68":6,"69":2,"70":1,"74":1,"77":1,"79":1,"80":1,"84":2,"87":2,"89":1,"93":1,"95":2,"103":1,"104":2,"105":1,"106":2,"111":1,"113":2,"117":1,"127":2,"129":2,"132":1,"133":4,"134":4,"135":1,"136":2,"143":1,"147":1,"149":2,"150":1,"151":1,"153":1,"154":2,"155":2,"156":1,"159":1,"160":1,"162":1,"164":3,"165":1,"166":1,"167":1,"169":3,"172":2,"175":1,"184":1,"191":2,"192":1,"194":4,"202":1,"204":1,"207":1,"208":2,"210":1,"211":1,"218":1,"222":1,"226":1,"233":2,"237":1,"243":2,"246":1,"249":1,"251":1,"260":1}}],["one2two",{"2":{"254":3}}],["onecold",{"2":{"189":2,"224":2,"232":2}}],["onehotbatch",{"2":{"187":1,"190":1,"221":2,"229":1}}],["onehotarrays",{"2":{"186":1,"220":1,"228":1}}],["onesc64",{"2":{"10":1}}],["onesc32",{"2":{"10":1}}],["onesc16",{"2":{"10":1}}],["ones64",{"2":{"10":1}}],["ones32",{"2":{"10":1}}],["ones16",{"2":{"10":1}}],["ones",{"2":{"10":7,"13":1,"59":1,"115":1,"166":2,"172":1,"236":1,"243":2,"248":1,"249":1}}],["onearray",{"2":{"7":3}}],["oneapidevice",{"2":{"4":2}}],["oneapi",{"2":{"2":1,"3":2,"7":1,"62":2,"80":1,"81":1,"83":2,"124":2,"158":1}}],["one",{"2":{"1":1,"33":1,"53":1,"56":1,"65":2,"69":1,"72":2,"73":2,"133":1,"143":1,"152":2,"163":1,"166":1,"202":1,"203":1,"249":1,"254":4}}],["official",{"2":{"148":1,"175":1,"184":1,"192":1,"203":1,"204":1,"210":1,"218":1,"226":1,"237":1,"246":1,"247":2,"251":1,"260":1}}],["off",{"2":{"138":1}}],["offending",{"2":{"110":9}}],["ofcourse",{"2":{"109":1}}],["often",{"2":{"3":1,"70":1,"88":2,"114":1,"132":1,"138":1,"141":1,"149":1,"152":2,"166":2,"204":1,"249":1}}],["of",{"0":{"120":1,"122":1,"123":1,"137":1},"2":{"1":1,"3":7,"4":3,"9":31,"10":36,"11":1,"12":2,"13":17,"14":3,"15":6,"16":4,"19":2,"20":2,"21":1,"22":4,"23":1,"24":8,"27":6,"28":6,"30":2,"31":6,"32":11,"33":2,"34":9,"37":2,"38":1,"44":7,"46":3,"47":1,"48":1,"49":7,"52":11,"53":13,"54":1,"55":10,"56":4,"57":10,"58":8,"59":16,"61":1,"65":32,"66":38,"67":3,"68":42,"69":37,"70":32,"71":23,"72":30,"73":13,"74":3,"77":5,"79":1,"81":2,"85":2,"89":1,"90":3,"95":2,"96":1,"97":2,"98":2,"100":2,"102":6,"104":2,"105":1,"106":6,"107":1,"109":1,"110":6,"114":2,"118":1,"120":2,"121":1,"122":3,"127":3,"129":3,"130":4,"131":1,"132":3,"134":1,"135":6,"136":1,"137":1,"138":6,"143":4,"146":2,"149":1,"152":4,"153":1,"155":1,"156":1,"157":1,"160":4,"161":1,"162":1,"164":1,"166":9,"167":1,"168":1,"170":4,"172":1,"174":3,"176":1,"178":1,"179":3,"180":2,"181":1,"183":1,"191":1,"194":1,"202":2,"203":2,"204":1,"217":2,"222":2,"227":1,"230":5,"234":1,"235":1,"236":1,"238":3,"240":1,"243":1,"247":2,"248":2,"249":6,"250":3,"254":5,"255":2,"256":3}}],["bfgs",{"2":{"204":1,"208":1,"258":2}}],["bs",{"2":{"187":1,"229":1}}],["bh2",{"2":{"254":1}}],["bh",{"2":{"254":1}}],["bhn",{"2":{"69":2}}],["bhz",{"2":{"69":2}}],["bhr",{"2":{"69":2}}],["bright",{"2":{"250":3}}],["break",{"2":{"244":1}}],["breaking",{"0":{"92":1,"95":1,"99":1,"102":1,"103":1,"104":1},"2":{"97":1}}],["branched",{"2":{"65":1}}],["branchlayer",{"2":{"65":5}}],["broken=true",{"2":{"48":2}}],["broken=false",{"2":{"48":2}}],["broken",{"2":{"30":1,"48":5,"49":4,"50":1}}],["broadcastfunction",{"2":{"106":1}}],["broadcastable",{"2":{"72":1}}],["broadcast",{"2":{"39":1,"67":1,"71":1,"106":1,"249":1}}],["broadcasted",{"2":{"20":1,"21":2,"39":2,"53":1}}],["broadcasting",{"2":{"18":1,"25":1,"166":1}}],["b=f",{"2":{"194":1}}],["b=layer",{"2":{"194":1}}],["b=",{"2":{"59":1}}],["b=zeros",{"2":{"59":1}}],["bc",{"2":{"242":7,"243":21,"244":120}}],["bce",{"2":{"53":5}}],["bcast",{"2":{"39":2}}],["bigger",{"2":{"166":1}}],["bigfloat",{"2":{"166":3}}],["bit",{"2":{"136":1,"208":1}}],["bibtex",{"2":{"74":2}}],["bilinear",{"2":{"70":3,"73":6,"104":1}}],["bidirectional",{"2":{"69":1}}],["bidirectionalrnn",{"2":{"69":1}}],["biz",{"2":{"69":1}}],["bir",{"2":{"69":1}}],["bin",{"2":{"53":7,"69":1}}],["binarycrossentropy",{"2":{"181":2}}],["binarycrossentropyloss",{"2":{"53":7,"181":1}}],["binaryfocalloss",{"2":{"53":3}}],["binary",{"2":{"53":3,"178":1}}],["bias=l",{"2":{"129":1}}],["bias=false",{"2":{"69":6,"70":3}}],["bias=true",{"2":{"66":4,"69":2,"70":6}}],["bias=nothing",{"2":{"66":1,"69":3,"70":3}}],["bias=ps",{"2":{"59":1}}],["bias=zeros32",{"2":{"66":1,"70":1,"72":4,"129":1,"256":3}}],["bias=zero",{"2":{"32":1}}],["bias",{"0":{"20":1},"2":{"20":12,"21":3,"23":3,"24":15,"31":1,"32":4,"34":4,"59":1,"66":12,"69":25,"70":22,"72":20,"80":15,"105":4,"106":12,"110":3,"116":2,"120":4,"121":2,"122":2,"123":1,"129":6,"131":4,"134":3,"136":4,"137":4,"152":5,"160":1,"174":1,"216":2,"236":30,"249":1,"256":3,"258":3}}],["biases",{"2":{"9":1,"249":1}}],["bn=batchnorm",{"2":{"32":1}}],["black",{"2":{"213":1,"217":2,"248":2}}],["blas",{"2":{"13":1,"23":1}}],["blue",{"2":{"206":1,"209":2,"213":1,"248":1}}],["block",{"2":{"59":3,"65":1,"68":3,"179":1}}],["blocks",{"2":{"52":1,"69":1,"179":1}}],["blog",{"2":{"46":1}}],["blisblas",{"2":{"23":1}}],["b",{"2":{"21":4,"22":2,"23":3,"28":3,"59":6,"68":6,"70":4,"109":2,"125":1,"131":2,"135":8,"152":2,"174":8,"194":9,"244":8,"254":4}}],["bulk",{"2":{"249":1}}],["builds",{"2":{"109":1,"116":3,"134":2,"236":3,"244":4}}],["buildkite",{"2":{"109":1,"116":3,"134":2,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"236":3,"237":1,"244":4,"246":1,"251":1,"260":1}}],["build",{"2":{"80":1,"157":3,"165":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["building",{"0":{"249":1},"2":{"52":1}}],["built",{"0":{"64":1},"1":{"65":1,"66":1,"67":1,"68":1,"69":1,"70":1,"71":1,"72":1,"73":1},"2":{"75":1,"127":2,"168":1,"191":1}}],["buffer",{"2":{"21":1,"23":1,"39":3}}],["bugs",{"2":{"84":1,"98":1,"131":1}}],["bug",{"2":{"16":1,"59":1,"164":1,"202":1}}],["but",{"2":{"3":2,"9":1,"12":2,"13":1,"18":1,"19":1,"20":1,"24":1,"27":2,"44":1,"46":3,"54":3,"55":5,"57":2,"59":4,"71":1,"73":2,"76":1,"77":2,"85":1,"88":3,"89":2,"95":1,"106":1,"109":3,"110":2,"124":1,"129":2,"130":1,"131":1,"134":3,"135":1,"136":1,"141":1,"142":1,"149":2,"154":1,"161":1,"164":1,"174":1,"179":2,"181":1,"207":1,"214":1,"233":1,"235":1,"243":1,"249":1,"256":1}}],["box",{"2":{"248":1}}],["boxing",{"2":{"58":1,"234":1,"236":1}}],["borrow",{"2":{"247":1}}],["borders",{"2":{"66":2,"68":3}}],["book",{"2":{"172":1}}],["boom",{"2":{"136":1}}],["bool=true",{"2":{"69":1}}],["bool=false",{"2":{"2":2,"44":2,"46":1,"48":1,"69":4,"231":1,"233":1}}],["boolean",{"2":{"13":1,"46":1,"50":1}}],["bool",{"2":{"2":1,"3":5,"13":3,"33":1,"46":1,"53":2,"54":1,"55":1,"236":26}}],["boilerplate",{"2":{"106":1,"107":1,"180":1}}],["boundaries",{"2":{"250":1}}],["boundary",{"2":{"242":1}}],["bound",{"2":{"66":6,"69":18,"70":9}}],["bounds",{"2":{"66":2,"69":6,"70":3}}],["bottom",{"2":{"59":1,"106":1}}],["both",{"2":{"2":1,"33":2,"53":2,"75":1,"80":1,"105":1,"110":3,"157":1,"160":1,"194":1,"203":1,"249":1}}],["body",{"2":{"59":2,"236":3,"254":9}}],["bypasses",{"2":{"59":2}}],["bypass",{"2":{"16":1,"130":1,"158":1}}],["by",{"0":{"121":1},"2":{"2":1,"9":5,"12":1,"13":2,"16":1,"18":2,"21":1,"23":1,"24":1,"46":1,"52":3,"53":2,"54":1,"56":1,"59":2,"65":2,"66":5,"68":3,"69":5,"70":5,"71":1,"72":1,"73":1,"77":2,"79":1,"80":1,"95":1,"102":1,"106":2,"110":1,"121":3,"122":2,"125":1,"126":1,"127":1,"133":3,"134":2,"137":1,"138":1,"143":3,"149":1,"150":1,"151":1,"157":1,"161":1,"166":2,"167":2,"168":1,"170":1,"176":1,"188":1,"203":1,"207":1,"216":2,"236":1,"247":1,"249":1,"250":1}}],["bayes",{"2":{"249":3}}],["bayesian",{"0":{"247":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"248":1,"250":1}}],["battery",{"2":{"77":1}}],["batches",{"2":{"187":1,"229":1}}],["batched",{"0":{"19":1,"28":1,"135":1},"2":{"16":1,"19":6,"28":1,"133":1,"135":8,"139":1,"140":1,"152":2}}],["batchsize=32",{"2":{"244":2}}],["batchsize=256",{"2":{"221":1}}],["batchsize=min",{"2":{"221":2}}],["batchsize=8",{"2":{"207":2}}],["batchsize=128",{"2":{"178":2}}],["batchsize=12",{"2":{"154":2}}],["batchsize=13",{"2":{"5":1}}],["batchsize",{"2":{"187":3,"221":4,"229":3}}],["batching",{"2":{"164":3}}],["batchlastindex",{"2":{"54":1,"69":2}}],["batchnorm",{"2":{"12":1,"24":1,"65":4,"72":8,"109":4,"110":4,"120":1,"123":1,"129":1,"132":1,"134":2,"135":1,"193":2,"196":1}}],["batch",{"2":{"5":1,"24":6,"28":3,"46":1,"66":3,"68":3,"69":13,"71":1,"72":6,"73":3,"92":1,"135":1,"164":6,"178":1,"207":1,"208":9,"244":8}}],["basis",{"2":{"66":2,"70":1}}],["basically",{"2":{"130":1}}],["basic",{"2":{"52":2,"165":1}}],["baseline",{"2":{"53":1}}],["base",{"2":{"48":3,"54":3,"67":1,"71":1,"102":1,"116":1,"131":1,"133":3,"136":1,"137":2,"204":1,"207":2,"231":1,"236":25,"256":2}}],["based",{"2":{"2":1,"3":2,"23":1,"24":1,"55":1,"65":1,"84":1,"104":2,"105":1,"134":1,"138":1,"165":1,"204":1}}],["bad",{"2":{"33":1,"149":1,"250":1}}],["ba",{"2":{"24":1}}],["backtracking",{"2":{"258":1}}],["background",{"2":{"129":1,"149":1,"255":1}}],["backward",{"2":{"33":2,"54":1,"69":11,"110":3,"111":1,"168":1}}],["back",{"2":{"2":1,"3":1,"18":1,"131":1}}],["backendtpu",{"2":{"63":1}}],["backendgpu",{"2":{"63":1}}],["backends=",{"2":{"49":2}}],["backends",{"0":{"36":1},"2":{"1":1,"2":4,"3":5,"5":1,"21":3,"23":4,"25":1,"27":6,"28":2,"49":7,"52":5,"76":2,"88":3,"100":1,"124":3,"154":1,"156":1,"164":1,"167":1,"172":1,"233":1}}],["backend",{"0":{"45":1,"125":1,"126":1,"158":1},"1":{"46":1},"2":{"1":17,"2":3,"3":1,"17":1,"27":6,"28":3,"36":5,"37":20,"38":4,"39":11,"40":1,"41":1,"49":1,"52":4,"63":4,"93":1,"125":5,"133":1,"143":14,"144":2,"146":1,"158":4,"164":6,"170":1,"204":1}}],["bernoulli",{"2":{"249":1}}],["became",{"2":{"149":1}}],["because",{"2":{"131":1,"258":1}}],["become",{"2":{"129":1,"176":1}}],["benchmarking",{"2":{"191":1}}],["benefits",{"2":{"106":1}}],["bengio",{"2":{"9":2,"71":1}}],["behind",{"2":{"95":1}}],["behaving",{"2":{"163":1}}],["behaviour",{"2":{"95":1}}],["behavior",{"2":{"3":2,"9":1,"12":2,"13":1,"16":2,"57":3,"69":1,"134":2,"149":1,"179":1}}],["behaves",{"2":{"65":1,"96":1,"154":1}}],["behave",{"2":{"54":1}}],["better",{"2":{"77":1,"84":1}}],["between",{"0":{"42":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"9":1,"34":1,"44":1,"53":1,"70":1,"77":1,"106":1,"146":1,"194":3,"250":3}}],["beginner",{"0":{"199":1}}],["begin",{"2":{"66":2,"68":3,"206":1,"209":1,"243":1,"245":1,"255":1,"256":1,"259":1}}],["beyond",{"2":{"65":1}}],["been",{"2":{"44":2,"77":1,"88":1,"92":2,"95":6,"99":3,"102":10,"103":5,"105":1,"137":1,"145":1,"165":1,"252":1}}],["below",{"2":{"32":1,"57":2,"65":1,"74":1,"230":1,"248":1,"249":2,"250":1}}],["being",{"2":{"20":1,"22":2,"24":2,"55":1,"72":4,"87":2,"93":1,"100":1,"133":1,"134":2,"147":1,"149":1,"191":1,"208":1}}],["best",{"2":{"18":2,"21":1,"23":1,"44":1,"59":2,"87":2,"88":1}}],["beware",{"2":{"10":1,"131":1}}],["before",{"2":{"9":2,"13":2,"37":1,"40":1,"41":1,"65":1,"81":1,"107":1,"129":1,"143":1,"169":1,"183":1}}],["be",{"2":{"1":2,"2":1,"3":10,"4":5,"9":8,"11":1,"12":4,"13":2,"15":1,"18":2,"19":1,"20":3,"21":1,"22":4,"23":1,"24":20,"25":1,"28":5,"31":2,"33":3,"34":5,"37":7,"39":2,"41":2,"44":5,"46":5,"48":3,"49":3,"50":4,"52":9,"53":5,"55":3,"58":2,"59":16,"60":2,"61":1,"65":13,"66":16,"67":4,"68":9,"69":17,"70":8,"71":8,"72":4,"73":3,"74":1,"75":1,"76":3,"77":2,"80":1,"84":1,"90":1,"98":1,"99":1,"102":1,"103":1,"106":3,"108":1,"111":1,"115":1,"119":1,"121":1,"124":2,"126":1,"127":1,"129":1,"131":2,"132":1,"133":2,"134":5,"135":2,"137":1,"139":1,"143":2,"144":1,"149":1,"153":1,"158":1,"160":3,"161":2,"163":3,"166":1,"168":1,"169":1,"170":1,"171":1,"176":1,"179":1,"191":2,"193":1,"194":6,"196":1,"202":1,"203":1,"208":1,"214":1,"223":1,"234":1,"238":2,"243":1,"248":1,"249":1,"254":4,"257":1}}],["ixy",{"2":{"254":2}}],["ixx",{"2":{"254":3}}],["iyy",{"2":{"254":3}}],["i∈",{"2":{"174":1}}],["iid",{"2":{"249":1}}],["iii",{"2":{"86":2,"88":1}}],["ii",{"2":{"86":2,"88":1}}],["ii+pi+p",{"2":{"66":1,"68":3}}],["i=σ",{"2":{"69":1}}],["ih",{"2":{"69":6,"105":1}}],["i+n",{"2":{"66":1,"68":3}}],["ignore",{"2":{"222":1,"256":1}}],["ignores",{"2":{"44":1}}],["ignored",{"2":{"2":1,"9":1,"70":3}}],["ioffe",{"2":{"24":1}}],["imgs",{"2":{"187":5,"221":4,"229":5}}],["im",{"0":{"168":1}}],["immutability",{"2":{"169":1}}],["immutable",{"2":{"18":1,"21":1,"23":1,"55":1,"76":1,"77":1,"168":1}}],["immediately",{"2":{"106":1}}],["imbalanced",{"2":{"53":1}}],["images",{"2":{"72":3,"73":2,"187":1,"229":1}}],["image",{"2":{"53":1,"66":2}}],["imagenet",{"2":{"9":2,"142":1}}],["imply",{"2":{"132":1}}],["implements",{"2":{"47":1}}],["implemented",{"2":{"44":1,"77":1,"133":1,"194":1}}],["implementations",{"2":{"18":1,"19":1,"23":1,"25":1,"52":1,"55":1,"146":1,"161":1}}],["implementation",{"0":{"115":1,"195":1,"234":1},"1":{"196":1},"2":{"16":4,"18":3,"21":4,"23":6,"31":1,"52":1,"55":5,"58":2,"65":1,"69":1,"95":1,"138":1,"146":1,"230":1}}],["implement",{"0":{"222":1},"2":{"12":1,"31":1,"129":2,"194":1,"247":1}}],["implementing",{"0":{"194":1},"2":{"12":2,"77":1,"114":1,"227":1,"247":1}}],["improving",{"2":{"89":1}}],["imports",{"0":{"177":1,"186":1,"205":1,"212":1,"220":1,"228":1,"239":1,"253":1}}],["importing",{"2":{"129":1,"165":1,"247":1}}],["important",{"0":{"195":1},"1":{"196":1},"2":{"124":1,"129":1,"134":1,"135":1,"143":2,"194":1}}],["importantly",{"2":{"107":1,"166":1,"250":1}}],["imported",{"2":{"59":1}}],["import",{"2":{"44":1,"46":1,"82":1,"247":1}}],["iclr",{"2":{"9":1}}],["ieee",{"2":{"9":2,"53":5}}],["i",{"2":{"5":10,"9":1,"16":1,"31":1,"44":1,"46":1,"52":2,"53":3,"57":1,"58":1,"59":1,"65":9,"66":6,"68":21,"70":4,"71":8,"73":2,"81":1,"86":4,"88":1,"95":2,"115":3,"132":1,"136":1,"143":1,"164":6,"169":4,"172":1,"174":4,"207":2,"244":3,"245":6,"248":12,"249":7,"250":10}}],["ith",{"2":{"65":1}}],["its",{"2":{"13":1,"61":1,"68":3,"70":1,"71":1,"72":1,"77":1,"250":1}}],["itself",{"2":{"3":1,"71":1,"109":1}}],["iter",{"2":{"81":4,"107":12,"208":2,"244":7,"257":1}}],["iteratively",{"2":{"65":1}}],["iterations",{"2":{"174":12,"249":2}}],["iteration",{"0":{"5":1},"2":{"5":2,"81":12,"107":4,"114":1,"154":1,"169":8,"207":1,"208":26,"244":101,"250":1,"257":1,"259":1}}],["iterate",{"2":{"5":1,"30":1}}],["iterates",{"2":{"5":1}}],["iterators",{"2":{"5":1,"179":1,"180":1,"243":1,"244":2,"245":1}}],["iterator",{"2":{"5":3,"107":1}}],["item",{"2":{"5":1}}],["it",{"2":{"2":1,"3":2,"4":5,"9":3,"12":3,"13":6,"16":2,"18":3,"20":2,"21":2,"22":1,"30":1,"31":1,"32":1,"34":1,"44":3,"47":2,"50":2,"52":1,"53":5,"54":1,"55":3,"57":3,"58":1,"59":5,"65":2,"66":4,"68":3,"69":8,"71":1,"72":7,"73":3,"74":1,"75":1,"77":4,"79":1,"81":1,"85":1,"89":3,"95":1,"103":1,"105":1,"106":5,"108":2,"109":1,"110":3,"113":1,"125":1,"127":2,"129":2,"130":5,"131":4,"133":3,"134":6,"135":2,"141":1,"143":2,"144":1,"149":1,"150":2,"151":1,"154":1,"158":1,"161":1,"164":2,"165":1,"166":2,"167":1,"168":2,"169":2,"172":1,"174":1,"179":4,"181":2,"202":1,"203":1,"204":1,"207":2,"208":2,"222":2,"230":1,"249":2,"254":1,"256":2}}],["inlined",{"2":{"244":4}}],["in2",{"2":{"70":6}}],["in12",{"2":{"70":3}}],["in1",{"2":{"70":8}}],["injection",{"2":{"65":5}}],["independent",{"2":{"135":1}}],["index",{"2":{"65":1,"70":1,"71":2,"164":1,"250":1}}],["indexing",{"0":{"150":1},"2":{"25":1,"65":1,"71":1,"150":1}}],["indexed",{"2":{"2":2,"4":2,"71":1,"84":1}}],["individual",{"2":{"71":1,"208":1}}],["individually",{"2":{"65":1,"73":2,"135":1}}],["indices",{"2":{"70":2}}],["inplace",{"2":{"52":2}}],["inputsize",{"2":{"69":1,"95":1}}],["inputs",{"0":{"135":1},"2":{"13":2,"16":1,"21":3,"23":1,"28":1,"32":1,"33":1,"52":1,"53":4,"55":1,"58":1,"65":9,"66":2,"67":3,"68":9,"69":11,"70":1,"71":6,"72":5,"73":2,"76":1,"103":1,"106":1,"133":1,"135":2,"136":1,"137":1,"146":1,"173":1,"216":1,"230":2,"249":1,"256":1}}],["input",{"0":{"112":1,"118":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1},"2":{"9":4,"13":2,"18":2,"20":1,"21":1,"22":3,"23":2,"24":9,"27":2,"28":1,"33":2,"46":3,"53":4,"58":1,"59":1,"65":30,"66":7,"67":2,"68":6,"69":20,"70":12,"71":7,"72":14,"73":6,"80":1,"106":1,"109":6,"110":14,"114":1,"129":1,"132":2,"135":3,"137":1,"143":1,"144":1,"146":1,"149":2,"159":1,"160":1,"163":1,"164":8,"179":1,"188":1,"230":1,"241":1,"256":1}}],["inner",{"2":{"31":2,"33":1,"121":1}}],["incase",{"2":{"256":1}}],["including",{"2":{"98":1,"106":1}}],["included",{"2":{"9":1,"77":1}}],["include",{"2":{"9":1,"12":1,"13":1}}],["inconvenient",{"2":{"113":1}}],["inconsistent",{"2":{"95":1}}],["incorrectly",{"2":{"149":1}}],["incorrect",{"0":{"109":1},"2":{"28":2,"59":1,"108":1,"111":1}}],["increase",{"2":{"66":1,"105":1}}],["ingredient",{"2":{"24":1,"72":1}}],["investigate",{"2":{"235":1}}],["investigated",{"2":{"147":1}}],["inversability",{"2":{"66":1}}],["inverse",{"2":{"22":1}}],["inv",{"2":{"66":2,"69":6,"70":3,"254":1}}],["involving",{"2":{"58":1}}],["invokes",{"2":{"32":1}}],["invoked",{"2":{"2":1,"179":1}}],["invariant",{"2":{"53":1}}],["invp",{"2":{"22":4}}],["informed",{"2":{"242":3}}],["informs",{"2":{"130":1}}],["information",{"2":{"13":1,"16":1,"22":1,"29":1,"33":1,"49":1,"57":1,"76":1,"77":1,"146":1,"151":1,"203":1,"204":1}}],["info",{"2":{"109":8,"110":20,"124":1,"126":1,"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2,"247":2,"251":2,"260":2}}],["inf",{"2":{"53":1,"134":2,"135":2,"136":2,"137":2,"141":4}}],["inference",{"0":{"196":1},"2":{"33":1,"67":3,"72":6,"74":1,"134":2,"164":2,"196":1,"249":2}}],["inferred",{"2":{"24":1}}],["infinity",{"2":{"22":1}}],["int=50000",{"2":{"244":1}}],["int=32",{"2":{"241":1,"244":1}}],["int=0",{"2":{"39":5,"244":1}}],["intro",{"2":{"165":1}}],["introduction",{"2":{"96":1}}],["introductory",{"2":{"84":1}}],["introducing",{"2":{"44":1}}],["introduces",{"2":{"165":1}}],["introduce",{"2":{"16":1,"58":1,"110":2}}],["introduced",{"2":{"9":1}}],["int64",{"2":{"53":3,"80":16,"106":6,"116":4,"166":3,"168":2,"207":2,"217":4,"236":99}}],["int",{"2":{"24":1,"66":2,"69":2,"71":1,"72":1,"73":1,"129":4,"174":1,"221":3,"230":2,"248":1}}],["into",{"0":{"143":1},"2":{"9":2,"13":1,"21":1,"33":1,"39":1,"46":1,"53":1,"54":1,"56":4,"59":2,"65":2,"66":2,"68":3,"69":4,"71":1,"73":1,"80":1,"95":1,"106":1,"127":1,"129":1,"164":2,"166":2,"179":1,"187":1,"229":1}}],["intentionally",{"2":{"193":1}}],["integral",{"2":{"53":1}}],["integrating",{"0":{"143":1}}],["integration",{"0":{"40":1,"41":1},"2":{"144":1}}],["integrated",{"2":{"13":1,"151":1}}],["integers",{"2":{"37":1,"66":11,"68":15,"70":5}}],["integer",{"2":{"2":3,"4":3,"9":11,"57":3,"66":12,"68":9,"70":1,"71":1,"72":5}}],["interpolatingadjoint",{"2":{"236":13}}],["intermediate",{"0":{"200":1},"2":{"244":4}}],["interactiveutils",{"2":{"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"228":1,"237":2,"246":2,"251":2,"260":2}}],["interactive",{"2":{"155":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["interested",{"2":{"135":1,"250":1}}],["interoperability",{"0":{"42":1},"1":{"43":1,"44":1,"45":1,"46":1}}],["inter",{"2":{"28":1}}],["internals",{"2":{"222":1,"249":1}}],["internally",{"2":{"44":1,"58":1,"95":1,"133":1,"164":1,"236":1}}],["internal",{"2":{"24":2,"25":3,"31":1,"46":1,"52":1,"58":2,"65":1,"71":2,"75":1,"77":1,"102":1,"143":1}}],["international",{"2":{"9":6,"24":1,"53":3}}],["interfacem",{"2":{"46":1}}],["interface",{"0":{"127":1,"128":1,"131":1,"132":1},"1":{"128":1,"129":2,"130":2,"131":1,"132":1},"2":{"12":1,"41":2,"46":2,"127":2,"129":1,"204":1,"230":1,"249":1}}],["interval",{"2":{"9":2}}],["intelligence",{"2":{"9":2}}],["intel",{"2":{"3":1}}],["initilly",{"2":{"67":1}}],["initial",{"2":{"14":1,"15":1,"59":2,"65":1,"69":11,"77":1,"95":1,"179":1,"194":1,"255":1,"256":1,"258":2}}],["initialstates",{"2":{"12":3,"13":1,"15":1,"129":2,"179":1,"194":2}}],["initialparameters",{"2":{"12":3,"13":1,"14":1,"129":2,"179":1,"194":2,"222":3}}],["initializing",{"0":{"162":1},"1":{"163":1},"2":{"59":1,"143":1}}],["initialize",{"0":{"223":1,"231":1},"2":{"37":5,"59":3,"80":1,"81":1,"143":3,"144":1,"146":1,"164":1,"174":1,"213":1,"249":1}}],["initialized",{"2":{"9":4,"37":2,"72":8}}],["initializers",{"2":{"9":1}}],["initializer",{"2":{"9":1,"59":3,"69":10,"70":7}}],["initializations",{"2":{"104":2}}],["initialization",{"0":{"37":1},"2":{"6":1,"9":2,"37":1,"46":1,"66":4,"104":1,"162":1,"179":1,"194":1,"222":1,"256":1}}],["init",{"2":{"9":7,"54":6,"59":6,"66":10,"69":26,"70":21,"72":16,"104":2,"105":1,"129":10,"144":1,"163":8,"179":2,"180":2,"194":4,"256":6}}],["inside",{"0":{"164":1},"2":{"32":1,"57":1,"59":1,"121":1,"124":1,"133":3,"164":3,"194":3}}],["instructions",{"2":{"79":1}}],["institute",{"2":{"74":1}}],["installation",{"0":{"79":1},"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["install",{"0":{"61":1},"2":{"61":1,"62":1,"63":1,"79":1,"80":1,"82":1}}],["installed",{"2":{"37":4,"41":1,"143":1,"167":1}}],["instability",{"2":{"58":1,"151":1,"160":1}}],["instabilities",{"0":{"151":1},"2":{"44":1,"106":1,"151":2}}],["instance",{"2":{"24":3,"69":1,"72":3,"249":1}}],["instancenorm",{"2":{"24":2,"72":7,"104":1,"105":1}}],["instead",{"2":{"3":2,"4":2,"13":1,"31":2,"36":2,"44":1,"48":1,"52":1,"57":1,"59":1,"66":3,"98":1,"99":1,"100":1,"102":1,"103":4,"104":2,"113":1,"114":1,"122":1,"127":1,"130":1,"131":1,"136":1,"143":1,"149":1,"152":1,"154":1,"161":1,"164":2,"166":1,"179":1,"194":2,"227":1,"244":4,"247":1}}],["inspiration",{"2":{"5":1}}],["in",{"0":{"64":1,"104":1,"113":1},"1":{"65":1,"66":1,"67":1,"68":1,"69":1,"70":1,"71":1,"72":1,"73":1},"2":{"2":2,"3":1,"4":2,"5":2,"9":15,"12":4,"13":5,"15":3,"18":3,"20":2,"22":3,"24":2,"27":2,"30":1,"31":2,"32":3,"33":3,"34":6,"37":1,"39":2,"40":1,"44":5,"46":1,"47":1,"48":1,"52":7,"53":3,"54":4,"55":7,"57":1,"58":4,"59":18,"61":3,"65":20,"66":22,"67":2,"68":3,"69":23,"70":13,"71":6,"72":10,"74":2,"75":2,"76":1,"77":3,"79":1,"81":7,"82":1,"84":2,"85":2,"87":1,"89":1,"90":1,"92":1,"95":1,"104":1,"105":1,"107":4,"108":1,"109":1,"110":11,"111":3,"114":1,"115":1,"119":2,"120":1,"121":1,"124":2,"125":1,"129":10,"132":1,"133":3,"134":3,"135":8,"136":1,"137":1,"138":2,"141":1,"143":4,"146":2,"147":1,"149":2,"150":2,"151":1,"154":5,"155":2,"161":1,"162":1,"163":1,"164":5,"165":2,"166":2,"168":1,"169":3,"170":1,"174":2,"176":1,"178":3,"179":6,"180":3,"182":3,"185":1,"189":1,"190":2,"193":2,"194":1,"197":2,"202":1,"204":2,"207":1,"208":4,"211":1,"217":1,"222":1,"224":1,"225":4,"232":1,"233":2,"235":1,"238":1,"243":1,"244":1,"245":2,"247":1,"248":6,"249":2,"250":13,"255":1,"256":3}}],["idx",{"2":{"224":2,"225":12}}],["ideal",{"2":{"191":1}}],["ideally",{"2":{"25":1,"203":1}}],["identity",{"2":{"9":13,"21":1,"24":5,"65":1,"80":3,"106":1,"116":2,"217":1,"236":9}}],["id",{"2":{"2":7,"4":3}}],["if",{"2":{"1":2,"2":12,"3":9,"4":3,"11":1,"12":1,"13":5,"16":2,"18":4,"20":2,"21":2,"22":7,"23":2,"24":6,"28":2,"31":3,"32":1,"33":5,"34":2,"37":2,"44":2,"46":1,"48":1,"49":2,"50":3,"52":2,"53":8,"54":4,"55":3,"57":1,"58":3,"59":6,"60":1,"61":1,"65":17,"66":8,"67":10,"69":28,"70":12,"71":5,"72":14,"73":3,"74":2,"77":1,"79":1,"80":1,"81":2,"87":2,"88":1,"95":2,"102":1,"106":3,"107":1,"109":1,"110":1,"111":1,"120":1,"121":1,"124":1,"125":4,"126":1,"127":1,"129":3,"130":3,"131":3,"133":3,"134":2,"137":1,"146":1,"147":1,"153":1,"157":3,"161":1,"162":2,"163":2,"164":1,"167":2,"169":1,"170":1,"174":1,"175":3,"184":3,"192":3,"196":1,"197":1,"202":1,"203":2,"208":2,"210":3,"217":1,"218":3,"226":3,"237":3,"244":1,"246":3,"251":3,"254":3,"260":3}}],["isdefined",{"2":{"175":3,"184":3,"192":3,"210":3,"218":3,"226":3,"237":3,"246":3,"251":3,"260":3}}],["isnan",{"2":{"244":1}}],["isn",{"2":{"95":1,"197":1,"254":1}}],["issuing",{"2":{"57":1}}],["issue",{"2":{"57":1,"87":1,"106":1,"111":1,"133":1,"134":2,"137":1,"147":1,"153":1,"202":1,"203":2}}],["issues",{"2":{"52":1,"59":1,"69":1,"77":1,"84":1,"88":2,"89":1,"133":1,"164":1}}],["isbitstype",{"2":{"55":1}}],["istraining",{"2":{"54":4}}],["iszero",{"2":{"32":1}}],["isa",{"2":{"9":1,"32":1,"59":1}}],["isleaf",{"2":{"3":5,"15":1}}],["is",{"2":{"0":1,"1":4,"2":19,"3":18,"4":10,"5":3,"6":1,"9":9,"11":1,"12":6,"13":12,"15":2,"16":4,"18":2,"19":1,"20":3,"21":4,"22":6,"23":2,"24":7,"25":4,"27":4,"31":5,"32":4,"33":6,"37":2,"41":2,"44":3,"46":7,"47":2,"50":2,"52":4,"53":36,"54":5,"55":9,"57":5,"58":3,"59":9,"60":2,"61":1,"65":23,"66":3,"67":7,"68":3,"69":38,"70":9,"71":10,"72":18,"73":3,"77":6,"79":1,"85":1,"87":2,"89":2,"90":1,"93":1,"95":1,"96":1,"98":1,"102":3,"104":1,"105":1,"106":4,"108":2,"109":4,"110":5,"111":1,"113":1,"121":2,"124":1,"125":9,"126":1,"127":4,"129":3,"130":6,"131":2,"133":3,"134":12,"135":2,"138":1,"139":1,"141":1,"142":1,"143":4,"144":1,"145":1,"146":2,"147":2,"149":2,"150":2,"151":1,"152":2,"154":3,"155":1,"157":2,"158":1,"163":3,"164":3,"165":1,"166":4,"167":3,"168":4,"170":3,"174":1,"178":1,"179":1,"180":1,"181":2,"185":1,"191":2,"194":6,"196":3,"204":2,"208":4,"216":1,"217":1,"222":1,"230":1,"233":2,"236":1,"238":1,"244":4,"247":3,"248":1,"249":8,"250":2,"252":1,"254":1,"256":2}}],["fθ",{"2":{"250":1}}],["fw",{"2":{"174":1}}],["fwiw",{"2":{"130":1}}],["fd",{"2":{"134":8,"135":8,"136":8,"137":8}}],["f2",{"2":{"129":2}}],["f=σ",{"2":{"69":1}}],["f=identity",{"2":{"24":1}}],["ft",{"2":{"58":3}}],["f64",{"2":{"56":1,"256":1}}],["f32",{"2":{"56":1}}],["f16",{"2":{"56":1}}],["f1",{"2":{"53":1,"129":2}}],["fn=neuralode",{"2":{"231":1}}],["fn",{"2":{"52":2,"53":2,"59":8,"231":1}}],["fp32",{"2":{"18":1}}],["f",{"2":{"18":4,"21":2,"23":2,"27":5,"28":7,"32":3,"48":2,"49":6,"52":2,"55":4,"59":1,"65":4,"71":6,"163":1,"171":3,"172":1,"173":1,"174":1,"194":1,"208":1}}],["flexibility",{"2":{"131":1}}],["flexible",{"2":{"12":1}}],["flow",{"2":{"106":2}}],["floating",{"0":{"56":1},"2":{"18":1,"56":3}}],["float16",{"2":{"10":8,"56":1,"57":6}}],["float32",{"2":{"5":3,"9":6,"10":8,"44":1,"46":1,"53":6,"56":2,"57":5,"65":2,"71":2,"80":57,"81":1,"106":28,"107":1,"109":9,"110":27,"116":6,"117":3,"118":3,"120":10,"123":6,"125":3,"126":1,"129":2,"130":2,"131":3,"134":10,"135":2,"136":11,"137":11,"141":2,"149":5,"162":5,"166":8,"167":1,"171":5,"172":4,"173":1,"174":8,"178":1,"187":1,"190":1,"193":2,"207":4,"213":3,"216":4,"229":1,"236":77,"243":5,"244":1,"248":8,"256":7}}],["float64",{"2":{"5":4,"10":8,"53":1,"56":1,"57":4,"59":4,"149":3,"164":7,"166":5,"169":1,"207":4,"249":13,"255":1,"256":1,"257":1,"258":2}}],["flipkernel=true",{"2":{"105":1}}],["flat",{"2":{"233":1}}],["flattening",{"2":{"102":1}}],["flatten",{"2":{"71":1}}],["flattens",{"2":{"71":1}}],["flattenlayer",{"2":{"46":1,"71":3,"188":3,"223":1,"231":1,"236":9}}],["flattened",{"2":{"44":1,"71":3,"247":1}}],["flaky",{"2":{"145":1}}],["flag",{"2":{"46":1}}],["fluxlinear",{"2":{"194":5}}],["fluxlayer",{"2":{"44":2}}],["fluxmpifluxmodel",{"2":{"145":1}}],["fluxmpi",{"0":{"144":1},"1":{"145":1,"146":1},"2":{"142":1,"144":3,"145":1,"146":2}}],["flux",{"0":{"44":1,"77":1,"193":1,"197":1},"1":{"194":1,"195":1,"196":1,"197":1},"2":{"44":14,"69":3,"75":1,"77":4,"129":2,"145":1,"165":1,"193":3,"194":5,"196":4,"197":1,"247":1,"252":1}}],["fmaps",{"2":{"13":1}}],["fmap",{"2":{"12":3,"13":2,"15":1,"32":1,"55":7,"249":1}}],["func",{"2":{"208":3}}],["functor",{"2":{"55":2,"56":3,"194":2}}],["functors",{"2":{"3":1,"5":1,"12":2,"15":3,"32":2,"55":2,"56":3,"95":1,"100":1,"103":1,"106":1,"120":1,"194":1,"247":1}}],["function3",{"2":{"137":4}}],["function2",{"2":{"136":4}}],["function1",{"2":{"134":5}}],["functions",{"0":{"9":1,"10":1,"25":1,"38":1,"53":1,"189":1,"224":1,"232":1,"242":1,"254":1},"2":{"7":1,"10":1,"12":1,"18":1,"44":1,"46":1,"52":1,"53":2,"54":1,"56":1,"65":1,"69":4,"75":1,"76":2,"77":1,"85":1,"95":1,"102":2,"106":1,"107":1,"124":2,"125":1,"129":3,"133":1,"137":1,"146":1,"160":1,"161":1,"166":1,"168":1,"172":2,"194":1,"216":1,"222":1,"254":1}}],["function",{"0":{"113":1,"134":1,"136":1,"137":1,"216":1},"1":{"135":1},"2":{"3":1,"4":4,"9":2,"13":4,"15":2,"16":3,"18":4,"19":1,"20":2,"21":1,"22":2,"23":1,"24":7,"27":4,"28":3,"31":1,"32":4,"36":2,"37":1,"46":1,"48":1,"49":1,"52":15,"53":2,"55":1,"57":3,"58":1,"59":11,"65":2,"66":4,"69":3,"70":4,"71":5,"73":1,"77":1,"81":3,"95":1,"102":2,"104":3,"106":9,"107":1,"110":2,"115":1,"117":1,"120":1,"121":3,"122":2,"125":3,"129":3,"130":1,"131":1,"133":5,"134":1,"135":6,"137":2,"139":1,"140":1,"141":1,"143":1,"163":1,"164":2,"167":2,"169":1,"171":1,"173":1,"174":1,"178":1,"180":1,"181":2,"187":1,"189":2,"190":1,"194":2,"197":1,"205":1,"208":4,"213":2,"216":3,"217":2,"221":2,"222":1,"223":1,"224":2,"229":1,"230":4,"231":1,"232":2,"233":2,"234":2,"238":1,"239":1,"241":2,"242":14,"244":1,"248":1,"249":5,"250":3,"254":16,"256":2,"257":7}}],["functionalities",{"2":{"33":1,"35":1}}],["functionality",{"0":{"33":1,"102":1,"103":1,"145":1},"2":{"11":1,"52":1,"69":1,"82":1,"95":2,"102":1,"127":2,"167":1}}],["functional",{"2":{"2":5,"3":5,"37":1,"126":2,"142":1,"160":1,"167":1,"175":2,"184":2,"192":2,"203":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":2,"260":2}}],["full",{"0":{"141":1},"2":{"135":1,"141":16}}],["fully",{"0":{"23":1},"2":{"9":1,"53":1,"55":1,"70":2,"88":1,"142":1}}],["future",{"2":{"133":1}}],["further",{"2":{"52":1}}],["fusion",{"2":{"21":1,"23":1}}],["fuse",{"2":{"21":1}}],["fuses",{"2":{"21":1}}],["fused",{"2":{"21":2,"23":2,"53":1,"152":3,"160":1}}],["fetch",{"2":{"131":1,"213":1}}],["feel",{"2":{"129":1}}],["feedforward",{"2":{"9":2,"249":1}}],["few",{"2":{"75":1,"166":1,"170":1}}],["fed",{"2":{"69":4}}],["features",{"0":{"30":1,"93":1,"96":1,"100":1,"105":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"30":3,"47":1,"76":1,"90":1,"102":1,"133":1,"147":2,"187":1,"229":1}}],["feature",{"2":{"2":1,"9":2,"58":1,"66":1,"68":6,"72":3,"89":1,"106":1,"111":1,"133":2,"164":2,"166":1}}],["framerate=10",{"2":{"245":1}}],["framework",{"2":{"75":1,"77":1,"127":1,"165":1,"168":1,"185":1}}],["frameworks",{"0":{"43":1},"1":{"44":1},"2":{"66":1,"69":1,"75":3,"127":1,"166":1,"256":1}}],["frequently",{"2":{"202":1}}],["frees",{"2":{"154":1}}],["freeze",{"2":{"31":5,"119":1,"120":6,"121":6,"122":5,"123":1}}],["freezing",{"0":{"31":1,"119":1,"120":1,"121":1,"122":1,"123":1},"1":{"120":1,"121":1,"122":1,"123":1},"2":{"119":1,"121":1,"122":2}}],["free",{"2":{"5":1,"9":1,"52":1,"254":1}}],["freeable",{"2":{"5":1}}],["friendly",{"2":{"75":2}}],["frozen",{"2":{"31":8,"120":9,"123":4}}],["frozenlayer",{"2":{"31":8}}],["fromfluxadaptor",{"2":{"44":5,"197":1}}],["from",{"0":{"43":1,"144":1,"193":1},"1":{"44":1,"145":1,"146":1,"194":1,"195":1,"196":1,"197":1},"2":{"4":1,"5":3,"9":6,"10":12,"12":2,"16":1,"18":1,"22":1,"25":1,"32":1,"44":1,"46":1,"52":5,"53":2,"54":1,"59":1,"65":1,"66":2,"67":1,"69":10,"70":1,"72":2,"77":1,"80":1,"81":1,"102":2,"103":1,"104":2,"124":1,"129":1,"130":1,"131":1,"134":2,"135":2,"136":1,"137":1,"149":1,"152":1,"154":3,"166":1,"174":2,"179":1,"185":1,"204":1,"208":1,"211":1,"213":1,"215":1,"227":1,"236":3,"238":1,"247":1,"249":3,"250":4,"252":2,"254":3,"256":1}}],["familiar",{"2":{"176":1,"222":1}}],["far",{"2":{"129":1}}],["farley",{"2":{"71":1}}],["fake",{"2":{"110":1}}],["favor",{"2":{"66":1}}],["fact",{"2":{"146":1}}],["factor",{"2":{"9":2,"22":2,"24":8,"53":2,"73":2}}],["facusapienza",{"2":{"134":1}}],["facilitates",{"2":{"58":1}}],["fausto",{"2":{"53":1}}],["failed",{"2":{"109":1}}],["failing",{"2":{"49":1}}],["failures",{"2":{"44":1}}],["fail",{"2":{"44":1,"49":4,"88":1}}],["fails",{"2":{"33":1,"48":1,"87":2}}],["fashionmnist",{"0":{"219":1},"1":{"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1},"2":{"221":1,"225":2}}],["fashion",{"2":{"37":1}}],["fastest",{"2":{"87":2,"139":1}}],["faster",{"0":{"152":1},"2":{"3":1,"18":2,"19":3,"87":1,"133":1,"152":2,"153":1}}],["fast",{"2":{"18":4,"20":2,"24":1,"25":1,"30":1,"46":1,"72":1,"102":1,"138":1,"152":2,"154":1,"172":1,"256":3}}],["fancy",{"2":{"106":1}}],["fan",{"2":{"9":6,"66":2}}],["fallback",{"2":{"12":1,"16":3,"21":2,"23":2,"25":1,"95":1,"197":1}}],["fall",{"2":{"3":1}}],["falls",{"2":{"2":1,"18":1}}],["false",{"2":{"2":2,"15":1,"22":1,"25":1,"33":1,"44":1,"46":2,"50":1,"53":5,"54":1,"55":1,"58":1,"65":2,"66":2,"69":10,"70":3,"104":1,"133":1,"150":1,"156":1,"157":1,"220":1,"228":1,"230":3,"234":2,"236":13,"239":1,"257":1}}],["fitting",{"0":{"211":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1}}],["fit",{"2":{"208":1,"211":1,"244":4,"250":1}}],["figure",{"2":{"206":1,"209":1,"213":1,"217":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["figured",{"2":{"110":1}}],["fig",{"2":{"206":3,"209":3,"213":3,"217":3,"245":6,"248":3,"250":6,"255":3,"256":3,"259":6}}],["fixing",{"2":{"98":1,"137":1,"162":1}}],["fix",{"2":{"88":1,"109":1,"110":2,"134":2,"138":1,"149":2,"202":1}}],["fix1",{"2":{"67":1,"71":1,"133":3,"136":1,"137":2,"231":1,"236":9,"256":2}}],["fixed=",{"2":{"59":1}}],["fixed",{"2":{"22":1,"58":1,"59":4,"103":2,"109":4,"168":1}}],["finetune",{"2":{"208":1}}],["fine",{"2":{"130":1}}],["finite",{"2":{"134":4,"135":2,"138":1}}],["finitediff",{"2":{"49":1,"133":1,"134":2,"135":2}}],["finish",{"2":{"108":1}}],["findmax",{"2":{"250":1}}],["find",{"2":{"59":1,"111":1,"164":1,"174":1,"202":1,"250":1}}],["final",{"2":{"59":2,"225":3}}],["finally",{"0":{"191":1},"2":{"37":1,"59":1,"129":1,"130":1,"143":1,"166":1,"174":1,"179":1,"217":1,"236":1,"259":1}}],["fields",{"2":{"52":1,"55":1,"65":10,"71":2,"131":1}}],["fieldnames",{"2":{"12":1,"179":1}}],["field",{"2":{"12":3,"13":1,"44":2,"54":1,"65":1,"179":1}}],["fill",{"2":{"53":1,"166":1}}],["filter",{"2":{"9":1}}],["file",{"2":{"1":1,"125":3}}],["first",{"2":{"2":1,"13":1,"32":1,"34":1,"44":1,"46":1,"59":6,"65":2,"68":3,"71":2,"80":1,"89":1,"106":2,"110":4,"125":1,"129":2,"131":2,"133":1,"134":1,"144":1,"149":2,"155":1,"164":2,"168":1,"169":2,"171":1,"174":1,"179":3,"189":1,"191":1,"193":1,"194":1,"217":1,"224":1,"230":1,"232":1,"240":1,"247":1,"248":2,"250":3,"255":1,"256":2,"257":1,"259":1}}],["footnotes",{"0":{"89":1}}],["foldl",{"2":{"54":3}}],["follow",{"2":{"107":1}}],["follows",{"2":{"3":1,"5":1,"65":1}}],["following",{"2":{"2":1,"5":1,"25":1,"28":1,"32":1,"55":1,"56":1,"57":2,"61":2,"62":2,"63":2,"107":1,"120":1,"124":1,"127":1,"133":1,"138":1,"148":1,"149":1,"153":1,"155":2,"157":1,"179":1,"193":1,"242":1,"250":1}}],["focuses",{"2":{"53":1}}],["focalloss",{"2":{"53":2}}],["focal",{"2":{"53":4}}],["four",{"2":{"166":1}}],["fourth",{"2":{"53":1}}],["found",{"2":{"2":1,"74":1,"203":1}}],["forum",{"2":{"133":1}}],["forget",{"2":{"108":1}}],["formulas",{"2":{"254":1}}],["format",{"2":{"166":1}}],["formats",{"2":{"65":4,"71":1}}],["form",{"2":{"59":1,"70":1,"161":1,"164":1,"168":2}}],["forms",{"2":{"53":1}}],["forwarded",{"2":{"48":1}}],["forward",{"2":{"33":1,"54":1,"59":5,"69":1,"70":2,"81":1,"86":2,"110":1,"111":1,"134":1,"136":1,"137":1,"170":1,"171":2,"172":2,"249":1,"250":5}}],["forwarddiff",{"2":{"25":1,"49":1,"55":1,"86":1,"133":3,"134":2,"136":4,"137":2,"141":1,"170":2,"171":4,"172":1}}],["forbidden",{"2":{"9":1}}],["forces",{"2":{"77":1}}],["force",{"2":{"2":6,"44":4,"48":1}}],["for",{"0":{"45":1,"118":1,"135":1,"153":1,"155":1,"165":1,"257":1},"1":{"46":1,"156":1,"157":1,"158":1,"159":1,"160":1,"161":1,"166":1,"167":1,"168":1,"169":1,"170":1,"171":1,"172":1,"173":1,"174":1,"175":1},"2":{"0":1,"1":1,"2":3,"3":14,"4":9,"5":3,"6":1,"9":11,"11":1,"12":11,"13":7,"16":1,"17":1,"18":1,"19":1,"21":5,"22":8,"23":5,"24":15,"25":3,"27":2,"28":3,"29":1,"31":5,"32":3,"33":8,"34":2,"36":2,"37":8,"38":2,"44":3,"46":5,"47":2,"48":1,"49":5,"52":5,"53":9,"54":5,"55":15,"56":1,"57":4,"58":2,"59":16,"60":3,"65":11,"66":10,"68":15,"69":24,"70":12,"71":6,"72":18,"73":8,"75":1,"76":1,"77":9,"79":1,"80":2,"81":7,"82":1,"83":5,"84":1,"85":2,"87":11,"88":2,"89":1,"92":2,"93":2,"95":2,"96":1,"97":1,"98":1,"99":1,"100":1,"102":1,"104":2,"105":1,"106":5,"107":4,"108":1,"109":1,"110":1,"113":1,"115":2,"119":1,"121":2,"124":1,"127":2,"129":6,"130":3,"131":3,"132":1,"133":6,"135":1,"136":2,"137":1,"138":1,"139":1,"141":2,"142":1,"143":1,"146":1,"147":2,"148":2,"149":3,"151":1,"153":1,"154":3,"156":2,"158":1,"159":1,"160":5,"161":1,"162":2,"163":1,"164":4,"165":1,"166":3,"167":1,"168":2,"169":2,"170":5,"171":2,"172":2,"174":3,"178":3,"179":4,"180":2,"181":1,"182":3,"185":1,"189":1,"190":2,"191":1,"193":1,"196":1,"203":1,"204":2,"208":3,"214":1,"217":2,"224":1,"225":3,"229":1,"230":2,"232":1,"233":5,"236":3,"238":1,"243":2,"244":1,"245":2,"248":5,"249":2,"250":5,"255":1}}],["tmp",{"2":{"254":4}}],["tmatch",{"2":{"13":3}}],["t∈",{"2":{"243":1}}],["ttest",{"2":{"225":2,"233":1}}],["ttraining",{"2":{"225":2,"233":1}}],["ttime",{"2":{"190":2,"225":3,"233":3}}],["td",{"0":{"118":1}}],["tdchain",{"0":{"116":1},"2":{"115":4,"117":1,"118":1}}],["tl",{"2":{"108":1}}],["tloss",{"2":{"107":1}}],["tpu",{"2":{"63":1,"106":1}}],["tsteps",{"2":{"255":5,"256":4,"259":6}}],["tstate",{"2":{"174":6,"217":9,"233":7}}],["tsit5",{"2":{"206":1,"208":1,"209":1,"236":13}}],["tspan=",{"2":{"230":2,"234":1}}],["tspan",{"2":{"206":5,"209":2,"230":5,"234":3,"236":7,"255":4,"256":1,"259":1}}],["tsung",{"2":{"53":2}}],["ts",{"2":{"52":16,"245":6,"248":1,"249":4}}],["t=rand",{"2":{"49":1}}],["t=float32",{"2":{"9":8}}],["tail",{"2":{"249":1}}],["tab>`",{"2":{"171":1}}],["table",{"2":{"70":1,"166":1}}],["tall",{"2":{"170":1}}],["tasklocalrng",{"2":{"80":1,"123":1,"165":1,"174":1}}],["tasks",{"2":{"53":3}}],["taking",{"2":{"53":1,"54":1,"71":1}}],["takeaway",{"2":{"131":1}}],["takes",{"2":{"13":1,"53":2,"59":1,"65":4,"81":1,"106":1,"208":1,"216":1,"250":1,"256":2}}],["taken",{"2":{"5":1,"9":1,"60":1,"66":2,"70":1,"104":1,"106":1,"137":1}}],["take",{"2":{"1":1,"10":1,"32":1,"52":1,"53":2,"60":1,"66":1,"80":1,"88":1,"114":1,"134":1,"163":1,"241":1,"243":1,"249":1}}],["targets",{"2":{"53":1,"187":1,"229":1}}],["target",{"2":{"48":10,"53":2,"189":3,"224":3,"232":3,"242":6,"243":12,"244":10}}],["tanh",{"2":{"34":2,"69":1,"80":11,"134":1,"135":1,"136":3,"137":3,"141":3,"208":2,"231":4,"236":36,"241":3,"249":3}}],["turing",{"2":{"247":6}}],["tutorials",{"0":{"198":1,"199":1,"200":1,"201":1,"203":1},"1":{"199":1,"200":1,"201":1,"202":1,"203":1},"2":{"59":1,"84":1,"203":1,"238":1,"244":4}}],["tutorial",{"2":{"58":1,"81":1,"165":3,"170":1,"176":2,"185":2,"203":1,"204":6,"211":1,"238":1,"247":2}}],["tu",{"2":{"27":1}}],["tuple=true",{"2":{"233":1}}],["tuples",{"2":{"12":2}}],["tuple",{"2":{"3":2,"9":3,"12":2,"24":2,"25":1,"31":4,"32":1,"46":1,"52":1,"53":1,"55":1,"65":7,"66":11,"68":15,"69":22,"70":5,"72":1,"73":3,"76":1,"80":12,"115":1,"136":1,"137":1,"164":1,"207":2,"231":2,"236":36,"249":1}}],["temporal",{"2":{"243":1}}],["te",{"2":{"190":4,"191":1,"233":2}}],["technology",{"2":{"74":1}}],["terminate",{"2":{"208":1}}],["terminology",{"2":{"208":1}}],["terms",{"2":{"89":1}}],["term",{"2":{"69":1,"134":1,"141":1,"249":2}}],["terrible",{"2":{"52":1}}],["tested",{"2":{"76":1,"77":1,"88":2,"170":1,"202":1}}],["test",{"0":{"50":1},"2":{"48":7,"49":10,"50":8,"67":3,"72":2,"77":1,"89":1,"106":1,"134":2,"164":1,"187":5,"190":3,"191":20,"221":5,"225":13,"229":5,"233":3,"255":1}}],["tests",{"2":{"48":2,"49":2,"88":1,"138":1,"147":2}}],["testing",{"0":{"48":1},"2":{"47":3,"76":1}}],["testmode`",{"2":{"134":2}}],["testmode",{"2":{"15":1,"67":3,"72":3,"106":3,"182":1,"189":1,"196":1,"224":1,"232":1,"244":1}}],["tends",{"2":{"22":1,"134":1,"135":1}}],["tensors",{"2":{"73":1}}],["tensorflow",{"2":{"69":1}}],["tensor",{"2":{"9":1,"21":3,"72":1,"166":1,"254":1}}],["tile",{"2":{"166":2}}],["tiles",{"2":{"166":1}}],["tightly",{"2":{"161":1}}],["tier",{"2":{"86":8,"88":3}}],["tied",{"0":{"34":1},"2":{"77":1}}],["title",{"2":{"74":2,"245":1,"250":1}}],["tips",{"2":{"148":2}}],["tip",{"2":{"13":1,"23":1,"44":1,"46":1,"48":1,"59":1,"69":1,"129":1,"133":2,"142":1,"203":1,"254":1}}],["timewrapper",{"2":{"207":7,"208":1}}],["timelastindex",{"2":{"54":1,"69":2}}],["timestep",{"2":{"255":1}}],["timespace",{"2":{"255":1}}],["times",{"2":{"47":1,"65":3,"166":1,"169":1,"204":1}}],["time",{"0":{"115":1},"2":{"3":1,"44":1,"69":2,"87":2,"106":1,"114":3,"117":5,"190":3,"191":20,"207":1,"225":2,"233":3,"235":1,"245":1,"255":2,"256":3,"257":1,"259":2}}],["typing",{"2":{"79":1}}],["typical",{"2":{"59":1}}],["typically",{"2":{"12":1,"69":1,"77":1,"136":1,"154":1,"181":1,"243":1,"256":1}}],["typed",{"2":{"171":1}}],["typeof",{"2":{"58":1,"80":8,"106":3,"110":1,"116":2,"129":4,"217":2,"236":80,"256":2}}],["types",{"0":{"7":1,"12":1,"112":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1},"2":{"3":1,"13":2,"25":2,"31":1,"55":6,"97":2,"100":1,"131":1,"132":1,"160":1,"162":2}}],["type",{"0":{"57":1,"149":1,"151":1,"236":1},"2":{"3":13,"4":8,"7":2,"9":7,"10":1,"12":6,"13":5,"21":1,"25":1,"37":5,"44":1,"46":1,"53":1,"55":6,"57":12,"58":3,"59":4,"65":2,"72":1,"106":1,"109":5,"110":13,"113":1,"120":1,"121":1,"130":1,"131":3,"132":1,"143":3,"149":2,"151":2,"160":2,"163":1,"179":2,"182":2,"221":1,"235":1,"236":1}}],["two",{"2":{"9":3,"55":2,"65":4,"70":1,"72":1,"73":3,"125":1,"129":1,"130":1,"131":1,"138":1,"163":1,"249":1,"254":4,"256":1}}],["tr",{"2":{"138":2,"141":11,"190":4,"233":2}}],["trying",{"2":{"129":1,"208":1}}],["try",{"2":{"87":1,"106":1,"109":2,"110":1,"116":1,"131":2,"141":1,"166":1,"174":1,"183":1,"203":1}}],["treat",{"2":{"49":1,"130":1}}],["treated",{"2":{"20":1,"59":1}}],["trivial",{"2":{"77":1,"127":1,"168":1,"208":1,"236":26}}],["trilinear",{"2":{"73":6}}],["tries",{"2":{"21":1}}],["trigger",{"2":{"3":4,"4":2,"124":1,"125":2,"143":1}}],["truth",{"2":{"136":1,"174":1}}],["truncation",{"2":{"102":2}}],["truncated",{"2":{"7":2,"9":2}}],["truly",{"2":{"77":1}}],["true",{"0":{"255":1},"2":{"2":2,"3":3,"9":2,"15":1,"22":6,"24":4,"32":1,"33":2,"34":1,"44":2,"46":1,"49":2,"53":40,"54":3,"55":1,"58":1,"59":4,"65":3,"66":2,"67":4,"69":11,"73":1,"76":1,"80":8,"104":1,"106":3,"109":1,"110":2,"116":2,"120":2,"123":2,"125":1,"126":1,"134":3,"135":1,"136":1,"137":1,"139":1,"140":1,"141":1,"157":2,"161":1,"174":2,"181":4,"189":1,"206":2,"208":2,"213":1,"217":3,"224":1,"232":1,"234":1,"236":58,"242":3,"244":2,"247":2,"249":1,"255":1,"256":2}}],["traditional",{"2":{"70":1,"249":1}}],["transitioned",{"2":{"124":1}}],["transposed",{"2":{"66":2}}],["transpose",{"2":{"66":3,"166":1}}],["transformation",{"2":{"72":1}}],["transformations",{"2":{"44":1}}],["transforms",{"2":{"68":3}}],["transform",{"2":{"44":1}}],["transformed",{"2":{"20":1}}],["transferred",{"2":{"3":1,"194":1}}],["transferring",{"2":{"0":1,"5":1,"154":1}}],["transfer",{"0":{"2":1,"154":1},"2":{"2":1,"154":2,"183":1,"194":1}}],["trace",{"0":{"138":1},"1":{"139":1,"140":1,"141":1},"2":{"138":7,"139":3,"140":2,"141":10,"254":4}}],["tracing",{"2":{"55":1,"106":1}}],["tracking",{"0":{"110":1},"2":{"105":1,"111":2}}],["track",{"2":{"33":1,"57":1,"65":2,"72":12,"109":1,"110":5,"149":1}}],["tracker",{"2":{"21":1,"23":1,"25":1,"49":1,"52":1,"55":1,"86":1,"116":1,"244":12,"247":1}}],["tracked",{"2":{"109":1}}],["trackedarray",{"2":{"13":1,"116":2}}],["trackedreals",{"2":{"13":1}}],["train=1024",{"2":{"221":1}}],["trainstate",{"0":{"107":1},"2":{"52":11,"80":3,"81":1,"102":1,"106":2,"107":3,"174":1,"182":1,"190":1,"204":1,"217":4,"225":1,"233":1,"241":1,"244":1}}],["trained2",{"2":{"182":2}}],["trained",{"2":{"46":1,"182":2,"183":8,"208":2,"209":2,"244":5,"245":1,"259":3}}],["train",{"0":{"235":1},"2":{"46":1,"52":6,"59":1,"69":24,"80":8,"81":7,"106":1,"107":10,"174":4,"178":4,"182":12,"185":1,"187":6,"190":15,"191":3,"204":2,"208":3,"217":1,"221":10,"225":24,"229":6,"233":8,"241":1,"243":1,"244":9}}],["trainmode",{"2":{"15":1}}],["trainable",{"2":{"12":1,"31":1,"52":2,"59":9,"66":2,"69":4,"70":3,"72":1,"194":9}}],["training=val",{"2":{"67":3}}],["training",{"0":{"52":1,"142":1,"176":1,"182":1,"190":1,"191":1,"196":1,"204":1,"206":1,"208":1,"217":1,"219":1,"225":1,"233":1,"238":1,"244":1,"252":1,"257":1,"258":1},"1":{"143":1,"144":1,"145":1,"146":1,"147":1,"177":1,"178":1,"179":1,"180":1,"181":1,"182":1,"183":1,"184":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1,"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"4":3,"9":2,"15":2,"22":9,"24":8,"33":1,"36":2,"37":4,"52":14,"53":2,"54":3,"67":6,"72":9,"74":1,"77":2,"80":8,"81":3,"102":1,"103":2,"106":1,"107":5,"109":1,"110":3,"120":2,"122":1,"123":2,"142":2,"174":5,"176":1,"182":2,"185":1,"190":4,"191":20,"196":2,"204":1,"216":1,"217":5,"225":2,"233":5,"235":1,"241":1,"244":2,"257":2,"258":1}}],["t",{"2":{"2":1,"3":5,"4":6,"9":20,"10":1,"13":1,"18":2,"21":1,"23":1,"32":1,"34":1,"44":1,"47":1,"49":1,"52":1,"54":1,"59":6,"69":1,"71":1,"77":1,"80":1,"81":1,"88":2,"89":1,"95":2,"98":1,"102":1,"108":1,"111":1,"113":1,"114":1,"115":4,"117":2,"129":4,"131":3,"132":1,"133":2,"134":1,"135":1,"136":10,"138":1,"146":1,"147":1,"164":1,"166":2,"168":2,"172":1,"174":1,"178":1,"179":4,"180":2,"183":1,"187":1,"190":3,"194":1,"196":1,"197":1,"202":1,"204":1,"206":6,"207":11,"208":8,"209":9,"217":1,"222":1,"223":1,"225":2,"229":1,"230":5,"234":3,"243":6,"244":12,"254":16,"255":1,"256":1,"257":2}}],["thousands",{"2":{"166":1}}],["though",{"2":{"129":1,"153":1,"170":1}}],["those",{"2":{"12":1,"13":1,"49":1,"73":1,"82":1,"110":1,"130":1,"147":1,"166":1,"230":1,"250":1}}],["thunk",{"2":{"106":1}}],["thumb",{"2":{"73":1}}],["thus",{"2":{"24":1,"65":1,"73":1}}],["threads",{"2":{"175":3,"184":3,"192":3,"210":3,"218":3,"226":3,"237":3,"246":3,"251":3,"260":3}}],["three",{"2":{"71":1,"166":1}}],["through",{"2":{"5":1,"13":1,"65":4,"72":4,"79":1,"84":2,"114":1,"131":1,"148":1,"168":1,"179":3,"194":1,"207":1,"250":1}}],["throws",{"2":{"50":1}}],["throw",{"2":{"2":1,"3":1,"33":1,"52":1,"57":1,"95":1,"244":1}}],["thrown",{"2":{"2":2,"31":1,"33":2}}],["than",{"2":{"9":2,"170":1,"233":1}}],["that",{"2":{"2":3,"3":8,"5":1,"9":3,"12":4,"13":5,"15":1,"16":1,"18":2,"21":1,"22":1,"23":2,"25":1,"30":1,"31":1,"33":1,"34":2,"39":2,"46":2,"47":1,"52":1,"53":1,"54":1,"55":3,"58":1,"59":5,"65":5,"66":2,"67":2,"68":6,"70":1,"71":4,"72":6,"77":2,"80":1,"82":1,"89":3,"90":2,"95":2,"98":2,"106":1,"107":2,"108":2,"109":3,"110":1,"111":2,"129":3,"130":4,"131":1,"132":2,"133":3,"134":2,"135":3,"136":2,"138":2,"141":3,"143":5,"144":1,"146":1,"149":3,"150":1,"155":1,"157":1,"161":1,"164":2,"166":1,"168":3,"169":1,"171":1,"172":1,"179":3,"183":2,"193":1,"194":2,"197":1,"202":1,"204":2,"208":4,"214":1,"216":1,"233":1,"236":1,"240":1,"241":1,"243":1,"249":1,"250":3,"256":3}}],["third",{"2":{"166":2}}],["thirteenth",{"2":{"9":2}}],["thing",{"2":{"71":2,"149":1,"164":1}}],["things",{"2":{"59":1,"129":1,"131":1,"166":1}}],["think",{"2":{"59":1,"129":1}}],["this",{"2":{"2":3,"3":8,"4":10,"5":1,"6":1,"9":5,"12":5,"13":6,"15":1,"16":6,"18":3,"19":1,"21":3,"23":3,"24":2,"25":4,"27":2,"28":2,"30":2,"32":2,"33":5,"36":2,"37":3,"40":1,"41":1,"44":6,"46":3,"47":2,"48":1,"49":2,"52":6,"53":4,"54":1,"55":12,"57":7,"58":6,"59":11,"65":3,"66":5,"68":3,"69":6,"70":5,"71":1,"72":5,"73":3,"74":2,"76":1,"77":2,"82":1,"85":1,"87":2,"89":1,"90":2,"95":6,"96":1,"97":1,"98":3,"103":1,"104":1,"105":1,"106":5,"107":3,"108":2,"109":3,"111":2,"119":2,"121":1,"124":1,"125":2,"127":1,"129":8,"131":6,"132":1,"133":6,"134":9,"135":5,"136":1,"137":1,"138":3,"139":1,"141":2,"143":3,"145":2,"147":2,"149":7,"150":1,"151":1,"153":1,"154":4,"156":1,"157":2,"158":3,"160":1,"161":1,"164":5,"165":1,"166":5,"168":4,"170":1,"174":1,"175":1,"176":2,"178":1,"180":2,"184":1,"185":1,"188":1,"191":1,"192":1,"194":2,"196":3,"204":3,"208":5,"210":1,"211":1,"214":1,"216":1,"218":1,"222":1,"226":1,"233":1,"234":1,"236":2,"237":1,"238":2,"246":1,"247":1,"249":4,"251":1,"252":1,"254":3,"256":2,"258":1,"260":1}}],["theoretical",{"2":{"254":1}}],["theoretically",{"2":{"170":1}}],["theorem",{"2":{"71":1}}],["thesis",{"2":{"74":1}}],["these",{"2":{"10":1,"12":5,"13":1,"30":1,"35":1,"37":1,"44":3,"46":3,"52":1,"72":1,"75":2,"77":1,"80":1,"88":7,"95":2,"102":3,"114":1,"129":2,"130":2,"133":1,"137":1,"147":1,"153":1,"157":1,"160":1,"161":3,"165":1,"168":1,"172":1,"202":3,"203":1,"227":1,"249":1}}],["they",{"2":{"52":1,"59":2,"110":1,"120":1,"166":1,"170":2,"194":1}}],["there",{"2":{"31":1,"46":1,"52":1,"80":1,"87":1,"89":1,"97":1,"110":1,"111":1,"133":1,"136":2,"143":1,"149":1,"153":1,"163":1,"166":1,"191":1}}],["them",{"0":{"148":1},"1":{"149":1,"150":1,"151":1,"152":1,"153":1,"154":1},"2":{"30":1,"32":1,"53":1,"59":1,"65":1,"70":1,"73":1,"75":1,"76":1,"80":1,"88":1,"95":1,"102":1,"149":1,"160":1,"164":1,"166":2,"169":1,"172":1,"203":2,"241":1,"249":1,"256":2}}],["their",{"2":{"12":2,"66":1,"84":1,"152":1,"166":1,"168":2}}],["then",{"2":{"1":2,"2":6,"3":1,"13":1,"22":2,"24":1,"28":2,"31":3,"33":2,"48":1,"49":2,"50":1,"53":6,"54":1,"55":2,"58":2,"60":1,"65":5,"66":4,"67":4,"69":9,"70":7,"71":1,"74":1,"77":1,"79":1,"106":1,"125":3,"129":1,"130":3,"138":2,"155":1}}],["the",{"0":{"107":1,"116":1,"117":1,"118":1,"122":1,"137":2,"139":1,"140":1,"141":1,"165":1,"180":1,"182":1,"183":1,"188":1,"190":1,"191":1,"207":1,"208":1,"209":1,"223":1,"230":1,"231":1,"235":1,"241":1,"242":1,"243":1,"245":1,"249":1,"255":1,"257":1,"258":1,"259":1},"1":{"166":1,"167":1,"168":1,"169":1,"170":1,"171":1,"172":1,"173":1,"174":1,"175":1},"2":{"1":6,"2":12,"3":31,"4":14,"5":13,"9":60,"10":25,"11":3,"12":19,"13":22,"14":4,"15":7,"16":6,"18":10,"19":2,"20":3,"21":11,"22":16,"23":6,"24":21,"25":5,"27":20,"28":13,"29":1,"30":1,"31":14,"32":22,"33":15,"34":14,"35":1,"37":10,"38":4,"39":11,"40":5,"41":3,"44":12,"46":9,"48":6,"49":9,"50":4,"52":35,"53":55,"54":4,"55":13,"56":6,"57":19,"58":18,"59":49,"60":6,"61":7,"62":2,"63":2,"65":46,"66":38,"67":6,"68":45,"69":69,"70":19,"71":33,"72":58,"73":14,"75":1,"76":5,"77":6,"79":3,"80":6,"81":8,"82":1,"84":4,"85":3,"87":7,"88":2,"89":1,"90":2,"92":2,"95":2,"96":2,"97":2,"98":4,"99":1,"102":11,"104":9,"105":4,"106":23,"107":7,"108":3,"109":6,"110":19,"113":2,"114":2,"115":1,"117":3,"118":2,"120":1,"121":7,"122":3,"124":3,"125":5,"127":6,"129":20,"130":10,"131":4,"132":5,"133":9,"134":5,"135":16,"136":5,"137":13,"138":11,"139":2,"141":7,"142":1,"143":22,"144":3,"145":1,"146":6,"148":1,"149":9,"151":1,"152":3,"153":2,"154":8,"155":4,"156":1,"157":1,"158":1,"159":1,"160":10,"161":4,"163":11,"164":10,"165":2,"166":12,"167":7,"168":2,"169":5,"170":6,"171":1,"172":4,"173":2,"174":6,"176":2,"178":6,"179":24,"180":3,"181":2,"182":6,"183":5,"185":2,"187":2,"188":3,"190":2,"191":5,"193":4,"194":6,"196":4,"197":1,"202":1,"203":6,"204":5,"207":8,"208":11,"213":4,"216":3,"217":2,"222":6,"225":1,"229":2,"230":13,"231":1,"233":3,"234":2,"235":3,"236":5,"238":2,"240":3,"241":2,"242":4,"243":3,"247":5,"248":3,"249":16,"250":22,"252":1,"254":7,"255":2,"256":10,"257":4,"258":2,"259":2}}],["top",{"2":{"106":1,"127":1}}],["topic",{"2":{"69":1}}],["toolchain",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["tools",{"2":{"33":2,"59":1,"108":1,"165":1,"170":1}}],["too",{"2":{"75":1,"170":1,"250":1}}],["towards",{"2":{"55":1,"168":1,"204":1}}],["toarray",{"2":{"46":1}}],["tosimplechainsadaptor",{"2":{"46":4,"77":1,"188":2}}],["together",{"2":{"32":1}}],["total",{"2":{"14":1,"15":1,"34":1,"38":2,"59":4,"65":4,"72":3,"80":1,"81":1,"109":1,"110":1,"115":1,"188":2,"189":6,"214":1,"224":6,"232":6,"244":3,"256":1}}],["to",{"0":{"44":1,"46":1,"50":1,"61":1,"84":1,"90":1,"143":1,"148":1,"193":1,"252":1},"1":{"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":1,"103":1,"104":1,"105":1,"149":1,"150":1,"151":1,"152":1,"153":1,"154":1,"194":1,"195":1,"196":1,"197":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"1":3,"2":7,"3":10,"4":5,"5":5,"9":14,"11":2,"12":4,"13":10,"15":2,"16":2,"18":7,"19":2,"20":4,"21":5,"22":13,"23":3,"24":16,"25":2,"27":8,"28":5,"30":1,"31":7,"32":2,"33":10,"34":5,"37":9,"39":6,"41":1,"44":14,"46":17,"47":2,"48":3,"49":10,"52":8,"53":14,"54":3,"55":9,"56":4,"57":4,"58":4,"59":21,"60":1,"61":2,"62":1,"63":1,"65":18,"66":19,"67":13,"68":9,"69":59,"70":7,"71":9,"72":22,"73":8,"74":1,"75":2,"76":1,"77":9,"79":3,"80":2,"81":2,"84":2,"88":3,"89":1,"90":5,"92":1,"95":5,"96":1,"97":1,"98":5,"99":2,"102":6,"103":3,"104":6,"105":3,"106":14,"107":8,"108":4,"109":1,"110":7,"113":2,"114":2,"115":1,"119":1,"120":1,"121":1,"122":1,"124":3,"125":2,"127":5,"129":7,"130":5,"131":4,"132":2,"133":7,"134":10,"135":6,"136":1,"137":4,"138":6,"141":1,"142":1,"143":5,"144":3,"146":4,"149":7,"150":1,"151":2,"152":1,"154":3,"155":4,"156":2,"157":3,"158":4,"160":4,"161":3,"162":1,"163":3,"164":9,"165":4,"166":7,"167":1,"168":3,"170":1,"171":1,"172":3,"174":2,"176":2,"178":3,"179":10,"181":1,"183":1,"185":2,"187":1,"188":3,"193":3,"194":8,"196":1,"197":3,"202":1,"203":6,"204":4,"207":6,"208":7,"216":1,"222":1,"223":1,"227":2,"229":1,"230":6,"231":1,"233":1,"234":1,"236":11,"238":2,"242":2,"243":2,"247":1,"248":3,"249":12,"250":5,"254":5,"256":6,"257":2,"258":2}}],["toml",{"2":{"1":1,"47":1}}],["l3",{"2":{"259":2}}],["lb",{"2":{"256":1,"259":1}}],["lbfgs",{"2":{"204":1,"208":3}}],["lr",{"2":{"244":2}}],["ld",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["llvm",{"2":{"175":2,"184":2,"192":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"260":1}}],["ll",{"2":{"165":1,"248":1,"249":1,"256":2}}],["lpnorm",{"2":{"105":1}}],["lppool",{"2":{"68":1,"105":1}}],["lp",{"2":{"68":4,"249":1,"250":1}}],["l=",{"2":{"53":1}}],["ls",{"2":{"53":4}}],["lstmcell",{"2":{"69":6,"104":1,"105":1,"179":3,"180":1}}],["lstm",{"0":{"176":1},"1":{"177":1,"178":1,"179":1,"180":1,"181":1,"182":1,"183":1,"184":1},"2":{"12":1,"69":1,"179":17,"180":4}}],["l2",{"2":{"34":5,"130":4,"256":2,"259":2}}],["l2=dense",{"2":{"34":1}}],["l1",{"2":{"34":5,"130":4,"256":2,"259":2}}],["l1=dense",{"2":{"34":1}}],["l",{"2":{"13":5,"14":2,"15":2,"31":13,"32":5,"44":2,"46":1,"65":2,"67":1,"115":5,"120":2,"121":2,"129":16,"174":1,"179":2,"194":4,"208":4,"213":2,"217":2,"230":4,"255":2,"257":3}}],["lag",{"2":{"244":1}}],["lazy",{"2":{"194":1}}],["latter",{"2":{"152":2}}],["later",{"2":{"248":1}}],["latentsize",{"2":{"69":3}}],["latest",{"2":{"61":1,"203":1}}],["language",{"2":{"55":2}}],["label=l",{"2":{"206":2,"209":4}}],["labels",{"2":{"53":2,"178":3,"187":2,"221":4,"229":2}}],["label",{"2":{"53":18}}],["larger",{"0":{"202":1},"2":{"77":1,"166":1,"202":1}}],["large",{"2":{"44":1,"77":3,"170":1}}],["layerfreezing",{"2":{"122":1}}],["layer2",{"2":{"65":1}}],["layer1",{"2":{"65":1}}],["layernorm",{"2":{"24":1,"72":4,"92":1}}],["layer",{"0":{"16":1,"32":1,"58":1,"59":1,"121":1,"128":1,"129":1,"130":1,"222":1,"230":1,"231":1,"234":1},"1":{"129":1,"130":1},"2":{"12":13,"13":9,"14":4,"15":4,"16":4,"24":3,"31":7,"32":6,"33":13,"44":10,"46":11,"57":5,"58":7,"59":11,"65":67,"66":5,"67":3,"68":12,"69":2,"70":5,"71":8,"72":24,"73":3,"77":2,"80":47,"102":2,"103":1,"104":1,"106":24,"108":2,"109":30,"110":69,"113":2,"115":3,"117":5,"120":9,"121":9,"122":5,"123":9,"129":8,"130":12,"133":1,"134":4,"135":1,"136":4,"137":6,"149":1,"160":1,"164":2,"174":1,"179":1,"188":18,"194":5,"214":2,"216":4,"217":4,"222":1,"223":1,"230":2,"236":211,"249":1,"256":12,"258":4}}],["layers",{"0":{"21":1,"23":1,"64":1,"66":1,"67":1,"68":1,"69":1,"70":1,"71":1,"72":1,"81":1,"120":1,"194":1,"197":1},"1":{"65":1,"66":1,"67":1,"68":1,"69":1,"70":1,"71":1,"72":1,"73":1},"2":{"9":8,"11":1,"12":9,"13":2,"32":3,"33":2,"44":2,"57":1,"58":2,"65":39,"66":1,"71":12,"76":4,"77":4,"81":1,"93":1,"102":2,"105":1,"109":9,"110":19,"115":4,"120":1,"127":2,"129":1,"130":3,"133":1,"149":1,"153":1,"159":1,"160":1,"179":1,"180":1,"193":1,"194":3,"222":1,"230":1,"235":1,"236":6,"249":2}}],["last",{"2":{"2":1,"9":1,"20":1,"58":1,"59":1,"69":1,"71":2,"73":2,"115":1,"135":1,"143":1,"230":1,"248":2}}],["lecture",{"2":{"227":1}}],["lecun",{"2":{"53":1}}],["leveraging",{"2":{"164":1}}],["levels=10",{"2":{"245":1}}],["level",{"2":{"9":2,"85":1,"107":1,"160":2,"164":1}}],["less",{"2":{"71":1,"75":1,"88":1,"170":1}}],["lets",{"2":{"131":1,"190":1,"225":1,"233":1,"247":1}}],["let",{"2":{"59":1,"106":2,"107":1,"108":1,"109":2,"110":7,"120":1,"129":1,"130":2,"131":2,"133":3,"134":2,"136":1,"137":1,"141":3,"144":1,"149":1,"164":1,"165":2,"166":1,"169":1,"171":1,"172":1,"173":1,"174":4,"179":1,"181":1,"183":1,"194":2,"208":2,"213":1,"214":1,"217":1,"233":1,"235":1,"249":1,"255":1,"256":1,"257":1,"259":2}}],["len",{"2":{"49":2,"178":1,"243":11}}],["length=25",{"2":{"250":4}}],["length=bc",{"2":{"243":3}}],["length=grid",{"2":{"243":1}}],["length=datasize",{"2":{"206":1,"255":1}}],["length=50",{"2":{"178":1}}],["length",{"2":{"9":7,"10":24,"34":2,"65":2,"66":2,"68":6,"129":8,"130":2,"178":5,"181":1,"189":1,"207":2,"224":1,"232":1,"245":8,"249":3,"254":2,"259":1}}],["leibler",{"2":{"53":1}}],["lei",{"2":{"24":1}}],["lempitsky",{"2":{"24":1,"72":1}}],["leakyrelu",{"2":{"104":1}}],["learn",{"2":{"166":1}}],["learned",{"2":{"77":1}}],["learnable",{"2":{"70":1,"72":4}}],["learning",{"2":{"2":1,"6":1,"9":3,"22":1,"24":1,"53":1,"138":1,"149":1,"152":1,"162":1,"163":1,"166":1,"174":1,"204":1,"254":1}}],["leads",{"2":{"131":1}}],["leading",{"2":{"73":1}}],["lead",{"2":{"18":1,"33":1,"44":1,"55":1,"59":2,"124":1,"164":1,"208":1}}],["leaf",{"2":{"3":1,"13":1,"55":2,"80":12}}],["leaves",{"2":{"3":1,"55":4}}],["luxlinear",{"2":{"194":7}}],["luxlib",{"0":{"17":1,"91":1},"1":{"18":1,"19":1,"20":1,"21":1,"22":1,"23":1,"24":1,"25":1,"92":1,"93":1},"2":{"18":2,"19":1,"20":2,"21":1,"22":2,"23":1,"24":4,"25":1,"48":2,"60":2,"92":1,"134":4,"152":8,"155":1,"160":2,"161":2,"256":2}}],["luxflux",{"2":{"193":1,"194":2}}],["luxdeviceutils",{"2":{"98":3}}],["luxdl",{"2":{"61":1,"69":1,"82":1}}],["luxops",{"0":{"54":1},"2":{"54":8,"179":1,"180":1}}],["luxtestutils",{"0":{"47":1},"1":{"48":1,"49":1,"50":1},"2":{"48":3,"49":2,"50":1}}],["luxcorearrayinterfacereversediffext",{"2":{"116":2}}],["luxcorearrayinterfacetrackerext",{"2":{"116":2}}],["luxcore",{"0":{"11":1,"94":1},"1":{"12":1,"13":1,"14":1,"15":1,"16":1,"95":1,"96":1},"2":{"11":2,"12":3,"13":7,"14":2,"15":5,"16":1,"60":2,"95":2,"116":5,"127":3,"129":14,"130":8,"131":1,"155":1,"160":3}}],["luxcudadevice",{"2":{"99":1}}],["luxcuda",{"2":{"3":1,"62":2,"80":1,"81":1,"83":1,"124":1,"162":1,"164":1,"167":3,"177":1,"205":1,"212":1,"220":1,"228":1,"239":1}}],["lux",{"0":{"42":1,"44":1,"45":1,"46":1,"61":1,"75":1,"77":1,"90":1,"101":1,"106":1,"108":1,"127":1,"155":1,"165":1,"193":1,"204":1},"1":{"43":1,"44":1,"45":1,"46":2,"76":1,"77":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"98":1,"99":1,"100":1,"101":1,"102":2,"103":2,"104":2,"105":2,"107":1,"109":1,"110":1,"111":1,"128":1,"129":1,"130":1,"131":1,"132":1,"156":1,"157":1,"158":1,"159":1,"160":1,"161":1,"166":1,"167":1,"168":1,"169":1,"170":1,"171":1,"172":1,"173":1,"174":1,"175":1,"194":1,"195":1,"196":1,"197":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"4":4,"11":4,"12":3,"13":4,"16":1,"17":1,"27":2,"28":1,"31":13,"32":4,"33":8,"34":3,"35":1,"36":2,"37":3,"38":2,"39":4,"40":1,"41":1,"44":11,"46":15,"47":1,"48":2,"52":9,"53":16,"54":9,"55":9,"56":4,"57":2,"58":5,"59":19,"60":1,"61":5,"62":4,"63":3,"65":11,"66":3,"67":7,"68":9,"69":9,"70":4,"71":12,"72":8,"73":2,"74":1,"75":1,"77":9,"79":4,"80":8,"81":4,"82":1,"83":1,"84":1,"85":2,"88":1,"89":2,"90":2,"95":3,"98":1,"99":2,"102":6,"103":8,"106":7,"107":1,"108":3,"109":6,"110":4,"111":1,"113":1,"115":3,"116":4,"117":4,"118":1,"120":6,"121":2,"122":2,"123":3,"124":3,"125":1,"127":7,"129":3,"130":2,"131":1,"133":4,"134":3,"135":1,"136":1,"137":1,"141":1,"142":1,"146":1,"149":6,"151":2,"155":3,"160":3,"163":1,"164":8,"165":3,"167":1,"168":1,"169":2,"170":1,"172":1,"174":2,"176":2,"177":1,"179":7,"180":1,"182":2,"185":1,"186":1,"188":4,"189":1,"190":1,"191":2,"193":3,"194":13,"196":2,"197":2,"202":1,"203":3,"204":2,"205":1,"208":1,"212":1,"216":2,"217":4,"220":1,"222":4,"223":2,"224":1,"225":1,"228":1,"230":5,"231":1,"232":1,"234":4,"236":96,"238":2,"239":1,"240":1,"241":1,"244":6,"247":4,"249":7,"253":1,"256":2}}],["lt",{"2":{"4":2,"5":1,"9":3,"53":1,"59":2,"82":1,"95":2,"125":1,"130":2,"133":12,"206":1,"209":1}}],["literate",{"2":{"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2,"251":2,"260":2}}],["literature",{"2":{"138":1}}],["little",{"2":{"106":1,"110":1}}],["lib64",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["libllvm",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["libopenlibm",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["libm",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["libraries",{"2":{"133":1,"163":1,"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1,"247":2,"249":1}}],["library",{"2":{"74":1,"168":1,"175":1,"183":1,"184":1,"192":1,"193":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["lib",{"2":{"109":1,"116":6,"134":4,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"236":3,"237":1,"244":4,"246":1}}],["linux",{"2":{"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2,"251":2,"260":2}}],["linked",{"2":{"203":1}}],["link",{"2":{"165":1}}],["linking",{"2":{"65":1}}],["lin",{"2":{"53":2}}],["liner",{"2":{"249":1}}],["linewidth=2",{"2":{"245":1,"255":1,"256":2,"259":3}}],["linewidth=3",{"2":{"213":1,"217":1,"250":3}}],["linewidth=4",{"2":{"206":2,"209":4,"259":1}}],["linesearch=linesearches",{"2":{"258":1}}],["linesearches",{"2":{"253":1}}],["linestyle=",{"2":{"206":2,"209":2}}],["lines",{"2":{"206":2,"209":4,"213":1,"217":1,"255":1,"256":2,"259":4}}],["line",{"2":{"49":1}}],["linearalgebra",{"2":{"133":1,"247":1}}],["linear",{"0":{"70":1,"174":1},"2":{"9":1,"59":4,"71":1,"73":1,"129":12,"130":23,"164":1,"174":1}}],["limits=extrema",{"2":{"245":1}}],["limiter",{"2":{"236":26}}],["limitations",{"2":{"44":1,"52":1,"77":1}}],["limit",{"2":{"18":1,"22":1,"54":1,"175":2,"184":2,"192":1,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"260":1}}],["like",{"2":{"9":1,"20":1,"21":2,"23":1,"25":1,"28":1,"47":1,"52":1,"53":3,"55":1,"59":3,"66":2,"69":3,"77":2,"96":1,"127":1,"129":2,"131":1,"154":1,"163":1,"166":3,"172":1,"193":1,"194":2,"204":1,"230":1,"233":1,"248":1,"255":1,"256":1}}],["lighter",{"2":{"127":1}}],["light",{"2":{"6":1}}],["lightweight",{"2":{"0":1}}],["lists",{"2":{"34":1,"85":1}}],["listed",{"2":{"30":1,"57":1}}],["list",{"2":{"3":1,"34":5,"37":1,"48":1,"49":3,"65":4,"71":1,"81":1,"90":2,"155":1,"162":1,"166":1,"203":1,"225":4}}],["lost",{"2":{"165":1}}],["lossfn",{"2":{"174":3,"181":2,"182":2}}],["lossfunctions",{"2":{"53":1}}],["losses",{"2":{"53":1,"257":2,"259":3}}],["loss",{"0":{"53":1,"134":1,"136":1,"137":1,"181":1,"216":1,"242":1},"1":{"135":1},"2":{"52":5,"53":35,"59":5,"80":2,"81":14,"106":5,"107":4,"133":1,"134":10,"135":9,"136":4,"137":6,"174":19,"181":4,"182":456,"189":1,"190":2,"208":30,"216":4,"217":10,"224":1,"225":1,"238":1,"240":1,"242":23,"244":436,"257":6,"258":1,"259":2}}],["lower",{"2":{"233":1}}],["low",{"2":{"107":1}}],["love",{"2":{"88":1}}],["lotka",{"2":{"206":2}}],["lot",{"2":{"77":1,"169":1}}],["logs",{"2":{"143":1}}],["logging",{"2":{"143":1,"247":1}}],["logarithm",{"2":{"53":1}}],["logarithmic",{"2":{"53":1}}],["log⁡",{"2":{"53":1}}],["log",{"2":{"53":2,"54":2,"249":1,"250":1}}],["logitcrossentropy",{"2":{"232":1,"233":1}}],["logitbinarycrossentropy",{"2":{"181":1}}],["logitbce",{"2":{"53":5}}],["logit",{"2":{"53":1}}],["logits=val",{"2":{"53":2,"189":1,"224":1,"232":1}}],["logits",{"2":{"53":4}}],["longer",{"2":{"102":2,"145":3}}],["long",{"2":{"44":1,"69":1,"131":1}}],["location",{"2":{"33":3,"109":4,"110":10}}],["local",{"2":{"4":1,"38":2,"59":1,"143":1,"144":1,"175":2,"184":2,"191":1,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2}}],["localpreferences",{"2":{"1":1,"125":2}}],["loosely",{"2":{"165":1}}],["looks",{"2":{"154":1}}],["lookup",{"2":{"70":1}}],["look",{"2":{"31":1,"73":1,"172":1,"179":1,"227":1}}],["loops",{"2":{"25":1,"55":1}}],["loopedarrayop",{"2":{"25":1}}],["loopvectorization",{"2":{"18":1,"19":2,"23":1,"25":1,"153":1,"161":1}}],["loop",{"0":{"161":1,"190":1},"2":{"18":1,"25":1,"153":1,"161":1,"217":1}}],["looping",{"2":{"5":1}}],["loadmnist",{"2":{"187":2,"190":1,"229":2,"233":1}}],["loader",{"2":{"107":1,"182":4}}],["loaded",{"2":{"2":2,"3":7,"4":2,"15":1,"16":1,"23":1,"41":1,"95":2,"125":2,"143":1,"161":1,"167":1,"197":1}}],["load",{"2":{"18":1,"19":1,"23":1,"41":1,"47":1,"102":1,"107":1,"124":1,"153":1,"183":1,"187":1,"221":4,"225":1,"229":1}}],["loading",{"0":{"154":1,"187":1,"221":1,"229":1},"2":{"12":1,"19":1,"23":1,"44":1,"46":1,"83":1,"154":1,"183":1}}],["lo",{"2":{"9":2}}],["=ϕwhere",{"2":{"255":1,"256":1}}],["=χu",{"2":{"255":1,"256":1}}],["=∫θp",{"2":{"250":1}}],["=wx+b",{"2":{"174":1}}],["=x",{"2":{"171":1}}],["=12xtx",{"2":{"171":1}}],["=1v∑i=1vvitjvinote",{"2":{"138":1}}],["=1v∑i=1vvitaviwe",{"2":{"138":1}}],["=e",{"2":{"138":1}}],["=i",{"2":{"138":1}}],["=vcat",{"2":{"69":1}}],["=val",{"2":{"46":1,"53":1}}],["=true",{"2":{"33":1}}],["=static",{"2":{"33":1}}],["=>",{"2":{"31":2,"32":3,"34":8,"44":2,"46":5,"59":8,"65":6,"66":2,"69":4,"70":4,"72":12,"80":6,"81":8,"106":6,"107":3,"109":17,"110":14,"115":4,"134":2,"135":2,"136":4,"137":4,"141":4,"149":1,"174":1,"179":2,"180":2,"188":15,"193":4,"214":4,"217":2,"223":1,"231":5,"241":4,"249":3,"256":6}}],["=0",{"2":{"9":2}}],["=nothing",{"2":{"2":1,"53":2,"69":1,"72":1}}],["==3",{"2":{"254":1}}],["===",{"2":{"34":4}}],["==",{"2":{"1":1,"3":1,"9":4,"22":1,"28":1,"54":2,"58":1,"59":1,"66":10,"68":15,"70":3,"73":2,"81":2,"107":2,"115":1,"121":1,"122":2,"131":2,"132":1,"143":1,"174":2,"181":1,"189":1,"208":1,"217":2,"224":1,"225":2,"232":1,"244":2,"249":1,"254":5}}],["=",{"2":{"1":2,"2":2,"3":1,"5":8,"9":14,"20":1,"22":3,"24":6,"28":1,"31":2,"32":5,"34":8,"36":3,"44":4,"46":5,"48":3,"49":2,"53":59,"55":1,"58":2,"59":46,"62":4,"63":3,"65":40,"66":7,"67":6,"68":9,"69":6,"70":14,"71":25,"72":15,"73":3,"74":13,"80":52,"81":26,"95":2,"106":61,"107":10,"109":25,"110":47,"115":10,"116":3,"117":3,"118":2,"120":29,"121":1,"123":23,"125":4,"126":7,"129":8,"130":9,"131":5,"134":21,"135":12,"136":20,"137":21,"139":2,"140":2,"141":12,"143":7,"144":1,"149":5,"154":6,"162":9,"163":6,"164":19,"165":1,"166":8,"167":1,"168":3,"169":3,"171":3,"172":4,"173":1,"174":17,"175":8,"178":6,"179":6,"180":5,"181":5,"182":13,"184":8,"187":7,"188":21,"189":5,"190":12,"191":1,"192":8,"193":7,"194":12,"205":2,"206":12,"207":3,"208":19,"209":6,"210":8,"213":8,"214":3,"215":1,"216":12,"217":13,"218":8,"221":4,"222":2,"223":3,"224":5,"225":21,"226":8,"229":7,"230":6,"231":5,"232":5,"233":10,"234":3,"236":136,"237":8,"239":2,"242":13,"243":19,"244":20,"245":11,"246":8,"248":21,"249":19,"250":19,"251":7,"254":55,"255":20,"256":38,"257":4,"258":14,"259":13,"260":7}}],["jointly",{"2":{"179":1}}],["journal",{"2":{"22":1}}],["jvi",{"2":{"138":1}}],["jvp",{"0":{"27":1},"2":{"27":1,"140":6,"141":10,"170":4,"172":4}}],["j∈rd×d",{"2":{"138":1}}],["jet`",{"2":{"48":1}}],["jet",{"0":{"48":1},"2":{"48":14}}],["jax",{"2":{"165":1}}],["jacobian`",{"2":{"134":1}}],["jacobian",{"0":{"134":1,"135":1,"137":1,"139":1,"140":1,"141":1,"172":1,"173":1},"1":{"135":1},"2":{"27":11,"28":6,"133":3,"134":3,"135":9,"137":5,"138":4,"139":1,"140":1,"141":17,"170":6,"172":2,"173":1}}],["jamie",{"2":{"24":1}}],["jimmy",{"2":{"24":1}}],["j",{"2":{"9":1,"28":2,"134":2,"135":3,"137":2,"138":1,"141":8}}],["just",{"2":{"3":1,"59":1,"65":1,"67":3,"72":3,"80":1,"125":1,"127":2,"146":1,"163":1,"166":1,"172":1,"233":1}}],["julia∂t",{"2":{"136":1}}],["julia∂x",{"2":{"134":1,"135":1,"137":1}}],["juliazygote",{"2":{"110":1}}],["juliazeros",{"2":{"166":2}}],["juliazerosc64",{"2":{"10":1}}],["juliazerosc32",{"2":{"10":1}}],["juliazerosc16",{"2":{"10":1}}],["juliazeros64",{"2":{"10":1}}],["juliazeros32",{"2":{"10":1}}],["juliazeros16",{"2":{"10":1}}],["juliaxt",{"2":{"117":1}}],["juliax",{"2":{"106":1,"125":1,"149":2,"166":11,"168":2,"174":1}}],["juliaxlogy",{"2":{"54":1}}],["juliaxlogx",{"2":{"54":1}}],["juliaxla",{"2":{"2":1}}],["juliaw",{"2":{"166":1,"174":1}}],["juliaweights",{"2":{"162":1,"163":3}}],["juliaweightnorm",{"2":{"72":1}}],["juliawrappedfunction",{"2":{"71":1}}],["juliann",{"2":{"164":1}}],["julian",{"2":{"81":1,"174":1}}],["julianooplayer",{"2":{"71":1}}],["juliancclbackend",{"2":{"36":1}}],["juliaembedding",{"2":{"70":1}}],["juliaeachslice",{"2":{"54":1}}],["juliavjp",{"2":{"173":1,"217":1}}],["juliavariationalhiddendropout",{"2":{"67":1}}],["juliavector",{"2":{"27":1}}],["juliay",{"2":{"65":2}}],["juliapkg>",{"2":{"155":1}}],["juliaprob",{"2":{"255":1,"256":1,"259":1}}],["juliaprintln",{"2":{"141":1,"171":1}}],["juliapred",{"2":{"106":2}}],["juliaps",{"2":{"110":1,"143":1,"174":1,"182":1}}],["juliapixelshuffle",{"2":{"73":1}}],["juliaparallel",{"2":{"65":1}}],["juliaparameterlength",{"2":{"14":1}}],["juliapairwisefusion",{"2":{"65":1}}],["juliapoissonloss",{"2":{"53":1}}],["juliamodel",{"2":{"106":2,"107":1,"109":2,"110":4,"118":1,"130":1,"135":1,"141":1,"174":1,"214":1,"236":2}}],["juliameanpool",{"2":{"68":1}}],["juliamaxout",{"2":{"71":1}}],["juliamaxpool",{"2":{"68":1}}],["juliamatch",{"2":{"57":1}}],["juliamaeloss",{"2":{"53":1}}],["juliamultigate",{"2":{"54":1}}],["juliamsleloss",{"2":{"53":1}}],["juliamseloss",{"2":{"53":1}}],["juliampibackend",{"2":{"36":1}}],["juliakldivergenceloss",{"2":{"53":1}}],["juliakaiming",{"2":{"9":2}}],["juliahuberloss",{"2":{"53":1}}],["juliahingeloss",{"2":{"53":1}}],["juliaupsample",{"2":{"73":1}}],["juliaupdate",{"2":{"15":1}}],["juliausing",{"2":{"48":1,"62":8,"63":4,"80":1,"81":1,"106":1,"109":1,"110":1,"115":1,"120":1,"123":1,"129":1,"131":1,"133":1,"149":1,"150":1,"155":1,"162":2,"164":1,"165":1,"167":1,"170":1,"174":1,"175":1,"177":1,"184":1,"186":1,"192":1,"193":2,"194":2,"205":1,"210":1,"212":1,"218":1,"220":1,"226":1,"228":1,"237":1,"239":1,"246":1,"251":1,"253":1,"260":1}}],["juliaunfreeze",{"2":{"31":2}}],["juliajvp",{"2":{"172":1}}],["juliajet",{"2":{"48":1}}],["juliajacobian",{"2":{"27":1}}],["juliajulia>",{"2":{"5":1,"9":2,"31":1,"32":1,"34":1,"44":1,"46":1,"48":1,"49":1,"53":15,"59":6,"61":2,"65":3,"71":4,"72":3}}],["julialux",{"2":{"188":1}}],["julialuxops",{"2":{"54":1}}],["julialength",{"2":{"166":1}}],["julialang",{"2":{"109":1,"116":3,"134":2,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"236":3,"237":1,"244":4,"246":1,"251":1,"260":1}}],["julialayer",{"2":{"32":1}}],["julialayernorm",{"2":{"24":1,"72":1}}],["julialstmcell",{"2":{"69":1}}],["julialppool",{"2":{"68":1}}],["julialossfn",{"2":{"174":1}}],["julialoss",{"2":{"106":1,"257":1}}],["julialocal",{"2":{"38":1}}],["julialoaded",{"2":{"3":1}}],["juliabegin",{"2":{"213":1,"217":1,"259":1}}],["juliabackend",{"2":{"143":1,"164":2}}],["juliabatchnorm",{"2":{"24":1,"72":1}}],["juliabatched",{"2":{"19":1,"28":1}}],["juliabranchlayer",{"2":{"65":1}}],["juliabilinear",{"2":{"70":1}}],["juliabidirectionalrnn",{"2":{"69":1}}],["juliabinaryfocalloss",{"2":{"53":1}}],["juliabinarycrossentropyloss",{"2":{"53":1}}],["juliabias",{"2":{"20":2}}],["juliabcast",{"2":{"39":1}}],["juliafig",{"2":{"250":2}}],["juliaf",{"2":{"171":1,"172":1}}],["juliafmap",{"2":{"106":1}}],["juliaflattenlayer",{"2":{"71":1}}],["juliafluxlayer",{"2":{"44":1}}],["juliaf64",{"2":{"56":1}}],["juliaf32",{"2":{"56":1}}],["juliaf16",{"2":{"56":1}}],["juliafor",{"2":{"169":1}}],["juliafoldl",{"2":{"54":1}}],["juliafocalloss",{"2":{"53":1}}],["juliafromfluxadaptor",{"2":{"44":1}}],["juliafrozenlayer",{"2":{"31":1}}],["juliafreeze",{"2":{"31":2}}],["juliafunction",{"2":{"106":2,"110":1,"117":1,"129":2,"134":1,"136":1,"137":1,"139":1,"140":1,"141":1,"174":1,"178":1,"179":2,"180":1,"182":1,"187":1,"190":1,"206":1,"208":1,"213":1,"217":1,"221":1,"222":2,"223":1,"225":1,"229":1,"230":2,"231":1,"233":1,"242":1,"244":1,"249":1,"254":4,"255":1,"256":1}}],["juliafunctional",{"2":{"3":1}}],["juliafused",{"2":{"21":1,"23":1}}],["juliafast",{"2":{"18":2}}],["juliats",{"2":{"245":1}}],["juliatstate",{"2":{"217":1}}],["juliatest",{"2":{"49":1}}],["juliatestmode",{"2":{"15":1}}],["juliatosimplechainsadaptor",{"2":{"46":1}}],["juliatotal",{"2":{"38":1}}],["juliatr",{"2":{"141":1,"191":1}}],["juliatry",{"2":{"109":1,"110":1}}],["juliatrain",{"2":{"191":1,"233":4,"235":1}}],["juliatrainstate",{"2":{"52":1}}],["juliatrainmode",{"2":{"15":1}}],["juliatruncated",{"2":{"9":1}}],["juliadudt",{"2":{"209":1}}],["juliadataloader",{"2":{"154":2}}],["juliadata",{"2":{"143":1}}],["juliadicecoeffloss",{"2":{"53":1}}],["juliadistributedutils",{"2":{"143":1}}],["juliadistributeddatacontainer",{"2":{"41":1}}],["juliadistributedoptimizer",{"2":{"40":1}}],["juliadisplay",{"2":{"13":1}}],["juliadropout",{"2":{"22":1,"67":1}}],["juliadense",{"2":{"70":1}}],["juliadebuglayer",{"2":{"33":1}}],["juliadeviceiterator",{"2":{"5":1}}],["juliadefault",{"2":{"3":1}}],["juliacdev",{"2":{"125":1,"126":1}}],["juliachain",{"2":{"65":1}}],["juliacheck",{"2":{"13":1}}],["juliacrossentropyloss",{"2":{"53":1}}],["juliaconst",{"2":{"106":1,"181":1,"189":1,"216":1,"224":1,"232":1,"249":1,"256":2,"257":2}}],["juliaconvtranspose",{"2":{"66":1}}],["juliaconv",{"2":{"66":1}}],["juliacontains",{"2":{"13":1}}],["juliacompute",{"2":{"52":1}}],["juliacpu",{"2":{"2":1}}],["juliaadtype",{"2":{"258":1}}],["juliaadaptor",{"2":{"188":1}}],["juliaadaptivemeanpool",{"2":{"68":1}}],["juliaadaptivemaxpool",{"2":{"68":1}}],["juliaadaptivelppool",{"2":{"68":1}}],["juliaadapt",{"2":{"44":1,"46":1}}],["juliaanalytical",{"2":{"243":1}}],["juliaallreduce",{"2":{"39":1}}],["juliaalphadropout",{"2":{"67":1}}],["juliaalpha",{"2":{"22":1}}],["juliaapply",{"2":{"13":1,"52":2}}],["juliaabstract",{"2":{"12":3}}],["juliarng",{"2":{"81":1,"109":1,"116":1,"129":1,"165":1,"169":1,"194":2,"213":1}}],["juliarnncell",{"2":{"69":1}}],["juliareversesequence",{"2":{"71":1}}],["juliarecurrence",{"2":{"69":1}}],["juliarecursive",{"2":{"55":6}}],["juliareshapelayer",{"2":{"71":1}}],["juliares",{"2":{"65":2}}],["juliareset",{"2":{"3":1}}],["juliarepeatedlayer",{"2":{"65":1}}],["juliareplicate",{"2":{"13":1}}],["juliareduce",{"2":{"39":1}}],["juliarandom",{"2":{"169":1}}],["juliarandc64",{"2":{"10":1}}],["juliarandc32",{"2":{"10":1}}],["juliarandc16",{"2":{"10":1}}],["juliarand64",{"2":{"10":1}}],["juliarandnc64",{"2":{"10":1}}],["juliarandnc32",{"2":{"10":1}}],["juliarandnc16",{"2":{"10":1}}],["juliarandn64",{"2":{"10":1}}],["juliarandn32",{"2":{"10":1}}],["juliarandn16",{"2":{"10":1}}],["juliarand32",{"2":{"10":1}}],["juliarand16",{"2":{"10":1}}],["juliaopt",{"2":{"143":1,"215":1}}],["juliaoutputsize",{"2":{"16":1}}],["juliaonesc64",{"2":{"10":1}}],["juliaonesc32",{"2":{"10":1}}],["juliaonesc16",{"2":{"10":1}}],["juliaones64",{"2":{"10":1}}],["juliaones32",{"2":{"10":1}}],["juliaones16",{"2":{"10":1}}],["juliaorthogonal",{"2":{"9":1}}],["juliaimport",{"2":{"79":1}}],["juliaistraining",{"2":{"54":1}}],["juliaisleaf",{"2":{"3":1}}],["juliainput",{"2":{"164":1}}],["juliainternal",{"2":{"25":1}}],["juliainstancenorm",{"2":{"24":1,"72":1}}],["juliainitialized",{"2":{"37":1}}],["juliainitialize",{"2":{"37":1}}],["juliainitialstates",{"2":{"15":1}}],["juliainitialparameters",{"2":{"14":1}}],["juliaidentity",{"2":{"9":1}}],["julia>",{"2":{"5":4,"9":3,"32":6,"34":3,"44":6,"46":7,"48":1,"49":2,"53":45,"59":34,"61":2,"65":3,"71":8}}],["juliastruct",{"2":{"117":1,"130":1,"131":1,"179":1,"207":1,"230":1,"234":1,"241":1}}],["juliastatefulrecurrentcell",{"2":{"69":1}}],["juliastatefulluxlayer",{"2":{"58":1}}],["juliastatelength",{"2":{"15":1}}],["juliastateless",{"2":{"13":1}}],["juliaselectdim",{"2":{"71":1}}],["juliasetup",{"2":{"13":1}}],["juliaset",{"2":{"4":2,"60":1}}],["juliascale",{"2":{"70":1}}],["juliaskipconnection",{"2":{"65":1}}],["juliasquaredhingeloss",{"2":{"53":1}}],["juliasize",{"2":{"166":1}}],["juliasiamesecontrastiveloss",{"2":{"53":1}}],["juliasingle",{"2":{"52":2}}],["juliasimplechainslayer",{"2":{"46":1}}],["juliasynchronize",{"2":{"39":1}}],["juliashare",{"2":{"34":1}}],["juliasparse",{"2":{"9":1}}],["juliasupported",{"2":{"3":1}}],["juliagdev",{"2":{"125":1,"164":1}}],["juliagrucell",{"2":{"69":1}}],["juliagroupnorm",{"2":{"24":1,"72":1}}],["juliaglobalmeanpool",{"2":{"68":1}}],["juliaglobalmaxpool",{"2":{"68":1}}],["juliagloballppool",{"2":{"68":1}}],["juliaglorot",{"2":{"9":2}}],["juliagenericlossfunction",{"2":{"53":1}}],["juliagetproperty",{"2":{"54":1}}],["juliaget",{"2":{"3":2,"37":1}}],["juliagpu",{"2":{"1":1,"2":1}}],["julia",{"0":{"165":1},"1":{"166":1,"167":1,"168":1,"169":1,"170":1,"171":1,"172":1,"173":1,"174":1,"175":1},"2":{"1":1,"21":1,"25":1,"33":1,"44":1,"48":1,"49":1,"50":1,"59":3,"60":1,"61":2,"74":1,"75":1,"77":1,"79":2,"80":3,"82":1,"84":1,"85":1,"102":2,"106":1,"121":2,"122":2,"124":1,"125":1,"129":1,"133":1,"134":1,"136":1,"137":1,"141":1,"148":1,"149":1,"161":2,"162":4,"164":2,"165":2,"166":7,"169":1,"170":1,"175":12,"183":2,"184":12,"192":9,"210":12,"218":12,"226":12,"236":2,"237":12,"242":1,"246":12,"247":1,"248":1,"249":4,"250":2,"251":9,"254":1,"260":9}}],["jld2",{"2":{"177":1,"183":3}}],["jll",{"2":{"175":2,"184":2,"210":2,"218":2,"226":2,"237":2,"246":2}}],["jl`",{"2":{"23":1,"134":2}}],["jl",{"0":{"40":1,"41":1,"48":1,"61":1,"91":1,"94":1,"97":1,"98":1,"101":1,"106":1,"144":1,"155":1,"204":1},"1":{"92":1,"93":1,"95":1,"96":1,"99":1,"100":1,"102":1,"103":1,"104":1,"105":1,"107":1,"145":1,"146":1,"156":1,"157":1,"158":1,"159":1,"160":1,"161":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"0":1,"2":3,"3":8,"7":2,"11":5,"12":2,"13":1,"15":2,"16":1,"17":1,"18":3,"19":1,"23":2,"25":2,"27":1,"28":1,"37":8,"40":1,"41":2,"44":1,"46":4,"47":1,"48":1,"49":6,"52":7,"53":2,"54":1,"60":1,"61":4,"69":1,"77":2,"79":2,"81":3,"82":1,"83":5,"86":8,"87":7,"89":1,"95":3,"98":5,"100":1,"102":1,"106":3,"107":1,"109":2,"116":6,"124":1,"127":5,"129":1,"134":4,"142":1,"143":1,"144":1,"145":1,"146":1,"151":2,"152":2,"153":2,"154":3,"155":3,"157":1,"161":2,"162":1,"164":3,"165":1,"167":1,"168":1,"170":4,"172":1,"174":1,"175":1,"176":2,"184":1,"185":4,"188":1,"191":1,"192":1,"203":3,"204":3,"208":3,"210":1,"215":1,"218":1,"226":1,"227":1,"230":1,"236":6,"237":1,"238":3,"244":4,"246":1,"247":2,"251":1,"252":1,"255":1,"256":1,"260":1}}],["p=res",{"2":{"259":1}}],["p=θ",{"2":{"257":1}}],["p=params",{"2":{"256":1}}],["p=2",{"2":{"68":3}}],["p^",{"2":{"255":1,"256":1}}],["p^2",{"2":{"255":1}}],["pdes",{"2":{"238":1}}],["pde",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"243":8,"244":2,"245":3}}],["physics",{"2":{"242":6,"244":108,"256":1}}],["philosophies",{"2":{"194":1}}],["phase",{"2":{"72":4}}],["pcie",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["pptional",{"2":{"80":1,"81":1}}],["pytorch",{"2":{"66":3,"70":1,"77":1,"104":3,"105":1,"165":1}}],["pkg>",{"2":{"155":2}}],["pkg",{"2":{"61":4,"62":9,"63":2,"79":2,"80":1,"82":2,"155":1,"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2,"251":2,"260":2}}],["p2",{"2":{"53":6}}],["p1",{"2":{"53":4}}],["pipe",{"2":{"207":1}}],["pipeline",{"2":{"118":1}}],["pipelines",{"2":{"52":1}}],["pitfalls",{"0":{"148":1},"1":{"149":1,"150":1,"151":1,"152":1,"153":1,"154":1}}],["pinn2dpde",{"2":{"244":8}}],["pinn",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"238":1,"241":4,"244":5,"245":1}}],["pinns",{"2":{"136":1,"238":1}}],["pin",{"2":{"108":2}}],["pixel",{"2":{"73":2}}],["pixelshuffle",{"2":{"73":1}}],["pixels",{"2":{"68":3,"73":2}}],["pi",{"2":{"53":1}}],["pmlr",{"2":{"24":1}}],["p",{"2":{"22":7,"44":1,"59":2,"67":12,"206":2,"208":4,"209":2,"230":6,"234":2,"250":2,"254":3,"255":6,"256":2,"258":1}}],["peel",{"2":{"179":1,"180":1}}],["peel`",{"2":{"179":1}}],["people",{"2":{"129":1}}],["pesky",{"2":{"110":1}}],["pended",{"2":{"102":1}}],["penalties",{"2":{"59":1}}],["penultimate",{"2":{"20":1,"21":1,"69":2}}],["perspective",{"2":{"254":1}}],["perceptron",{"2":{"211":1}}],["per",{"2":{"72":4,"135":1,"249":2}}],["periodic",{"2":{"66":1}}],["period",{"2":{"30":1}}],["performs",{"2":{"125":1,"208":1}}],["performing",{"2":{"68":3}}],["performed",{"2":{"46":2,"66":1,"249":1}}],["perform",{"2":{"13":1,"39":2,"52":2,"66":3,"194":1,"249":1,"254":1}}],["performance",{"0":{"148":1,"153":1},"1":{"149":1,"150":1,"151":1,"152":1,"153":1,"154":1},"2":{"9":2,"18":1,"33":1,"52":1,"59":3,"87":2,"88":2,"89":1,"124":2,"148":2,"149":2,"153":1,"164":1}}],["perfect",{"2":{"9":1}}],["plot",{"2":{"217":1,"248":3,"250":9,"255":1,"256":1,"259":1}}],["plotting",{"0":{"209":1}}],["plugin",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["plus",{"2":{"31":1,"34":1,"59":3,"65":6,"72":7,"80":1,"81":1,"109":2,"110":2,"115":1,"188":2,"214":1,"256":1}}],["platform",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["plan",{"2":{"133":1}}],["places",{"2":{"59":2}}],["place",{"2":{"18":1,"20":2,"55":5,"124":1,"146":1}}],["please",{"2":{"4":2,"74":2,"79":1,"81":1,"84":1,"89":1,"111":1,"133":1,"147":1,"179":1,"202":1,"203":3,"204":1,"238":1}}],["push",{"2":{"257":1}}],["pullback",{"2":{"110":2}}],["pull",{"2":{"84":1,"95":1}}],["publisher",{"2":{"74":1}}],["public",{"2":{"3":1,"53":1}}],["purpose",{"2":{"170":1}}],["purposes",{"2":{"57":1,"131":1,"208":1,"229":1}}],["pure",{"2":{"13":1,"75":1,"76":1,"77":1,"168":1}}],["pseudorandom",{"2":{"169":1}}],["pseudo",{"2":{"67":3,"165":1}}],["ps",{"2":{"12":2,"13":5,"31":2,"32":18,"34":17,"39":2,"44":4,"46":2,"53":2,"57":1,"58":5,"59":22,"65":4,"71":9,"80":3,"81":6,"106":16,"107":6,"109":5,"110":12,"113":3,"115":2,"116":2,"117":6,"118":2,"120":9,"121":7,"122":6,"123":2,"129":7,"130":7,"131":10,"133":3,"134":9,"135":9,"136":8,"137":11,"139":2,"140":2,"141":9,"143":2,"149":5,"164":7,"174":7,"179":4,"181":2,"182":3,"183":3,"189":2,"190":2,"193":5,"194":7,"208":4,"209":1,"216":1,"217":1,"222":2,"224":2,"225":2,"230":2,"231":5,"232":2,"233":2,"234":3,"236":15,"242":4,"244":3,"249":10,"250":1,"256":2}}],["power",{"2":{"250":1}}],["powerful",{"2":{"166":1}}],["pooling",{"0":{"68":1},"2":{"68":21,"105":1}}],["poisson",{"2":{"53":1}}],["poissonloss",{"2":{"53":2}}],["points",{"2":{"174":1,"194":1,"213":1,"248":4,"249":1,"250":1}}],["point",{"0":{"56":1},"2":{"4":2,"18":1,"56":3,"108":2,"119":1,"124":1,"143":1,"166":1,"255":1,"256":1}}],["populated",{"2":{"179":2}}],["populate",{"2":{"33":1}}],["polynomial",{"0":{"211":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1},"2":{"211":1,"213":1}}],["polyester",{"2":{"25":1}}],["polyalgorithm",{"2":{"23":1}}],["position=",{"2":{"206":1,"209":1,"256":1,"259":1}}],["position",{"2":{"71":2,"254":1}}],["possibly",{"2":{"66":2,"68":3,"87":1}}],["possible",{"2":{"1":1,"2":1,"12":1,"16":1,"18":3,"20":1,"21":2,"23":1,"37":2,"44":2,"88":1,"134":1}}],["posterior",{"2":{"250":2}}],["posted",{"2":{"133":1}}],["post",{"2":{"12":1,"46":1}}],["potentially",{"2":{"12":1,"16":1,"46":1}}],["painful",{"2":{"108":1}}],["pairwisefusion",{"2":{"65":3}}],["pairs",{"2":{"53":1,"236":13}}],["pair",{"2":{"53":1,"66":2,"69":1}}],["past",{"2":{"77":1}}],["passes",{"2":{"65":2,"111":1,"168":1}}],["passed",{"2":{"15":1,"24":1,"34":2,"48":1,"49":1,"59":1,"65":6,"69":5,"71":6,"76":1,"104":1,"130":1}}],["passing",{"2":{"13":1,"59":2,"65":1,"72":1,"143":2,"162":2}}],["pass",{"2":{"13":1,"27":2,"49":1,"50":1,"54":2,"56":1,"59":3,"65":1,"66":1,"70":2,"72":1,"77":1,"81":1,"102":1,"110":3,"129":1,"143":1,"162":1,"179":3}}],["pal2023efficient",{"2":{"74":1}}],["pal2023lux",{"2":{"74":1}}],["pal",{"2":{"74":2}}],["pattern",{"2":{"53":1,"106":1,"154":1,"248":1}}],["path",{"2":{"15":1,"65":1,"103":2,"120":2,"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2,"251":1,"254":4,"260":1}}],["page",{"2":{"29":1,"30":1,"31":1,"58":1,"90":1,"106":1,"108":1,"164":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["pad",{"2":{"66":6,"68":3}}],["pad=0",{"2":{"66":2,"68":3}}],["padding",{"2":{"9":1,"66":9,"68":12}}],["pads",{"2":{"9":1}}],["parse",{"2":{"135":1}}],["parallelism",{"2":{"154":1}}],["parallel=true",{"2":{"154":2}}],["parallel",{"0":{"142":1},"1":{"143":1,"144":1,"145":1,"146":1,"147":1},"2":{"32":1,"65":12,"71":1}}],["params=nothing",{"2":{"254":2}}],["params",{"2":{"31":14,"32":6,"52":2,"72":3,"120":3,"123":1,"254":4,"255":3,"256":6,"257":2,"258":1,"259":1}}],["parameterized",{"2":{"250":1}}],["parameterization",{"2":{"74":1,"247":1}}],["parameterlength",{"2":{"12":1,"14":1,"129":4,"130":1,"223":1,"249":3}}],["parameter",{"0":{"31":1,"131":1},"2":{"9":3,"31":2,"34":2,"53":1,"59":6,"66":4,"71":1,"72":2,"76":1,"77":4,"80":1,"110":1,"122":2,"129":5,"130":2,"131":5,"143":1,"194":1,"208":2,"233":1,"247":1,"249":2}}],["parameters",{"0":{"14":1,"34":1,"119":1,"122":1,"137":1},"1":{"120":1,"121":1,"122":1,"123":1},"2":{"3":1,"9":1,"12":5,"13":1,"14":2,"31":13,"32":7,"33":1,"34":23,"40":1,"44":4,"46":1,"52":9,"53":1,"56":2,"57":4,"58":5,"59":17,"65":24,"66":2,"69":9,"70":4,"71":2,"72":31,"75":1,"77":2,"80":5,"81":7,"95":2,"105":1,"107":1,"109":6,"110":7,"115":5,"119":2,"122":3,"127":1,"129":5,"130":3,"131":7,"133":1,"137":1,"143":1,"170":1,"174":6,"179":2,"182":2,"183":1,"188":12,"190":2,"194":6,"208":5,"214":3,"216":1,"217":3,"222":1,"225":4,"230":1,"233":2,"244":1,"249":79,"250":2,"256":6}}],["parts",{"2":{"208":1}}],["party",{"0":{"203":1}}],["particle",{"2":{"255":2,"256":1}}],["particles",{"2":{"254":1}}],["participate",{"0":{"113":1}}],["particular",{"0":{"120":1},"2":{"108":1,"120":1}}],["partition",{"2":{"41":1}}],["part",{"0":{"122":1,"123":1},"2":{"3":1,"54":1,"90":1,"134":1,"202":1,"204":1,"230":1}}],["packages",{"0":{"42":1,"82":1,"205":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"3":3,"27":4,"28":2,"46":1,"52":2,"60":1,"80":1,"81":1,"82":3,"83":1,"85":2,"88":3,"89":2,"95":1,"98":1,"124":1,"148":1,"153":1,"160":1,"161":3,"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1,"247":1}}],["package",{"0":{"177":1,"186":1,"212":1,"220":1,"228":1,"239":1,"253":1},"2":{"0":1,"3":1,"4":3,"6":1,"7":1,"40":1,"47":3,"62":1,"63":1,"79":1,"82":1,"85":1,"86":1,"89":1,"98":3,"102":1,"125":2,"158":1,"163":1,"167":1,"203":1,"238":1}}],["practitioners",{"2":{"208":1}}],["prngs",{"2":{"169":1}}],["prng",{"2":{"165":1,"169":2}}],["prs",{"2":{"133":1}}],["primarily",{"2":{"249":1}}],["primitives",{"0":{"39":1,"152":1},"2":{"152":1,"249":1}}],["principles",{"0":{"76":1}}],["printf",{"2":{"81":2,"106":1,"107":1,"174":2,"177":1,"182":2,"186":1,"190":1,"205":1,"208":1,"212":1,"217":1,"220":1,"225":2,"228":1,"233":1,"239":1,"244":1,"253":1,"257":1}}],["printouts",{"2":{"77":1}}],["printout",{"2":{"59":1}}],["print",{"2":{"57":1,"59":1}}],["printing",{"2":{"46":1,"71":1,"77":1,"81":1}}],["println",{"2":{"32":1,"109":2,"110":2,"129":2,"130":3,"131":2,"134":2,"135":2,"136":2,"137":2,"141":6,"168":2,"169":2,"171":2,"172":1,"173":1,"174":3,"175":2,"184":2,"192":2,"210":2,"218":2,"225":1,"226":2,"237":2,"246":2,"251":2,"260":2}}],["prints",{"2":{"4":2,"110":1}}],["printed",{"2":{"2":1,"13":1,"44":1,"59":1}}],["priority",{"2":{"88":2}}],["prior",{"2":{"66":1,"249":1}}],["pr",{"2":{"16":1,"202":1,"203":1}}],["precompile",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["precision",{"0":{"56":1},"2":{"21":2,"56":1,"149":3,"166":1}}],["press",{"2":{"155":1}}],["pressing",{"2":{"79":1}}],["presence",{"2":{"59":1}}],["presented",{"2":{"172":1}}],["presently",{"2":{"106":1}}],["present",{"2":{"44":1,"46":1,"48":1,"54":1,"66":2,"69":10,"70":2,"102":1,"109":1,"110":1,"125":2}}],["preserving",{"2":{"46":1,"73":1}}],["preservation",{"2":{"44":1}}],["preserved",{"2":{"46":1}}],["preserve",{"2":{"44":7,"46":1}}],["pretty",{"2":{"71":1,"77":1,"81":1,"134":1,"144":1}}],["preds",{"2":{"249":2}}],["pred=ŷ",{"2":{"181":1}}],["predictive",{"2":{"250":1}}],["predictions",{"2":{"217":1,"249":2,"250":2}}],["prediction",{"0":{"250":1},"2":{"53":2,"249":1,"250":1}}],["predictor",{"2":{"245":1}}],["predict",{"2":{"178":1,"250":6}}],["predicted",{"2":{"53":3,"189":2,"224":2,"232":2,"250":2}}],["pred",{"2":{"53":8,"106":4,"181":5,"208":2,"209":3,"217":2,"245":4,"257":4}}],["preprint",{"2":{"24":2,"72":1}}],["preprocessing",{"2":{"13":1}}],["prevents",{"2":{"102":1}}],["prevent",{"2":{"22":2,"37":1,"52":1,"53":1,"102":1,"129":1}}],["previously",{"2":{"12":1,"16":1,"77":1,"95":1,"104":1,"145":1}}],["previous",{"2":{"5":1,"12":1,"77":1,"135":1}}],["pre",{"2":{"12":1,"16":1,"66":1,"79":1,"80":1,"102":1}}],["prefer",{"2":{"152":1}}],["preferably",{"2":{"25":1}}],["preferred",{"2":{"5":1,"25":1}}],["preferencetools",{"2":{"155":2}}],["preference",{"2":{"1":2,"2":1,"57":1,"60":2,"133":1,"149":1,"151":1,"155":6,"158":2,"159":1,"160":3,"161":1}}],["preferences",{"0":{"1":1,"155":1},"1":{"156":1,"157":1,"158":1,"159":1,"160":1,"161":1},"2":{"57":1,"60":4,"102":1,"151":2,"155":4,"157":3,"160":2}}],["progress",{"2":{"247":3}}],["programming",{"2":{"165":1}}],["proto",{"2":{"190":4}}],["probabilistic",{"2":{"249":2}}],["probability",{"2":{"22":3,"53":1,"67":3}}],["prob",{"2":{"206":2,"208":9,"209":2,"230":4,"234":2,"255":1,"256":1,"257":1,"259":1}}],["problem",{"0":{"240":1},"2":{"109":1,"110":1,"134":1,"137":1,"168":1,"174":2,"208":1,"214":1,"234":1,"236":1,"240":1}}],["problematic",{"2":{"108":1,"109":1}}],["problems",{"0":{"109":1},"2":{"75":1,"106":1}}],["promotion",{"0":{"149":1},"2":{"149":1}}],["promotions",{"2":{"57":1,"149":1}}],["promote",{"2":{"21":1,"149":1}}],["project",{"2":{"47":1}}],["propagating",{"2":{"58":1}}],["propagated",{"2":{"65":1}}],["propagate",{"2":{"13":1}}],["proper",{"2":{"118":1,"165":1}}],["properly",{"2":{"33":1}}],["properties",{"2":{"28":1,"65":1}}],["proportion",{"2":{"9":3}}],["produce",{"2":{"54":1,"95":1}}],["products",{"2":{"27":2}}],["product",{"0":{"139":1,"140":1,"172":1,"173":1},"2":{"27":8,"133":2,"138":4,"139":1,"140":1,"170":5,"172":2,"173":1,"243":1,"245":1}}],["prod",{"2":{"9":1}}],["proceeding",{"2":{"129":1}}],["proceedings",{"2":{"9":5,"24":1,"53":2}}],["proceeds",{"2":{"69":8}}],["processor",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["processes",{"2":{"40":1,"41":1,"143":3,"154":1}}],["processing",{"2":{"22":1,"69":1}}],["process",{"2":{"4":1,"143":1,"154":1,"187":1,"229":1,"243":1}}],["provides",{"2":{"46":1,"49":1,"58":1,"80":1,"100":1,"127":1,"155":1,"162":1,"172":1,"204":1,"234":1}}],["provide",{"2":{"9":1,"16":1,"44":1,"52":2,"59":1,"82":1,"85":1,"138":1,"167":1}}],["provided",{"2":{"4":2,"5":1,"22":1,"30":1,"32":2,"54":1,"69":3,"127":1,"143":1,"167":1,"216":1,"250":1}}],["providing",{"2":{"6":1,"188":1}}],["ddot",{"2":{"254":2}}],["ddp",{"2":{"142":1}}],["dt^2",{"2":{"254":1}}],["dt2",{"2":{"254":3}}],["dt",{"2":{"254":21,"255":4,"256":2,"257":2,"259":2}}],["dset",{"2":{"221":4}}],["dnns",{"2":{"108":1}}],["dl",{"2":{"54":1,"256":1}}],["dynamic",{"2":{"47":1}}],["dynamics",{"2":{"9":1}}],["d2",{"2":{"34":5,"115":1,"116":3,"254":3}}],["d2=dense",{"2":{"34":1,"115":1,"118":1}}],["d3",{"2":{"34":9}}],["d3=chain",{"2":{"34":1}}],["d1",{"2":{"34":5,"115":1,"116":3}}],["d1=dense",{"2":{"34":1,"115":1,"118":1}}],["d1×",{"2":{"24":2,"72":2}}],["duration",{"2":{"249":2}}],["during",{"2":{"3":1,"72":4}}],["dudt",{"2":{"208":2,"209":1,"230":4,"234":2}}],["du",{"2":{"206":3}}],["due",{"2":{"135":1,"164":1,"236":1}}],["dummy",{"2":{"80":1}}],["dual",{"2":{"25":1}}],["dmitry",{"2":{"24":1,"72":1}}],["driver",{"2":{"175":3,"184":3,"210":3,"218":3,"226":3,"237":3,"246":3}}],["dr",{"2":{"108":1}}],["drop",{"2":{"53":1,"133":1}}],["dropped",{"2":{"22":2,"27":2}}],["dropout",{"0":{"22":1,"67":1},"2":{"22":14,"67":17,"77":1,"120":1,"123":1,"132":1,"196":1}}],["drawn",{"2":{"9":5,"250":1}}],["d",{"2":{"13":1,"59":11,"66":4,"68":6,"71":2,"73":4,"88":1,"120":2,"121":5,"122":6,"131":4,"178":4,"233":2,"254":2}}],["domain",{"2":{"243":1}}],["domainerror",{"2":{"33":1,"110":2}}],["doubt",{"2":{"129":1}}],["dot",{"2":{"109":1,"116":3,"134":2,"206":2,"209":2,"236":3,"244":4}}],["doi",{"2":{"74":2,"254":2}}],["doing",{"2":{"55":4,"107":1,"222":1,"249":1}}],["documented",{"2":{"102":1}}],["document",{"2":{"85":1}}],["documentations",{"2":{"148":1}}],["documentation",{"2":{"12":1,"13":1,"19":1,"49":1,"60":1,"73":1,"82":1,"84":1,"104":2,"129":1,"159":1,"160":1,"191":1,"203":1}}],["doctor",{"0":{"160":1},"2":{"60":4,"151":1,"160":3}}],["docstrings",{"2":{"129":1}}],["docstring",{"2":{"102":1}}],["docs",{"2":{"59":2,"92":1,"204":1,"244":4,"247":2}}],["downstream",{"2":{"77":1,"89":1,"203":1}}],["downright",{"2":{"77":1,"127":1}}],["down",{"0":{"110":1},"2":{"53":2,"57":1,"109":1,"110":1,"111":1,"130":1,"149":1,"166":2}}],["does",{"2":{"18":1,"24":2,"57":1,"71":1}}],["doesn",{"2":{"18":2,"32":1,"34":1,"54":1,"59":3,"69":1,"113":1,"129":2,"131":1,"132":1,"134":1,"196":1,"223":1}}],["do",{"2":{"11":1,"59":14,"65":1,"71":2,"79":1,"81":2,"97":1,"106":2,"107":1,"129":1,"130":1,"131":1,"132":1,"133":1,"138":2,"149":1,"154":3,"164":1,"166":2,"168":1,"179":1,"180":1,"188":1,"196":1,"208":2,"214":1,"222":1,"230":1,"245":1,"249":1,"250":1,"254":1,"256":1}}],["done",{"2":{"18":2,"32":1,"57":1,"80":1,"125":1,"135":1,"143":1,"161":1}}],["don",{"2":{"4":2,"10":1,"21":1,"23":1,"47":1,"59":2,"77":1,"80":1,"88":2,"89":1,"108":1,"111":1,"129":2,"131":1,"133":1,"135":1,"146":1,"147":1,"166":2,"168":2,"172":1,"178":1,"179":1,"183":1,"187":1,"194":1,"207":1,"222":1,"229":1}}],["digits=2",{"2":{"225":4}}],["digits",{"2":{"166":2}}],["direactly",{"2":{"102":1}}],["direct",{"2":{"102":1}}],["direction",{"2":{"72":2}}],["directly",{"2":{"3":1,"5":1,"11":1,"13":1,"22":1,"31":1,"36":2,"52":1,"55":2,"65":2,"71":1,"73":1,"99":1,"103":1,"126":1,"133":1,"143":1,"162":1,"227":1}}],["diagonal",{"2":{"70":1,"249":1}}],["divisible",{"2":{"66":2}}],["divides",{"2":{"73":1}}],["divide",{"2":{"66":2}}],["diverges",{"2":{"53":1}}],["divergence",{"2":{"53":2}}],["dilation",{"2":{"66":2,"68":3}}],["dilation=1",{"2":{"66":2,"68":3}}],["dice",{"2":{"53":2}}],["dicecoeffloss",{"2":{"53":4}}],["diffeqsol",{"2":{"230":3,"231":1,"236":9}}],["diffeqflux",{"2":{"227":1}}],["differ",{"2":{"170":1}}],["differern",{"2":{"135":1}}],["differences",{"0":{"146":1},"2":{"134":2,"142":1,"194":1}}],["difference",{"2":{"67":1,"95":1,"106":3,"130":1,"134":2,"135":2,"138":1,"235":1,"254":2}}],["differentiate",{"2":{"106":1,"141":1,"168":1}}],["differentiationinterface",{"2":{"133":1,"172":2}}],["differentiation",{"0":{"26":1,"85":1,"133":1,"156":1,"170":1},"1":{"27":1,"28":1,"29":1,"86":1,"87":1,"88":1,"89":1,"134":1,"135":1,"136":1,"137":1,"138":1,"139":1,"140":1,"141":1,"171":1,"172":1,"173":1},"2":{"29":1,"106":1,"131":1,"133":1,"136":1,"156":2,"164":1,"165":1,"172":1}}],["differential",{"2":{"74":1,"257":1}}],["differently",{"2":{"65":1}}],["different",{"0":{"45":1},"1":{"46":1},"2":{"23":1,"25":1,"32":1,"53":1,"60":1,"65":1,"104":1,"135":1,"159":1,"169":1,"194":1,"208":1}}],["diff",{"2":{"134":1,"136":1,"137":1,"170":1}}],["diffractor",{"2":{"86":1}}],["difficulty",{"2":{"9":2}}],["dim",{"2":{"71":6,"107":1,"174":7}}],["dimwhere",{"2":{"53":1}}],["dimensionmismatch",{"2":{"109":2}}],["dimensionality",{"2":{"53":1}}],["dimensional",{"2":{"24":2,"70":3,"73":2,"172":1}}],["dimension",{"0":{"109":1},"2":{"20":2,"21":1,"24":1,"46":1,"54":1,"66":4,"67":2,"68":6,"69":8,"70":1,"71":9,"72":14,"92":1,"108":1,"115":1,"118":1,"163":1}}],["dimensions",{"2":{"9":11,"24":2,"28":2,"46":1,"66":3,"68":9,"69":2,"70":10,"71":5,"72":4,"73":8,"105":1,"174":1,"188":1}}],["dims=3",{"2":{"178":1}}],["dims=ndims",{"2":{"115":1}}],["dims=colon",{"2":{"72":1}}],["dims=2",{"2":{"69":1}}],["dims=",{"2":{"67":2}}],["dims=pad",{"2":{"66":1}}],["dims=1",{"2":{"24":1,"53":1,"243":5}}],["dims",{"2":{"9":11,"22":2,"24":3,"46":2,"53":2,"54":1,"66":1,"67":4,"69":31,"70":55,"71":7,"72":3,"92":1,"129":12,"163":4,"179":9,"180":7,"241":11,"244":2}}],["disabling",{"0":{"161":1},"2":{"153":1}}],["disabled",{"2":{"13":1,"66":2,"70":3,"150":1}}],["disable",{"2":{"13":1,"33":1,"60":3,"102":2,"110":1,"133":1,"150":1,"156":1,"160":1,"161":2}}],["disallow",{"2":{"150":1}}],["disaster",{"2":{"149":1}}],["discrete",{"2":{"230":1,"233":1}}],["discouraged",{"2":{"134":2}}],["discourse",{"2":{"133":1,"134":1,"136":1}}],["discuss",{"2":{"204":1}}],["discussed",{"2":{"111":1}}],["discussions",{"2":{"84":1}}],["discussion",{"2":{"69":1,"136":1,"172":1}}],["disruptive",{"2":{"75":1}}],["dissimilar",{"2":{"53":1}}],["distinguish",{"2":{"194":2}}],["distinction",{"2":{"77":1}}],["distinct",{"2":{"69":1}}],["distance",{"2":{"53":1}}],["distributions",{"2":{"53":1,"249":2}}],["distribution",{"2":{"9":8,"10":12,"53":5,"66":2,"69":6,"70":3,"250":1}}],["distributeddatacontainer",{"2":{"41":1,"143":2,"144":1}}],["distributedoptimizer",{"2":{"40":2,"143":2,"144":1}}],["distributedutils",{"0":{"143":1},"2":{"4":2,"35":1,"36":2,"37":3,"38":2,"39":6,"40":1,"41":1,"142":1,"143":11,"144":5,"145":1,"146":3}}],["distributed",{"0":{"35":1,"142":1},"1":{"36":1,"37":1,"38":1,"39":1,"40":1,"41":1,"143":1,"144":1,"145":1,"146":1,"147":1},"2":{"4":3,"9":2,"36":4,"37":7,"77":1,"143":2}}],["disjoint",{"2":{"34":1}}],["displays",{"2":{"248":1,"250":1}}],["displayed",{"2":{"125":1}}],["display",{"2":{"13":1,"77":1,"130":1}}],["dispatch₋₋₋",{"2":{"236":3}}],["dispatch=",{"2":{"222":1}}],["dispatch=nothing",{"2":{"59":1}}],["dispatching",{"0":{"112":1},"1":{"113":1,"114":1,"115":1,"116":1,"117":1,"118":1}}],["dispatchdoctor",{"2":{"13":1,"60":1,"151":1}}],["dispatches",{"2":{"3":1,"13":1,"25":2,"54":1,"59":1,"127":1}}],["dispatch",{"0":{"113":1,"117":1,"160":1},"2":{"3":1,"47":1,"59":2,"60":4,"113":2,"117":1,"118":1,"132":1,"151":1,"160":3,"164":1}}],["date",{"2":{"203":2}}],["datapoints",{"2":{"213":1}}],["datasize",{"2":{"206":1,"255":1}}],["datasets",{"0":{"221":1},"2":{"178":1,"221":2,"225":1}}],["dataset",{"0":{"178":1,"213":1},"2":{"41":1,"154":2,"178":7,"187":3,"208":1,"213":2,"221":2,"229":3,"248":2}}],["datatype",{"2":{"13":1}}],["dataloaders",{"2":{"178":3,"182":2,"225":3}}],["dataloader",{"0":{"207":1},"2":{"5":8,"100":1,"107":4,"154":5,"178":5,"187":3,"189":2,"190":5,"207":5,"208":4,"221":2,"224":2,"225":9,"228":1,"229":3,"232":2,"233":5,"244":6}}],["data",{"0":{"2":1,"142":1,"154":1,"206":1,"243":1,"248":1},"1":{"143":1,"144":1,"145":1,"146":1,"147":1},"2":{"0":1,"2":1,"3":3,"5":1,"41":2,"52":6,"53":1,"59":10,"66":5,"68":9,"69":1,"71":1,"72":6,"73":2,"77":1,"81":10,"100":1,"106":1,"107":2,"129":1,"143":3,"149":1,"154":5,"164":1,"174":1,"178":8,"187":6,"206":3,"207":3,"208":3,"209":2,"211":1,"213":4,"216":1,"217":5,"224":2,"225":16,"229":6,"242":5,"243":10,"244":113,"248":6,"249":1,"250":7,"255":3,"256":2,"257":1,"259":2}}],["danger",{"2":{"2":1,"4":2,"28":1,"37":1}}],["deadlocks",{"2":{"147":1}}],["deal",{"2":{"136":1,"230":1}}],["deactivate",{"2":{"69":3}}],["derivative",{"2":{"136":1}}],["derivatives",{"2":{"134":1,"135":1,"172":1,"240":1,"256":1}}],["demonstration",{"2":{"208":1,"229":1,"238":1}}],["demonstrative",{"2":{"131":1}}],["demonstrate",{"2":{"134":1,"174":1,"185":1,"194":1}}],["de",{"2":{"134":2}}],["decides",{"2":{"196":1}}],["decision",{"2":{"155":1}}],["decouples",{"2":{"72":1}}],["declared",{"2":{"59":1}}],["decrease",{"2":{"18":1}}],["debuglayer",{"2":{"33":3,"109":4,"110":4}}],["debug",{"2":{"33":5,"103":1,"108":4,"109":6,"110":10,"149":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["debugging",{"0":{"33":1,"108":1},"1":{"109":1,"110":1,"111":1},"2":{"33":3,"57":1,"108":2}}],["density",{"2":{"249":1}}],["denselayerparameters",{"2":{"131":3}}],["dense",{"2":{"23":2,"31":2,"32":18,"34":4,"44":2,"46":3,"53":2,"59":10,"65":12,"70":2,"71":1,"72":12,"80":17,"81":6,"93":1,"104":1,"106":9,"107":3,"109":17,"110":14,"115":6,"116":2,"118":2,"120":9,"121":3,"123":3,"129":1,"131":2,"134":2,"135":2,"136":4,"137":4,"141":4,"149":1,"152":1,"160":1,"164":2,"174":2,"179":2,"180":1,"188":9,"193":5,"208":3,"214":4,"217":4,"223":4,"231":5,"236":45,"241":4,"249":4,"256":6}}],["denom",{"2":{"255":3,"256":3}}],["denominator",{"2":{"24":4,"72":4}}],["denotes",{"2":{"3":4}}],["detour",{"2":{"170":1}}],["detection",{"2":{"53":2}}],["detected",{"2":{"33":1,"110":2,"244":1}}],["deterministic",{"2":{"9":1,"77":1}}],["determined",{"2":{"80":1}}],["determines",{"2":{"46":1}}],["determine",{"2":{"3":1,"22":2,"24":2,"55":1,"146":1,"249":1}}],["details",{"0":{"195":1},"1":{"196":1},"2":{"19":1,"22":2,"24":4,"33":1,"34":1,"44":1,"46":2,"49":1,"58":1,"59":2,"60":1,"72":1,"73":1,"92":1,"95":1,"97":1,"102":1,"104":2,"129":1,"156":1,"159":1,"160":1,"167":1}}],["detailed",{"2":{"12":1,"31":1,"109":1}}],["delta=0",{"2":{"53":1}}],["delta",{"2":{"53":2}}],["delving",{"2":{"9":2}}],["deleted",{"2":{"1":1}}],["descent",{"2":{"174":2}}],["describe",{"2":{"164":1}}],["describes",{"2":{"46":1,"108":1,"255":1,"256":1}}],["described",{"2":{"9":3,"65":2}}],["despite",{"2":{"52":1}}],["destructure",{"2":{"44":3,"77":2,"197":1}}],["desirable",{"2":{"161":1,"249":1}}],["desired",{"2":{"1":1,"66":1,"105":1}}],["design",{"0":{"76":1},"2":{"5":1,"75":1,"194":1}}],["depots",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["depot",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["depwarn",{"2":{"102":1}}],["deprecated",{"2":{"92":1,"244":4}}],["deprecation",{"2":{"30":1,"98":2,"134":2}}],["deprecations",{"2":{"2":1,"102":1}}],["depthwise",{"2":{"66":2}}],["depths",{"2":{"32":1}}],["dependent",{"0":{"115":1},"2":{"114":1}}],["dependencies",{"0":{"153":1},"2":{"47":1,"95":1,"161":1,"249":1}}],["dependency",{"2":{"6":1,"11":1,"47":1,"127":1,"135":1}}],["depend",{"2":{"11":1,"24":1,"127":1,"129":1}}],["depending",{"2":{"9":1,"11":1,"13":1}}],["defiing",{"0":{"256":1}}],["definition",{"0":{"240":1}}],["definitions",{"2":{"130":1}}],["definitely",{"2":{"141":1}}],["defining",{"0":{"81":1,"181":1},"2":{"0":1,"3":2,"16":1,"59":1,"81":1,"106":1,"113":1,"127":1,"129":1,"130":1,"131":1,"179":1,"180":2,"188":1,"222":1}}],["define",{"0":{"188":1,"190":1,"207":1,"224":1,"230":1,"232":1,"241":1,"242":1,"254":1},"2":{"12":1,"25":1,"81":1,"110":2,"113":2,"114":1,"117":1,"127":2,"129":3,"130":1,"131":2,"164":1,"174":1,"179":3,"180":1,"181":1,"194":2,"207":1,"208":1,"230":1,"241":1,"242":1,"249":3,"254":2,"256":2,"257":2}}],["defines",{"2":{"11":1,"12":2,"55":1,"254":1,"255":1}}],["defined",{"2":{"3":1,"13":1,"16":1}}],["defer",{"2":{"172":1}}],["defaults",{"0":{"104":1},"2":{"58":1,"69":3,"104":4,"256":1}}],["default",{"2":{"3":2,"7":5,"9":8,"10":24,"13":1,"24":10,"32":1,"44":1,"46":3,"56":1,"57":1,"59":1,"63":4,"69":1,"71":4,"72":1,"77":1,"80":1,"81":1,"92":1,"95":2,"102":1,"104":2,"106":1,"107":1,"123":1,"129":1,"131":5,"149":1,"150":1,"157":1,"160":1,"161":1,"162":4,"163":1,"165":1,"166":1,"168":1,"174":1,"175":1,"179":1,"184":1,"192":1,"193":2,"194":2,"196":2,"207":1,"208":1,"210":1,"218":1,"222":1,"226":1,"231":1,"236":1,"237":1,"244":1,"246":1,"248":1,"251":1,"256":1,"260":1}}],["dev=gpu",{"2":{"231":1}}],["developer",{"2":{"106":1}}],["developed",{"2":{"77":1,"203":1}}],["deviate",{"2":{"256":1}}],["deviation",{"2":{"9":3,"72":1}}],["devicememory",{"2":{"5":3,"80":18,"125":1,"162":2,"164":2,"167":1,"236":18}}],["deviceiterator",{"2":{"5":3,"100":2,"107":2,"207":1}}],["device",{"0":{"154":1},"2":{"2":25,"3":32,"4":11,"5":3,"25":1,"37":1,"62":5,"63":4,"80":2,"81":1,"98":1,"99":3,"103":2,"106":5,"107":5,"125":8,"126":2,"146":4,"154":5,"164":1,"175":1,"182":2,"184":1,"194":3,"205":2,"207":1,"210":1,"216":2,"217":1,"218":1,"225":1,"226":1,"231":1,"233":2,"236":1,"237":1,"239":2,"246":1}}],["devices",{"2":{"0":1,"37":4,"194":1}}],["dev",{"2":{"2":1,"3":1,"4":2,"5":4,"25":2,"62":4,"63":3,"80":5,"81":4,"182":3,"216":3,"217":2,"225":4,"231":2,"233":3}}],["deepcopy",{"2":{"52":1}}],["deep",{"2":{"2":1,"6":1,"9":6,"24":1,"74":1,"77":1,"149":1,"152":1,"162":1,"163":1}}],["axes",{"2":{"222":3}}],["axislegend",{"2":{"206":1,"209":1,"213":1,"217":1,"255":1,"256":1,"259":1}}],["axis",{"2":{"206":1,"209":1,"213":1,"217":1,"236":44,"245":1,"248":1,"250":1,"255":1,"256":1,"259":2}}],["ax",{"2":{"206":4,"209":6,"213":4,"217":5,"245":4,"248":3,"255":4,"256":6,"259":11}}],["a=layer",{"2":{"194":1}}],["a×b×x",{"2":{"194":1}}],["a100",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["ai",{"2":{"172":1}}],["aims",{"2":{"9":1}}],["aware",{"0":{"157":1},"2":{"146":2,"147":1,"157":4}}],["a∈rd×d",{"2":{"138":1}}],["ah",{"2":{"110":1}}],["ahmadi",{"2":{"53":1}}],["ahmad",{"2":{"53":1}}],["author",{"2":{"74":2}}],["autojacvec=reversediffvjp",{"2":{"233":1}}],["autojacvec=zygotevjp",{"2":{"231":1,"233":1}}],["auto",{"2":{"149":1,"175":1,"184":1,"192":1,"196":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["autoreactant",{"2":{"52":1}}],["autoreversediff",{"2":{"49":1,"52":2}}],["autoenzyme",{"2":{"49":1,"52":1,"107":2}}],["autofinitediff",{"2":{"49":1}}],["autoforwarddiff",{"2":{"27":1,"28":1,"49":1,"135":1,"140":1,"172":2}}],["autotracker",{"2":{"49":1,"52":1}}],["autozygote",{"2":{"27":1,"28":1,"49":1,"52":1,"80":2,"81":1,"139":1,"173":1,"174":1,"182":1,"190":2,"208":1,"217":2,"225":1,"233":1,"244":1,"258":1}}],["autodiff",{"2":{"22":2,"24":2,"75":1,"131":1,"133":1,"134":2,"170":2}}],["autoselection",{"2":{"2":1}}],["automatically",{"2":{"5":1,"12":2,"13":1,"22":3,"24":2,"69":2,"133":1,"146":1,"153":1,"178":1,"179":2,"180":1,"187":1,"196":1,"197":1,"229":1,"249":1}}],["automatic",{"0":{"26":1,"85":1,"125":1,"133":1,"156":1,"159":1,"170":1},"1":{"27":1,"28":1,"29":1,"86":1,"87":1,"88":1,"89":1,"134":1,"135":1,"136":1,"137":1,"138":1,"139":1,"140":1,"141":1,"171":1,"172":1,"173":1},"2":{"2":1,"3":1,"13":1,"29":1,"106":1,"125":2,"126":1,"131":1,"133":2,"156":4,"158":1,"164":1,"165":1,"172":1}}],["affected",{"2":{"138":1}}],["affects",{"2":{"66":1}}],["affine",{"2":{"72":1}}],["affine=false",{"2":{"72":5,"104":1}}],["affine=true",{"2":{"65":2,"72":20,"104":1,"109":1,"110":2}}],["afterwards",{"2":{"72":1}}],["after",{"2":{"22":2,"33":1,"59":1,"65":2,"67":1,"68":3,"72":4,"77":1,"108":1,"143":1,"146":1,"154":1,"174":14,"179":1}}],["ab7d",{"2":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["abyss",{"2":{"165":1}}],["ability",{"2":{"127":1}}],["able",{"2":{"95":1,"176":1}}],["above",{"2":{"54":1,"65":2,"79":1,"137":1,"163":1,"167":1,"250":1}}],["about",{"2":{"33":1,"57":1,"77":1,"129":1,"166":1,"169":1,"179":1,"243":1,"250":1}}],["abstol",{"2":{"236":13}}],["abstol=1",{"2":{"231":1}}],["abstraction",{"2":{"98":1}}],["abstractions",{"2":{"25":1}}],["abstractexplicitcontainerlayer",{"2":{"95":2,"96":1}}],["abstractexplicitlayer",{"2":{"95":1}}],["abstractrecurrentcell",{"2":{"69":2}}],["abstractrng",{"2":{"9":3,"12":2,"13":2,"14":1,"15":1,"22":4,"129":2,"194":2,"213":1,"222":1}}],["abstractrng=utils",{"2":{"9":8,"10":24}}],["abstracttimeseriesdatabatchordering=batchlastindex",{"2":{"69":2}}],["abstractadtype",{"2":{"27":2,"28":1,"52":1}}],["abstractarrays",{"2":{"49":1}}],["abstractarray",{"2":{"9":16,"10":60,"18":2,"21":2,"24":1,"25":1,"28":1,"54":1,"55":1,"67":3,"70":6,"71":6,"72":1,"117":1,"133":3,"137":1,"179":1,"180":1,"194":2,"208":1,"209":1,"242":3}}],["abstractmatrix",{"2":{"23":2,"71":1,"129":1,"130":1,"230":1}}],["abstractvector",{"2":{"21":1,"23":1,"71":1,"249":1,"254":2}}],["abstractluxdistributedbacked",{"2":{"40":1}}],["abstractluxdistributedbackend",{"2":{"37":3,"38":2,"39":7,"41":1}}],["abstractluxwrapperlayer",{"2":{"12":4,"32":1,"95":1,"96":2,"115":1,"130":1,"230":1,"234":1}}],["abstractluxcontainerlayer",{"2":{"12":5,"95":1,"130":3,"179":2,"241":1}}],["abstractluxlayers",{"2":{"65":1}}],["abstractluxlayer",{"2":{"12":5,"13":2,"31":4,"32":3,"33":1,"46":3,"58":1,"65":4,"72":1,"95":1,"113":2,"116":3,"117":2,"129":2,"194":1,"222":2,"230":3,"234":2}}],["abstract",{"0":{"12":1},"2":{"3":2,"11":1,"12":3,"70":4,"208":1}}],["abstractdevice",{"2":{"2":1,"3":9,"4":4,"5":1}}],["abstractgpudevice",{"2":{"1":1,"126":1}}],["absolute",{"2":{"53":1,"149":1}}],["absolutely",{"2":{"52":1}}],["abs2",{"2":{"49":1,"53":1,"59":3,"134":2,"135":2,"136":2,"137":1,"242":3,"249":1}}],["abs",{"2":{"9":1,"71":1,"245":2,"254":1}}],["amazing",{"2":{"203":1}}],["am",{"2":{"136":1,"256":1}}],["ambiguous",{"2":{"55":1,"95":1}}],["amp",{"0":{"27":1,"148":1,"165":1},"1":{"149":1,"150":1,"151":1,"152":1,"153":1,"154":1,"166":1,"167":1,"168":1,"169":1,"170":1,"171":1,"172":1,"173":1,"174":1,"175":1},"2":{"27":2,"28":1,"52":1,"66":4,"69":1,"70":1,"71":1,"72":1,"75":2}}],["amd",{"2":{"3":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["amdgpudevice",{"2":{"4":2,"126":2,"146":1,"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["amdgpu",{"2":{"2":2,"3":1,"7":2,"37":1,"62":2,"80":1,"81":1,"83":2,"88":1,"124":2,"147":1,"175":2,"184":2,"192":2,"210":2,"218":2,"226":2,"237":2,"246":2,"251":2,"260":2}}],["average",{"2":{"250":3}}],["averages",{"2":{"39":2,"40":1}}],["avik",{"2":{"74":2}}],["avg",{"2":{"39":2}}],["avoids",{"2":{"234":1}}],["avoiding",{"2":{"135":1}}],["avoid",{"2":{"22":1,"69":1,"77":1,"146":1,"234":1,"236":1,"249":2}}],["available",{"2":{"18":2,"21":1,"23":1,"35":1,"41":1,"79":1,"82":1,"85":1,"106":1,"125":1,"126":1,"153":1,"175":1,"184":1,"197":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["aka",{"2":{"21":1,"23":1}}],["adjust",{"2":{"244":1}}],["adjoint",{"2":{"208":2}}],["adam",{"2":{"59":1,"80":15,"81":1,"107":1,"182":1,"190":1,"208":4,"215":3,"217":1,"225":1,"233":1,"244":1}}],["adapted",{"2":{"252":2}}],["adaptive=false",{"2":{"255":1,"256":1,"257":1,"259":1}}],["adaptivemeanpool",{"2":{"68":1}}],["adaptivemaxpool",{"2":{"68":1}}],["adaptive",{"2":{"68":3}}],["adaptivelppool",{"2":{"68":1,"105":1}}],["adaptor",{"2":{"46":4,"99":1,"188":1}}],["adapt",{"2":{"3":4,"44":6,"46":6}}],["ads",{"2":{"55":1}}],["adtype=autotracker",{"2":{"249":1}}],["adtypes",{"2":{"52":1,"133":1,"177":1,"186":1,"212":1,"217":1,"220":1,"239":1}}],["adtype",{"2":{"49":1,"258":1}}],["advances",{"2":{"22":1}}],["advancedvi",{"2":{"247":1}}],["advanced",{"0":{"201":1},"2":{"12":2,"106":1}}],["ad",{"0":{"28":1,"29":1},"2":{"21":1,"23":1,"27":6,"28":1,"29":1,"33":2,"52":3,"53":1,"55":2,"58":2,"76":1,"77":3,"80":2,"85":4,"86":2,"89":2,"133":4,"136":2,"148":1,"156":1,"165":1,"168":1,"170":2,"171":5,"172":2,"217":1,"233":1,"238":1,"240":2,"242":1,"245":1}}],["adding",{"2":{"147":1}}],["additional",{"0":{"82":1},"2":{"48":1,"49":1,"54":1,"82":1,"83":1,"167":1,"172":1,"174":1,"193":1}}],["additionally",{"2":{"12":1,"52":1,"53":1,"54":2,"85":1,"127":1,"138":1,"154":1,"164":1,"183":1,"194":1,"207":1,"242":1}}],["addition",{"2":{"20":1,"55":1}}],["address",{"2":{"75":1}}],["add",{"2":{"47":1,"55":3,"61":2,"62":5,"63":1,"79":2,"80":1,"82":1,"88":1,"105":1,"134":1,"155":3,"166":1,"167":1,"203":1}}],["added",{"2":{"20":1,"24":4,"53":2,"66":2,"68":3,"71":1,"72":4,"90":1,"105":1}}],["adds",{"2":{"19":1,"33":1,"166":1}}],["academic",{"2":{"74":1}}],["acc",{"2":{"182":2,"190":8,"191":2,"225":13,"233":4}}],["accumulated",{"2":{"208":1}}],["accumulates",{"2":{"72":2}}],["accuracy",{"0":{"181":1},"2":{"18":1,"181":2,"182":102,"189":2,"190":4,"191":40,"224":2,"225":212,"232":2,"233":94,"235":18}}],["account",{"2":{"59":1}}],["accordingly",{"2":{"24":2,"72":2}}],["according",{"2":{"9":1}}],["accidental",{"2":{"57":1}}],["accelerate",{"2":{"161":1}}],["accelerating",{"2":{"24":1}}],["acceptance",{"2":{"249":1}}],["accept",{"2":{"131":1,"249":1}}],["accepts",{"2":{"73":1}}],["access",{"2":{"59":1,"62":1,"63":1,"65":2,"129":1,"130":1,"131":1}}],["accessors",{"2":{"34":1}}],["accessing",{"2":{"32":1,"44":1,"46":1}}],["acts",{"2":{"249":1}}],["action",{"2":{"60":1}}],["activates",{"2":{"160":1}}],["activated",{"2":{"69":4}}],["activate",{"2":{"49":1}}],["activation=tanh",{"2":{"69":1}}],["activation=identity",{"2":{"66":2,"70":4,"72":4}}],["activations",{"2":{"18":1,"21":1,"249":1}}],["activation",{"0":{"18":1,"20":1},"2":{"18":7,"20":12,"21":4,"23":3,"24":5,"59":1,"66":6,"69":3,"70":10,"72":12,"102":3,"104":3,"152":7,"160":1,"249":1,"256":3}}],["activity",{"2":{"49":2}}],["actively",{"2":{"93":1,"106":1}}],["active",{"2":{"44":2}}],["act=relu",{"2":{"59":2,"81":1}}],["actually",{"2":{"95":1,"127":1}}],["actual",{"2":{"33":1,"171":2,"217":1}}],["act",{"2":{"18":2,"24":2,"59":8,"81":5,"241":4}}],["achieved",{"2":{"66":1,"135":1}}],["achieve",{"2":{"9":1,"69":1}}],["across",{"2":{"0":1,"24":2,"39":2,"40":1,"41":1,"76":2,"143":3,"166":2,"250":1}}],["at=train",{"2":{"187":1,"229":1}}],["at=0",{"2":{"178":1}}],["atleast",{"2":{"24":1}}],["attempts",{"2":{"19":1,"23":1,"44":1}}],["at",{"2":{"4":2,"22":1,"31":1,"39":2,"53":1,"59":1,"66":2,"68":3,"69":1,"73":1,"80":1,"106":1,"109":4,"110":10,"119":1,"124":1,"131":1,"147":1,"160":2,"172":1,"179":1,"203":2,"244":4,"250":2,"254":2,"255":1}}],["april",{"2":{"74":1}}],["append",{"2":{"248":2}}],["appendix",{"0":{"175":1,"184":1,"192":1,"210":1,"218":1,"226":1,"237":1,"246":1,"251":1,"260":1}}],["appreciate",{"2":{"59":1}}],["approach",{"0":{"125":1},"2":{"106":1,"208":1}}],["approximation",{"2":{"71":1}}],["approximately",{"2":{"9":1}}],["approx",{"2":{"49":1}}],["appropriate",{"2":{"9":1,"13":1}}],["applychain",{"2":{"236":3}}],["apply",{"0":{"18":1},"2":{"13":7,"52":5,"55":1,"65":1,"66":2,"67":2,"68":3,"80":3,"113":1,"115":1,"116":4,"117":3,"129":1,"130":1,"160":1,"164":1,"217":1,"249":1}}],["applying",{"2":{"9":1,"22":2,"65":1}}],["applications",{"2":{"77":1,"168":1}}],["applicable",{"2":{"2":1,"4":1}}],["applies",{"2":{"20":1,"24":1,"65":1,"72":6,"73":2,"143":1}}],["applied",{"2":{"9":3,"13":1,"53":2,"55":1,"65":1,"67":4,"72":4,"194":2}}],["appleaccelerate",{"2":{"23":1}}],["apple",{"2":{"3":1,"83":1}}],["api",{"0":{"8":1,"52":1,"107":1,"180":1},"1":{"9":1,"10":1},"2":{"3":1,"18":2,"19":1,"20":2,"21":1,"22":2,"23":1,"24":4,"39":3,"44":1,"52":2,"53":2,"65":9,"71":2,"80":3,"90":1,"92":2,"97":1,"106":2,"107":1,"119":1,"174":1,"175":1,"176":1,"180":1,"184":1,"185":1,"193":1,"204":1,"210":1,"216":1,"218":1,"226":1,"230":1,"234":1,"236":1,"237":1,"246":1,"256":2}}],["astroinformatics",{"2":{"252":1}}],["ask",{"2":{"166":3}}],["asymmetric",{"2":{"66":2,"68":3}}],["assert",{"2":{"169":1,"249":1,"254":5}}],["associated",{"2":{"75":1}}],["assume",{"2":{"106":1,"254":1}}],["assumed",{"2":{"49":1}}],["assumes",{"2":{"16":1,"65":1,"104":1}}],["assigned",{"2":{"59":1}}],["assign",{"2":{"37":1}}],["as",{"2":{"3":2,"4":2,"5":3,"9":5,"12":4,"18":2,"20":2,"22":2,"24":3,"27":2,"33":1,"34":1,"44":1,"46":1,"48":2,"49":2,"50":4,"52":2,"53":8,"54":2,"55":1,"57":2,"59":10,"61":1,"65":10,"66":1,"68":3,"69":3,"71":3,"72":1,"73":1,"74":1,"77":2,"81":2,"87":1,"90":1,"102":2,"103":2,"106":6,"107":1,"110":1,"114":1,"130":3,"131":2,"134":1,"135":1,"136":2,"137":1,"141":1,"143":1,"144":1,"146":2,"149":2,"159":1,"163":2,"164":2,"165":1,"166":1,"168":1,"169":2,"171":1,"180":1,"185":2,"191":1,"204":1,"208":1,"222":1,"230":1,"236":1,"247":1,"249":6,"250":1,"256":2}}],["agent",{"2":{"109":1,"116":3,"134":2,"236":3,"244":4}}],["aggressive",{"2":{"98":1}}],["aggregation",{"2":{"53":1}}],["aggregated",{"2":{"53":1}}],["agg=sum",{"2":{"53":2}}],["agg=mean",{"2":{"53":1}}],["agg",{"2":{"53":26}}],["agnostic",{"2":{"3":3,"39":3,"154":1}}],["against",{"2":{"88":1,"138":1}}],["again",{"2":{"3":1,"135":1,"143":1}}],["algebras",{"2":{"172":1}}],["algebraic",{"2":{"135":1}}],["algorithm",{"2":{"2":1,"138":1,"247":1}}],["aligned",{"2":{"73":1}}],["align",{"2":{"73":2,"104":2,"105":1}}],["alias",{"2":{"54":1}}],["aliased",{"2":{"52":3}}],["aliases",{"2":{"20":1}}],["alternate",{"0":{"234":1}}],["alternatives",{"2":{"152":1}}],["alternatively",{"2":{"49":1,"69":2,"73":1,"79":1,"143":1,"149":1}}],["alternative",{"2":{"3":1}}],["alts",{"2":{"71":2}}],["already",{"2":{"60":1,"75":1,"80":1,"169":1,"216":1}}],["almost",{"2":{"44":1}}],["always",{"2":{"27":2,"30":1,"31":1,"44":1,"53":1,"55":1,"104":1,"132":1}}],["alpha=0",{"2":{"213":1,"217":2,"255":2,"256":4,"259":7}}],["alphadropout",{"2":{"67":4}}],["alpha",{"2":{"22":4,"249":2}}],["al",{"2":{"9":2,"22":2,"53":2,"252":1}}],["along",{"2":{"9":1,"20":1,"24":1,"53":1,"54":1,"65":1,"66":1,"67":2,"69":2,"72":1,"77":1,"115":1}}],["also",{"2":{"3":1,"9":2,"12":1,"20":2,"31":1,"37":2,"54":1,"55":2,"57":1,"59":1,"65":2,"67":5,"71":1,"72":7,"73":2,"77":2,"79":1,"90":1,"98":1,"129":2,"162":1,"166":1,"180":1,"182":1,"230":1,"233":2,"249":1,"250":1}}],["allocate",{"2":{"164":1}}],["allow",{"2":{"34":1,"49":1,"55":1,"98":1,"102":1,"149":1}}],["allowscalar",{"2":{"150":1,"220":1,"228":1,"239":1}}],["allows",{"2":{"11":1,"12":1,"13":1,"27":2,"39":2,"55":1,"59":1,"65":1,"69":1,"71":1,"84":1,"105":1,"130":1,"131":1,"233":1}}],["allreduce",{"2":{"39":3,"40":1,"145":1}}],["all",{"2":{"3":1,"9":1,"11":1,"12":1,"13":1,"15":3,"30":1,"31":3,"32":2,"39":5,"46":2,"48":2,"49":1,"52":2,"55":1,"56":1,"58":1,"65":3,"66":2,"68":6,"70":1,"71":2,"73":2,"76":3,"77":3,"80":1,"82":2,"98":1,"100":1,"106":1,"122":2,"129":1,"143":3,"146":1,"154":1,"160":1,"161":1,"163":1,"166":3,"167":1,"168":1,"207":1,"241":1,"248":1,"249":2}}],["artifact",{"2":{"175":1,"184":1,"210":1,"218":1,"226":1,"237":1,"246":1}}],["artificially",{"2":{"110":1}}],["artificial",{"2":{"9":2,"248":2}}],["arranged",{"2":{"166":1,"248":1}}],["arrayandtime",{"2":{"117":8,"118":1}}],["array",{"2":{"7":3,"9":9,"13":1,"18":5,"22":4,"24":5,"25":1,"28":2,"46":7,"53":1,"55":2,"59":1,"66":5,"68":6,"69":2,"70":4,"71":6,"72":16,"106":1,"107":2,"117":3,"135":1,"136":1,"163":1,"166":8,"167":1,"168":6,"189":1,"206":1,"208":1,"230":3,"231":1,"236":9,"248":4,"249":1,"255":1,"256":1,"257":1,"259":1}}],["arrays",{"0":{"150":1,"166":1,"167":1},"1":{"167":1},"2":{"3":1,"16":1,"21":2,"23":3,"25":7,"55":1,"56":1,"65":1,"70":1,"71":1,"73":5,"106":1,"129":1,"164":2,"166":4,"168":1,"194":2}}],["architecture",{"2":{"129":2,"194":1}}],["architectures",{"2":{"77":1}}],["architectural",{"2":{"129":1}}],["arbitrary",{"2":{"73":1,"233":1}}],["args",{"2":{"48":2,"49":4,"55":2,"57":6,"103":1}}],["argumenterror",{"2":{"244":1}}],["argument",{"2":{"10":1,"13":2,"59":1,"65":4,"69":1,"71":1,"95":1,"104":1,"105":1,"121":1}}],["arguments",{"2":{"2":2,"4":2,"9":3,"13":1,"18":2,"20":1,"21":1,"22":2,"23":1,"24":4,"27":2,"28":1,"31":1,"33":3,"34":1,"40":1,"44":2,"46":2,"48":3,"49":5,"52":3,"58":1,"59":2,"65":11,"66":4,"67":5,"68":9,"69":7,"70":8,"71":7,"72":9,"73":2,"92":1,"129":1,"163":1,"236":3}}],["around",{"2":{"19":1,"27":2,"44":1,"66":2,"68":3,"77":1,"130":1,"137":1}}],["arxiv",{"2":{"9":1,"24":4,"71":1,"72":2}}],["aren",{"2":{"202":1}}],["are",{"2":{"2":1,"3":2,"9":4,"11":1,"12":2,"13":1,"16":1,"18":1,"23":1,"24":3,"25":1,"28":2,"30":2,"31":6,"33":4,"34":1,"35":1,"37":2,"46":2,"49":2,"52":6,"53":6,"54":2,"55":1,"57":2,"59":2,"60":1,"70":3,"71":2,"72":5,"73":2,"76":2,"77":2,"79":1,"81":1,"82":1,"87":3,"88":5,"89":2,"95":1,"103":1,"105":1,"106":2,"107":1,"108":1,"110":1,"116":1,"120":1,"124":1,"129":3,"130":2,"131":2,"132":1,"133":2,"134":2,"135":1,"138":1,"141":3,"143":1,"146":1,"147":1,"149":2,"152":1,"153":2,"154":1,"157":2,"158":1,"160":1,"161":4,"164":1,"166":1,"168":1,"170":1,"179":2,"202":2,"203":2,"208":1,"222":1,"230":1,"249":1,"250":1,"255":1,"256":1}}],["animation",{"2":{"250":1}}],["analytical",{"2":{"243":4,"245":1}}],["anticlockwise",{"2":{"176":1,"178":4}}],["anonymous",{"2":{"102":1}}],["another",{"2":{"70":1,"130":1}}],["answers",{"2":{"84":1}}],["anything",{"2":{"129":1}}],["anywhere",{"2":{"95":1}}],["any",{"2":{"13":2,"16":1,"21":1,"23":1,"30":1,"33":1,"46":1,"49":1,"52":2,"53":3,"55":2,"59":2,"65":1,"69":3,"71":1,"76":1,"89":1,"98":2,"106":3,"107":1,"109":1,"111":1,"116":7,"127":2,"129":2,"131":1,"133":1,"146":1,"161":1,"164":1,"167":1,"168":2,"169":1,"170":1,"183":1,"194":1,"203":1,"216":1,"223":1,"233":1,"236":15}}],["an",{"2":{"2":5,"3":2,"4":1,"5":1,"9":19,"10":36,"13":3,"18":1,"22":4,"24":3,"27":2,"33":1,"36":2,"39":1,"40":1,"50":3,"52":2,"53":2,"55":1,"57":1,"59":1,"65":10,"66":3,"67":3,"68":3,"69":2,"70":8,"71":2,"72":7,"77":1,"80":1,"85":1,"87":1,"89":1,"95":1,"102":1,"104":2,"105":1,"106":1,"107":1,"108":1,"110":1,"111":2,"114":1,"125":1,"133":1,"134":2,"135":1,"137":1,"147":1,"149":1,"153":1,"155":2,"166":3,"169":1,"185":1,"202":1,"203":3,"208":1,"248":1,"249":1,"250":1}}],["andrea",{"2":{"24":1,"72":1}}],["and",{"0":{"42":1,"154":1,"181":1,"219":1,"223":1,"231":1},"1":{"43":1,"44":1,"45":1,"46":1,"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1},"2":{"1":1,"2":10,"3":3,"4":7,"5":4,"9":13,"12":7,"13":7,"18":1,"19":1,"20":1,"21":4,"22":2,"23":3,"24":20,"28":1,"30":2,"31":4,"32":7,"33":5,"34":3,"37":4,"39":2,"41":2,"44":5,"46":2,"47":2,"48":2,"50":2,"52":8,"53":19,"54":2,"55":6,"56":1,"57":4,"58":1,"59":5,"60":1,"65":10,"66":9,"68":3,"69":31,"70":8,"71":3,"72":22,"73":8,"75":3,"76":2,"77":6,"79":1,"80":4,"81":5,"84":2,"87":4,"88":2,"95":5,"100":2,"102":5,"103":3,"104":3,"105":4,"106":6,"107":2,"109":3,"110":6,"111":3,"118":1,"120":1,"124":1,"125":3,"126":1,"127":2,"129":9,"130":8,"131":3,"132":1,"133":2,"134":5,"136":1,"137":1,"138":2,"139":2,"141":1,"143":6,"144":2,"146":1,"147":1,"149":3,"152":2,"153":1,"154":5,"158":2,"160":2,"161":4,"163":2,"164":5,"166":7,"167":2,"168":1,"169":3,"170":3,"171":1,"173":1,"174":3,"176":2,"178":3,"179":10,"180":1,"183":3,"187":1,"188":1,"191":1,"194":10,"196":1,"202":1,"203":3,"208":3,"213":1,"216":2,"217":1,"229":1,"230":2,"233":2,"236":1,"238":1,"241":1,"243":2,"248":1,"249":12,"250":3,"254":3,"255":2,"256":6}}],["a",{"0":{"45":1,"120":1,"123":1,"176":1,"179":1,"211":1,"219":1,"222":1,"238":1,"252":1,"256":1},"1":{"46":1,"177":1,"178":1,"179":1,"180":1,"181":1,"182":1,"183":1,"184":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"220":1,"221":1,"222":1,"223":1,"224":1,"225":1,"226":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"0":1,"1":2,"2":5,"3":5,"4":5,"5":3,"6":1,"9":20,"10":13,"11":1,"12":10,"13":12,"15":1,"16":4,"19":1,"20":3,"21":2,"22":4,"23":2,"24":2,"27":2,"28":4,"30":1,"31":4,"32":1,"33":3,"34":4,"37":2,"39":3,"40":1,"44":9,"46":9,"47":2,"48":1,"49":4,"50":5,"52":5,"53":4,"54":5,"55":5,"57":3,"58":4,"59":21,"62":1,"63":1,"65":27,"66":21,"68":18,"69":43,"70":8,"71":10,"72":20,"73":6,"75":2,"76":1,"77":4,"79":1,"80":2,"81":3,"84":1,"90":2,"95":9,"96":2,"97":2,"100":3,"106":4,"107":3,"108":1,"109":3,"110":5,"113":2,"114":1,"117":3,"118":1,"119":1,"120":1,"122":2,"124":1,"125":7,"127":3,"129":8,"130":8,"131":2,"133":6,"134":7,"135":2,"136":3,"137":1,"138":10,"141":1,"142":2,"143":1,"146":1,"147":1,"149":5,"154":1,"157":1,"158":1,"162":2,"163":2,"164":4,"165":3,"166":12,"167":1,"169":3,"170":12,"171":1,"172":2,"174":4,"176":1,"178":2,"179":3,"180":1,"185":2,"191":1,"194":17,"196":1,"197":1,"202":1,"204":3,"207":1,"208":6,"211":2,"214":1,"217":2,"223":1,"230":3,"233":2,"234":1,"235":1,"238":2,"241":3,"242":1,"243":1,"247":1,"248":3,"249":7,"250":5,"254":9,"256":3,"257":1,"258":1}}]],"serializationVersion":2}';export{e as default}; diff --git a/previews/PR1000/assets/chunks/@localSearchIndexroot.CXVX33RZ.js b/previews/PR1000/assets/chunks/@localSearchIndexroot.CXVX33RZ.js deleted file mode 100644 index e292e53aab..0000000000 --- a/previews/PR1000/assets/chunks/@localSearchIndexroot.CXVX33RZ.js +++ /dev/null @@ -1 +0,0 @@ -const e='{"documentCount":261,"nextId":261,"documentIds":{"0":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#MLDataDevices-API","1":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#preferences","2":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#Data-Transfer","3":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#miscellaneous","4":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#Multi-GPU-Support","5":"/previews/PR1000/api/Accelerator_Support/MLDataDevices#iteration","6":"/previews/PR1000/api/Building_Blocks/LuxCore#luxcore","7":"/previews/PR1000/api/Building_Blocks/LuxCore#Abstract-Types","8":"/previews/PR1000/api/Building_Blocks/LuxCore#general","9":"/previews/PR1000/api/Building_Blocks/LuxCore#parameters","10":"/previews/PR1000/api/Building_Blocks/LuxCore#states","11":"/previews/PR1000/api/Building_Blocks/LuxCore#Layer-size","12":"/previews/PR1000/api/Building_Blocks/LuxLib#LuxLib-API","13":"/previews/PR1000/api/Building_Blocks/LuxLib#Apply-Activation","14":"/previews/PR1000/api/Building_Blocks/LuxLib#Batched-Operations","15":"/previews/PR1000/api/Building_Blocks/LuxLib#Bias-Activation","16":"/previews/PR1000/api/Building_Blocks/LuxLib#Convolutional-Layers","17":"/previews/PR1000/api/Building_Blocks/LuxLib#dropout","18":"/previews/PR1000/api/Building_Blocks/LuxLib#Fully-Connected-Layers","19":"/previews/PR1000/api/Building_Blocks/LuxLib#normalization","20":"/previews/PR1000/api/Building_Blocks/LuxLib#Helper-Functions","21":"/previews/PR1000/api/Building_Blocks/WeightInitializers#WeightInitializers-API","22":"/previews/PR1000/api/Building_Blocks/WeightInitializers#Supported-RNG-Types-WeightInit","23":"/previews/PR1000/api/Building_Blocks/WeightInitializers#API-Reference","24":"/previews/PR1000/api/Building_Blocks/WeightInitializers#Main-Functions","25":"/previews/PR1000/api/Building_Blocks/WeightInitializers#Other-Convenience-Functions","26":"/previews/PR1000/api/Lux/autodiff#autodiff-lux-helpers","27":"/previews/PR1000/api/Lux/autodiff#JVP-and-VJP-Wrappers","28":"/previews/PR1000/api/Lux/autodiff#Batched-AD","29":"/previews/PR1000/api/Lux/autodiff#Nested-2nd-Order-AD","30":"/previews/PR1000/api/Lux/interop#Interoperability-between-Lux-and-other-packages","31":"/previews/PR1000/api/Lux/interop#Switching-from-older-frameworks","32":"/previews/PR1000/api/Lux/interop#flux-to-lux-migrate-api","33":"/previews/PR1000/api/Lux/interop#Using-a-different-backend-for-Lux","34":"/previews/PR1000/api/Lux/interop#Lux-Models-to-Simple-Chains","35":"/previews/PR1000/api/Lux/distributed_utils#Distributed-Utils","36":"/previews/PR1000/api/Lux/distributed_utils#communication-backends","37":"/previews/PR1000/api/Lux/distributed_utils#initialization","38":"/previews/PR1000/api/Lux/distributed_utils#Helper-Functions","39":"/previews/PR1000/api/Lux/distributed_utils#Communication-Primitives","40":"/previews/PR1000/api/Lux/distributed_utils#Optimizers.jl-Integration","41":"/previews/PR1000/api/Lux/distributed_utils#MLUtils.jl-Integration","42":"/previews/PR1000/api/Lux/contrib#Experimental-Features","43":"/previews/PR1000/api/Lux/contrib#Parameter-Freezing","44":"/previews/PR1000/api/Lux/contrib#Map-over-Layer","45":"/previews/PR1000/api/Lux/contrib#Debugging-Functionality","46":"/previews/PR1000/api/Lux/contrib#Tied-Parameters","47":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#luxtestutils","48":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#Testing-using-JET.jl","49":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#Gradient-Correctness","50":"/previews/PR1000/api/Testing_Functionality/LuxTestUtils#Extensions-to-@test","51":"/previews/PR1000/api/Lux/utilities#utilities","52":"/previews/PR1000/api/Lux/utilities#Training-API","53":"/previews/PR1000/api/Lux/utilities#Loss-Functions","54":"/previews/PR1000/api/Lux/utilities#LuxOps-Module","55":"/previews/PR1000/api/Lux/utilities#Recursive-Operations","56":"/previews/PR1000/api/Lux/utilities#Updating-Floating-Point-Precision","57":"/previews/PR1000/api/Lux/utilities#Element-Type-Matching","58":"/previews/PR1000/api/Lux/utilities#Stateful-Layer","59":"/previews/PR1000/api/Lux/utilities#Compact-Layer","60":"/previews/PR1000/api/Lux/utilities#miscellaneous","61":"/previews/PR1000/api/Lux/layers#Built-In-Layers","62":"/previews/PR1000/api/Lux/layers#containers","63":"/previews/PR1000/api/Lux/layers#Convolutional-Layers","64":"/previews/PR1000/api/Lux/layers#Dropout-Layers","65":"/previews/PR1000/api/Lux/layers#Pooling-Layers","66":"/previews/PR1000/api/Lux/layers#Recurrent-Layers","67":"/previews/PR1000/api/Lux/layers#Linear-Layers","68":"/previews/PR1000/api/Lux/layers#Misc.-Helper-Layers","69":"/previews/PR1000/api/Lux/layers#Normalization-Layers","70":"/previews/PR1000/api/Lux/layers#upsampling","71":"/previews/PR1000/#How-to-Install-Lux.jl?","72":"/previews/PR1000/#Want-GPU-Support?","73":"/previews/PR1000/#Want-XLA-Support?","74":"/previews/PR1000/introduction/citation#citation","75":"/previews/PR1000/introduction/#getting-started","76":"/previews/PR1000/introduction/#installation","77":"/previews/PR1000/introduction/#quickstart","78":"/previews/PR1000/introduction/#Defining-Custom-Layers","79":"/previews/PR1000/introduction/#Additional-Packages","80":"/previews/PR1000/introduction/#GPU-Support","81":"/previews/PR1000/introduction/resources#Resources-to-Get-Started","82":"/previews/PR1000/introduction/updating_to_v1#updating-to-v1","83":"/previews/PR1000/introduction/updating_to_v1#LuxLib.jl","84":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes","85":"/previews/PR1000/introduction/updating_to_v1#New-Major-Features","86":"/previews/PR1000/introduction/updating_to_v1#LuxCore.jl","87":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-2","88":"/previews/PR1000/introduction/updating_to_v1#New-Major-Features-2","89":"/previews/PR1000/introduction/updating_to_v1#WeightInitializers.jl","90":"/previews/PR1000/introduction/updating_to_v1#MLDataDevices.jl","91":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-3","92":"/previews/PR1000/introduction/updating_to_v1#New-Major-Features-3","93":"/previews/PR1000/introduction/updating_to_v1#Lux.jl","94":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-(Removed-Functionality)","95":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-(Moved-Functionality)","96":"/previews/PR1000/introduction/updating_to_v1#Breaking-Changes-(Changes-in-Defaults)","97":"/previews/PR1000/introduction/updating_to_v1#New-Features","98":"/previews/PR1000/introduction/overview#Why-we-wrote-Lux?","99":"/previews/PR1000/introduction/overview#Design-Principles","100":"/previews/PR1000/introduction/overview#Why-use-Lux-over-Flux?","101":"/previews/PR1000/manual/autodiff#autodiff-lux","102":"/previews/PR1000/manual/autodiff#overview","103":"/previews/PR1000/manual/autodiff#autodiff-recommendations","104":"/previews/PR1000/manual/autodiff#Support-Class","105":"/previews/PR1000/manual/autodiff#footnotes","106":"/previews/PR1000/manual/compiling_lux_models#reactant-compilation","107":"/previews/PR1000/manual/compiling_lux_models#compile_lux_model_trainstate","108":"/previews/PR1000/manual/dispatch_custom_input#Dispatching-on-Custom-Input-Types","109":"/previews/PR1000/manual/dispatch_custom_input#Which-function-should-participate-in-dispatch?","110":"/previews/PR1000/manual/dispatch_custom_input#Concrete-Example","111":"/previews/PR1000/manual/dispatch_custom_input#Time-Dependent-Chain-Implementation","112":"/previews/PR1000/manual/dispatch_custom_input#Running-the-TDChain","113":"/previews/PR1000/manual/dispatch_custom_input#Writing-the-Correct-Dispatch-Rules","114":"/previews/PR1000/manual/dispatch_custom_input#Using-the-Same-Input-for-Non-TD-Models","115":"/previews/PR1000/manual/debugging#debug-lux-layers","116":"/previews/PR1000/manual/debugging#Incorrect-Model-Specification:-Dimension-Mismatch-Problems","117":"/previews/PR1000/manual/debugging#Tracking-down-NaNs","118":"/previews/PR1000/manual/debugging#conclusion","119":"/previews/PR1000/manual/distributed_utils#Distributed-Data-Parallel-Training","120":"/previews/PR1000/manual/distributed_utils#Guide-to-Integrating-DistributedUtils-into-your-code","121":"/previews/PR1000/manual/distributed_utils#Migration-Guide-from-FluxMPI.jl","122":"/previews/PR1000/manual/distributed_utils#Removed-Functionality","123":"/previews/PR1000/manual/distributed_utils#Key-Differences","124":"/previews/PR1000/manual/distributed_utils#Known-Shortcomings","125":"/previews/PR1000/manual/freezing_model_parameters#freezing-model-parameters","126":"/previews/PR1000/manual/freezing_model_parameters#Freezing-Layers-of-a-Particular-Kind","127":"/previews/PR1000/manual/freezing_model_parameters#Freezing-by-Layer-Name","128":"/previews/PR1000/manual/freezing_model_parameters#Freezing-Part-of-the-Parameters","129":"/previews/PR1000/manual/freezing_model_parameters#Freezing-Part-of-a-Chain","130":"/previews/PR1000/manual/gpu_management#GPU-Management","131":"/previews/PR1000/manual/gpu_management#Automatic-Backend-Management-(Recommended-Approach)","132":"/previews/PR1000/manual/gpu_management#Manual-Backend-Management","133":"/previews/PR1000/manual/interface#lux-interface","134":"/previews/PR1000/manual/interface#Layer-Interface","135":"/previews/PR1000/manual/interface#Singular-Layer","136":"/previews/PR1000/manual/interface#Container-Layer","137":"/previews/PR1000/manual/interface#Parameter-Interface","138":"/previews/PR1000/manual/interface#State-Interface","139":"/previews/PR1000/manual/migrate_from_flux#migrate-from-flux","140":"/previews/PR1000/manual/migrate_from_flux#Implementing-Custom-Layers","141":"/previews/PR1000/manual/migrate_from_flux#Certain-Important-Implementation-Details","142":"/previews/PR1000/manual/migrate_from_flux#Training/Inference-Mode","143":"/previews/PR1000/manual/migrate_from_flux#Can-we-still-use-Flux-Layers?","144":"/previews/PR1000/manual/nested_autodiff#nested_autodiff","145":"/previews/PR1000/manual/nested_autodiff#Loss-Function-containing-Jacobian-Computation","146":"/previews/PR1000/manual/nested_autodiff#Using-Batched-Jacobian-for-Multiple-Inputs","147":"/previews/PR1000/manual/nested_autodiff#Loss-Function-contains-Gradient-Computation","148":"/previews/PR1000/manual/nested_autodiff#Loss-Function-computing-the-Jacobian-of-the-Parameters","149":"/previews/PR1000/manual/nested_autodiff#Hutchinson-Trace-Estimation","150":"/previews/PR1000/manual/nested_autodiff#Computing-using-the-Vector-Jacobian-Product","151":"/previews/PR1000/manual/nested_autodiff#Computing-using-the-Jacobian-Vector-Product","152":"/previews/PR1000/manual/nested_autodiff#Computing-using-the-Full-Jacobian","153":"/previews/PR1000/manual/nn_inside_gpu_kernels#Neural-Networks-Inside-GPU-Kernels","154":"/previews/PR1000/manual/performance_pitfalls#Performance-Pitfalls-and-How-to-Catch-Them","155":"/previews/PR1000/manual/performance_pitfalls#Spurious-Type-Promotion","156":"/previews/PR1000/manual/performance_pitfalls#Scalar-Indexing-on-GPU-Arrays","157":"/previews/PR1000/manual/performance_pitfalls#Type-Instabilities","158":"/previews/PR1000/manual/performance_pitfalls#Faster-Primitives","159":"/previews/PR1000/manual/performance_pitfalls#Optional-Dependencies-for-Performance","160":"/previews/PR1000/manual/performance_pitfalls#Data-Loading-and-Device-Transfer","161":"/previews/PR1000/manual/preferences#Preferences-for-Lux.jl","162":"/previews/PR1000/manual/preferences#Nested-Automatic-Differentiation","163":"/previews/PR1000/manual/preferences#gpu-aware-mpi-preferences","164":"/previews/PR1000/manual/preferences#GPU-Backend-Selection","165":"/previews/PR1000/manual/preferences#automatic-eltypes-preference","166":"/previews/PR1000/manual/preferences#dispatch-doctor-preference","167":"/previews/PR1000/manual/preferences#disable_loop_vectorization","168":"/previews/PR1000/manual/weight_initializers#Initializing-Weights","169":"/previews/PR1000/manual/weight_initializers#Quick-examples","170":"/previews/PR1000/tutorials/beginner/1_Basics#Julia-and-Lux-for-the-Uninitiated","171":"/previews/PR1000/tutorials/beginner/1_Basics#arrays","172":"/previews/PR1000/tutorials/beginner/1_Basics#CUDA-Arrays","173":"/previews/PR1000/tutorials/beginner/1_Basics#im-mutability","174":"/previews/PR1000/tutorials/beginner/1_Basics#Managing-Randomness","175":"/previews/PR1000/tutorials/beginner/1_Basics#Automatic-Differentiation","176":"/previews/PR1000/tutorials/beginner/1_Basics#gradients","177":"/previews/PR1000/tutorials/beginner/1_Basics#Jacobian-Vector-Product","178":"/previews/PR1000/tutorials/beginner/1_Basics#Vector-Jacobian-Product","179":"/previews/PR1000/tutorials/beginner/1_Basics#Linear-Regression","180":"/previews/PR1000/tutorials/beginner/1_Basics#appendix","181":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Training-a-Simple-LSTM","182":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Package-Imports","183":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#dataset","184":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Creating-a-Classifier","185":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Using-the-@compact-API","186":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Defining-Accuracy,-Loss-and-Optimiser","187":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Training-the-Model","188":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#Saving-the-Model","189":"/previews/PR1000/tutorials/beginner/3_SimpleRNN#appendix","190":"/previews/PR1000/tutorials/beginner/4_SimpleChains#MNIST-Classification-with-SimpleChains","191":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Package-Imports","192":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Loading-MNIST","193":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Define-the-Model","194":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Helper-Functions","195":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Define-the-Training-Loop","196":"/previews/PR1000/tutorials/beginner/4_SimpleChains#Finally-Training-the-Model","197":"/previews/PR1000/tutorials/beginner/4_SimpleChains#appendix","198":"/previews/PR1000/tutorials/#tutorials","199":"/previews/PR1000/tutorials/#beginner-tutorials","200":"/previews/PR1000/tutorials/#intermediate-tutorials","201":"/previews/PR1000/tutorials/#advanced-tutorials","202":"/previews/PR1000/tutorials/#larger-models","203":"/previews/PR1000/tutorials/#selected-3rd-party-tutorials","204":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Optimization-Lux-Tutorial","205":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Imports-packages","206":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Generate-some-training-data","207":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Define-the-DataLoader","208":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Training-the-model","209":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#Plotting-the-results","210":"/previews/PR1000/tutorials/beginner/5_OptimizationIntegration#appendix","211":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#MNIST-Classification-using-Neural-ODEs","212":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Package-Imports","213":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Loading-MNIST","214":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Define-the-Neural-ODE-Layer","215":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Create-and-Initialize-the-Neural-ODE-Layer","216":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Define-Utility-Functions","217":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#training","218":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Alternate-Implementation-using-Stateful-Layer","219":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Train-the-new-Stateful-Neural-ODE","220":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#Type-Stability","221":"/previews/PR1000/tutorials/intermediate/1_NeuralODE#appendix","222":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Training-a-HyperNetwork-on-MNIST-and-FashionMNIST","223":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Package-Imports","224":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Loading-Datasets","225":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Implement-a-HyperNet-Layer","226":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Create-and-Initialize-the-HyperNet","227":"/previews/PR1000/tutorials/intermediate/3_HyperNet#Define-Utility-Functions","228":"/previews/PR1000/tutorials/intermediate/3_HyperNet#training","229":"/previews/PR1000/tutorials/intermediate/3_HyperNet#appendix","230":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Fitting-a-Polynomial-using-MLP","231":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Package-Imports","232":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#dataset","233":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Neural-Network","234":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#optimizer","235":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#Loss-Function","236":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#training","237":"/previews/PR1000/tutorials/beginner/2_PolynomialFitting#appendix","238":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Training-a-PINN-on-2D-PDE","239":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Package-Imports","240":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Problem-Definition","241":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Define-the-Neural-Networks","242":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Define-the-Loss-Functions","243":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Generate-the-Data","244":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#training","245":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#Visualizing-the-Results","246":"/previews/PR1000/tutorials/intermediate/4_PINN2DPDE#appendix","247":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Bayesian-Neural-Network","248":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Generating-data","249":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Building-the-Neural-Network","250":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#Prediction-Visualization","251":"/previews/PR1000/tutorials/intermediate/2_BayesianNN#appendix","252":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Training-a-Neural-ODE-to-Model-Gravitational-Waveforms","253":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Package-Imports","254":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Define-some-Utility-Functions","255":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Simulating-the-True-Model","256":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Defiing-a-Neural-Network-Model","257":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Setting-Up-for-Training-the-Neural-Network","258":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Training-the-Neural-Network","259":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#Visualizing-the-Results","260":"/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm#appendix"},"fieldIds":{"title":0,"titles":1,"text":2},"fieldLength":{"0":[1,1,14],"1":[1,1,46],"2":[2,1,156],"3":[1,1,194],"4":[3,1,93],"5":[1,1,114],"6":[1,1,38],"7":[2,1,160],"8":[1,1,214],"9":[1,1,19],"10":[1,1,62],"11":[2,1,77],"12":[1,1,5],"13":[2,1,100],"14":[2,1,44],"15":[2,1,68],"16":[2,1,128],"17":[1,1,160],"18":[3,1,132],"19":[1,1,232],"20":[2,1,91],"21":[1,1,16],"22":[3,1,27],"23":[2,1,1],"24":[2,3,342],"25":[3,3,92],"26":[3,1,1],"27":[4,3,72],"28":[2,3,86],"29":[4,3,13],"30":[6,1,1],"31":[4,6,1],"32":[4,10,172],"33":[6,6,1],"34":[5,11,188],"35":[2,1,11],"36":[1,2,29],"37":[1,2,81],"38":[2,2,18],"39":[2,2,58],"40":[3,2,31],"41":[3,2,35],"42":[2,1,54],"43":[2,2,131],"44":[3,2,124],"45":[2,2,140],"46":[2,2,118],"47":[1,1,48],"48":[4,1,91],"49":[2,1,127],"50":[3,1,33],"51":[1,1,1],"52":[2,1,228],"53":[2,1,449],"54":[2,1,134],"55":[2,1,164],"56":[4,1,45],"57":[3,1,108],"58":[2,1,112],"59":[2,1,358],"60":[1,1,54],"61":[3,1,1],"62":[1,3,292],"63":[2,3,257],"64":[2,3,115],"65":[2,3,168],"66":[2,3,361],"67":[2,3,191],"68":[3,3,243],"69":[2,3,294],"70":[1,3,165],"71":[6,1,47],"72":[4,1,34],"73":[4,1,29],"74":[1,1,67],"75":[2,1,1],"76":[1,2,51],"77":[1,2,402],"78":[3,2,181],"79":[2,2,36],"80":[2,2,17],"81":[4,1,48],"82":[4,1,41],"83":[2,4,1],"84":[2,6,27],"85":[3,6,22],"86":[2,4,1],"87":[2,6,119],"88":[3,6,19],"89":[2,4,30],"90":[2,4,48],"91":[2,6,22],"92":[3,6,30],"93":[2,4,1],"94":[5,5,117],"95":[5,5,54],"96":[5,5,68],"97":[2,5,63],"98":[5,1,63],"99":[2,5,58],"100":[6,5,218],"101":[2,1,35],"102":[1,2,34],"103":[1,2,50],"104":[2,2,65],"105":[1,2,67],"106":[6,1,493],"107":[4,6,156],"108":[5,1,1],"109":[7,5,36],"110":[2,5,40],"111":[4,7,72],"112":[3,7,73],"113":[5,7,47],"114":[8,7,46],"115":[3,1,76],"116":[6,3,156],"117":[3,3,190],"118":[1,3,51],"119":[4,1,27],"120":[7,4,122],"121":[5,4,42],"122":[2,9,28],"123":[2,9,79],"124":[2,4,52],"125":[3,1,27],"126":[6,3,123],"127":[4,3,53],"128":[5,3,43],"129":[5,3,49],"130":[2,1,71],"131":[6,2,89],"132":[3,2,44],"133":[2,1,80],"134":[2,2,1],"135":[2,3,239],"136":[2,3,158],"137":[2,2,151],"138":[2,2,47],"139":[5,1,67],"140":[3,5,164],"141":[4,5,1],"142":[3,9,49],"143":[7,5,34],"144":[3,1,169],"145":[5,3,265],"146":[6,8,155],"147":[5,3,219],"148":[7,3,216],"149":[3,3,88],"150":[6,6,46],"151":[6,6,31],"152":[5,6,113],"153":[5,1,286],"154":[7,1,18],"155":[3,7,149],"156":[5,7,32],"157":[2,7,39],"158":[2,7,43],"159":[4,7,39],"160":[5,7,75],"161":[4,1,49],"162":[3,4,21],"163":[4,4,40],"164":[3,4,33],"165":[3,4,23],"166":[2,4,61],"167":[4,4,58],"168":[2,1,133],"169":[2,2,64],"170":[6,1,77],"171":[1,6,286],"172":[2,7,86],"173":[3,6,110],"174":[2,6,130],"175":[2,6,96],"176":[1,8,72],"177":[3,8,106],"178":[3,8,30],"179":[2,6,253],"180":[1,6,150],"181":[4,1,38],"182":[2,4,12],"183":[1,4,95],"184":[3,4,160],"185":[4,4,74],"186":[5,4,56],"187":[3,4,530],"188":[3,4,48],"189":[1,4,150],"190":[4,1,33],"191":[2,4,15],"192":[2,4,60],"193":[3,4,64],"194":[2,4,38],"195":[4,4,73],"196":[4,4,131],"197":[1,4,110],"198":[1,1,1],"199":[2,1,1],"200":[2,1,1],"201":[2,1,1],"202":[2,1,41],"203":[4,1,65],"204":[6,1,78],"205":[2,6,29],"206":[4,6,58],"207":[3,6,70],"208":[3,6,243],"209":[3,6,47],"210":[1,6,150],"211":[5,1,24],"212":[2,5,23],"213":[2,5,64],"214":[5,5,139],"215":[7,5,60],"216":[3,5,37],"217":[1,5,239],"218":[5,5,60],"219":[6,5,70],"220":[2,5,180],"221":[1,5,150],"222":[7,1,1],"223":[2,7,18],"224":[2,7,41],"225":[4,7,76],"226":[5,7,42],"227":[3,7,39],"228":[1,7,248],"229":[1,7,150],"230":[5,1,16],"231":[2,5,12],"232":[1,5,274],"233":[2,5,37],"234":[1,5,18],"235":[2,5,112],"236":[1,5,150],"237":[1,5,150],"238":[6,1,45],"239":[2,6,31],"240":[2,6,26],"241":[4,6,55],"242":[4,6,71],"243":[3,6,86],"244":[1,6,1041],"245":[3,6,73],"246":[1,6,150],"247":[3,1,72],"248":[2,3,105],"249":[4,3,548],"250":[2,3,174],"251":[1,3,101],"252":[8,1,21],"253":[2,8,13],"254":[4,8,202],"255":[4,8,124],"256":[5,8,1354],"257":[7,8,73],"258":[4,8,1250],"259":[3,8,83],"260":[1,8,103]},"averageFieldLength":[2.954022988505747,3.6091954022988504,107.4176245210728],"storedFields":{"0":{"title":"MLDataDevices","titles":[]},"1":{"title":"Preferences","titles":["MLDataDevices"]},"2":{"title":"Data Transfer","titles":["MLDataDevices"]},"3":{"title":"Miscellaneous","titles":["MLDataDevices"]},"4":{"title":"Multi-GPU Support","titles":["MLDataDevices"]},"5":{"title":"Iteration","titles":["MLDataDevices"]},"6":{"title":"LuxCore","titles":[]},"7":{"title":"Abstract Types","titles":["LuxCore"]},"8":{"title":"General","titles":["LuxCore"]},"9":{"title":"Parameters","titles":["LuxCore"]},"10":{"title":"States","titles":["LuxCore"]},"11":{"title":"Layer size","titles":["LuxCore"]},"12":{"title":"LuxLib","titles":[]},"13":{"title":"Apply Activation","titles":["LuxLib"]},"14":{"title":"Batched Operations","titles":["LuxLib"]},"15":{"title":"Bias Activation","titles":["LuxLib"]},"16":{"title":"Convolutional Layers","titles":["LuxLib"]},"17":{"title":"Dropout","titles":["LuxLib"]},"18":{"title":"Fully Connected Layers","titles":["LuxLib"]},"19":{"title":"Normalization","titles":["LuxLib"]},"20":{"title":"Helper Functions","titles":["LuxLib"]},"21":{"title":"WeightInitializers","titles":[]},"22":{"title":"Supported RNG Types","titles":["WeightInitializers"]},"23":{"title":"API Reference","titles":["WeightInitializers"]},"24":{"title":"Main Functions","titles":["WeightInitializers","API Reference"]},"25":{"title":"Other Convenience Functions","titles":["WeightInitializers","API Reference"]},"26":{"title":"Automatic Differentiation Helpers","titles":[]},"27":{"title":"JVP & VJP Wrappers","titles":["Automatic Differentiation Helpers"]},"28":{"title":"Batched AD","titles":["Automatic Differentiation Helpers"]},"29":{"title":"Nested 2nd Order AD","titles":["Automatic Differentiation Helpers"]},"30":{"title":"Interoperability between Lux and other packages","titles":[]},"31":{"title":"Switching from older frameworks","titles":["Interoperability between Lux and other packages"]},"32":{"title":"Flux Models to Lux Models","titles":["Interoperability between Lux and other packages","Switching from older frameworks"]},"33":{"title":"Using a different backend for Lux","titles":["Interoperability between Lux and other packages"]},"34":{"title":"Lux Models to Simple Chains","titles":["Interoperability between Lux and other packages","Using a different backend for Lux"]},"35":{"title":"Distributed Utils","titles":[]},"36":{"title":"Backends","titles":["Distributed Utils"]},"37":{"title":"Initialization","titles":["Distributed Utils"]},"38":{"title":"Helper Functions","titles":["Distributed Utils"]},"39":{"title":"Communication Primitives","titles":["Distributed Utils"]},"40":{"title":"Optimizers.jl Integration","titles":["Distributed Utils"]},"41":{"title":"MLUtils.jl Integration","titles":["Distributed Utils"]},"42":{"title":"Experimental Features","titles":[]},"43":{"title":"Parameter Freezing","titles":["Experimental Features"]},"44":{"title":"Map over Layer","titles":["Experimental Features"]},"45":{"title":"Debugging Functionality","titles":["Experimental Features"]},"46":{"title":"Tied Parameters","titles":["Experimental Features"]},"47":{"title":"LuxTestUtils","titles":[]},"48":{"title":"Testing using JET.jl","titles":["LuxTestUtils"]},"49":{"title":"Gradient Correctness","titles":["LuxTestUtils"]},"50":{"title":"Extensions to @test","titles":["LuxTestUtils"]},"51":{"title":"Utilities","titles":[]},"52":{"title":"Training API","titles":["Utilities"]},"53":{"title":"Loss Functions","titles":["Utilities"]},"54":{"title":"LuxOps Module","titles":["Utilities"]},"55":{"title":"Recursive Operations","titles":["Utilities"]},"56":{"title":"Updating Floating Point Precision","titles":["Utilities"]},"57":{"title":"Element Type Matching","titles":["Utilities"]},"58":{"title":"Stateful Layer","titles":["Utilities"]},"59":{"title":"Compact Layer","titles":["Utilities"]},"60":{"title":"Miscellaneous","titles":["Utilities"]},"61":{"title":"Built-In Layers","titles":[]},"62":{"title":"Containers","titles":["Built-In Layers"]},"63":{"title":"Convolutional Layers","titles":["Built-In Layers"]},"64":{"title":"Dropout Layers","titles":["Built-In Layers"]},"65":{"title":"Pooling Layers","titles":["Built-In Layers"]},"66":{"title":"Recurrent Layers","titles":["Built-In Layers"]},"67":{"title":"Linear Layers","titles":["Built-In Layers"]},"68":{"title":"Misc. Helper Layers","titles":["Built-In Layers"]},"69":{"title":"Normalization Layers","titles":["Built-In Layers"]},"70":{"title":"Upsampling","titles":["Built-In Layers"]},"71":{"title":"How to Install Lux.jl?","titles":[]},"72":{"title":"Want GPU Support?","titles":[]},"73":{"title":"Want XLA Support?","titles":[]},"74":{"title":"Citation","titles":[]},"75":{"title":"Getting Started","titles":[]},"76":{"title":"Installation","titles":["Getting Started"]},"77":{"title":"Quickstart","titles":["Getting Started"]},"78":{"title":"Defining Custom Layers","titles":["Getting Started"]},"79":{"title":"Additional Packages","titles":["Getting Started"]},"80":{"title":"GPU Support","titles":["Getting Started"]},"81":{"title":"Resources to Get Started","titles":[]},"82":{"title":"Updating to Lux v1","titles":[]},"83":{"title":"LuxLib.jl","titles":["Updating to Lux v1"]},"84":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxLib.jl"]},"85":{"title":"New Major Features","titles":["Updating to Lux v1","LuxLib.jl"]},"86":{"title":"LuxCore.jl","titles":["Updating to Lux v1"]},"87":{"title":"Breaking Changes","titles":["Updating to Lux v1","LuxCore.jl"]},"88":{"title":"New Major Features","titles":["Updating to Lux v1","LuxCore.jl"]},"89":{"title":"WeightInitializers.jl","titles":["Updating to Lux v1"]},"90":{"title":"MLDataDevices.jl","titles":["Updating to Lux v1"]},"91":{"title":"Breaking Changes","titles":["Updating to Lux v1","MLDataDevices.jl"]},"92":{"title":"New Major Features","titles":["Updating to Lux v1","MLDataDevices.jl"]},"93":{"title":"Lux.jl","titles":["Updating to Lux v1"]},"94":{"title":"Breaking Changes (Removed Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"95":{"title":"Breaking Changes (Moved Functionality)","titles":["Updating to Lux v1","Lux.jl"]},"96":{"title":"Breaking Changes (Changes in Defaults)","titles":["Updating to Lux v1","Lux.jl"]},"97":{"title":"New Features","titles":["Updating to Lux v1","Lux.jl"]},"98":{"title":"Why we wrote Lux?","titles":[]},"99":{"title":"Design Principles","titles":["Why we wrote Lux?"]},"100":{"title":"Why use Lux over Flux?","titles":["Why we wrote Lux?"]},"101":{"title":"Automatic Differentiation","titles":[]},"102":{"title":"Overview","titles":["Automatic Differentiation"]},"103":{"title":"Recommendations","titles":["Automatic Differentiation"]},"104":{"title":"Support Class","titles":["Automatic Differentiation"]},"105":{"title":"Footnotes","titles":["Automatic Differentiation"]},"106":{"title":"Compiling Lux Models using Reactant.jl","titles":[]},"107":{"title":"Using the TrainState API","titles":["Compiling Lux Models using Reactant.jl"]},"108":{"title":"Dispatching on Custom Input Types","titles":[]},"109":{"title":"Which function should participate in dispatch?","titles":["Dispatching on Custom Input Types"]},"110":{"title":"Concrete Example","titles":["Dispatching on Custom Input Types"]},"111":{"title":"Time-Dependent Chain Implementation","titles":["Dispatching on Custom Input Types","Concrete Example"]},"112":{"title":"Running the TDChain","titles":["Dispatching on Custom Input Types","Concrete Example"]},"113":{"title":"Writing the Correct Dispatch Rules","titles":["Dispatching on Custom Input Types","Concrete Example"]},"114":{"title":"Using the Same Input for Non-TD Models","titles":["Dispatching on Custom Input Types","Concrete Example"]},"115":{"title":"Debugging Lux Models","titles":[]},"116":{"title":"Incorrect Model Specification: Dimension Mismatch Problems","titles":["Debugging Lux Models"]},"117":{"title":"Tracking down NaNs","titles":["Debugging Lux Models"]},"118":{"title":"Conclusion","titles":["Debugging Lux Models"]},"119":{"title":"Distributed Data Parallel Training","titles":[]},"120":{"title":"Guide to Integrating DistributedUtils into your code","titles":["Distributed Data Parallel Training"]},"121":{"title":"Migration Guide from FluxMPI.jl","titles":["Distributed Data Parallel Training"]},"122":{"title":"Removed Functionality","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"123":{"title":"Key Differences","titles":["Distributed Data Parallel Training","Migration Guide from FluxMPI.jl"]},"124":{"title":"Known Shortcomings","titles":["Distributed Data Parallel Training"]},"125":{"title":"Freezing Model Parameters","titles":[]},"126":{"title":"Freezing Layers of a Particular Kind","titles":["Freezing Model Parameters"]},"127":{"title":"Freezing by Layer Name","titles":["Freezing Model Parameters"]},"128":{"title":"Freezing Part of the Parameters","titles":["Freezing Model Parameters"]},"129":{"title":"Freezing Part of a Chain","titles":["Freezing Model Parameters"]},"130":{"title":"GPU Management","titles":[]},"131":{"title":"Automatic Backend Management (Recommended Approach)","titles":["GPU Management"]},"132":{"title":"Manual Backend Management","titles":["GPU Management"]},"133":{"title":"Lux Interface","titles":[]},"134":{"title":"Layer Interface","titles":["Lux Interface"]},"135":{"title":"Singular Layer","titles":["Lux Interface","Layer Interface"]},"136":{"title":"Container Layer","titles":["Lux Interface","Layer Interface"]},"137":{"title":"Parameter Interface","titles":["Lux Interface"]},"138":{"title":"State Interface","titles":["Lux Interface"]},"139":{"title":"Migrating from Flux to Lux","titles":[]},"140":{"title":"Implementing Custom Layers","titles":["Migrating from Flux to Lux"]},"141":{"title":"Certain Important Implementation Details","titles":["Migrating from Flux to Lux"]},"142":{"title":"Training/Inference Mode","titles":["Migrating from Flux to Lux","Certain Important Implementation Details"]},"143":{"title":"Can we still use Flux Layers?","titles":["Migrating from Flux to Lux"]},"144":{"title":"Nested Automatic Differentiation","titles":[]},"145":{"title":"Loss Function containing Jacobian Computation","titles":["Nested Automatic Differentiation"]},"146":{"title":"Using Batched Jacobian for Multiple Inputs","titles":["Nested Automatic Differentiation","Loss Function containing Jacobian Computation"]},"147":{"title":"Loss Function contains Gradient Computation","titles":["Nested Automatic Differentiation"]},"148":{"title":"Loss Function computing the Jacobian of the Parameters","titles":["Nested Automatic Differentiation"]},"149":{"title":"Hutchinson Trace Estimation","titles":["Nested Automatic Differentiation"]},"150":{"title":"Computing using the Vector-Jacobian Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"151":{"title":"Computing using the Jacobian-Vector Product","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"152":{"title":"Computing using the Full Jacobian","titles":["Nested Automatic Differentiation","Hutchinson Trace Estimation"]},"153":{"title":"Neural Networks Inside GPU Kernels","titles":[]},"154":{"title":"Performance Pitfalls & How to Catch Them","titles":[]},"155":{"title":"Spurious Type-Promotion","titles":["Performance Pitfalls & How to Catch Them"]},"156":{"title":"Scalar Indexing on GPU Arrays","titles":["Performance Pitfalls & How to Catch Them"]},"157":{"title":"Type Instabilities","titles":["Performance Pitfalls & How to Catch Them"]},"158":{"title":"Faster Primitives","titles":["Performance Pitfalls & How to Catch Them"]},"159":{"title":"Optional Dependencies for Performance","titles":["Performance Pitfalls & How to Catch Them"]},"160":{"title":"Data Loading and Device Transfer","titles":["Performance Pitfalls & How to Catch Them"]},"161":{"title":"Preferences for Lux.jl","titles":[]},"162":{"title":"Nested Automatic Differentiation","titles":["Preferences for Lux.jl"]},"163":{"title":"GPU-Aware MPI Support","titles":["Preferences for Lux.jl"]},"164":{"title":"GPU Backend Selection","titles":["Preferences for Lux.jl"]},"165":{"title":"Automatic Eltype Conversion","titles":["Preferences for Lux.jl"]},"166":{"title":"Dispatch Doctor","titles":["Preferences for Lux.jl"]},"167":{"title":"Disabling Loop Vectorization / Octavian","titles":["Preferences for Lux.jl"]},"168":{"title":"Initializing Weights","titles":[]},"169":{"title":"Quick examples","titles":["Initializing Weights"]},"170":{"title":"Julia & Lux for the Uninitiated","titles":[]},"171":{"title":"Arrays","titles":["Julia & Lux for the Uninitiated"]},"172":{"title":"CUDA Arrays","titles":["Julia & Lux for the Uninitiated","Arrays"]},"173":{"title":"(Im)mutability","titles":["Julia & Lux for the Uninitiated"]},"174":{"title":"Managing Randomness","titles":["Julia & Lux for the Uninitiated"]},"175":{"title":"Automatic Differentiation","titles":["Julia & Lux for the Uninitiated"]},"176":{"title":"Gradients","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"177":{"title":"Jacobian-Vector Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"178":{"title":"Vector-Jacobian Product","titles":["Julia & Lux for the Uninitiated","Automatic Differentiation"]},"179":{"title":"Linear Regression","titles":["Julia & Lux for the Uninitiated"]},"180":{"title":"Appendix","titles":["Julia & Lux for the Uninitiated"]},"181":{"title":"Training a Simple LSTM","titles":[]},"182":{"title":"Package Imports","titles":["Training a Simple LSTM"]},"183":{"title":"Dataset","titles":["Training a Simple LSTM"]},"184":{"title":"Creating a Classifier","titles":["Training a Simple LSTM"]},"185":{"title":"Using the @compact API","titles":["Training a Simple LSTM"]},"186":{"title":"Defining Accuracy, Loss and Optimiser","titles":["Training a Simple LSTM"]},"187":{"title":"Training the Model","titles":["Training a Simple LSTM"]},"188":{"title":"Saving the Model","titles":["Training a Simple LSTM"]},"189":{"title":"Appendix","titles":["Training a Simple LSTM"]},"190":{"title":"MNIST Classification with SimpleChains","titles":[]},"191":{"title":"Package Imports","titles":["MNIST Classification with SimpleChains"]},"192":{"title":"Loading MNIST","titles":["MNIST Classification with SimpleChains"]},"193":{"title":"Define the Model","titles":["MNIST Classification with SimpleChains"]},"194":{"title":"Helper Functions","titles":["MNIST Classification with SimpleChains"]},"195":{"title":"Define the Training Loop","titles":["MNIST Classification with SimpleChains"]},"196":{"title":"Finally Training the Model","titles":["MNIST Classification with SimpleChains"]},"197":{"title":"Appendix","titles":["MNIST Classification with SimpleChains"]},"198":{"title":"Tutorials","titles":[]},"199":{"title":"Beginner Tutorials","titles":["Tutorials"]},"200":{"title":"Intermediate Tutorials","titles":["Tutorials"]},"201":{"title":"Advanced Tutorials","titles":["Tutorials"]},"202":{"title":"Larger Models","titles":["Tutorials"]},"203":{"title":"Selected 3rd Party Tutorials","titles":["Tutorials"]},"204":{"title":"Training Lux Models using Optimization.jl","titles":[]},"205":{"title":"Imports packages","titles":["Training Lux Models using Optimization.jl"]},"206":{"title":"Generate some training data","titles":["Training Lux Models using Optimization.jl"]},"207":{"title":"Define the DataLoader","titles":["Training Lux Models using Optimization.jl"]},"208":{"title":"Training the model","titles":["Training Lux Models using Optimization.jl"]},"209":{"title":"Plotting the results","titles":["Training Lux Models using Optimization.jl"]},"210":{"title":"Appendix","titles":["Training Lux Models using Optimization.jl"]},"211":{"title":"MNIST Classification using Neural ODEs","titles":[]},"212":{"title":"Package Imports","titles":["MNIST Classification using Neural ODEs"]},"213":{"title":"Loading MNIST","titles":["MNIST Classification using Neural ODEs"]},"214":{"title":"Define the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"215":{"title":"Create and Initialize the Neural ODE Layer","titles":["MNIST Classification using Neural ODEs"]},"216":{"title":"Define Utility Functions","titles":["MNIST Classification using Neural ODEs"]},"217":{"title":"Training","titles":["MNIST Classification using Neural ODEs"]},"218":{"title":"Alternate Implementation using Stateful Layer","titles":["MNIST Classification using Neural ODEs"]},"219":{"title":"Train the new Stateful Neural ODE","titles":["MNIST Classification using Neural ODEs"]},"220":{"title":"Type Stability","titles":["MNIST Classification using Neural ODEs"]},"221":{"title":"Appendix","titles":["MNIST Classification using Neural ODEs"]},"222":{"title":"Training a HyperNetwork on MNIST and FashionMNIST","titles":[]},"223":{"title":"Package Imports","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"224":{"title":"Loading Datasets","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"225":{"title":"Implement a HyperNet Layer","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"226":{"title":"Create and Initialize the HyperNet","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"227":{"title":"Define Utility Functions","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"228":{"title":"Training","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"229":{"title":"Appendix","titles":["Training a HyperNetwork on MNIST and FashionMNIST"]},"230":{"title":"Fitting a Polynomial using MLP","titles":[]},"231":{"title":"Package Imports","titles":["Fitting a Polynomial using MLP"]},"232":{"title":"Dataset","titles":["Fitting a Polynomial using MLP"]},"233":{"title":"Neural Network","titles":["Fitting a Polynomial using MLP"]},"234":{"title":"Optimizer","titles":["Fitting a Polynomial using MLP"]},"235":{"title":"Loss Function","titles":["Fitting a Polynomial using MLP"]},"236":{"title":"Training","titles":["Fitting a Polynomial using MLP"]},"237":{"title":"Appendix","titles":["Fitting a Polynomial using MLP"]},"238":{"title":"Training a PINN on 2D PDE","titles":[]},"239":{"title":"Package Imports","titles":["Training a PINN on 2D PDE"]},"240":{"title":"Problem Definition","titles":["Training a PINN on 2D PDE"]},"241":{"title":"Define the Neural Networks","titles":["Training a PINN on 2D PDE"]},"242":{"title":"Define the Loss Functions","titles":["Training a PINN on 2D PDE"]},"243":{"title":"Generate the Data","titles":["Training a PINN on 2D PDE"]},"244":{"title":"Training","titles":["Training a PINN on 2D PDE"]},"245":{"title":"Visualizing the Results","titles":["Training a PINN on 2D PDE"]},"246":{"title":"Appendix","titles":["Training a PINN on 2D PDE"]},"247":{"title":"Bayesian Neural Network","titles":[]},"248":{"title":"Generating data","titles":["Bayesian Neural Network"]},"249":{"title":"Building the Neural Network","titles":["Bayesian Neural Network"]},"250":{"title":"Prediction Visualization","titles":["Bayesian Neural Network"]},"251":{"title":"Appendix","titles":["Bayesian Neural Network"]},"252":{"title":"Training a Neural ODE to Model Gravitational Waveforms","titles":[]},"253":{"title":"Package Imports","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"254":{"title":"Define some Utility Functions","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"255":{"title":"Simulating the True Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"256":{"title":"Defiing a Neural Network Model","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"257":{"title":"Setting Up for Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"258":{"title":"Training the Neural Network","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"259":{"title":"Visualizing the Results","titles":["Training a Neural ODE to Model Gravitational Waveforms"]},"260":{"title":"Appendix","titles":["Training a Neural ODE to Model Gravitational Waveforms"]}},"dirtCount":0,"index":[["π",{"2":{"254":2,"255":1}}],["√",{"2":{"254":2}}],["√t",{"2":{"24":2}}],["↦",{"2":{"254":1}}],["ϕ̇",{"2":{"255":2,"256":2}}],["ϕ",{"2":{"254":5,"255":1,"256":1}}],["χ̇",{"2":{"255":2,"256":2}}],["χ",{"2":{"254":4,"255":4,"256":2}}],["₋₋₋kwargs₋₋₋",{"2":{"220":4}}],["₋₋₋no",{"2":{"220":3}}],["│",{"2":{"220":3,"244":4}}],["─",{"2":{"220":3}}],["θ|x",{"2":{"250":2}}],["θ",{"2":{"208":2,"249":2,"250":9,"257":3}}],["÷",{"2":{"183":4}}],["∥22we",{"2":{"179":1}}],["⟶∑i=1k12∥yi−fw",{"2":{"179":1}}],["∇f",{"2":{"176":3}}],["∇offending",{"2":{"117":2}}],["$",{"2":{"245":1}}],["$i",{"2":{"174":2,"250":1}}],["$name",{"2":{"44":1}}],["⋮",{"2":{"153":4}}],["∞",{"2":{"145":4,"146":4,"147":4,"148":4,"152":8}}],["∘",{"2":{"144":4,"147":1,"242":3}}],["└──",{"2":{"220":3}}],["└",{"2":{"116":1,"145":2,"244":4}}],["┌",{"2":{"116":1,"145":2,"244":4}}],["~",{"2":{"112":3,"145":2,"249":2}}],["∂w",{"2":{"242":2}}],["∂v",{"2":{"242":2}}],["∂y",{"2":{"242":4}}],["∂u",{"2":{"242":10}}],["∂t",{"2":{"147":7,"242":2}}],["∂xyt",{"2":{"242":4}}],["∂x",{"2":{"145":7,"146":7,"148":7,"152":11,"242":4}}],["∂ps",{"2":{"106":4,"145":8,"146":8,"147":8,"148":8,"152":11}}],["∂f∂x",{"2":{"27":2}}],["↩︎",{"2":{"105":8}}],["❌",{"2":{"102":5}}],["❓",{"2":{"102":6}}],["\\tfashionmnist\\ttraining",{"2":{"228":1}}],["\\tfashionmnist\\ttime",{"2":{"228":50}}],["\\ttest",{"2":{"217":45,"219":9,"228":102}}],["\\ttime",{"2":{"217":45,"219":9}}],["\\tloss",{"2":{"107":11}}],["\\t",{"2":{"78":11,"196":60,"228":51,"236":6,"244":400}}],["quadrupole",{"2":{"254":10}}],["quadratic",{"2":{"232":1,"236":1}}],["quantiles",{"2":{"249":1}}],["quick",{"0":{"169":1},"2":{"170":1}}],["quickstart",{"0":{"77":1},"2":{"81":1}}],["quite",{"2":{"98":1,"173":1,"175":1}}],["questions",{"2":{"81":3,"144":1,"175":1}}],["quoting",{"2":{"106":1}}],["quot",{"2":{"1":2,"8":2,"17":4,"18":1,"19":8,"24":12,"53":8,"55":2,"57":16,"59":2,"60":6,"62":4,"68":2,"69":2,"77":4,"123":2,"140":4,"155":2,"164":8}}],["⋅n+z⋅hprevarguments",{"2":{"66":1}}],["↗",{"2":{"62":2}}],["↘",{"2":{"62":2}}],["→",{"2":{"62":5}}],["832085764855876e",{"2":{"258":1}}],["8320256f",{"2":{"256":1}}],["830446680937386e",{"2":{"258":1}}],["830501f",{"2":{"256":1}}],["8313955715777306e",{"2":{"258":1}}],["831456f",{"2":{"256":1}}],["8317274f",{"2":{"179":1}}],["838436281599909e",{"2":{"258":1}}],["838341f",{"2":{"256":1}}],["8381s\\ttraining",{"2":{"217":1}}],["8376",{"2":{"249":1}}],["8374606",{"2":{"235":1}}],["837991",{"2":{"148":1}}],["834219f",{"2":{"256":1}}],["8346457",{"2":{"232":2}}],["8340728",{"2":{"148":1}}],["83",{"2":{"196":5,"217":6,"219":1,"228":2}}],["836",{"2":{"193":2}}],["8393",{"2":{"171":1}}],["8393034",{"2":{"171":1}}],["839303",{"2":{"171":2}}],["833518",{"2":{"172":1}}],["8337032",{"2":{"126":1}}],["833333333333333",{"2":{"53":1}}],["890794034731576e",{"2":{"258":1}}],["8904",{"2":{"249":1}}],["8955284424237366e",{"2":{"258":1}}],["8953523f",{"2":{"256":1}}],["894151625187261e",{"2":{"258":1}}],["8946",{"2":{"249":1}}],["8913923156868715e",{"2":{"258":1}}],["891327f",{"2":{"256":1}}],["891999920667637e",{"2":{"258":1}}],["8919454f",{"2":{"256":1}}],["891627f",{"2":{"256":1}}],["893293005081802e",{"2":{"258":1}}],["8938728f",{"2":{"256":1}}],["893465f",{"2":{"256":1}}],["898235510879696e",{"2":{"258":1}}],["898543471366949e",{"2":{"258":1}}],["898576f",{"2":{"256":1}}],["89858943",{"2":{"147":1}}],["898498f",{"2":{"256":1}}],["8984",{"2":{"249":1}}],["897456642107079e",{"2":{"258":1}}],["8979493365287074e",{"2":{"258":1}}],["8976296f",{"2":{"256":1}}],["8976705f",{"2":{"256":1}}],["8976378",{"2":{"232":2}}],["8972864",{"2":{"232":1}}],["8928s\\ttraining",{"2":{"217":1}}],["89",{"2":{"196":3,"228":1}}],["896853419589179e",{"2":{"258":1}}],["8969367f",{"2":{"256":1}}],["8969011",{"2":{"232":1}}],["896",{"2":{"193":2}}],["8996232018785184e",{"2":{"258":1}}],["8996743f",{"2":{"256":1}}],["8990",{"2":{"249":1}}],["899766",{"2":{"171":1}}],["8999977",{"2":{"148":1}}],["81375522695677e",{"2":{"258":1}}],["815267908639125e",{"2":{"258":1}}],["815592773796477e",{"2":{"258":1}}],["8153116f",{"2":{"256":1}}],["815336",{"2":{"148":1}}],["812922f",{"2":{"256":1}}],["811666816744006e",{"2":{"258":1}}],["811957190679273e",{"2":{"258":1}}],["8114002f",{"2":{"256":1}}],["811721f",{"2":{"256":1}}],["8110236",{"2":{"232":2}}],["816304f",{"2":{"256":1}}],["816281",{"2":{"232":1}}],["8100",{"2":{"249":1}}],["81779129679041e",{"2":{"258":1}}],["8177601f",{"2":{"256":1}}],["8178138f",{"2":{"256":1}}],["817417128355986e",{"2":{"258":1}}],["817477f",{"2":{"256":1}}],["8174s\\ttraining",{"2":{"217":1}}],["8173783982243132",{"2":{"153":2}}],["81",{"2":{"196":1,"217":7,"219":1,"228":29,"249":1}}],["81s",{"2":{"196":1}}],["8145244687099784e",{"2":{"258":1}}],["8148033f",{"2":{"256":1}}],["81481",{"2":{"217":2,"219":1}}],["8146",{"2":{"249":1}}],["8146973e",{"2":{"147":2}}],["81446123",{"2":{"235":1}}],["8147778",{"2":{"106":1}}],["881015585657028e",{"2":{"258":1}}],["8812",{"2":{"249":1}}],["8879156189621935e",{"2":{"258":1}}],["8882713f",{"2":{"256":1}}],["88889",{"2":{"217":1}}],["8804897098564516e",{"2":{"258":1}}],["880749f",{"2":{"256":1}}],["8805148f",{"2":{"256":1}}],["8843950898789558e",{"2":{"258":1}}],["884335f",{"2":{"256":1}}],["8848036f",{"2":{"256":1}}],["885226546454502e",{"2":{"258":1}}],["8852221",{"2":{"148":1}}],["8850536f",{"2":{"256":1}}],["8821303",{"2":{"232":1}}],["88214725",{"2":{"148":1}}],["88",{"2":{"228":17}}],["889705011615516e",{"2":{"258":1}}],["889705f",{"2":{"256":1}}],["88951594",{"2":{"232":1}}],["889",{"2":{"221":1}}],["8892s\\ttraining",{"2":{"219":1}}],["883664f0",{"2":{"145":1}}],["8611780260861976e",{"2":{"258":1}}],["861005f",{"2":{"256":1}}],["865734997496769e",{"2":{"258":1}}],["8656395f",{"2":{"256":1}}],["8656",{"2":{"249":1}}],["864719048598573e",{"2":{"258":1}}],["8647954f",{"2":{"256":1}}],["8646086939959324e",{"2":{"258":1}}],["8645542f",{"2":{"256":1}}],["860829740139632e",{"2":{"258":1}}],["8606568f",{"2":{"256":1}}],["8601828553599953",{"2":{"174":1}}],["8601870861800127",{"2":{"153":2}}],["8689",{"2":{"249":1}}],["8688824",{"2":{"232":1}}],["8638",{"2":{"249":1}}],["86614174",{"2":{"232":2}}],["86",{"2":{"196":2,"228":3}}],["8672",{"2":{"249":1}}],["867202",{"2":{"145":1}}],["8674763752817414",{"2":{"153":1}}],["8674763752817416",{"2":{"153":1}}],["869632",{"2":{"145":1}}],["862480861009647",{"2":{"153":2}}],["86290836",{"2":{"126":1}}],["8626451f",{"2":{"106":5}}],["855557021001359e",{"2":{"258":1}}],["8553897f",{"2":{"256":1}}],["858894467179369e",{"2":{"258":1}}],["85899f",{"2":{"256":1}}],["85615832181301e",{"2":{"258":1}}],["85999624010108e",{"2":{"258":1}}],["8599526f",{"2":{"256":1}}],["859405",{"2":{"147":1,"148":1}}],["854153375533071e",{"2":{"258":1}}],["8541596",{"2":{"145":1}}],["854168439435897e",{"2":{"258":1}}],["8547740239996884e",{"2":{"258":1}}],["8547255f",{"2":{"256":1}}],["8543207f",{"2":{"256":1}}],["8543445f",{"2":{"256":1}}],["853039f",{"2":{"256":1}}],["8534",{"2":{"249":1}}],["85344f",{"2":{"77":1}}],["851428507103351e",{"2":{"258":1}}],["851123f",{"2":{"256":1}}],["8513s\\ttraining",{"2":{"217":1}}],["85185",{"2":{"217":2}}],["85",{"2":{"196":2,"217":4,"228":4}}],["8501",{"2":{"244":1}}],["85050875",{"2":{"232":1}}],["850",{"2":{"193":2}}],["857273311000907e",{"2":{"258":1}}],["857257",{"2":{"172":1}}],["8576512",{"2":{"232":1}}],["857126",{"2":{"172":1}}],["8573820474674845",{"2":{"171":1}}],["8c79",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["807874035963595e",{"2":{"258":1}}],["8077785f",{"2":{"256":1}}],["8018370574615e",{"2":{"258":1}}],["8013359",{"2":{"232":1}}],["803273056227972e",{"2":{"258":1}}],["8031547",{"2":{"232":1}}],["8031496",{"2":{"232":2}}],["806462575972689e",{"2":{"258":1}}],["8066725f",{"2":{"256":1}}],["806367f",{"2":{"256":1}}],["8067698",{"2":{"147":1}}],["8050",{"2":{"249":1}}],["80",{"2":{"180":1,"189":1,"196":2,"210":1,"217":10,"219":2,"221":1,"228":1,"229":1,"237":1,"246":1,"248":1}}],["8048109",{"2":{"235":1}}],["80465555",{"2":{"179":1}}],["80423f",{"2":{"77":1}}],["809717199346536e",{"2":{"258":1}}],["809742im",{"2":{"168":1}}],["809148925132326e",{"2":{"258":1}}],["809692f",{"2":{"256":1}}],["8090885f",{"2":{"256":1}}],["8094",{"2":{"249":1}}],["809456",{"2":{"171":1}}],["800677645962824e",{"2":{"258":1}}],["8009728f",{"2":{"256":1}}],["8001",{"2":{"179":1,"244":1}}],["800792614736468",{"2":{"153":2}}],["8003627",{"2":{"147":1}}],["800",{"2":{"107":1}}],["8f",{"2":{"107":1,"179":1}}],["8732520022523096e",{"2":{"258":1}}],["8735186f",{"2":{"256":1}}],["878401409079714e",{"2":{"258":1}}],["8787",{"2":{"249":1}}],["8766903805779695e",{"2":{"258":1}}],["8766418f",{"2":{"256":1}}],["875082",{"2":{"232":1}}],["8740871f",{"2":{"256":1}}],["8740157",{"2":{"232":2}}],["87431f",{"2":{"77":1}}],["8776770210147576e",{"2":{"258":1}}],["877626f",{"2":{"256":1}}],["8770994f",{"2":{"256":1}}],["87702435",{"2":{"106":1}}],["8774464f",{"2":{"256":1}}],["877497",{"2":{"177":1,"178":1}}],["877159842473196e",{"2":{"258":1}}],["8771",{"2":{"249":1}}],["87",{"2":{"116":1,"228":2}}],["87143785f0",{"2":{"77":1}}],["822332640641795e",{"2":{"258":1}}],["822247f",{"2":{"256":1}}],["82293516",{"2":{"232":1}}],["8251677544339422e",{"2":{"258":1}}],["825154f",{"2":{"256":1}}],["825093374578964e",{"2":{"258":1}}],["82593214102676e",{"2":{"258":1}}],["82576f",{"2":{"256":1}}],["826957f",{"2":{"256":1}}],["8245838158569814e",{"2":{"258":1}}],["82454f",{"2":{"256":1}}],["8248566f",{"2":{"256":1}}],["824650e",{"2":{"208":1}}],["827235734405655e",{"2":{"258":1}}],["8277",{"2":{"249":1}}],["8271f",{"2":{"77":1}}],["82030004",{"2":{"232":1}}],["82",{"2":{"196":1,"217":6,"219":2,"228":2}}],["82805836",{"2":{"126":1}}],["82945824",{"2":{"77":1}}],["8",{"2":{"53":1,"62":2,"68":1,"77":27,"106":28,"145":1,"147":2,"148":3,"171":2,"173":4,"183":1,"187":15,"196":2,"208":4,"217":5,"219":1,"228":2,"232":1,"234":1,"236":1,"249":8,"256":60,"258":61,"260":1}}],["8481759317291004e",{"2":{"258":1}}],["8481105f",{"2":{"256":1}}],["8434707310424074e",{"2":{"258":1}}],["8438937777196597e",{"2":{"258":1}}],["843547f",{"2":{"256":1}}],["847838721998037e",{"2":{"258":1}}],["847904f",{"2":{"256":1}}],["847106",{"2":{"148":1}}],["841763622047724e",{"2":{"258":1}}],["841602f",{"2":{"256":1}}],["841231",{"2":{"148":1}}],["846065107124997e",{"2":{"258":1}}],["8462672f",{"2":{"256":1}}],["846457",{"2":{"148":1}}],["8499767",{"2":{"232":1}}],["8490221",{"2":{"137":1}}],["8424188",{"2":{"232":1}}],["8425196",{"2":{"232":2}}],["84s",{"2":{"196":1}}],["845849662922129e",{"2":{"258":1}}],["8458933f",{"2":{"256":1}}],["8454118",{"2":{"148":1}}],["84576",{"2":{"77":1}}],["840588",{"2":{"145":1}}],["84",{"2":{"34":2,"193":6,"196":1,"217":1,"219":1,"228":4}}],["δ",{"2":{"53":1,"117":3,"206":2}}],["ϵ",{"2":{"53":1}}],["ŷ",{"2":{"53":6}}],["∈",{"2":{"53":2,"254":1}}],["∗y+α∗size",{"2":{"53":1}}],["∗y+α∗0",{"2":{"53":1}}],["∗y^−logσ",{"2":{"53":1}}],["∗log⁡",{"2":{"53":1}}],["−log⁡",{"2":{"53":1}}],["−∑y~log⁡",{"2":{"53":1}}],["−",{"2":{"53":1}}],["−y~∗log⁡",{"2":{"53":1}}],["−x",{"2":{"24":1}}],["≈∑θ∼p",{"2":{"250":1}}],["≈",{"2":{"53":25,"59":1,"131":1,"132":1,"174":2}}],["9f",{"2":{"244":8}}],["938655602260798e",{"2":{"258":1}}],["938612f",{"2":{"256":1}}],["9358576575056893e",{"2":{"258":1}}],["9359122f",{"2":{"256":1}}],["932017226481967e",{"2":{"258":1}}],["932017f",{"2":{"256":1}}],["9340946f",{"2":{"256":1}}],["9340093",{"2":{"147":1}}],["937422807626698e",{"2":{"258":1}}],["937291160236234e",{"2":{"258":1}}],["9373574f",{"2":{"256":1}}],["9370079",{"2":{"232":2}}],["9393311049239834e",{"2":{"258":1}}],["9393967f",{"2":{"256":1}}],["939551267745315e",{"2":{"258":1}}],["9396538625362964e",{"2":{"258":1}}],["9392455f",{"2":{"256":1}}],["9365283514297854e",{"2":{"258":1}}],["936928f",{"2":{"256":1}}],["936497f",{"2":{"256":1}}],["933783438113124e",{"2":{"258":1}}],["933842236721743e",{"2":{"258":1}}],["9338737",{"2":{"148":1}}],["9336104666087008e",{"2":{"258":1}}],["9332135f",{"2":{"256":1}}],["9335634f",{"2":{"256":1}}],["9307",{"2":{"249":1}}],["93",{"2":{"228":5}}],["91730478174166e",{"2":{"258":1}}],["9170667f",{"2":{"256":1}}],["919483039624893e",{"2":{"258":1}}],["9194394f",{"2":{"256":1}}],["919269f",{"2":{"256":1}}],["91223164863911e",{"2":{"258":1}}],["912166f",{"2":{"256":1}}],["912781",{"2":{"152":1}}],["9127817",{"2":{"152":1}}],["912782",{"2":{"152":1}}],["9180890281496175e",{"2":{"258":1}}],["918204216237859e",{"2":{"258":1}}],["9181544f",{"2":{"256":1}}],["918128f",{"2":{"256":1}}],["918918",{"2":{"168":1}}],["911473490773644e",{"2":{"258":1}}],["9110552f",{"2":{"256":1}}],["9116936f",{"2":{"256":1}}],["9111102f",{"2":{"256":1}}],["9113503754948588e",{"2":{"258":1}}],["911330343683025e",{"2":{"258":1}}],["9113382f",{"2":{"256":1}}],["9113927629197256e",{"2":{"258":1}}],["9113s\\ttraining",{"2":{"217":1}}],["9144042337643907e",{"2":{"258":1}}],["914130477114876e",{"2":{"258":1}}],["9141305f",{"2":{"256":1}}],["9145657f",{"2":{"256":1}}],["910206f",{"2":{"256":1}}],["910251",{"2":{"147":1}}],["9109",{"2":{"249":1}}],["9108207",{"2":{"232":1}}],["91",{"2":{"228":4}}],["916942741803085e",{"2":{"258":1}}],["9169415f",{"2":{"256":1}}],["9163630895748854e",{"2":{"258":1}}],["9162975f",{"2":{"256":1}}],["916298im",{"2":{"168":1}}],["9166",{"2":{"249":1}}],["9161865",{"2":{"232":1}}],["9161s\\ttraining",{"2":{"217":1}}],["9164896",{"2":{"148":1}}],["913439463193834e",{"2":{"258":1}}],["9132796f",{"2":{"256":1}}],["913106e",{"2":{"208":1}}],["9139614",{"2":{"148":1}}],["9130077",{"2":{"106":2}}],["95844632393345e",{"2":{"258":1}}],["9582808320888176e",{"2":{"258":1}}],["958512f",{"2":{"256":1}}],["9585927f",{"2":{"256":1}}],["959446144919472e",{"2":{"258":1}}],["959421f",{"2":{"256":1}}],["959182",{"2":{"148":1}}],["956327838988016e",{"2":{"258":1}}],["9566823482047338e",{"2":{"258":1}}],["956277f",{"2":{"256":1}}],["956731f",{"2":{"256":1}}],["953958f",{"2":{"256":1}}],["9532s\\ttraining",{"2":{"217":1}}],["951879758423589e",{"2":{"258":1}}],["9514474f",{"2":{"256":1}}],["9515",{"2":{"249":1}}],["9517",{"2":{"249":1}}],["9511775",{"2":{"147":1}}],["954669886169323e",{"2":{"258":1}}],["954913434050791e",{"2":{"258":1}}],["954842f",{"2":{"256":1}}],["9547",{"2":{"249":1}}],["95401394",{"2":{"232":1}}],["95",{"2":{"228":10}}],["950787883423187e",{"2":{"258":1}}],["950722f",{"2":{"256":1}}],["9500",{"2":{"249":1}}],["9500444",{"2":{"147":1}}],["9501",{"2":{"244":1}}],["9501405",{"2":{"148":1}}],["9577397",{"2":{"147":1}}],["9572f",{"2":{"77":1}}],["9778820917328787e",{"2":{"258":1}}],["977256359378199e",{"2":{"258":1}}],["977523f",{"2":{"256":1}}],["97941223851332e",{"2":{"258":1}}],["979176f",{"2":{"256":1}}],["9781487f",{"2":{"256":1}}],["9711788694204284e",{"2":{"258":1}}],["9712992481006527e",{"2":{"258":1}}],["971281807812836e",{"2":{"258":1}}],["971239f",{"2":{"256":1}}],["9713647f",{"2":{"256":1}}],["971899",{"2":{"116":1}}],["9702146387616684e",{"2":{"258":1}}],["9702458f",{"2":{"256":1}}],["9708",{"2":{"249":1}}],["9747",{"2":{"249":1}}],["9742292",{"2":{"232":1}}],["97378695",{"2":{"232":1}}],["9734574",{"2":{"145":1}}],["976422781044965e",{"2":{"258":1}}],["9762614f",{"2":{"256":1}}],["9767632",{"2":{"232":1}}],["9766699015845924",{"2":{"174":4}}],["97667",{"2":{"171":1}}],["97",{"2":{"196":1,"228":4,"249":1}}],["975649",{"2":{"172":1}}],["9727589197669095e",{"2":{"258":1}}],["9727281",{"2":{"147":1}}],["972661216849735e",{"2":{"258":1}}],["9726102f",{"2":{"256":1}}],["9722208594516065e",{"2":{"258":1}}],["9722645f",{"2":{"256":1}}],["97281f",{"2":{"256":1}}],["9721554734769875",{"2":{"153":2}}],["906633952651798e",{"2":{"258":1}}],["906678f",{"2":{"256":1}}],["9021242084925348e",{"2":{"258":1}}],["902982902377702",{"2":{"153":2}}],["903749366049605e",{"2":{"258":1}}],["9037423f",{"2":{"256":1}}],["9019628f",{"2":{"256":1}}],["9019142280758281",{"2":{"153":2}}],["904941f",{"2":{"256":1}}],["9045",{"2":{"249":1}}],["9043228",{"2":{"145":1}}],["9051138961393236e",{"2":{"258":1}}],["9055119",{"2":{"232":2}}],["90508",{"2":{"126":1,"129":1}}],["90",{"2":{"228":10}}],["90910274",{"2":{"148":1}}],["9094505",{"2":{"145":1}}],["907540237105236e",{"2":{"258":1}}],["9075253f",{"2":{"256":1}}],["907459839356695e",{"2":{"258":1}}],["9071",{"2":{"249":1}}],["9076505",{"2":{"235":1}}],["9076533",{"2":{"145":1}}],["9079035",{"2":{"232":1}}],["90779305",{"2":{"232":1}}],["90772897",{"2":{"126":1}}],["9001",{"2":{"179":1,"244":1}}],["900",{"2":{"107":1}}],["961602458283923e",{"2":{"258":1}}],["961646f",{"2":{"256":1}}],["966221140068937e",{"2":{"258":1}}],["9661726f",{"2":{"256":1}}],["9681711f",{"2":{"256":1}}],["968504",{"2":{"232":2}}],["964016713326844e",{"2":{"258":1}}],["964024f",{"2":{"256":1}}],["9640",{"2":{"249":1}}],["9694",{"2":{"249":1}}],["9693878",{"2":{"148":1}}],["965008",{"2":{"235":1}}],["965117",{"2":{"126":1}}],["967892221732617e",{"2":{"258":1}}],["967811",{"2":{"172":1}}],["9679338",{"2":{"232":1}}],["96062994",{"2":{"232":2}}],["9604645f",{"2":{"106":4}}],["96",{"2":{"228":9}}],["96296",{"2":{"219":1}}],["9626998",{"2":{"147":1}}],["98886090397015e",{"2":{"258":1}}],["988985f",{"2":{"256":1}}],["9862820274541477e",{"2":{"258":1}}],["986889344950431e",{"2":{"258":1}}],["9863071f",{"2":{"256":1}}],["9869205f",{"2":{"256":1}}],["983126998097565e",{"2":{"258":1}}],["9835850698482504e",{"2":{"258":1}}],["9836163f",{"2":{"256":1}}],["98396176",{"2":{"117":1}}],["9847",{"2":{"249":1}}],["9849229",{"2":{"235":1}}],["9849193f",{"2":{"106":1}}],["9853",{"2":{"249":1}}],["9856883",{"2":{"232":1}}],["9828212f",{"2":{"256":1}}],["9823",{"2":{"249":1}}],["982162476",{"2":{"244":2}}],["98",{"2":{"228":1}}],["987609e",{"2":{"208":1}}],["987689",{"2":{"145":1}}],["981339",{"2":{"171":1}}],["980564996009565e",{"2":{"258":1}}],["980341f",{"2":{"256":1}}],["9803549",{"2":{"232":1}}],["9806255f",{"2":{"256":1}}],["980289823909054e",{"2":{"258":1}}],["9802s\\ttraining",{"2":{"217":1}}],["9802322f",{"2":{"106":2}}],["98023f",{"2":{"106":4}}],["980",{"2":{"159":1}}],["94386997369688e",{"2":{"258":1}}],["9436797",{"2":{"179":1}}],["9486002238647907e",{"2":{"258":1}}],["9486604f",{"2":{"256":1}}],["9489941",{"2":{"232":1}}],["944634276201806e",{"2":{"258":1}}],["94403f",{"2":{"256":1}}],["944269",{"2":{"168":1}}],["9460",{"2":{"249":1}}],["9414",{"2":{"249":1}}],["9413",{"2":{"249":1}}],["9418648",{"2":{"232":1}}],["94259596",{"2":{"235":1}}],["94",{"2":{"196":1,"228":4}}],["945808366232355e",{"2":{"258":1}}],["945713f",{"2":{"256":1}}],["9452923791531558",{"2":{"153":2}}],["94557f",{"2":{"77":1}}],["9470896",{"2":{"129":1}}],["9402",{"2":{"249":1}}],["9405848223512736",{"2":{"174":4}}],["940585",{"2":{"171":1}}],["940697f",{"2":{"106":1}}],["9407f",{"2":{"106":1}}],["9g",{"2":{"78":1}}],["9230",{"2":{"249":1}}],["923950547913545",{"2":{"153":2}}],["9248211",{"2":{"232":1}}],["924721e",{"2":{"208":1}}],["9253391656059496e",{"2":{"258":1}}],["9253392f",{"2":{"256":1}}],["9253495",{"2":{"232":1}}],["92593",{"2":{"217":1,"219":1}}],["929093054133669e",{"2":{"258":1}}],["9290606f",{"2":{"256":1}}],["92913383",{"2":{"232":2}}],["929f",{"2":{"77":1}}],["92",{"2":{"228":2}}],["928609",{"2":{"145":1}}],["92832f",{"2":{"77":1}}],["9206433",{"2":{"145":1}}],["92155594",{"2":{"176":4}}],["9214196",{"2":{"148":1}}],["921",{"2":{"59":1}}],["990011344791509e",{"2":{"258":1}}],["9900364f",{"2":{"256":1}}],["9943606731133943e",{"2":{"258":1}}],["9945206f",{"2":{"256":1}}],["9914185f",{"2":{"256":1}}],["991352908374989e",{"2":{"258":1}}],["9913",{"2":{"249":1}}],["9916",{"2":{"249":1}}],["9983",{"2":{"249":1}}],["997621817709457e",{"2":{"258":1}}],["997779913308183e",{"2":{"258":1}}],["9978844f",{"2":{"256":1}}],["9978455f",{"2":{"256":1}}],["9971247",{"2":{"235":1}}],["997003",{"2":{"77":6}}],["9934639",{"2":{"232":1}}],["9930804",{"2":{"232":1}}],["992126",{"2":{"232":2}}],["992662",{"2":{"145":1}}],["99",{"2":{"148":1,"196":2,"228":10}}],["9995849836970368e",{"2":{"258":1}}],["9995304f",{"2":{"256":1}}],["9999881",{"2":{"117":2}}],["999986",{"2":{"116":1}}],["99998605",{"2":{"116":1}}],["999",{"2":{"77":7,"234":1,"236":1}}],["99900760833609",{"2":{"53":1}}],["9",{"2":{"53":4,"62":2,"77":15,"94":1,"106":25,"126":2,"129":2,"145":1,"147":1,"148":2,"152":1,"171":1,"179":1,"187":14,"192":1,"195":2,"196":2,"208":2,"213":1,"217":52,"219":10,"224":2,"228":2,"234":1,"236":1,"249":3,"256":55,"258":72}}],["✖",{"2":{"49":4}}],["✔️",{"2":{"102":13}}],["✔",{"2":{"49":8}}],["`θ`",{"2":{"250":1}}],["`x`",{"2":{"250":1}}],["`circbuff",{"2":{"244":4}}],["`carry`",{"2":{"184":2}}],["`p",{"2":{"214":1}}],["`ps",{"2":{"136":1}}],["`iterators",{"2":{"184":1}}],["`eachslice`",{"2":{"184":1}}],["`∇`",{"2":{"176":1}}],["`autoforwarddiff",{"2":{"146":1}}],["`autozygote",{"2":{"146":1}}],["`a`",{"2":{"140":3}}],["`val",{"2":{"145":2}}],["`training`",{"2":{"145":2}}],["`zygote",{"2":{"145":1}}],["`b`",{"2":{"140":2}}],["`denselayerparameters`",{"2":{"137":2}}],["`namedtuple`",{"2":{"137":2}}],["`model",{"2":{"136":1}}],["`st`",{"2":{"136":1,"256":2}}],["`st",{"2":{"136":1}}],["`lag",{"2":{"244":4}}],["`lstm",{"2":{"184":1}}],["`linear",{"2":{"136":2}}],["`l",{"2":{"135":1}}],["`luxcore",{"2":{"145":2}}],["`luxcore`",{"2":{"135":1}}],["`lux",{"2":{"145":2}}],["`lux``",{"2":{"172":1}}],["`lux`",{"2":{"135":1}}],["`",{"2":{"48":1,"77":2,"135":1,"136":1,"145":2,"146":2,"176":1,"244":8}}],["`octavian",{"2":{"18":1}}],["×",{"2":{"28":6,"180":1,"183":2,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["×dn−2×1×1`",{"2":{"69":1}}],["×dn−2×1×1",{"2":{"19":1}}],["×dn−2×1×dn",{"2":{"19":1,"69":1}}],["≥",{"2":{"28":2,"54":1,"244":1}}],["znver2",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["zip",{"2":{"107":1,"244":1}}],["zips",{"2":{"44":1}}],["zyg",{"2":{"106":2}}],["zygotevjp",{"2":{"220":13}}],["zygote",{"2":{"27":1,"28":1,"49":1,"52":1,"53":1,"59":2,"77":3,"78":1,"100":1,"102":1,"103":3,"106":3,"117":2,"139":2,"140":2,"144":3,"145":1,"146":1,"147":3,"148":3,"152":3,"170":1,"173":1,"175":3,"176":4,"181":1,"182":1,"191":1,"212":1,"223":1,"231":1,"236":1,"239":1,"242":3}}],["zenodo",{"2":{"74":3}}],["zeroing",{"2":{"44":4,"55":1}}],["zero",{"2":{"24":3,"44":3,"53":2,"54":2,"55":9,"63":1,"64":3,"67":1,"107":1,"146":1,"171":1}}],["zeroed",{"2":{"24":3}}],["zerosc64",{"2":{"25":1}}],["zerosc32",{"2":{"25":1}}],["zerosc16",{"2":{"25":1}}],["zeros64",{"2":{"25":1}}],["zeros32",{"2":{"25":1,"135":2}}],["zeros16",{"2":{"25":1}}],["zeros",{"2":{"24":3,"25":6,"153":2,"171":1,"243":3,"248":1,"249":1}}],["z=σ",{"2":{"66":1}}],["z=",{"2":{"49":1}}],["z",{"2":{"49":2,"67":3,"249":2,"250":8}}],["≤",{"2":{"24":2,"49":2,"67":2}}],["6e",{"2":{"208":1}}],["6d",{"2":{"179":1}}],["660376532935051e",{"2":{"258":1}}],["660772765357732e",{"2":{"258":1}}],["660613f",{"2":{"256":1}}],["665910215948288e",{"2":{"258":1}}],["665616f",{"2":{"256":1}}],["6657373f",{"2":{"256":1}}],["665241",{"2":{"53":10}}],["664751354708432e",{"2":{"258":1}}],["664719f",{"2":{"256":1}}],["6642563709240284",{"2":{"153":2}}],["66",{"2":{"249":2,"256":1}}],["6685824158024457e",{"2":{"258":1}}],["6685279f",{"2":{"256":1}}],["66852736",{"2":{"235":1}}],["6684s\\ttraining",{"2":{"219":1}}],["667572",{"2":{"232":1}}],["663725478906726e",{"2":{"258":1}}],["6637242",{"2":{"147":1}}],["6630197f",{"2":{"256":1}}],["663036",{"2":{"172":1}}],["663553f",{"2":{"256":1}}],["6631958128077746e",{"2":{"258":1}}],["6631",{"2":{"249":1}}],["6660078765665115e",{"2":{"258":1}}],["6663792108896785e",{"2":{"258":1}}],["6663244",{"2":{"147":1}}],["66667",{"2":{"217":15,"219":4}}],["666682",{"2":{"147":1}}],["666965",{"2":{"168":1}}],["6819804465232837e",{"2":{"258":1}}],["681728",{"2":{"172":1}}],["682207231900662e",{"2":{"258":1}}],["682513f",{"2":{"256":1}}],["688773505313301e",{"2":{"258":1}}],["6887058029987743e",{"2":{"258":1}}],["688511f",{"2":{"256":1}}],["6886734f",{"2":{"256":1}}],["6886741",{"2":{"126":1}}],["688665",{"2":{"148":1}}],["689584608618977e",{"2":{"258":1}}],["6893357f",{"2":{"256":1}}],["689811f",{"2":{"256":1}}],["689767437074972e",{"2":{"258":1}}],["6897085f",{"2":{"256":1}}],["6897",{"2":{"249":1}}],["6847330007470475e",{"2":{"258":1}}],["6844086f",{"2":{"256":1}}],["6846",{"2":{"249":1}}],["6862267636589728e",{"2":{"258":1}}],["6862592f",{"2":{"256":1}}],["6867142",{"2":{"148":1}}],["68",{"2":{"217":2,"219":2,"228":17,"249":1}}],["6830945313415872",{"2":{"174":1}}],["6852",{"2":{"249":1}}],["6856",{"2":{"249":1}}],["6859683",{"2":{"232":1}}],["6850394",{"2":{"232":2}}],["685079",{"2":{"171":1}}],["6853896",{"2":{"148":1}}],["687753065833284e",{"2":{"258":1}}],["687813f",{"2":{"256":1}}],["6875",{"2":{"249":1}}],["687611",{"2":{"168":1}}],["687264",{"2":{"148":1}}],["687216",{"2":{"131":2}}],["680748",{"2":{"126":1}}],["692928915939261e",{"2":{"258":1}}],["6920981",{"2":{"232":1}}],["6960414138666134e",{"2":{"258":1}}],["6930075f",{"2":{"256":1}}],["691628635000494e",{"2":{"258":1}}],["691924f",{"2":{"256":1}}],["691207f",{"2":{"256":1}}],["6913705",{"2":{"145":1}}],["6958052f",{"2":{"256":1}}],["6970",{"2":{"249":1}}],["694813897298262e",{"2":{"258":1}}],["6947888f",{"2":{"256":1}}],["69472f",{"2":{"77":1}}],["6943",{"2":{"249":1}}],["6904922",{"2":{"232":1}}],["6909446526419574",{"2":{"153":2}}],["698632108539192e",{"2":{"258":1}}],["698606e",{"2":{"208":1}}],["69843f",{"2":{"256":1}}],["6983237",{"2":{"232":1}}],["6983s\\ttraining",{"2":{"217":1}}],["69s",{"2":{"196":1}}],["69",{"2":{"111":1,"196":1,"217":4,"228":2}}],["616636744536835e",{"2":{"258":1}}],["616932f",{"2":{"256":1}}],["6157609749355256e",{"2":{"258":1}}],["6156655f",{"2":{"256":1}}],["619961029693845e",{"2":{"258":1}}],["619813979774715e",{"2":{"258":1}}],["6197484f",{"2":{"256":1}}],["61964",{"2":{"187":1}}],["612883732914699e",{"2":{"258":1}}],["6128222f",{"2":{"256":1}}],["612652992366777e",{"2":{"258":1}}],["612653f",{"2":{"256":1}}],["6120",{"2":{"24":1}}],["6189",{"2":{"249":1}}],["61375986520706e",{"2":{"258":1}}],["6131280006125398e",{"2":{"258":1}}],["613493f",{"2":{"256":1}}],["6134",{"2":{"249":1}}],["6133",{"2":{"249":1}}],["6138451",{"2":{"106":1}}],["61384505",{"2":{"106":1}}],["6117",{"2":{"249":1}}],["6115",{"2":{"249":1}}],["61155146",{"2":{"232":1}}],["611584190904272",{"2":{"153":2}}],["6142140561750679e",{"2":{"258":1}}],["6141",{"2":{"249":1}}],["61417323",{"2":{"232":2}}],["6143286",{"2":{"126":1}}],["61",{"2":{"196":1,"228":2}}],["6107s\\ttraining",{"2":{"217":1}}],["610",{"2":{"148":1}}],["6587045347085305e",{"2":{"258":1}}],["6586156f",{"2":{"256":1}}],["658656f",{"2":{"256":1}}],["6594765471427673e",{"2":{"258":1}}],["659688f",{"2":{"256":1}}],["659702f",{"2":{"256":1}}],["6598257783694784e",{"2":{"258":1}}],["6598398f",{"2":{"256":1}}],["6598s\\ttraining",{"2":{"217":1}}],["654769556061554e",{"2":{"258":1}}],["6544744f",{"2":{"256":1}}],["6543269f",{"2":{"256":1}}],["6541584",{"2":{"106":1}}],["6541586",{"2":{"106":1}}],["6537222857452056e",{"2":{"258":1}}],["653540215202865e",{"2":{"258":1}}],["6535434",{"2":{"232":2}}],["6534965f",{"2":{"256":1}}],["6574890913135e",{"2":{"258":1}}],["657458f",{"2":{"256":1}}],["6571946",{"2":{"235":1}}],["657670184695808",{"2":{"153":2}}],["65",{"2":{"196":2,"228":12}}],["6556360448565952",{"2":{"174":1}}],["655881",{"2":{"171":1}}],["65178",{"2":{"147":1}}],["656613f",{"2":{"106":1}}],["650577240955891e",{"2":{"258":1}}],["65075f",{"2":{"256":1}}],["650139914756272e",{"2":{"258":1}}],["6501",{"2":{"244":1}}],["650",{"2":{"69":3}}],["630274419652513e",{"2":{"258":1}}],["63288923365011e",{"2":{"258":1}}],["6323408",{"2":{"126":1}}],["63160809479066e",{"2":{"258":1}}],["6316737f",{"2":{"256":1}}],["633184466347801e",{"2":{"258":1}}],["6330246f",{"2":{"256":1}}],["639988800741287e",{"2":{"258":1}}],["6399032f",{"2":{"256":1}}],["63922",{"2":{"187":1}}],["636719023475917e",{"2":{"258":1}}],["636546f",{"2":{"256":1}}],["636308",{"2":{"106":1}}],["6389992",{"2":{"232":1}}],["63838756",{"2":{"147":1}}],["63",{"2":{"228":1}}],["634202530999464e",{"2":{"258":1}}],["6345785334601886e",{"2":{"258":1}}],["634549",{"2":{"168":1}}],["63437f",{"2":{"256":1}}],["634942f",{"2":{"256":1}}],["634965f",{"2":{"256":1}}],["6349",{"2":{"249":1}}],["6340362477836592",{"2":{"174":1}}],["637753878908047e",{"2":{"258":1}}],["6370531107315014",{"2":{"153":2}}],["6372185",{"2":{"148":1}}],["63795453",{"2":{"106":1}}],["6230438084068423e",{"2":{"258":1}}],["6230875f",{"2":{"256":1}}],["620743f",{"2":{"256":1}}],["6202236f",{"2":{"256":1}}],["6202136",{"2":{"232":1}}],["629096931148694e",{"2":{"258":1}}],["629122f",{"2":{"256":1}}],["62963",{"2":{"217":2}}],["626286399747263e",{"2":{"258":1}}],["62625877100596",{"2":{"171":1}}],["626892726669568e",{"2":{"258":1}}],["6260574f",{"2":{"256":1}}],["62641f",{"2":{"256":1}}],["62186052855715e",{"2":{"258":1}}],["6214",{"2":{"249":1}}],["621958",{"2":{"171":1}}],["6223006443002694e",{"2":{"258":1}}],["6221407f",{"2":{"256":1}}],["6220472",{"2":{"232":2}}],["622865",{"2":{"172":1}}],["62",{"2":{"196":1,"228":19}}],["627069f",{"2":{"256":1}}],["627",{"2":{"180":1}}],["6242370661773574e",{"2":{"258":1}}],["6249428919618555e",{"2":{"258":1}}],["6244044f",{"2":{"256":1}}],["6246s\\ttraining",{"2":{"217":1}}],["624383955429775",{"2":{"174":1}}],["624384",{"2":{"171":1}}],["62450767",{"2":{"106":1}}],["625460438220049e",{"2":{"258":1}}],["625794892509532e",{"2":{"258":1}}],["625631f",{"2":{"256":1}}],["6251043f",{"2":{"256":1}}],["625",{"2":{"53":1}}],["603920272009079e",{"2":{"258":1}}],["6036577f",{"2":{"256":1}}],["6059146276209827e",{"2":{"258":1}}],["605977f",{"2":{"256":1}}],["605866f",{"2":{"256":1}}],["602745740313698e",{"2":{"258":1}}],["6029056f",{"2":{"256":1}}],["6022s\\ttraining",{"2":{"219":1}}],["609350097255151e",{"2":{"258":1}}],["609",{"2":{"237":1}}],["60",{"2":{"196":1,"228":1,"249":1}}],["606037329305128e",{"2":{"258":1}}],["60676",{"2":{"187":1}}],["6064619",{"2":{"106":1}}],["6049256",{"2":{"148":1}}],["6003383",{"2":{"232":1}}],["6001",{"2":{"179":1,"244":1}}],["600",{"2":{"107":1}}],["6076053f0",{"2":{"53":2}}],["6730006732762932e",{"2":{"258":1}}],["673478589786585e",{"2":{"258":1}}],["673646f",{"2":{"256":1}}],["672932497396827e",{"2":{"258":1}}],["672867f",{"2":{"256":1}}],["6726513",{"2":{"145":1}}],["6717555857054658e",{"2":{"258":1}}],["6716919876302937e",{"2":{"258":1}}],["6718411f",{"2":{"256":1}}],["6719978f",{"2":{"256":1}}],["6712",{"2":{"249":1}}],["677444643462762e",{"2":{"258":1}}],["677242340888547e",{"2":{"258":1}}],["677286f",{"2":{"256":1}}],["6773845f",{"2":{"256":1}}],["6771653",{"2":{"232":2}}],["6776220912718907",{"2":{"174":1}}],["6759801412865855e",{"2":{"258":1}}],["67595f",{"2":{"77":1}}],["6758072f",{"2":{"256":1}}],["6758s\\ttraining",{"2":{"217":1}}],["6752",{"2":{"249":1}}],["67",{"2":{"196":1,"217":1,"228":1}}],["67dffc4a8ae",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["6788684f",{"2":{"256":1}}],["6788",{"2":{"249":1}}],["67861",{"2":{"171":1}}],["6784163",{"2":{"77":1}}],["67909",{"2":{"171":1}}],["67908657",{"2":{"171":1}}],["679087",{"2":{"171":2}}],["6798847",{"2":{"147":1}}],["67094",{"2":{"168":1}}],["6702257",{"2":{"145":1}}],["6",{"2":{"24":1,"34":2,"62":3,"77":7,"106":1,"145":1,"147":5,"148":7,"152":1,"171":2,"173":3,"179":1,"180":5,"187":14,"189":5,"193":8,"196":2,"197":1,"208":2,"210":5,"217":5,"219":1,"221":5,"228":2,"229":5,"232":6,"237":5,"244":4,"246":5,"249":11,"250":6,"251":1,"255":2,"256":161,"258":161,"260":1}}],["649845f",{"2":{"256":1}}],["641964f",{"2":{"256":1}}],["6433709535667355e",{"2":{"258":1}}],["6434365f",{"2":{"256":1}}],["6431680510403082e",{"2":{"258":1}}],["6431195f",{"2":{"256":1}}],["6431484",{"2":{"235":1}}],["647251920260201e",{"2":{"258":1}}],["6471756f",{"2":{"256":1}}],["6477456",{"2":{"145":1}}],["645297002663145e",{"2":{"258":1}}],["645576f",{"2":{"256":1}}],["6456693",{"2":{"232":2}}],["6456416023530093",{"2":{"174":1}}],["64495987",{"2":{"232":1}}],["6448233",{"2":{"232":1}}],["642088009313119e",{"2":{"258":1}}],["6423511984038721",{"2":{"153":2}}],["6424197",{"2":{"147":1}}],["6465652342208385e",{"2":{"258":1}}],["6466415f",{"2":{"256":1}}],["64644",{"2":{"232":1}}],["6469914",{"2":{"126":1}}],["64605f",{"2":{"77":1}}],["6406990034034946e",{"2":{"258":1}}],["6407535f",{"2":{"256":1}}],["6409345",{"2":{"147":1}}],["6409691",{"2":{"106":1}}],["64058f",{"2":{"77":1}}],["64",{"2":{"18":1,"69":18,"78":1,"180":2,"189":2,"197":2,"210":2,"221":2,"226":2,"229":2,"237":2,"246":2,"251":3,"256":1,"260":2}}],["β",{"2":{"19":4,"69":1,"206":2}}],["γ=0",{"2":{"53":2}}],["γ",{"2":{"19":4,"53":1,"69":1,"206":2}}],["7651367f",{"2":{"256":1}}],["7624133951519924e",{"2":{"258":1}}],["7624674f",{"2":{"256":1}}],["7624897f",{"2":{"256":1}}],["7609525343764992e",{"2":{"258":1}}],["760977e",{"2":{"208":1}}],["760427422715573e",{"2":{"258":1}}],["760447f",{"2":{"256":1}}],["768967092895072e",{"2":{"258":1}}],["7689272356710357",{"2":{"153":2}}],["768313f",{"2":{"256":1}}],["76158035",{"2":{"232":1}}],["764638737529744e",{"2":{"258":1}}],["7646063f",{"2":{"256":1}}],["764463548183539e",{"2":{"258":1}}],["764152f",{"2":{"256":1}}],["76425457",{"2":{"232":1}}],["7649314",{"2":{"148":1}}],["7663455",{"2":{"232":1}}],["76s",{"2":{"196":1}}],["76",{"2":{"196":2,"208":2,"217":6,"219":2,"228":2}}],["767996f",{"2":{"256":1}}],["7673453",{"2":{"126":1}}],["7672513",{"2":{"106":1}}],["731966267225899e",{"2":{"258":1}}],["731059",{"2":{"53":2}}],["7373078571754834e",{"2":{"258":1}}],["7370",{"2":{"249":1}}],["739095023535343e",{"2":{"258":1}}],["739751602749602e",{"2":{"258":1}}],["7397516f",{"2":{"256":1}}],["735090023190146e",{"2":{"258":1}}],["7352",{"2":{"249":1}}],["732200674826759e",{"2":{"258":1}}],["732115f",{"2":{"256":1}}],["732052f",{"2":{"256":1}}],["7346220608212875e",{"2":{"258":1}}],["73406131587131e",{"2":{"258":1}}],["734798f",{"2":{"256":1}}],["7342343f",{"2":{"256":1}}],["7348538f",{"2":{"256":1}}],["733465620020983e",{"2":{"258":1}}],["733466f",{"2":{"256":1}}],["73341",{"2":{"232":1}}],["73",{"2":{"196":2,"217":1,"219":1,"228":3}}],["7385816896544564e",{"2":{"258":1}}],["738736799869232e",{"2":{"258":1}}],["738614370580132e",{"2":{"258":1}}],["7386144f",{"2":{"256":1}}],["7386605f",{"2":{"256":1}}],["738859f",{"2":{"256":1}}],["738496f",{"2":{"256":1}}],["7384",{"2":{"249":1}}],["7384486",{"2":{"232":1}}],["73806",{"2":{"172":1}}],["738356",{"2":{"171":1}}],["730941",{"2":{"148":1}}],["752449346610743e",{"2":{"258":1}}],["751714181455874e",{"2":{"258":1}}],["751939f",{"2":{"256":1}}],["7515543f",{"2":{"256":1}}],["751377",{"2":{"147":1}}],["758773689184584e",{"2":{"258":1}}],["758543213589722e",{"2":{"258":1}}],["7586665f",{"2":{"256":1}}],["758307f",{"2":{"256":1}}],["7580993408473766",{"2":{"17":1}}],["759619100026308e",{"2":{"258":1}}],["7598553f",{"2":{"256":1}}],["759412",{"2":{"148":1}}],["7533",{"2":{"249":1}}],["7560",{"2":{"249":1}}],["756177e",{"2":{"208":1}}],["7559453",{"2":{"232":1}}],["75",{"2":{"217":2,"219":1,"228":23,"249":1,"255":1,"256":2,"259":4}}],["7502",{"2":{"251":1,"260":1}}],["750194624824808e",{"2":{"258":1}}],["7501",{"2":{"244":1}}],["750",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["7541456",{"2":{"232":1}}],["754747",{"2":{"148":1}}],["754463",{"2":{"148":1}}],["7718179375088544e",{"2":{"258":1}}],["77165353",{"2":{"232":2}}],["775208708426028e",{"2":{"258":1}}],["77526884507643e",{"2":{"258":1}}],["7752743f",{"2":{"256":1}}],["7759933246341794e",{"2":{"258":1}}],["775398577173693e",{"2":{"258":1}}],["775345f",{"2":{"256":1}}],["7757571f",{"2":{"256":1}}],["773003886737635e",{"2":{"258":1}}],["7730693f",{"2":{"256":1}}],["7730363f",{"2":{"256":1}}],["7733535276924052",{"2":{"174":4}}],["773354",{"2":{"171":1}}],["7781809077523993e",{"2":{"258":1}}],["7781497f",{"2":{"256":1}}],["7789",{"2":{"249":1}}],["7795275",{"2":{"232":2}}],["779426e",{"2":{"208":1}}],["7794657",{"2":{"126":1}}],["77",{"2":{"217":1,"228":1}}],["77778",{"2":{"217":2}}],["7741s\\ttraining",{"2":{"217":1}}],["7746656838366666",{"2":{"174":1}}],["7766",{"2":{"171":1}}],["776598",{"2":{"171":2}}],["77207",{"2":{"148":1}}],["770813849111468e",{"2":{"258":1}}],["770885f",{"2":{"256":1}}],["770358496073841e",{"2":{"258":1}}],["7703",{"2":{"249":1}}],["77037",{"2":{"147":1}}],["7701346738750905",{"2":{"153":2}}],["7700396",{"2":{"147":1}}],["7110655302891605e",{"2":{"258":1}}],["711329",{"2":{"168":1}}],["715464643035007e",{"2":{"258":1}}],["715701f",{"2":{"256":1}}],["7103025f",{"2":{"256":1}}],["7108229f",{"2":{"256":1}}],["714292507921566e",{"2":{"258":1}}],["714238f",{"2":{"256":1}}],["714094f",{"2":{"256":1}}],["7149",{"2":{"249":1}}],["717424224382949e",{"2":{"258":1}}],["7174554f",{"2":{"256":1}}],["7179456f",{"2":{"256":1}}],["717862265170678e",{"2":{"258":1}}],["717863343114228",{"2":{"153":2}}],["7178s\\ttraining",{"2":{"217":1}}],["7199",{"2":{"249":1}}],["7195462169922173",{"2":{"153":1}}],["7195462169922175",{"2":{"153":1}}],["7121",{"2":{"249":1}}],["7127",{"2":{"249":1}}],["7123551",{"2":{"232":1}}],["712979078066394",{"2":{"153":2}}],["7183194",{"2":{"232":1}}],["7181164",{"2":{"145":1}}],["71s",{"2":{"196":1}}],["71",{"2":{"196":1,"217":2,"228":11}}],["71681050840083e",{"2":{"258":1}}],["7168105f",{"2":{"256":1}}],["7165354",{"2":{"232":2}}],["7165512278088038",{"2":{"153":2}}],["716134755899883",{"2":{"153":2}}],["713058120829044e",{"2":{"258":1}}],["713364f",{"2":{"256":1}}],["713316",{"2":{"106":1}}],["713",{"2":{"236":1}}],["71315+0",{"2":{"168":1}}],["713567516238075",{"2":{"153":2}}],["741878603182283e",{"2":{"258":1}}],["741348",{"2":{"232":1}}],["7452810343679628e",{"2":{"258":1}}],["7453644f",{"2":{"256":1}}],["743726519704837e",{"2":{"258":1}}],["743678f",{"2":{"256":1}}],["747605783095499e",{"2":{"258":1}}],["7474338f",{"2":{"256":1}}],["7473106f",{"2":{"256":1}}],["7498834f",{"2":{"256":1}}],["7421070270380343e",{"2":{"258":1}}],["7421926f",{"2":{"256":1}}],["7426",{"2":{"249":1}}],["7429947",{"2":{"129":1}}],["7480315",{"2":{"232":2}}],["7400992173019836e",{"2":{"258":1}}],["740639f",{"2":{"256":1}}],["740405f",{"2":{"256":1}}],["7401575",{"2":{"232":2}}],["74074",{"2":{"217":3}}],["7402",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["74",{"2":{"196":1,"217":5,"228":1}}],["746749184499562e",{"2":{"258":1}}],["7466636f",{"2":{"256":1}}],["7468008914093891",{"2":{"174":4}}],["746801",{"2":{"171":1}}],["74694291453392",{"2":{"174":4}}],["746943",{"2":{"171":1}}],["744489",{"2":{"145":1}}],["709861174396069e",{"2":{"258":1}}],["7092237618801464e",{"2":{"258":1}}],["70926f",{"2":{"256":1}}],["704435244334782e",{"2":{"258":1}}],["704262f",{"2":{"256":1}}],["70714088884325e",{"2":{"258":1}}],["707154f",{"2":{"256":1}}],["70786051052727e",{"2":{"258":1}}],["707813494580927e",{"2":{"258":1}}],["70740747390149e",{"2":{"258":1}}],["707998788399833e",{"2":{"258":1}}],["7079844f",{"2":{"256":1}}],["707534",{"2":{"171":1}}],["7062434919532456e",{"2":{"258":1}}],["7062184f",{"2":{"256":1}}],["706277f",{"2":{"256":1}}],["706858618952844e",{"2":{"258":1}}],["7068297f",{"2":{"256":1}}],["7068062f",{"2":{"256":1}}],["701348139444165e",{"2":{"258":1}}],["701661750316573e",{"2":{"258":1}}],["701655f",{"2":{"256":1}}],["701846f",{"2":{"256":1}}],["7018592",{"2":{"145":1}}],["7024",{"2":{"249":1}}],["7081604043003266e",{"2":{"258":1}}],["7081s\\ttraining",{"2":{"217":1}}],["7082115f",{"2":{"256":1}}],["7080156f",{"2":{"256":1}}],["70866144",{"2":{"232":2}}],["7089496198123055",{"2":{"153":2}}],["70",{"2":{"217":1,"228":2}}],["70s",{"2":{"196":7}}],["7054554",{"2":{"147":1}}],["7031091f",{"2":{"256":1}}],["70370",{"2":{"217":1}}],["70390546",{"2":{"126":1}}],["70330536",{"2":{"77":1}}],["700202567847338e",{"2":{"258":1}}],["700263f",{"2":{"256":1}}],["7005s\\ttraining",{"2":{"217":1}}],["7001468",{"2":{"232":1}}],["7001",{"2":{"179":1,"244":1}}],["700",{"2":{"107":1}}],["7217091029342412e",{"2":{"258":1}}],["7211657f",{"2":{"256":1}}],["723347007773074e",{"2":{"258":1}}],["7237308297251812",{"2":{"174":1}}],["72373",{"2":{"171":1}}],["723731",{"2":{"171":2}}],["726351164446e",{"2":{"258":1}}],["7263f",{"2":{"256":1}}],["726808143903494e",{"2":{"258":1}}],["7262075",{"2":{"147":1}}],["722984f",{"2":{"256":1}}],["7270444f",{"2":{"256":1}}],["7273",{"2":{"249":1}}],["7274",{"2":{"249":1}}],["7200",{"2":{"249":1}}],["7259s\\ttraining",{"2":{"217":1}}],["7252903f",{"2":{"106":8}}],["72",{"2":{"196":1,"217":3}}],["724291174986106e",{"2":{"258":1}}],["72428167",{"2":{"126":1}}],["724115f",{"2":{"256":1}}],["724099",{"2":{"168":1}}],["72471225",{"2":{"147":1}}],["7295348785206703e",{"2":{"258":1}}],["7295855f",{"2":{"256":1}}],["7296810323607505e",{"2":{"258":1}}],["7296474",{"2":{"147":1}}],["729386077417963e",{"2":{"258":1}}],["729355f",{"2":{"256":1}}],["7294",{"2":{"249":1}}],["72908926",{"2":{"148":1}}],["7292344",{"2":{"145":1}}],["729",{"2":{"77":6}}],["7922867808978655e",{"2":{"258":1}}],["79250443",{"2":{"147":1}}],["798890823729545e",{"2":{"258":1}}],["798891f",{"2":{"256":1}}],["798027",{"2":{"147":1}}],["7977",{"2":{"249":1}}],["794170328027556e",{"2":{"258":1}}],["7942015f",{"2":{"256":1}}],["7942",{"2":{"249":1}}],["794908",{"2":{"147":1}}],["7990667",{"2":{"235":1}}],["79",{"2":{"196":2,"217":4,"228":2,"249":1}}],["7911112",{"2":{"147":1}}],["793159",{"2":{"171":1}}],["7939677f",{"2":{"106":1}}],["793463f",{"2":{"77":1}}],["79656f",{"2":{"77":1}}],["7886926096957105e",{"2":{"258":1}}],["78818554",{"2":{"232":1}}],["78814f",{"2":{"106":1}}],["786864492114564e",{"2":{"258":1}}],["786319f",{"2":{"256":1}}],["7829989662953562e",{"2":{"258":1}}],["7829227f",{"2":{"256":1}}],["78226817",{"2":{"135":1}}],["785606996131011e",{"2":{"258":1}}],["7852259233548545e",{"2":{"258":1}}],["785301f",{"2":{"256":1}}],["7854622f",{"2":{"256":1}}],["787924651986792e",{"2":{"258":1}}],["7879247f",{"2":{"256":1}}],["787159",{"2":{"145":1}}],["783470620238665e",{"2":{"258":1}}],["783416f",{"2":{"256":1}}],["7834",{"2":{"249":1}}],["7834251",{"2":{"147":1}}],["7891",{"2":{"249":1}}],["7897024",{"2":{"232":1}}],["78",{"2":{"196":3,"217":4,"219":1,"228":16}}],["7817315740767116",{"2":{"174":1}}],["7811481",{"2":{"145":1}}],["78000563",{"2":{"136":1}}],["7808904",{"2":{"74":2}}],["784083609884784e",{"2":{"258":1}}],["784082f",{"2":{"256":1}}],["7840384829517084e",{"2":{"258":1}}],["7840232f",{"2":{"256":1}}],["784768",{"2":{"147":1,"148":1}}],["784",{"2":{"69":6,"215":1,"220":6,"226":1}}],["7",{"2":{"19":9,"53":5,"62":4,"77":17,"106":10,"145":5,"147":2,"148":5,"152":1,"173":3,"179":1,"180":4,"187":14,"189":4,"196":2,"197":1,"208":1,"210":4,"217":5,"219":1,"220":3,"221":4,"228":2,"229":4,"232":5,"237":4,"246":4,"249":6,"251":1,"256":72,"258":74,"260":1}}],["5g",{"2":{"236":1}}],["5gb",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["5fs",{"2":{"228":1}}],["5f",{"2":{"187":3,"217":2}}],["5f0",{"2":{"126":1,"129":1,"152":1,"186":1,"248":12}}],["541732726041285e",{"2":{"258":1}}],["5410349f",{"2":{"256":1}}],["5496321184933076e",{"2":{"258":1}}],["5497177f",{"2":{"256":1}}],["5451092420729488e",{"2":{"258":1}}],["5452692f",{"2":{"256":1}}],["546812990455935e",{"2":{"258":1}}],["54682f",{"2":{"256":1}}],["5462926f",{"2":{"256":1}}],["5429377526735596e",{"2":{"258":1}}],["5429378f",{"2":{"256":1}}],["5424696",{"2":{"232":1}}],["542428",{"2":{"168":1}}],["540280074355407e",{"2":{"258":1}}],["5402196f",{"2":{"256":1}}],["540443387642542e",{"2":{"258":1}}],["5409693080034265e",{"2":{"258":1}}],["540062523231966e",{"2":{"258":1}}],["540509f",{"2":{"256":1}}],["540",{"2":{"220":8}}],["54075",{"2":{"187":1}}],["54",{"2":{"217":1}}],["544287e",{"2":{"208":1}}],["54452",{"2":{"172":1}}],["5476424498276177",{"2":{"174":4}}],["547642",{"2":{"171":1}}],["5×3",{"2":{"171":8,"172":1}}],["5×5",{"2":{"24":1,"171":3}}],["585594747124718e",{"2":{"258":1}}],["5852",{"2":{"249":1}}],["5891253316160807e",{"2":{"258":1}}],["5891324f",{"2":{"256":1}}],["5842924f",{"2":{"256":1}}],["584939f",{"2":{"256":1}}],["5840",{"2":{"249":1}}],["583997238264792e",{"2":{"258":1}}],["5833908f",{"2":{"256":1}}],["583076f",{"2":{"256":1}}],["5834562210438314e",{"2":{"258":1}}],["5834",{"2":{"249":1}}],["5812",{"2":{"249":1}}],["582914752732503e",{"2":{"258":1}}],["5821",{"2":{"249":1}}],["58263564",{"2":{"235":1}}],["5826772",{"2":{"232":2}}],["580818565085058e",{"2":{"258":1}}],["5808671f",{"2":{"256":1}}],["58030628269383e",{"2":{"258":1}}],["5806",{"2":{"249":1}}],["5809",{"2":{"249":1}}],["58",{"2":{"196":1,"217":3,"219":1,"228":1}}],["586636+0",{"2":{"168":1}}],["5865265807660498",{"2":{"153":2}}],["587973597486665e",{"2":{"258":1}}],["5879",{"2":{"249":1}}],["5879354f",{"2":{"106":1}}],["58721",{"2":{"171":1}}],["58720636",{"2":{"171":1}}],["58720636f0",{"2":{"171":1}}],["587206",{"2":{"171":2}}],["5878153",{"2":{"147":1}}],["5206532995873412e",{"2":{"258":1}}],["5204919f",{"2":{"256":1}}],["52024",{"2":{"232":1}}],["528007f",{"2":{"256":1}}],["5283",{"2":{"249":1}}],["528130970815243e",{"2":{"258":1}}],["5281",{"2":{"74":2}}],["52628386",{"2":{"232":1}}],["5267549745189349",{"2":{"153":2}}],["527559",{"2":{"232":2}}],["52717",{"2":{"147":1}}],["5257s\\ttraining",{"2":{"219":1}}],["525211e",{"2":{"208":1}}],["5219s\\ttraining",{"2":{"217":1}}],["5210742834996898",{"2":{"153":2}}],["5228735603273347e",{"2":{"258":1}}],["52232",{"2":{"187":1}}],["5221656",{"2":{"179":1}}],["522139",{"2":{"148":1}}],["5246345f",{"2":{"256":1}}],["5240s\\ttraining",{"2":{"217":1}}],["524008",{"2":{"171":1}}],["524781",{"2":{"148":1}}],["570309106577303e",{"2":{"258":1}}],["570129f",{"2":{"256":1}}],["575690225357303e",{"2":{"258":1}}],["5756973f",{"2":{"256":1}}],["571982f",{"2":{"256":1}}],["5718",{"2":{"254":2}}],["5792185290342753e",{"2":{"258":1}}],["579302f",{"2":{"256":1}}],["5794",{"2":{"249":1}}],["57950",{"2":{"187":1}}],["5789171421300255e",{"2":{"258":1}}],["578166980682285e",{"2":{"258":1}}],["578554f",{"2":{"256":1}}],["5780056f",{"2":{"256":1}}],["578374863",{"2":{"244":2}}],["57882756",{"2":{"232":1}}],["57",{"2":{"217":5,"228":1}}],["577555891661407e",{"2":{"258":1}}],["577581f",{"2":{"256":1}}],["577486f",{"2":{"256":1}}],["577181",{"2":{"171":1}}],["5779067839062536",{"2":{"153":2}}],["577653355970464e",{"2":{"258":1}}],["5776367e",{"2":{"148":1}}],["5776052f0",{"2":{"53":1}}],["57671577",{"2":{"137":1}}],["566240536224125e",{"2":{"258":1}}],["5663529032065214e",{"2":{"258":1}}],["566336f",{"2":{"256":1}}],["5663",{"2":{"249":1}}],["565380973885296e",{"2":{"258":1}}],["5651",{"2":{"249":1}}],["564503463515998e",{"2":{"258":1}}],["5647397f",{"2":{"256":1}}],["569926771309863e",{"2":{"258":1}}],["563478992310894e",{"2":{"258":1}}],["5638423f",{"2":{"256":1}}],["563782f",{"2":{"256":1}}],["5614860364583e",{"2":{"258":1}}],["5615815f",{"2":{"256":1}}],["561137f",{"2":{"106":1}}],["56",{"2":{"196":2,"217":1,"228":2}}],["5673",{"2":{"249":1}}],["5675557670686644",{"2":{"174":1}}],["56779",{"2":{"171":1}}],["567794",{"2":{"171":2}}],["56046",{"2":{"187":1}}],["560",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["56055",{"2":{"171":1}}],["56069607",{"2":{"106":1}}],["5680085614623662",{"2":{"153":2}}],["5627227",{"2":{"145":1}}],["593560692391147e",{"2":{"258":1}}],["593646f",{"2":{"256":1}}],["5925883083114988e",{"2":{"258":1}}],["59259",{"2":{"219":1}}],["5926716f",{"2":{"256":1}}],["591687335918585e",{"2":{"258":1}}],["5913045916933624e",{"2":{"258":1}}],["5913533750184917e",{"2":{"258":1}}],["5913297f",{"2":{"256":1}}],["591397f",{"2":{"256":1}}],["5910316f",{"2":{"256":1}}],["598798028322887e",{"2":{"258":1}}],["5988298f",{"2":{"256":1}}],["5988305f",{"2":{"256":1}}],["5983315626392106e",{"2":{"258":1}}],["59838f",{"2":{"256":1}}],["5983",{"2":{"249":1}}],["598505080",{"2":{"244":2}}],["590150866902443e",{"2":{"258":1}}],["5900963f",{"2":{"256":1}}],["59006",{"2":{"187":1}}],["5905511",{"2":{"232":2}}],["5956",{"2":{"249":1}}],["59543324",{"2":{"235":1}}],["5954s\\ttraining",{"2":{"217":1}}],["59538s\\ttraining",{"2":{"228":1}}],["59",{"2":{"148":1,"228":1}}],["594261",{"2":{"148":1}}],["5979974",{"2":{"145":1}}],["5975106",{"2":{"77":1}}],["5990931993431986e",{"2":{"258":1}}],["5991409648578535e",{"2":{"258":1}}],["5991",{"2":{"249":1}}],["599194",{"2":{"168":1}}],["5999714",{"2":{"126":1}}],["5994072",{"2":{"106":1}}],["5994074",{"2":{"106":1}}],["533782032335001e",{"2":{"258":1}}],["5330s\\ttraining",{"2":{"219":1}}],["534246509897024e",{"2":{"258":1}}],["534332f",{"2":{"256":1}}],["5345716",{"2":{"77":1}}],["5305499f",{"2":{"256":1}}],["530473586395223e",{"2":{"258":1}}],["5304",{"2":{"249":1}}],["539738f",{"2":{"256":1}}],["5355s\\ttraining",{"2":{"217":1}}],["5381063096584814e",{"2":{"258":1}}],["5381063f",{"2":{"256":1}}],["5381101016248151",{"2":{"153":2}}],["5386394131105624e",{"2":{"258":1}}],["5386",{"2":{"249":1}}],["5382",{"2":{"249":1}}],["5388155f",{"2":{"256":1}}],["5388",{"2":{"249":1}}],["53851",{"2":{"187":1}}],["5310851",{"2":{"179":1}}],["53144f",{"2":{"77":1}}],["5368397f",{"2":{"256":1}}],["536743e",{"2":{"152":1}}],["5367613",{"2":{"147":1}}],["5365157",{"2":{"148":1}}],["53",{"2":{"117":1,"196":1,"228":4,"249":1}}],["5378918",{"2":{"77":1}}],["5d",{"2":{"70":1,"208":1,"244":2,"257":1}}],["510208869835473e",{"2":{"258":1}}],["510067697768531e",{"2":{"258":1}}],["510381f",{"2":{"256":1}}],["513376400969248e",{"2":{"258":1}}],["5137206f",{"2":{"256":1}}],["5135363f",{"2":{"256":1}}],["519940153693118e",{"2":{"258":1}}],["519306138440145e",{"2":{"258":1}}],["5193824f",{"2":{"256":1}}],["5198546f",{"2":{"256":1}}],["51968503",{"2":{"232":2}}],["5147155368109755e",{"2":{"258":1}}],["5141768055814625e",{"2":{"258":1}}],["5148395f",{"2":{"256":1}}],["5142531f",{"2":{"256":1}}],["5149345f",{"2":{"256":1}}],["514340e",{"2":{"208":1}}],["5177831116016903e",{"2":{"258":1}}],["5176876f",{"2":{"256":1}}],["5178",{"2":{"249":1}}],["516279314626611e",{"2":{"258":1}}],["5165745f",{"2":{"256":1}}],["5167208",{"2":{"235":1}}],["5164s\\ttraining",{"2":{"217":1}}],["5164032",{"2":{"179":1}}],["51165134",{"2":{"235":1}}],["5115s\\ttraining",{"2":{"219":1}}],["518618352769437e",{"2":{"258":1}}],["518714f",{"2":{"256":1}}],["51878",{"2":{"187":1}}],["51852",{"2":{"217":1}}],["5151074552430909e",{"2":{"258":1}}],["515673021915105e",{"2":{"258":1}}],["5157055f",{"2":{"256":1}}],["5157s\\ttraining",{"2":{"217":1}}],["51524514",{"2":{"179":1}}],["51",{"2":{"69":3,"116":1,"208":2,"217":1,"236":1}}],["512935825921665e",{"2":{"258":1}}],["512425803036386e",{"2":{"258":1}}],["5128704f",{"2":{"256":1}}],["512089f",{"2":{"256":1}}],["5123946f",{"2":{"256":1}}],["512",{"2":{"59":3,"243":1}}],["5057718931371288e",{"2":{"258":1}}],["5054546280606808e",{"2":{"258":1}}],["5058574f",{"2":{"256":1}}],["5051434f",{"2":{"256":1}}],["5053",{"2":{"249":1}}],["5081822506421394e",{"2":{"258":1}}],["5081386f",{"2":{"256":1}}],["50837",{"2":{"187":1}}],["504250457378289e",{"2":{"258":1}}],["5048051573805486e",{"2":{"258":1}}],["5044234f",{"2":{"256":1}}],["5044s\\ttraining",{"2":{"217":1}}],["5047506f",{"2":{"256":1}}],["50120934186983e",{"2":{"258":1}}],["5011439f",{"2":{"256":1}}],["501",{"2":{"244":1}}],["5016s\\ttraining",{"2":{"217":1}}],["5016017",{"2":{"147":1}}],["50745459057368e",{"2":{"258":1}}],["507118520375801e",{"2":{"258":1}}],["507195f",{"2":{"256":1}}],["50752f",{"2":{"256":1}}],["5070s\\ttraining",{"2":{"217":1}}],["507971",{"2":{"168":1}}],["50798f",{"2":{"77":1}}],["5034714f",{"2":{"256":1}}],["5032s\\ttraining",{"2":{"217":1}}],["503186",{"2":{"172":1}}],["5031421",{"2":{"106":1}}],["5063s\\ttraining",{"2":{"217":1}}],["506485",{"2":{"172":1}}],["5090226537188095e",{"2":{"258":1}}],["5096s\\ttraining",{"2":{"217":1}}],["509523",{"2":{"168":1}}],["5000×30×1",{"2":{"249":1}}],["50000",{"2":{"244":100}}],["5000",{"2":{"244":1,"249":5}}],["5009556",{"2":{"235":1}}],["5001",{"2":{"179":1,"244":1}}],["500",{"2":{"107":1,"183":2,"244":1}}],["50",{"2":{"63":2,"69":3,"196":9,"228":112,"236":1,"249":1}}],["502312852219817",{"2":{"53":1}}],["5∗δ",{"2":{"53":1}}],["5∗|y−y^|2if",{"2":{"53":1}}],["557459384335777e",{"2":{"258":1}}],["552214310573785e",{"2":{"258":1}}],["5520544f",{"2":{"256":1}}],["5586695777104e",{"2":{"258":1}}],["5586767f",{"2":{"256":1}}],["550369611522905e",{"2":{"258":1}}],["550772451230695e",{"2":{"258":1}}],["550829691992387e",{"2":{"258":1}}],["5505995f",{"2":{"256":1}}],["5501",{"2":{"244":1}}],["5561503449666485e",{"2":{"258":1}}],["556622556946641e",{"2":{"258":1}}],["55602685978259e",{"2":{"258":1}}],["55602f",{"2":{"256":1}}],["5562014f",{"2":{"256":1}}],["556568f",{"2":{"256":1}}],["55658394",{"2":{"232":1}}],["5564",{"2":{"249":1}}],["5511811",{"2":{"232":2}}],["555338492493487e",{"2":{"258":1}}],["5553797706980106",{"2":{"174":1}}],["55556",{"2":{"217":2}}],["5594571",{"2":{"232":1}}],["5590551",{"2":{"232":2}}],["5595843665394066",{"2":{"174":1}}],["55989707",{"2":{"148":1}}],["553913528497993e",{"2":{"258":1}}],["55399f",{"2":{"256":1}}],["5532",{"2":{"171":1}}],["553631",{"2":{"147":1}}],["5530689",{"2":{"147":1}}],["55476f",{"2":{"77":1}}],["55",{"2":{"53":1,"219":1}}],["5where",{"2":{"53":1}}],["5",{"2":{"19":9,"24":8,"34":4,"44":2,"53":15,"59":11,"63":2,"69":6,"74":1,"77":16,"94":1,"102":1,"106":5,"111":5,"129":1,"130":1,"147":1,"148":2,"168":6,"171":29,"172":1,"173":3,"177":3,"179":6,"180":3,"187":14,"189":3,"193":14,"196":2,"205":1,"206":1,"210":3,"217":5,"218":2,"219":1,"220":40,"221":3,"228":2,"229":3,"232":8,"236":2,"237":3,"239":1,"246":3,"248":4,"249":19,"250":1,"254":4,"255":2,"256":736,"258":699,"259":3}}],["^2",{"2":{"255":2,"256":1}}],["^",{"2":{"19":9,"59":1,"78":1,"106":1,"107":1,"254":2}}],["λβ",{"2":{"17":1}}],["α",{"2":{"17":3,"53":3,"206":2,"250":3}}],["+=",{"2":{"194":2,"216":2,"227":2,"244":1,"249":1}}],["+ϵ∗γ+βwhere",{"2":{"69":1}}],["+ϵ∗γ+βand",{"2":{"19":1}}],["+",{"2":{"16":1,"18":1,"24":3,"32":1,"44":1,"49":2,"52":1,"55":1,"59":4,"62":5,"63":4,"65":6,"67":4,"70":1,"85":1,"135":2,"145":1,"146":1,"158":2,"171":4,"179":3,"183":2,"206":1,"232":1,"242":4,"243":3,"245":1,"248":4,"249":1,"254":13,"255":1,"256":2}}],["ys",{"2":{"245":6}}],["y∈",{"2":{"243":1}}],["ylabel=",{"2":{"232":1,"236":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["y=x2−2x",{"2":{"232":1}}],["y=x−e",{"2":{"19":1,"69":1}}],["yᵢ",{"2":{"107":2}}],["yes",{"2":{"99":1}}],["year",{"2":{"74":2}}],["yet",{"2":{"32":1,"106":1}}],["y3",{"2":{"62":1}}],["y2",{"2":{"62":1,"248":2}}],["y1",{"2":{"62":1,"248":2}}],["yann",{"2":{"53":1}}],["y+ϵ",{"2":{"53":1}}],["yi∈rm",{"2":{"179":1}}],["yi",{"2":{"53":2,"179":1}}],["y~=",{"2":{"53":2}}],["y~",{"2":{"53":4}}],["y^2+y∗max",{"2":{"53":1}}],["y^−y∗log⁡",{"2":{"53":1}}],["y^−y",{"2":{"53":1}}],["y^",{"2":{"53":7}}],["y^+ϵ",{"2":{"53":3}}],["ŷ",{"2":{"53":11,"145":2,"146":2,"147":2,"186":2,"187":3}}],["yoshua",{"2":{"24":2}}],["yourself",{"2":{"203":1}}],["your",{"0":{"120":1},"2":{"32":2,"47":1,"59":2,"69":3,"82":1,"115":2,"117":1,"124":1,"133":1,"135":2,"136":2,"137":3,"143":2,"144":1,"145":2,"157":1,"161":1,"163":2,"188":1}}],["you",{"2":{"2":1,"6":2,"32":1,"45":2,"47":1,"48":1,"52":3,"59":5,"71":3,"74":2,"76":3,"77":1,"78":1,"79":1,"94":1,"105":1,"106":2,"107":1,"115":3,"116":2,"117":2,"118":1,"120":2,"124":1,"127":1,"133":1,"135":8,"136":6,"137":5,"139":2,"143":2,"144":5,"145":4,"146":1,"153":1,"156":1,"157":2,"163":2,"168":1,"171":1,"172":3,"173":1,"175":1,"181":1,"185":1,"188":1,"202":1,"203":1,"217":2,"225":1,"233":1,"243":1,"256":1}}],["yuxin",{"2":{"19":1}}],["y",{"2":{"14":2,"15":2,"19":1,"24":3,"28":5,"49":3,"53":63,"54":3,"55":6,"58":1,"59":10,"62":8,"63":2,"65":9,"67":10,"68":6,"69":4,"77":1,"78":4,"106":11,"107":2,"113":4,"117":2,"135":3,"136":4,"145":7,"146":7,"153":2,"160":5,"179":11,"183":4,"184":5,"185":3,"186":12,"187":5,"192":6,"194":2,"195":4,"206":4,"213":6,"216":2,"217":2,"224":4,"225":2,"227":2,"228":2,"232":5,"236":5,"243":7,"245":1,"248":1,"254":6,"256":3}}],["σ=identity",{"2":{"19":1}}],["σ",{"2":{"13":10,"15":5,"16":3,"18":3,"19":7,"158":3}}],["know",{"2":{"104":1,"116":1,"144":1,"256":1}}],["known",{"0":{"124":1},"2":{"24":2,"52":1,"104":1}}],["knet",{"2":{"98":1}}],["k",{"2":{"63":6,"67":4,"179":1,"214":2,"218":2}}],["kwarg",{"2":{"94":2,"95":1}}],["kwargs",{"2":{"25":24,"37":2,"45":1,"49":4,"59":3,"62":9,"68":2,"95":1,"168":2,"169":2,"195":1,"214":7,"217":2,"218":4,"220":4}}],["kw",{"2":{"59":2}}],["kullback",{"2":{"53":1}}],["kl",{"2":{"53":1}}],["kldivergenceloss",{"2":{"53":5}}],["klambauer",{"2":{"17":1}}],["ki−1",{"2":{"63":1,"65":3}}],["kind",{"0":{"126":1},"2":{"34":1,"46":1,"59":2,"62":1,"116":1,"126":1}}],["kinds",{"2":{"8":1}}],["kiros",{"2":{"19":1}}],["kaiming",{"2":{"19":1,"24":4,"63":2,"67":1,"168":6}}],["keith",{"2":{"252":1}}],["keep",{"2":{"142":1}}],["kept",{"2":{"139":1}}],["kernelabstractions",{"2":{"20":1,"153":7}}],["kernels",{"0":{"153":1},"2":{"16":1,"153":3}}],["kernel",{"2":{"16":1,"20":1,"24":1,"63":5,"153":6}}],["keys",{"2":{"94":1,"111":1,"138":2}}],["keypath=keypath",{"2":{"45":1}}],["keypath",{"2":{"44":7,"94":1,"95":1,"116":4,"117":10,"127":2,"128":4}}],["key",{"0":{"123":1},"2":{"10":2,"119":1}}],["keywords",{"2":{"59":1,"70":1,"169":1}}],["keyword",{"2":{"2":1,"10":1,"32":1,"45":2,"48":1,"49":2,"62":5,"63":2,"64":4,"65":3,"66":2,"67":4,"68":2,"69":4,"70":2,"84":1,"96":1,"97":1,"169":1}}],["hₓ",{"2":{"254":4}}],["h₊",{"2":{"254":4}}],["h12",{"2":{"254":12}}],["h11",{"2":{"254":12}}],["h22",{"2":{"254":12}}],["hmc",{"2":{"249":2}}],["hcat",{"2":{"243":1,"249":1}}],["hn",{"2":{"225":2}}],["hnew",{"2":{"66":6}}],["hnew=activation",{"2":{"66":1}}],["hnew=",{"2":{"66":1}}],["hypernet",{"0":{"225":1,"226":1},"2":{"225":4,"226":1}}],["hypernetwork",{"0":{"222":1},"1":{"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1}}],["hh",{"2":{"66":6,"97":1}}],["h",{"2":{"65":6,"66":3,"70":4,"192":1,"213":1,"254":14}}],["home",{"2":{"137":1}}],["hosts",{"2":{"79":1}}],["hold",{"2":{"77":1,"99":1}}],["hot",{"2":{"55":1}}],["how",{"0":{"71":1,"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"34":1,"53":2,"55":1,"62":2,"66":3,"69":8,"76":1,"77":1,"100":1,"106":1,"107":1,"115":1,"117":2,"125":1,"135":1,"137":1,"140":1,"144":1,"148":1,"153":1,"157":1,"161":2,"171":2,"177":1,"190":1,"204":1,"214":1,"225":1,"247":1,"249":1,"250":1}}],["however",{"2":{"8":1,"11":1,"55":1,"59":1,"62":1,"77":1,"96":1,"98":1,"105":1,"118":1,"123":1,"148":1,"167":1,"173":2,"175":1,"202":1,"204":3,"214":1,"238":1,"254":1,"256":2}}],["hutchinson",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":2,"150":3,"151":2,"152":8}}],["huber",{"2":{"53":1}}],["huberloss",{"2":{"53":3}}],["human",{"2":{"24":2}}],["https",{"2":{"24":1,"66":1,"68":1,"71":1,"74":1,"177":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"254":2,"260":1}}],["heatmap",{"2":{"245":1}}],["head",{"2":{"100":1,"184":1}}],["heavily",{"2":{"58":1}}],["heavy",{"2":{"6":1}}],["height",{"2":{"63":1}}],["helps",{"2":{"214":1}}],["help",{"2":{"42":1,"43":1,"44":1,"45":2,"53":2,"57":1,"59":1,"62":6,"63":2,"65":3,"66":2,"68":1,"69":4,"70":1,"115":1,"157":1}}],["helpers",{"0":{"26":1},"1":{"27":1,"28":1,"29":1},"2":{"54":1}}],["helper",{"0":{"20":1,"38":1,"68":1,"194":1},"2":{"52":1,"57":1,"250":1}}],["here",{"2":{"42":1,"45":1,"52":2,"54":1,"59":3,"77":1,"98":1,"104":1,"110":1,"120":1,"136":1,"145":3,"146":1,"147":1,"160":1,"161":1,"171":3,"177":2,"208":1,"214":1,"241":1,"248":1,"249":2}}],["hence",{"2":{"32":1,"47":1,"140":1,"142":1,"150":1,"174":1,"175":1,"196":1,"208":1,"214":1,"256":1}}],["hessian",{"2":{"24":1}}],["he",{"2":{"19":1,"24":2}}],["historical",{"2":{"214":1}}],["historically",{"2":{"42":1}}],["hidden",{"2":{"66":43,"184":6,"185":3,"241":11,"244":2,"249":1}}],["hierarchically",{"2":{"59":1}}],["hinge",{"2":{"53":2}}],["hingeloss",{"2":{"53":2}}],["hinton",{"2":{"19":1}}],["highlight",{"2":{"123":1}}],["highly",{"2":{"53":1}}],["higher",{"2":{"70":1,"147":1,"155":1}}],["highest",{"2":{"16":1,"104":1,"155":1,"250":1}}],["high",{"2":{"62":1,"153":1,"171":1,"177":1}}],["hi",{"2":{"24":2}}],["hit",{"2":{"8":1}}],["hamiltonian",{"2":{"249":2}}],["hat",{"2":{"209":2}}],["had",{"2":{"175":1}}],["hadsell",{"2":{"53":1}}],["hand",{"2":{"171":1}}],["handles",{"2":{"100":1,"185":1}}],["handle",{"2":{"66":1,"130":1,"153":1,"214":1,"249":1}}],["handling",{"2":{"54":1,"57":1,"133":1,"155":1,"165":1}}],["happened",{"2":{"155":1}}],["happening",{"2":{"146":1,"160":1}}],["happens",{"2":{"116":1,"144":1,"160":1,"165":1}}],["haven",{"2":{"32":1}}],["have",{"2":{"24":1,"28":1,"42":2,"63":1,"66":1,"68":1,"81":1,"87":1,"91":2,"94":3,"95":1,"96":2,"97":1,"100":2,"104":3,"105":1,"116":1,"117":2,"118":2,"124":1,"135":1,"136":1,"137":1,"139":2,"143":1,"144":2,"145":1,"146":1,"152":1,"155":1,"172":1,"173":1,"174":1,"175":3,"184":3,"208":1,"226":1,"249":1,"250":2,"254":1}}],["having",{"2":{"6":1,"46":1,"59":1}}],["harder",{"2":{"100":1,"115":1}}],["hardware",{"2":{"99":1}}],["hard",{"2":{"11":1,"53":1,"117":1,"135":1,"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["has",{"2":{"8":3,"32":2,"43":1,"59":1,"63":1,"65":3,"70":1,"84":2,"87":6,"91":1,"94":7,"95":4,"96":1,"97":1,"98":1,"100":1,"115":1,"116":4,"119":1,"122":1,"130":1,"136":1,"148":1,"170":1,"171":2,"175":1,"249":1,"252":1,"256":1}}],["2f",{"2":{"195":2,"228":4}}],["2fs",{"2":{"195":1}}],["2`",{"2":{"136":1}}],["2789384409150318e",{"2":{"258":1}}],["27891335f",{"2":{"256":1}}],["27312967677122e",{"2":{"258":1}}],["27373654",{"2":{"126":1}}],["2727593133643193e",{"2":{"258":1}}],["27297f",{"2":{"256":1}}],["271933652062365e",{"2":{"258":1}}],["2718573f",{"2":{"256":1}}],["271291f",{"2":{"256":1}}],["271452574910546e",{"2":{"258":1}}],["2714",{"2":{"249":1}}],["2717",{"2":{"249":1}}],["27471745",{"2":{"235":1}}],["2758766310549294e",{"2":{"258":1}}],["27501",{"2":{"244":1}}],["2755905",{"2":{"232":2}}],["27545732",{"2":{"106":1}}],["27",{"2":{"217":3,"228":2,"249":1}}],["276422f",{"2":{"256":1}}],["27641",{"2":{"187":1}}],["276",{"2":{"208":1}}],["27659",{"2":{"187":1}}],["2709",{"2":{"249":1}}],["27001",{"2":{"244":1}}],["2701178",{"2":{"232":1}}],["27035674",{"2":{"179":1}}],["27051234",{"2":{"179":1}}],["27993",{"2":{"172":1}}],["279856",{"2":{"168":1}}],["2797048270773437",{"2":{"153":2}}],["2791573282471997",{"2":{"153":2}}],["2774433209671578e",{"2":{"258":1}}],["2774684f",{"2":{"256":1}}],["2774315",{"2":{"145":1}}],["27774",{"2":{"187":1}}],["2771789",{"2":{"179":1}}],["2771862617010155",{"2":{"153":2}}],["27th",{"2":{"24":1}}],["2×4",{"2":{"173":1}}],["2×5",{"2":{"168":6}}],["2×32",{"2":{"106":2}}],["2×2",{"2":{"53":2,"171":1}}],["2949409f",{"2":{"256":1}}],["2951023424746916e",{"2":{"258":1}}],["2950",{"2":{"249":1}}],["29501",{"2":{"244":1}}],["29561827",{"2":{"179":1}}],["2931565175423867e",{"2":{"258":1}}],["29315922",{"2":{"235":1}}],["2934191f",{"2":{"256":1}}],["2932029",{"2":{"235":1}}],["293211",{"2":{"106":1}}],["2938747",{"2":{"235":1}}],["2923174f",{"2":{"256":1}}],["292066480289389e",{"2":{"258":1}}],["29206026",{"2":{"235":1}}],["292002",{"2":{"168":1}}],["2922556",{"2":{"232":1}}],["29",{"2":{"196":1,"228":2,"249":1}}],["29s",{"2":{"196":1}}],["29001",{"2":{"244":1}}],["29006",{"2":{"187":1}}],["2904677",{"2":{"129":1}}],["298581043181066e",{"2":{"258":1}}],["29855f",{"2":{"256":1}}],["29828635",{"2":{"179":1}}],["298787",{"2":{"148":1}}],["2996369736091477e",{"2":{"258":1}}],["299464f",{"2":{"256":1}}],["29944375",{"2":{"179":1}}],["2991",{"2":{"249":1}}],["2992126",{"2":{"232":2}}],["29978",{"2":{"187":1}}],["2990853",{"2":{"179":1}}],["29955",{"2":{"168":1}}],["29630",{"2":{"217":2,"219":1}}],["296372",{"2":{"168":1}}],["296496",{"2":{"171":1}}],["297959481822617",{"2":{"153":2}}],["2913357",{"2":{"106":1}}],["266779763609969e",{"2":{"258":1}}],["266863f",{"2":{"256":1}}],["26657984",{"2":{"106":1}}],["263860589954412e",{"2":{"258":1}}],["263723400316271e",{"2":{"258":1}}],["263777f",{"2":{"256":1}}],["263985f",{"2":{"256":1}}],["26311",{"2":{"187":1}}],["26311737",{"2":{"106":1}}],["2695",{"2":{"249":1}}],["260147643001434e",{"2":{"258":1}}],["26001",{"2":{"244":1}}],["260598",{"2":{"235":1}}],["2606924",{"2":{"148":1}}],["26842650413899e",{"2":{"258":1}}],["268265f",{"2":{"256":1}}],["2681656",{"2":{"232":1}}],["268675e",{"2":{"208":1}}],["2689521",{"2":{"106":2}}],["268941",{"2":{"53":2}}],["26",{"2":{"208":2,"228":2,"249":3}}],["264070190253982e",{"2":{"258":1}}],["264597377353835e",{"2":{"258":1}}],["264759f",{"2":{"256":1}}],["2647",{"2":{"249":1}}],["26477236",{"2":{"106":2}}],["2648",{"2":{"249":1}}],["26425",{"2":{"187":1}}],["26136",{"2":{"187":1}}],["2615936",{"2":{"147":1}}],["261273",{"2":{"131":2}}],["2672",{"2":{"249":1}}],["26793814",{"2":{"235":1}}],["26771653",{"2":{"232":2}}],["26718",{"2":{"187":1}}],["267644",{"2":{"147":1,"148":1}}],["26700416",{"2":{"106":1}}],["26700422",{"2":{"106":1}}],["26501",{"2":{"244":1}}],["265788",{"2":{"168":1}}],["265372",{"2":{"168":1}}],["265",{"2":{"78":1}}],["2278924605019604e",{"2":{"258":1}}],["2276903f",{"2":{"256":1}}],["2271",{"2":{"249":1}}],["227513",{"2":{"168":1}}],["228003449814601e",{"2":{"258":1}}],["22807482",{"2":{"232":1}}],["228266f",{"2":{"256":1}}],["2287",{"2":{"249":1}}],["22846f",{"2":{"77":1}}],["2205666153078866e",{"2":{"258":1}}],["2208332f",{"2":{"256":1}}],["22001",{"2":{"244":1}}],["220",{"2":{"220":6}}],["22010",{"2":{"187":1}}],["226490694214039e",{"2":{"258":1}}],["2261",{"2":{"249":1}}],["226",{"2":{"208":1}}],["2263952f",{"2":{"256":1}}],["22635892",{"2":{"179":1}}],["226381",{"2":{"77":1}}],["22501",{"2":{"244":1}}],["225",{"2":{"184":1,"207":2,"214":1,"220":18,"241":1}}],["22543387",{"2":{"77":1}}],["2290224145974982",{"2":{"153":2}}],["22222",{"2":{"217":2,"219":1}}],["2227837233928576",{"2":{"153":2}}],["222528",{"2":{"148":1}}],["2221465",{"2":{"148":1}}],["2228831",{"2":{"106":1}}],["22",{"2":{"145":1,"187":14,"196":3,"217":1,"228":4,"254":14}}],["223385178875912e",{"2":{"258":1}}],["2233576638856505e",{"2":{"258":1}}],["22388883044749e",{"2":{"258":1}}],["2235525f",{"2":{"256":1}}],["2234122f",{"2":{"256":1}}],["2236526f",{"2":{"256":1}}],["22363997",{"2":{"106":1}}],["2236399",{"2":{"106":1}}],["2237958",{"2":{"77":1}}],["2245800985857894e",{"2":{"258":1}}],["22459084",{"2":{"77":1}}],["2247449737817038e",{"2":{"258":1}}],["2247887f",{"2":{"256":1}}],["224269f",{"2":{"256":1}}],["2243",{"2":{"249":1}}],["22406094",{"2":{"106":1}}],["255059f",{"2":{"256":1}}],["25501",{"2":{"244":1}}],["2557",{"2":{"249":1}}],["2555",{"2":{"249":1}}],["25587f",{"2":{"77":1}}],["2581186",{"2":{"232":1}}],["2586653",{"2":{"106":1}}],["2594698567493444e",{"2":{"258":1}}],["259477f",{"2":{"256":1}}],["25926",{"2":{"217":3,"219":1}}],["25963",{"2":{"187":1}}],["251941f",{"2":{"256":1}}],["251",{"2":{"208":1}}],["25s",{"2":{"196":1}}],["25001",{"2":{"244":1}}],["2501",{"2":{"244":1}}],["250",{"2":{"236":2,"250":1,"255":1}}],["25087",{"2":{"187":1}}],["25055695",{"2":{"177":1,"178":1}}],["25",{"2":{"171":1,"187":15,"208":2,"228":32,"249":1}}],["25419125201117e",{"2":{"258":1}}],["254935059427534e",{"2":{"258":1}}],["254694619052582e",{"2":{"258":1}}],["254646f",{"2":{"256":1}}],["254024f",{"2":{"256":1}}],["2547im",{"2":{"168":1}}],["2545935",{"2":{"77":1}}],["252298f",{"2":{"256":1}}],["252304767366444e",{"2":{"258":1}}],["252327e",{"2":{"208":1}}],["2523673",{"2":{"148":1}}],["25278285",{"2":{"179":1}}],["2525357728685884",{"2":{"153":2}}],["252985",{"2":{"148":1}}],["252925",{"2":{"148":1}}],["25378025",{"2":{"106":2}}],["25389904",{"2":{"106":1}}],["25385493",{"2":{"106":1}}],["2571059060761017e",{"2":{"258":1}}],["25752",{"2":{"187":1}}],["257",{"2":{"77":1}}],["25647524364796e",{"2":{"258":1}}],["256927f",{"2":{"256":1}}],["2562022f",{"2":{"256":1}}],["25618f",{"2":{"256":1}}],["256829e",{"2":{"208":1}}],["25662464",{"2":{"126":1}}],["256",{"2":{"34":1,"59":1,"77":6,"193":3,"226":2}}],["243830175005229e",{"2":{"258":1}}],["2461244419524062e",{"2":{"258":1}}],["2464001",{"2":{"106":1}}],["245906754784296e",{"2":{"258":1}}],["24564f",{"2":{"256":1}}],["24501",{"2":{"244":1}}],["249567112553275e",{"2":{"258":1}}],["2492364f",{"2":{"256":1}}],["249394f",{"2":{"256":1}}],["2440945",{"2":{"232":2}}],["244728",{"2":{"53":10}}],["24735+1",{"2":{"168":1}}],["24",{"2":{"111":2,"180":3,"187":14,"189":3,"197":2,"210":3,"221":3,"228":2,"229":3,"237":3,"246":3,"249":2}}],["241937",{"2":{"232":1}}],["2415948",{"2":{"148":1}}],["24150778",{"2":{"107":1}}],["24169",{"2":{"106":1}}],["240796210469526e",{"2":{"258":1}}],["240796f",{"2":{"256":1}}],["24001",{"2":{"244":1}}],["24099025",{"2":{"77":1}}],["240",{"2":{"69":3}}],["234997808674383e",{"2":{"258":1}}],["2331194315476374e",{"2":{"258":1}}],["233036f",{"2":{"256":1}}],["23325463",{"2":{"126":1}}],["238586386123643e",{"2":{"258":1}}],["238324f",{"2":{"256":1}}],["23835",{"2":{"187":1}}],["238063f",{"2":{"256":1}}],["23234403",{"2":{"235":1}}],["23242",{"2":{"147":1}}],["235979f",{"2":{"256":1}}],["2352f",{"2":{"256":1}}],["23501",{"2":{"244":1}}],["2354317",{"2":{"232":1}}],["2351742f",{"2":{"106":3}}],["2376997434390403e",{"2":{"258":1}}],["237496739926018e",{"2":{"258":1}}],["23747",{"2":{"187":1}}],["237191f",{"2":{"256":1}}],["23710957",{"2":{"137":1}}],["2392274956113632e",{"2":{"258":1}}],["23921265",{"2":{"147":1}}],["2396877248705143e",{"2":{"258":1}}],["2398599f",{"2":{"256":1}}],["23992883",{"2":{"179":1}}],["239756",{"2":{"172":1}}],["23057879547136e",{"2":{"258":1}}],["2307387f",{"2":{"256":1}}],["23001",{"2":{"244":1}}],["23095",{"2":{"171":1}}],["230954",{"2":{"171":2}}],["23021114",{"2":{"106":1}}],["231723",{"2":{"168":1}}],["23162955",{"2":{"106":1}}],["236302412504309e",{"2":{"258":1}}],["23604f",{"2":{"256":1}}],["2366",{"2":{"249":1}}],["23622048",{"2":{"232":2}}],["23626f",{"2":{"77":1}}],["2361472",{"2":{"126":1}}],["23",{"2":{"62":2,"180":1,"187":14,"189":1,"197":1,"210":1,"221":1,"228":4,"229":1,"237":1,"246":1,"249":2,"251":1,"260":1}}],["207059602444299e",{"2":{"258":1}}],["207062f",{"2":{"256":1}}],["207358072783739e",{"2":{"258":1}}],["207327f",{"2":{"256":1}}],["2072318f",{"2":{"256":1}}],["2050",{"2":{"249":1}}],["20501",{"2":{"244":1}}],["2058456",{"2":{"235":1}}],["20579764",{"2":{"235":1}}],["20548",{"2":{"168":1}}],["20472442",{"2":{"232":2}}],["2042089",{"2":{"106":1}}],["203133050548713e",{"2":{"258":1}}],["2030108141596797e",{"2":{"258":1}}],["203036",{"2":{"106":1}}],["203257f",{"2":{"256":1}}],["2034096f",{"2":{"256":1}}],["20382",{"2":{"187":1}}],["20882",{"2":{"187":1}}],["20881107",{"2":{"126":1}}],["2029622f",{"2":{"256":1}}],["2021",{"2":{"252":1}}],["2024",{"2":{"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["20277858",{"2":{"179":1}}],["2023",{"2":{"74":2}}],["206902051274912e",{"2":{"258":1}}],["206943215129883e",{"2":{"258":1}}],["206741f",{"2":{"256":1}}],["2067",{"2":{"249":1}}],["20661",{"2":{"187":1}}],["20668754",{"2":{"147":1}}],["20609",{"2":{"187":1}}],["206476",{"2":{"168":1}}],["2008955f",{"2":{"256":1}}],["2000563f",{"2":{"256":1}}],["20001",{"2":{"244":1}}],["20001543",{"2":{"179":1}}],["2000",{"2":{"192":1}}],["20098",{"2":{"187":1}}],["2001217574943265e",{"2":{"258":1}}],["2001",{"2":{"179":1,"244":1}}],["200",{"2":{"107":1,"220":18}}],["20053944",{"2":{"106":1}}],["2006",{"2":{"53":2}}],["20",{"2":{"62":1,"69":3,"77":1,"112":1,"147":1,"171":2,"179":3,"187":14,"215":5,"217":1,"220":24,"228":3,"249":7}}],["201",{"2":{"208":1,"220":18,"236":1}}],["20114",{"2":{"171":1}}],["201145",{"2":{"171":2}}],["201067676576273e",{"2":{"258":1}}],["2010",{"2":{"24":3}}],["2016",{"2":{"19":2,"53":2,"69":1}}],["2018106096849766e",{"2":{"258":1}}],["2018543f",{"2":{"256":1}}],["20188306",{"2":{"106":1}}],["2018",{"2":{"19":1}}],["2015",{"2":{"19":1,"24":2}}],["2014",{"2":{"17":1,"24":1}}],["2017",{"2":{"17":1,"53":2}}],["214317030765395e",{"2":{"258":1}}],["2146228f",{"2":{"256":1}}],["2120659713766265e",{"2":{"258":1}}],["2125984",{"2":{"232":2}}],["21252f",{"2":{"77":1}}],["2176147652786665e",{"2":{"258":1}}],["2171858f",{"2":{"256":1}}],["2177103f",{"2":{"256":1}}],["217015242889768e",{"2":{"258":1}}],["2170",{"2":{"249":1}}],["215454970903387e",{"2":{"258":1}}],["215379f",{"2":{"256":1}}],["21501",{"2":{"244":1}}],["21591",{"2":{"187":1}}],["211211167982715e",{"2":{"258":1}}],["211492628945743e",{"2":{"258":1}}],["2114489f",{"2":{"256":1}}],["2111507f",{"2":{"256":1}}],["2111273",{"2":{"148":1}}],["2119045f",{"2":{"256":1}}],["211",{"2":{"220":6}}],["210932752946492e",{"2":{"258":1}}],["2108494f",{"2":{"256":1}}],["21001",{"2":{"244":1}}],["210",{"2":{"220":18}}],["21820",{"2":{"187":1}}],["21856f",{"2":{"77":1}}],["2130",{"2":{"249":1}}],["21342",{"2":{"187":1}}],["21375",{"2":{"187":1}}],["216923232501825e",{"2":{"258":1}}],["2169828f",{"2":{"256":1}}],["21690917",{"2":{"179":1}}],["21672015",{"2":{"126":1}}],["219263176270675e",{"2":{"258":1}}],["2192146f",{"2":{"256":1}}],["2192001",{"2":{"148":1}}],["2191",{"2":{"249":1}}],["21940619",{"2":{"232":1}}],["21984",{"2":{"187":1}}],["2197981041108443",{"2":{"171":1}}],["21950458",{"2":{"106":1}}],["21",{"2":{"48":1,"69":1,"187":14,"217":1,"228":5,"249":4}}],["2=dense",{"2":{"44":1}}],["2877",{"2":{"249":1}}],["28725442",{"2":{"106":1}}],["2858675911986464e",{"2":{"258":1}}],["2852",{"2":{"249":1}}],["28501",{"2":{"244":1}}],["2859s\\ttraining",{"2":{"217":1}}],["280750462220311e",{"2":{"258":1}}],["2805775f",{"2":{"256":1}}],["28001",{"2":{"244":1}}],["28042415",{"2":{"106":1}}],["2899156",{"2":{"235":1}}],["28946856",{"2":{"235":1}}],["2896426",{"2":{"232":1}}],["2896142",{"2":{"147":1}}],["281252",{"2":{"232":1}}],["281053",{"2":{"168":1}}],["2839582608552799e",{"2":{"258":1}}],["2833",{"2":{"249":1}}],["2835",{"2":{"249":1}}],["28355134",{"2":{"106":1}}],["28368",{"2":{"187":1}}],["28308",{"2":{"187":1}}],["286035f",{"2":{"256":1}}],["28617",{"2":{"187":1}}],["2869913410456831",{"2":{"153":2}}],["2864522",{"2":{"148":1}}],["28682f",{"2":{"77":1}}],["284408233119904e",{"2":{"258":1}}],["284469f",{"2":{"256":1}}],["2849667",{"2":{"148":1}}],["28425345f",{"2":{"256":1}}],["2842",{"2":{"147":1}}],["28289196",{"2":{"77":1}}],["28",{"2":{"34":4,"180":1,"189":1,"193":2,"195":2,"197":1,"210":1,"217":1,"220":2,"221":1,"224":4,"228":2,"229":1,"237":1,"246":1,"249":1,"251":1,"260":1}}],["2nd",{"0":{"29":1},"2":{"102":1,"111":1,"240":2,"242":1}}],["2d",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"24":2,"63":3,"64":2,"65":3,"195":2,"238":1}}],["2",{"2":{"5":5,"24":5,"28":2,"32":4,"34":4,"43":5,"44":4,"46":8,"48":1,"49":1,"53":22,"59":13,"62":30,"63":9,"65":15,"66":11,"67":2,"68":15,"69":6,"70":3,"77":49,"78":5,"102":7,"106":32,"107":4,"111":1,"112":1,"113":2,"116":21,"117":41,"126":3,"127":2,"128":2,"129":4,"131":2,"132":1,"135":3,"136":12,"137":2,"139":6,"140":6,"145":7,"146":4,"147":13,"148":13,"152":1,"153":23,"155":5,"168":8,"171":20,"172":1,"173":4,"174":3,"176":1,"177":1,"180":4,"183":5,"184":1,"185":1,"187":15,"188":1,"189":4,"192":1,"193":18,"196":2,"197":2,"206":6,"208":7,"209":5,"210":4,"213":1,"214":1,"215":1,"217":5,"219":1,"220":67,"221":4,"226":1,"228":4,"229":5,"232":14,"233":1,"235":6,"236":3,"237":4,"242":4,"243":10,"245":4,"246":4,"248":2,"249":34,"254":50,"255":8,"256":100,"258":98}}],["v=v",{"2":{"242":1}}],["vtav",{"2":{"149":1}}],["vvt",{"2":{"149":1}}],["v∈rd",{"2":{"149":1}}],["vs",{"2":{"133":1,"208":1}}],["v0",{"2":{"74":1,"94":1,"130":1,"218":1}}],["voila",{"2":{"116":1}}],["vocabulary",{"2":{"67":2}}],["volterra",{"2":{"206":2}}],["vol",{"2":{"53":1}}],["volumetric",{"2":{"53":1}}],["vcat",{"2":{"66":1,"183":1,"254":1}}],["v",{"2":{"27":2,"53":1,"54":3,"69":1,"87":1,"150":5,"151":5,"152":10,"176":4,"177":2,"178":1,"241":4,"242":10,"254":21}}],["vjp",{"0":{"27":1},"2":{"27":1,"150":7,"152":16,"175":3,"178":4,"236":3}}],["visualization",{"0":{"250":1}}],["visualizing",{"0":{"245":1,"259":1}}],["visualize",{"2":{"232":1,"250":1,"259":1}}],["vision",{"2":{"19":1,"24":2,"53":4}}],["virtual",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["vi",{"2":{"149":1}}],["vitj",{"2":{"149":1}}],["viewaxis",{"2":{"220":68}}],["views",{"2":{"214":2,"242":1,"254":1}}],["view",{"2":{"68":4,"173":1,"249":1,"250":1}}],["victor",{"2":{"19":1,"69":1}}],["via",{"2":{"3":1,"5":2,"6":1,"7":1,"18":1,"24":1,"35":1,"52":2,"57":2,"79":2,"94":1,"95":1,"97":1,"100":2,"106":1,"120":1,"123":1,"153":1,"174":1,"176":4,"177":1,"249":1}}],["vanilla",{"2":{"106":1}}],["various",{"2":{"79":1,"100":1,"101":1,"204":1}}],["variants",{"2":{"158":1}}],["variance",{"2":{"17":2,"19":8,"69":6,"249":1}}],["variationalhiddendropout",{"2":{"64":5}}],["variable",{"2":{"59":2,"94":1,"249":1}}],["variables",{"2":{"52":2,"59":1,"77":1,"100":2,"241":1,"249":1,"254":1}}],["var",{"2":{"19":5,"59":1,"69":7,"116":2,"117":2,"126":1,"129":1,"184":1,"207":2,"214":1,"220":24,"241":1,"244":4}}],["validate",{"2":{"187":1}}],["validated",{"2":{"1":1}}],["validation",{"2":{"183":1,"187":101}}],["valid",{"2":{"69":1,"135":1,"164":1}}],["val",{"2":{"10":2,"17":4,"19":2,"34":1,"45":3,"53":4,"54":5,"55":4,"62":7,"64":1,"116":1,"117":2,"126":2,"129":2,"183":4,"184":1,"185":1,"187":2,"207":1,"220":13,"243":8,"245":3}}],["valued",{"2":{"176":1}}],["value>",{"2":{"161":3}}],["value",{"2":{"3":1,"8":1,"10":2,"17":1,"19":4,"39":2,"45":1,"50":1,"53":6,"54":1,"55":3,"59":6,"62":1,"66":5,"69":6,"117":1,"146":2,"179":16,"183":1,"241":1,"244":4,"249":1,"250":4}}],["valuestorage",{"2":{"220":3}}],["values",{"2":{"3":2,"37":2,"52":4,"53":1,"55":1,"56":3,"70":1,"235":1}}],["vec",{"2":{"152":2,"184":1,"185":1,"214":4,"225":1,"243":1,"245":1}}],["vectors",{"2":{"66":1,"67":2,"69":3,"149":1,"174":6,"247":1}}],["vectorization",{"0":{"167":1},"2":{"20":1,"159":1,"167":1}}],["vectorize",{"2":{"13":1}}],["vector",{"0":{"150":1,"151":1,"177":1,"178":1},"2":{"15":1,"18":1,"24":2,"27":9,"46":2,"48":1,"53":3,"59":2,"66":16,"67":5,"100":1,"112":2,"144":2,"149":4,"150":1,"151":1,"153":3,"171":4,"174":2,"175":5,"176":1,"177":3,"178":1,"179":1,"188":1,"207":1,"214":3,"217":1,"249":4,"250":1,"254":1}}],["vendor",{"2":{"20":1}}],["vedaldi",{"2":{"19":1,"69":1}}],["verbatim",{"2":{"59":1}}],["verified",{"2":{"152":1}}],["verification",{"2":{"8":1}}],["verify",{"2":{"55":1,"145":1,"146":1,"147":1,"148":1,"152":1}}],["versioninfo",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"229":3,"237":3,"246":3,"251":3,"260":3}}],["versioning",{"2":{"145":2}}],["versions",{"2":{"94":1,"159":1,"167":2}}],["version",{"2":{"43":1,"52":2,"55":1,"59":1,"71":3,"74":1,"76":1,"87":1,"110":1,"135":1,"150":1,"180":1,"189":1,"197":1,"203":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["very",{"2":{"7":1,"67":1,"77":1,"100":1,"106":2,"115":1,"116":1,"117":2,"139":1,"140":1,"170":1,"171":1,"254":1,"258":1}}],["v1",{"0":{"82":1},"1":{"83":1,"84":1,"85":1,"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1},"2":{"7":1,"63":2,"76":4,"82":2,"85":1,"130":1}}],["||",{"2":{"78":1,"107":1,"179":1,"236":1,"244":1}}],["|>",{"2":{"77":2,"78":3,"106":4,"107":2,"131":2,"132":3,"153":2,"160":2,"179":1,"187":3,"207":1,"208":2,"209":1,"215":2,"217":1,"225":2,"228":3,"235":1,"236":1,"244":3,"256":1}}],["|p|−di×",{"2":{"63":1,"65":3}}],["|y^−y|",{"2":{"53":1}}],["|y−y^|−0",{"2":{"53":1}}],["|y−y^|≤δδ∗",{"2":{"53":1}}],["|",{"2":{"3":4,"92":1,"116":5,"117":13,"245":1}}],["x~",{"2":{"250":1}}],["x~|θ",{"2":{"250":1}}],["x~|x",{"2":{"250":1}}],["x∈",{"2":{"243":1}}],["xyt",{"2":{"242":17,"243":18,"244":10}}],["xi∈rn",{"2":{"179":1}}],["xi",{"2":{"179":2}}],["xᵢ",{"2":{"107":4}}],["xdev",{"2":{"106":5,"107":3}}],["x3",{"2":{"62":1}}],["x3c",{"2":{"3":3,"4":2,"7":2,"16":1,"18":1,"37":3,"63":4,"66":2,"111":2,"113":2,"135":1,"136":1,"140":1,"161":6,"176":1,"184":1,"208":1,"214":2,"218":2,"220":2,"241":1,"244":2,"254":2}}],["x2s",{"2":{"248":8}}],["x2",{"2":{"62":3,"248":2,"250":18}}],["x26",{"2":{"46":6,"74":1,"127":2,"128":4,"180":4,"189":4,"197":4,"208":2,"210":4,"221":4,"229":4,"237":4,"244":4,"246":4,"251":4,"254":4,"260":4}}],["x1s",{"2":{"248":8}}],["x1",{"2":{"62":3,"248":2,"250":18}}],["xlogy",{"2":{"54":1}}],["xlogx",{"2":{"54":1}}],["xlabel=",{"2":{"232":1,"236":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["xladevice",{"2":{"2":2}}],["xla",{"0":{"73":1},"2":{"2":1,"73":3,"106":5,"107":1}}],["x=",{"2":{"49":1}}],["xoshiro",{"2":{"24":3,"46":1,"59":5,"78":1,"116":1,"126":2,"153":1,"155":1,"174":3,"187":1,"228":1}}],["xavier",{"2":{"24":4}}],["xs",{"2":{"20":2,"245":6,"248":1,"249":3}}],["xt0s",{"2":{"248":5}}],["xt1s",{"2":{"248":5}}],["xt",{"2":{"19":2,"113":1,"114":1}}],["x86",{"2":{"18":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["x",{"2":{"3":12,"5":12,"7":3,"8":12,"11":2,"13":10,"14":2,"15":10,"16":3,"17":8,"18":3,"19":26,"20":1,"24":7,"27":6,"28":10,"32":2,"34":2,"49":5,"53":2,"54":18,"55":16,"58":2,"59":31,"62":24,"63":23,"64":9,"65":27,"66":20,"67":15,"68":30,"69":13,"70":14,"77":4,"78":12,"106":14,"107":3,"109":3,"111":9,"112":2,"113":9,"116":4,"117":10,"121":3,"126":3,"129":2,"131":4,"132":9,"135":6,"136":5,"137":7,"139":6,"140":10,"144":6,"145":10,"146":12,"148":8,"150":2,"151":2,"152":9,"153":3,"155":3,"158":3,"160":5,"171":7,"172":2,"173":4,"176":7,"177":5,"178":1,"179":11,"183":6,"184":8,"185":8,"186":2,"187":4,"192":6,"194":2,"195":4,"206":4,"213":6,"214":10,"216":2,"217":2,"218":2,"220":13,"224":4,"225":2,"227":2,"228":2,"232":9,"236":8,"243":7,"245":1,"248":1,"249":5,"250":4,"254":6,"258":2}}],["3+0",{"2":{"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["3732278432340951e",{"2":{"258":1}}],["3734039f",{"2":{"256":1}}],["377044970092395e",{"2":{"258":1}}],["377014f",{"2":{"256":1}}],["375104870183805e",{"2":{"258":1}}],["3751904f",{"2":{"256":1}}],["37501",{"2":{"244":1}}],["37152559101506e",{"2":{"258":1}}],["371465f",{"2":{"256":1}}],["3719285",{"2":{"148":1}}],["372211719502685e",{"2":{"258":1}}],["3727",{"2":{"249":1}}],["37275374",{"2":{"232":1}}],["3729234",{"2":{"147":1}}],["37049280635499e",{"2":{"258":1}}],["3705473f",{"2":{"256":1}}],["3705",{"2":{"249":1}}],["37001",{"2":{"244":1}}],["3700787",{"2":{"232":2}}],["37037",{"2":{"217":2}}],["37078",{"2":{"187":1}}],["3707023",{"2":{"145":1}}],["37",{"2":{"217":5,"219":1,"228":2}}],["379788887468307e",{"2":{"258":1}}],["3797056f",{"2":{"256":1}}],["379848302981961e",{"2":{"258":1}}],["379944f",{"2":{"256":1}}],["379412e",{"2":{"208":1}}],["37969",{"2":{"187":1}}],["376929340349078e",{"2":{"258":1}}],["376904f",{"2":{"256":1}}],["376274136426115e",{"2":{"258":1}}],["3762087f",{"2":{"256":1}}],["376",{"2":{"208":1}}],["376383",{"2":{"148":1}}],["3742405e",{"2":{"179":1}}],["3rd",{"0":{"203":1},"2":{"147":1}}],["348845453668481e",{"2":{"258":1}}],["34855f",{"2":{"77":1}}],["3460542365913066e",{"2":{"258":1}}],["347116f",{"2":{"256":1}}],["34717593",{"2":{"179":1}}],["3459997f",{"2":{"256":1}}],["34501",{"2":{"244":1}}],["345025",{"2":{"148":1}}],["340545978743833e",{"2":{"258":1}}],["3408347863458833e",{"2":{"258":1}}],["3408023f",{"2":{"256":1}}],["3401",{"2":{"249":1}}],["34001",{"2":{"244":1}}],["3402s\\ttraining",{"2":{"217":1}}],["34",{"2":{"228":3,"249":1}}],["34446",{"2":{"187":1}}],["34254",{"2":{"171":1}}],["34253937",{"2":{"171":1}}],["342539",{"2":{"171":2}}],["343188178223385e",{"2":{"258":1}}],["3434386f",{"2":{"256":1}}],["343951f",{"2":{"256":1}}],["34307",{"2":{"187":1}}],["34351700231383653",{"2":{"174":1}}],["3436353225087703",{"2":{"153":2}}],["34383082",{"2":{"126":1}}],["3410565",{"2":{"136":1}}],["3490",{"2":{"249":1}}],["349",{"2":{"131":2}}],["3616014914551076e",{"2":{"258":1}}],["361536f",{"2":{"256":1}}],["36182",{"2":{"187":1}}],["363366887590736e",{"2":{"258":1}}],["3632055f",{"2":{"256":1}}],["3637495",{"2":{"232":1}}],["3675303f",{"2":{"256":1}}],["3669",{"2":{"249":1}}],["360023810559931e",{"2":{"258":1}}],["360017f",{"2":{"256":1}}],["36001",{"2":{"244":1}}],["360640014055303e",{"2":{"258":1}}],["36064f",{"2":{"256":1}}],["3602331",{"2":{"147":1}}],["3645461",{"2":{"232":1}}],["36428708",{"2":{"145":1}}],["36",{"2":{"217":1,"228":2}}],["3685295932451826e",{"2":{"258":1}}],["36856",{"2":{"187":1}}],["3686637f",{"2":{"256":1}}],["36815",{"2":{"187":1}}],["36829436",{"2":{"179":1}}],["3699043885450762e",{"2":{"258":1}}],["369959f",{"2":{"256":1}}],["3695515683559355e",{"2":{"258":1}}],["3694",{"2":{"249":1}}],["3690",{"2":{"249":1}}],["3693867",{"2":{"232":1}}],["36938f",{"2":{"77":1}}],["3691778381831775",{"2":{"174":1}}],["369178",{"2":{"171":2}}],["36918",{"2":{"171":1}}],["3653",{"2":{"249":1}}],["36501",{"2":{"244":1}}],["3650193059617517",{"2":{"153":2}}],["36542922",{"2":{"147":1}}],["3623130303215395e",{"2":{"258":1}}],["362544f",{"2":{"256":1}}],["362593",{"2":{"106":1}}],["3628393201392015e",{"2":{"258":1}}],["3628",{"2":{"249":1}}],["36220473",{"2":{"232":2}}],["36222499022985155",{"2":{"153":2}}],["36279",{"2":{"187":1}}],["319090588851249e",{"2":{"258":1}}],["3192628f",{"2":{"256":1}}],["3192",{"2":{"249":1}}],["3164238f",{"2":{"256":1}}],["313013f",{"2":{"256":1}}],["313226f",{"2":{"106":3}}],["317806676836575e",{"2":{"258":1}}],["3176812858269634e",{"2":{"258":1}}],["3177996f",{"2":{"256":1}}],["3170582",{"2":{"232":1}}],["314996676743906e",{"2":{"258":1}}],["3149896f",{"2":{"256":1}}],["3147238",{"2":{"77":1}}],["3155",{"2":{"249":1}}],["31550723",{"2":{"106":1}}],["31501",{"2":{"244":1}}],["310654773994394e",{"2":{"258":1}}],["3105712844388004e",{"2":{"258":1}}],["310388f",{"2":{"256":1}}],["31001",{"2":{"244":1}}],["3108876",{"2":{"147":1}}],["31",{"2":{"228":3,"249":2}}],["31807",{"2":{"187":1}}],["31861955",{"2":{"145":1}}],["31119",{"2":{"187":1}}],["31128f",{"2":{"77":1}}],["357306471016276e",{"2":{"258":1}}],["357115651971301e",{"2":{"258":1}}],["35704f",{"2":{"256":1}}],["3574214f",{"2":{"256":1}}],["359415916236027e",{"2":{"258":1}}],["359601592163036e",{"2":{"258":1}}],["359718f",{"2":{"256":1}}],["35951",{"2":{"187":1}}],["352131f",{"2":{"256":1}}],["3524",{"2":{"249":1}}],["3523193",{"2":{"179":1}}],["35501",{"2":{"244":1}}],["3558193508137715",{"2":{"153":2}}],["35001",{"2":{"244":1}}],["3501",{"2":{"244":1}}],["35395628",{"2":{"232":1}}],["356815459286549e",{"2":{"258":1}}],["35653025",{"2":{"232":1}}],["35638642",{"2":{"147":1}}],["3547942f",{"2":{"256":1}}],["35426",{"2":{"187":1}}],["354464",{"2":{"106":1}}],["35",{"2":{"180":2,"189":2,"210":2,"217":1,"221":2,"228":3,"229":2,"237":2,"246":2,"249":1}}],["35825",{"2":{"187":1}}],["35817",{"2":{"171":1}}],["35836592",{"2":{"77":1}}],["351958217200008e",{"2":{"258":1}}],["351425402965557e",{"2":{"258":1}}],["35149138733595564",{"2":{"174":4}}],["351491",{"2":{"171":1}}],["351189f",{"2":{"256":1}}],["351",{"2":{"208":1}}],["351662",{"2":{"168":1}}],["35181466",{"2":{"126":1}}],["35188f",{"2":{"77":1}}],["35152f",{"2":{"77":1}}],["385125984744453e",{"2":{"258":1}}],["38501",{"2":{"244":1}}],["3850993",{"2":{"126":1}}],["383134492329044e",{"2":{"258":1}}],["3837554f",{"2":{"256":1}}],["3830744f",{"2":{"256":1}}],["3834447782571341",{"2":{"153":1}}],["3834447782571344",{"2":{"153":1}}],["384567611450428e",{"2":{"258":1}}],["384502f",{"2":{"256":1}}],["384959f",{"2":{"256":1}}],["38415",{"2":{"187":1}}],["3841858f",{"2":{"106":2}}],["389381406751195e",{"2":{"258":1}}],["3893502f",{"2":{"256":1}}],["38911813",{"2":{"106":1}}],["3891182",{"2":{"106":1}}],["3864",{"2":{"249":1}}],["3886",{"2":{"249":1}}],["387575327149608e",{"2":{"258":1}}],["387699f",{"2":{"256":1}}],["387152676013409e",{"2":{"258":1}}],["3871598f",{"2":{"256":1}}],["3871873",{"2":{"235":1}}],["38798f",{"2":{"77":1}}],["38",{"2":{"228":6}}],["382899838799854e",{"2":{"258":1}}],["3827805804582414e",{"2":{"258":1}}],["3825785f",{"2":{"256":1}}],["38200346",{"2":{"232":1}}],["382971",{"2":{"168":1}}],["38242024",{"2":{"148":1}}],["380496649651563e",{"2":{"258":1}}],["3807100240288665e",{"2":{"258":1}}],["3807705f",{"2":{"256":1}}],["380777f0",{"2":{"146":1}}],["38001",{"2":{"244":1}}],["38068f",{"2":{"77":1}}],["38187847",{"2":{"126":1}}],["3=dense",{"2":{"44":1}}],["3270",{"2":{"249":1}}],["3279041356366077",{"2":{"153":2}}],["32501",{"2":{"244":1}}],["328542",{"2":{"232":1}}],["32839986131789933",{"2":{"153":2}}],["321",{"2":{"220":6}}],["321506",{"2":{"168":1}}],["320179818424497e",{"2":{"258":1}}],["32018f",{"2":{"256":1}}],["32001",{"2":{"244":1}}],["320",{"2":{"220":6}}],["326546499981479e",{"2":{"258":1}}],["3266429639777084e",{"2":{"258":1}}],["326183f",{"2":{"256":1}}],["3267385f",{"2":{"256":1}}],["326",{"2":{"208":1}}],["32623518",{"2":{"126":1}}],["324320920212499e",{"2":{"258":1}}],["324386f",{"2":{"256":1}}],["324151e",{"2":{"208":1}}],["32480",{"2":{"187":1}}],["3294",{"2":{"249":1}}],["3291591",{"2":{"235":1}}],["32950",{"2":{"187":1}}],["32984197",{"2":{"77":1}}],["32322538",{"2":{"126":1}}],["32",{"2":{"32":2,"59":2,"78":13,"106":9,"107":2,"116":1,"117":1,"193":2,"196":1,"206":1,"208":4,"217":2,"226":2,"228":2,"233":1,"244":1,"251":1,"256":8,"260":1}}],["3dv",{"2":{"53":1}}],["3d",{"2":{"19":1,"28":1,"53":1,"146":1,"187":1,"228":2,"236":1}}],["303216417650039e",{"2":{"258":1}}],["3031727f",{"2":{"256":1}}],["3035",{"2":{"249":1}}],["3085242545998626e",{"2":{"258":1}}],["3087605f",{"2":{"256":1}}],["30866",{"2":{"187":1}}],["3049548510657626e",{"2":{"258":1}}],["3040",{"2":{"249":1}}],["3048",{"2":{"249":1}}],["305788f",{"2":{"256":1}}],["30501",{"2":{"244":1}}],["3050475",{"2":{"235":1}}],["30569053",{"2":{"235":1}}],["305844",{"2":{"147":1,"148":1}}],["3070866",{"2":{"232":2}}],["3071294",{"2":{"148":1}}],["3024367f",{"2":{"256":1}}],["30269",{"2":{"187":1}}],["30215",{"2":{"171":1}}],["30673835",{"2":{"232":1}}],["30674052",{"2":{"179":1}}],["306641",{"2":{"168":1}}],["306147",{"2":{"168":1}}],["309413119754534e",{"2":{"258":1}}],["3093645f",{"2":{"256":1}}],["3092",{"2":{"249":1}}],["3096",{"2":{"249":1}}],["3097294",{"2":{"179":1}}],["309",{"2":{"145":2}}],["3008",{"2":{"249":1}}],["30001",{"2":{"244":1}}],["3001",{"2":{"179":1,"244":1}}],["300",{"2":{"107":1}}],["3014009690716666e",{"2":{"258":1}}],["301577f",{"2":{"256":1}}],["301",{"2":{"77":1,"208":1}}],["30",{"2":{"17":1,"24":1,"217":1,"228":2,"249":1}}],["3×2",{"2":{"59":2,"131":2}}],["3×5",{"2":{"53":4,"59":1}}],["3×3×1×1",{"2":{"24":1}}],["3×7",{"2":{"5":2}}],["3×13",{"2":{"5":4}}],["331787786529424e",{"2":{"258":1}}],["331477834438955e",{"2":{"258":1}}],["3315434f",{"2":{"256":1}}],["3386972073830855e",{"2":{"258":1}}],["3386461f",{"2":{"256":1}}],["3380056183907456e",{"2":{"258":1}}],["3385826",{"2":{"232":2}}],["3398447121235416e",{"2":{"258":1}}],["3394814f",{"2":{"256":1}}],["339079",{"2":{"148":1}}],["3378295f",{"2":{"256":1}}],["33758628",{"2":{"114":1}}],["336042507189592e",{"2":{"258":1}}],["33601f",{"2":{"256":1}}],["3367505",{"2":{"126":1}}],["3335",{"2":{"249":1}}],["3333333333333335",{"2":{"249":1}}],["33333",{"2":{"217":15,"219":2}}],["335581708591681e",{"2":{"258":1}}],["3355213f",{"2":{"256":1}}],["3356",{"2":{"249":1}}],["33501",{"2":{"244":1}}],["33544478",{"2":{"232":1}}],["330420267771681e",{"2":{"258":1}}],["33001",{"2":{"244":1}}],["33070865",{"2":{"232":2}}],["3347854f",{"2":{"256":1}}],["3343754",{"2":{"232":1}}],["3344351",{"2":{"148":1}}],["33442986",{"2":{"106":1}}],["33",{"2":{"5":1,"77":2,"78":1,"196":1,"228":2}}],["3994697838172852e",{"2":{"258":1}}],["3994042f",{"2":{"256":1}}],["3913409077181379e",{"2":{"258":1}}],["3916997494893138e",{"2":{"258":1}}],["3916467f",{"2":{"256":1}}],["3918982346833022e",{"2":{"258":1}}],["3915383f",{"2":{"256":1}}],["3919468f",{"2":{"256":1}}],["395606542684836e",{"2":{"258":1}}],["3956",{"2":{"249":1}}],["3952281f",{"2":{"256":1}}],["3952",{"2":{"249":1}}],["39501",{"2":{"244":1}}],["39588368",{"2":{"232":1}}],["395306",{"2":{"145":1}}],["3934064234487803e",{"2":{"258":1}}],["393877f",{"2":{"256":1}}],["393502f",{"2":{"256":1}}],["3935",{"2":{"249":1}}],["39300445",{"2":{"235":1}}],["39370078",{"2":{"232":2}}],["3939896",{"2":{"148":1}}],["39s",{"2":{"196":2}}],["398038",{"2":{"232":1}}],["39800483",{"2":{"148":1}}],["39899",{"2":{"187":1}}],["396497f",{"2":{"256":1}}],["3960",{"2":{"249":1}}],["39610",{"2":{"187":1}}],["396323",{"2":{"168":1}}],["3969839f",{"2":{"106":1}}],["390488475148462e",{"2":{"258":1}}],["390499133461278e",{"2":{"258":1}}],["3902441f",{"2":{"256":1}}],["390200114697191",{"2":{"153":2}}],["3907394f",{"2":{"256":1}}],["39053f",{"2":{"256":1}}],["39001",{"2":{"244":1}}],["3903124",{"2":{"106":1}}],["3942307655311696",{"2":{"153":2}}],["3940224213371761",{"2":{"153":2}}],["3977319",{"2":{"148":1}}],["3928175",{"2":{"145":1}}],["39",{"2":{"2":1,"4":2,"8":1,"10":1,"13":2,"16":1,"18":1,"25":1,"32":1,"44":1,"46":1,"47":1,"52":1,"53":1,"54":1,"57":3,"59":7,"62":1,"66":1,"68":1,"69":3,"70":1,"77":2,"87":2,"90":1,"94":1,"100":2,"104":3,"105":1,"106":2,"107":1,"109":1,"110":1,"116":1,"117":1,"118":1,"121":2,"123":1,"124":1,"126":1,"135":5,"136":1,"137":3,"138":1,"140":2,"142":1,"143":1,"144":4,"145":4,"146":1,"147":2,"148":1,"152":3,"153":2,"155":1,"170":5,"171":8,"173":2,"176":1,"177":1,"179":1,"184":3,"186":1,"188":2,"196":1,"204":2,"207":1,"214":1,"217":1,"225":1,"228":3,"232":1,"233":1,"236":1,"248":1,"249":4,"254":1,"255":1,"256":2}}],["3",{"2":{"2":1,"5":3,"24":5,"32":2,"34":1,"44":7,"48":1,"52":1,"53":30,"59":27,"62":17,"63":2,"64":2,"66":4,"68":4,"69":3,"77":10,"78":4,"102":2,"106":16,"107":1,"111":3,"112":1,"114":1,"116":12,"117":7,"126":4,"129":6,"131":1,"132":1,"135":1,"137":1,"145":4,"147":12,"148":7,"153":2,"171":23,"172":1,"173":3,"174":7,"180":6,"184":1,"185":1,"187":14,"189":6,"192":1,"193":9,"195":1,"196":3,"206":1,"208":1,"210":6,"213":1,"215":2,"217":5,"219":1,"220":56,"221":7,"228":7,"229":6,"232":7,"235":4,"237":6,"241":3,"242":2,"243":1,"244":2,"246":6,"249":16,"250":1,"254":12,"255":1,"256":91,"258":95}}],["4t",{"2":{"243":1}}],["4th",{"2":{"127":1}}],["4fs",{"2":{"217":1}}],["4x",{"2":{"196":1}}],["469735168084071e",{"2":{"258":1}}],["46944f",{"2":{"256":1}}],["462714667449189e",{"2":{"258":1}}],["462887f",{"2":{"256":1}}],["463542835188122e",{"2":{"258":1}}],["463371f",{"2":{"256":1}}],["46327f",{"2":{"77":1}}],["4673",{"2":{"249":1}}],["46788",{"2":{"187":1}}],["4667",{"2":{"249":1}}],["46667",{"2":{"187":1}}],["4644",{"2":{"249":1}}],["464567",{"2":{"232":2}}],["4659068196387454e",{"2":{"258":1}}],["465938f",{"2":{"256":1}}],["46501",{"2":{"244":1}}],["46514034",{"2":{"179":1}}],["46",{"2":{"196":1,"228":4,"249":1}}],["4683193628608677e",{"2":{"258":1}}],["4684915f",{"2":{"256":1}}],["46845",{"2":{"187":1}}],["4687857",{"2":{"148":1}}],["460925377231771e",{"2":{"258":1}}],["460613686117172e",{"2":{"258":1}}],["46061087",{"2":{"232":1}}],["460674f",{"2":{"256":1}}],["46001",{"2":{"244":1}}],["46056",{"2":{"187":1}}],["4604845",{"2":{"148":1}}],["461231f",{"2":{"256":1}}],["461908",{"2":{"171":1}}],["4619778701480337",{"2":{"153":2}}],["4614",{"2":{"147":1}}],["4890337906226898e",{"2":{"258":1}}],["489481735777707e",{"2":{"258":1}}],["4894685552956632e",{"2":{"258":1}}],["489358f",{"2":{"256":1}}],["48939225f",{"2":{"256":1}}],["48983693",{"2":{"232":1}}],["487281574293196e",{"2":{"258":1}}],["48712015f",{"2":{"256":1}}],["4877s\\ttraining",{"2":{"217":1}}],["48078199742359e",{"2":{"258":1}}],["480865f",{"2":{"256":1}}],["48001",{"2":{"244":1}}],["48027995",{"2":{"232":1}}],["480",{"2":{"220":3}}],["48501",{"2":{"244":1}}],["4850s\\ttraining",{"2":{"219":1}}],["4853s\\ttraining",{"2":{"219":1}}],["4851346",{"2":{"137":1}}],["4847416702318032e",{"2":{"258":1}}],["484157256610741e",{"2":{"258":1}}],["484870642040789e",{"2":{"258":1}}],["4844751f",{"2":{"256":1}}],["484223f",{"2":{"256":1}}],["4845918f",{"2":{"256":1}}],["484",{"2":{"210":1,"246":1}}],["484056",{"2":{"168":1}}],["48370",{"2":{"187":1}}],["48351598",{"2":{"179":1}}],["48",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"228":5,"229":3,"237":3,"246":3}}],["482763740355533e",{"2":{"258":1}}],["4827036f",{"2":{"256":1}}],["482248041235874e",{"2":{"258":1}}],["482217f",{"2":{"256":1}}],["48263642",{"2":{"179":1}}],["48203f",{"2":{"77":1}}],["486475501107193e",{"2":{"258":1}}],["4863922f",{"2":{"256":1}}],["486214",{"2":{"168":1}}],["486550215883336",{"2":{"153":2}}],["4889482f",{"2":{"256":1}}],["4880",{"2":{"249":1}}],["488028235902512",{"2":{"153":2}}],["48818898",{"2":{"232":2}}],["488105",{"2":{"147":1,"148":1}}],["488623",{"2":{"148":1}}],["48148",{"2":{"217":6,"219":1}}],["48193252",{"2":{"145":1}}],["4810884",{"2":{"145":1}}],["48137343",{"2":{"126":1}}],["4280112618469538e",{"2":{"258":1}}],["427086028943335e",{"2":{"258":1}}],["427308391863947e",{"2":{"258":1}}],["4273795f",{"2":{"256":1}}],["4279862f",{"2":{"256":1}}],["425686458755322e",{"2":{"258":1}}],["4258464f",{"2":{"256":1}}],["42501",{"2":{"244":1}}],["42519686",{"2":{"232":2}}],["4267803f",{"2":{"256":1}}],["426",{"2":{"208":1}}],["42669478",{"2":{"106":1}}],["422697903624872e",{"2":{"258":1}}],["422574f",{"2":{"256":1}}],["4225",{"2":{"249":1}}],["422",{"2":{"189":1}}],["422918",{"2":{"171":1}}],["42438",{"2":{"187":1}}],["4242767",{"2":{"179":1}}],["421541212782677e",{"2":{"258":1}}],["421374f",{"2":{"256":1}}],["42189494",{"2":{"179":1}}],["421916im",{"2":{"168":1}}],["4216236",{"2":{"126":1}}],["42",{"2":{"168":2,"196":1,"228":3}}],["42001",{"2":{"244":1}}],["420058350197999",{"2":{"153":1}}],["42005835019799886",{"2":{"153":1}}],["420698",{"2":{"147":1,"148":1}}],["403889494878001e",{"2":{"258":1}}],["403594f",{"2":{"256":1}}],["40354207",{"2":{"179":1}}],["408660673005281e",{"2":{"258":1}}],["408726f",{"2":{"256":1}}],["409567252336606e",{"2":{"258":1}}],["409519f",{"2":{"256":1}}],["409785f",{"2":{"256":1}}],["4059",{"2":{"249":1}}],["40501",{"2":{"244":1}}],["4051151",{"2":{"176":4}}],["406639886091372e",{"2":{"258":1}}],["4066515",{"2":{"148":1}}],["4067254f",{"2":{"256":1}}],["406751f",{"2":{"256":1}}],["4067s\\ttraining",{"2":{"217":1}}],["4061",{"2":{"249":1}}],["40623155",{"2":{"235":1}}],["40",{"2":{"217":5,"219":2,"228":4}}],["4017",{"2":{"249":1}}],["4018757",{"2":{"232":1}}],["4015749",{"2":{"232":2}}],["401",{"2":{"208":1}}],["401215",{"2":{"113":1}}],["40gb",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["407645013452556e",{"2":{"258":1}}],["407668f",{"2":{"256":1}}],["407840679998861e",{"2":{"258":1}}],["40789893",{"2":{"179":1}}],["407521f",{"2":{"256":1}}],["40727592",{"2":{"235":1}}],["40741",{"2":{"217":3}}],["407158+0",{"2":{"168":1}}],["40473",{"2":{"171":1}}],["404728",{"2":{"171":2}}],["40229",{"2":{"171":1}}],["40001",{"2":{"244":1}}],["400129928959714e",{"2":{"258":1}}],["4001",{"2":{"179":1,"244":1}}],["4007292",{"2":{"147":1}}],["4007905",{"2":{"145":1}}],["400",{"2":{"107":1}}],["458242800821592e",{"2":{"258":1}}],["4582062f",{"2":{"256":1}}],["4581461109865615e",{"2":{"258":1}}],["458119f",{"2":{"256":1}}],["4585489",{"2":{"126":1}}],["4564485849501682e",{"2":{"258":1}}],["45649965f",{"2":{"256":1}}],["4566929",{"2":{"232":2}}],["457147349079619e",{"2":{"258":1}}],["457368094578844e",{"2":{"258":1}}],["45738",{"2":{"187":1}}],["457661750053847e",{"2":{"258":1}}],["457243f",{"2":{"256":1}}],["4574167f",{"2":{"256":1}}],["4574596f",{"2":{"256":1}}],["454064863858324e",{"2":{"258":1}}],["4549306f",{"2":{"256":1}}],["4543",{"2":{"249":1}}],["45461",{"2":{"187":1}}],["454679",{"2":{"168":1}}],["4532623550054356e",{"2":{"258":1}}],["45325348",{"2":{"135":1}}],["4534",{"2":{"249":1}}],["45501",{"2":{"244":1}}],["45583528",{"2":{"232":1}}],["4552384158732863",{"2":{"174":4}}],["455238",{"2":{"171":1}}],["45",{"2":{"228":3}}],["4511840832783175e",{"2":{"258":1}}],["4510533f",{"2":{"256":1}}],["4510227f",{"2":{"256":1}}],["4510427",{"2":{"126":1}}],["451",{"2":{"208":1}}],["45s",{"2":{"196":1}}],["452677590444851e",{"2":{"258":1}}],["452951f",{"2":{"256":1}}],["45298263",{"2":{"77":1}}],["4524014",{"2":{"232":1}}],["45247704",{"2":{"179":1}}],["4593922",{"2":{"176":4}}],["459019",{"2":{"171":1}}],["4507907253758794e",{"2":{"258":1}}],["45001",{"2":{"244":1}}],["4500963619011972",{"2":{"153":2}}],["4501",{"2":{"244":1}}],["45016125",{"2":{"179":1}}],["450581f",{"2":{"106":5}}],["4176036566396015e",{"2":{"258":1}}],["417967f",{"2":{"256":1}}],["4185236978195102e",{"2":{"258":1}}],["41806373185002e",{"2":{"258":1}}],["4184581f",{"2":{"256":1}}],["41848",{"2":{"187":1}}],["41501",{"2":{"244":1}}],["4158693023143286",{"2":{"153":2}}],["415858",{"2":{"106":1}}],["410063510377457e",{"2":{"258":1}}],["41001",{"2":{"244":1}}],["4106286403838673e",{"2":{"258":1}}],["4107241f",{"2":{"256":1}}],["4101033",{"2":{"147":1}}],["41",{"2":{"228":3}}],["4163321612661705e",{"2":{"258":1}}],["4163393f",{"2":{"256":1}}],["416",{"2":{"193":2}}],["4115727934031092e",{"2":{"258":1}}],["4114983462099916e",{"2":{"258":1}}],["4118353f",{"2":{"256":1}}],["411761f",{"2":{"256":1}}],["41116",{"2":{"187":1}}],["4112944",{"2":{"106":1}}],["4112945",{"2":{"106":1}}],["41919118",{"2":{"179":1}}],["414769",{"2":{"171":1}}],["4144732",{"2":{"148":1}}],["41238892",{"2":{"147":1}}],["41246277",{"2":{"126":1}}],["436354039633233e",{"2":{"258":1}}],["436059f",{"2":{"256":1}}],["43676835",{"2":{"147":1}}],["434507243337986e",{"2":{"258":1}}],["434593f",{"2":{"256":1}}],["43470",{"2":{"187":1}}],["435243961483699e",{"2":{"258":1}}],["435068f",{"2":{"256":1}}],["43501",{"2":{"244":1}}],["4353992",{"2":{"232":1}}],["430005684148854e",{"2":{"258":1}}],["4300912f",{"2":{"256":1}}],["43001",{"2":{"244":1}}],["4305115e",{"2":{"152":1}}],["4305781",{"2":{"126":1}}],["4371308",{"2":{"232":1}}],["4376572711003852",{"2":{"153":2}}],["4330709",{"2":{"232":2}}],["4339262",{"2":{"179":1}}],["433656",{"2":{"168":1}}],["43350348",{"2":{"147":1}}],["43190",{"2":{"187":1}}],["4319346874291314",{"2":{"171":1}}],["4314566",{"2":{"148":1}}],["4318977",{"2":{"126":1}}],["439761129679035e",{"2":{"258":1}}],["4397957",{"2":{"145":1}}],["4395102f",{"2":{"256":1}}],["4391564",{"2":{"106":1}}],["43af",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["43",{"2":{"87":1,"228":4}}],["4322f",{"2":{"77":1}}],["438082204853424e",{"2":{"258":1}}],["438335f",{"2":{"256":1}}],["4381146f",{"2":{"256":1}}],["4382721",{"2":{"232":1}}],["438698060122472e",{"2":{"258":1}}],["43860",{"2":{"187":1}}],["43866f",{"2":{"77":1}}],["43846482",{"2":{"179":1}}],["438519",{"2":{"106":1}}],["4389",{"2":{"68":1}}],["47034f",{"2":{"256":1}}],["47001",{"2":{"244":1}}],["4781626f",{"2":{"256":1}}],["47872233",{"2":{"145":1}}],["4732353004606066e",{"2":{"258":1}}],["4732864f",{"2":{"256":1}}],["4730743722547668",{"2":{"153":2}}],["4717597825169725e",{"2":{"258":1}}],["471329766709e",{"2":{"258":1}}],["4718792f",{"2":{"256":1}}],["471506f",{"2":{"256":1}}],["47123",{"2":{"187":1}}],["47789603479828e",{"2":{"258":1}}],["4778283",{"2":{"232":1}}],["477288631326389e",{"2":{"258":1}}],["477129f",{"2":{"256":1}}],["475126774066839e",{"2":{"258":1}}],["4759154972248e",{"2":{"258":1}}],["475299f",{"2":{"256":1}}],["47501",{"2":{"244":1}}],["47551f",{"2":{"77":1}}],["4743s\\ttraining",{"2":{"217":1}}],["476",{"2":{"208":1}}],["4763277207638013",{"2":{"153":1}}],["4763277207638008",{"2":{"153":1}}],["4797095957732168e",{"2":{"258":1}}],["479661f",{"2":{"256":1}}],["4791",{"2":{"249":1}}],["479297e",{"2":{"208":1}}],["47988",{"2":{"187":1}}],["47s",{"2":{"196":1}}],["47",{"2":{"193":2,"196":1,"228":2}}],["4721743924473833e",{"2":{"258":1}}],["472123725236716e",{"2":{"258":1}}],["4720129409459e",{"2":{"258":1}}],["4720404f",{"2":{"256":1}}],["472366419096616e",{"2":{"258":1}}],["472367f",{"2":{"256":1}}],["4725690907474887e",{"2":{"258":1}}],["4724155f",{"2":{"256":1}}],["472462f",{"2":{"256":1}}],["472",{"2":{"66":1}}],["4d",{"2":{"70":3,"107":2}}],["491552737970756e",{"2":{"258":1}}],["4912466076377036e",{"2":{"258":1}}],["4912409773203008e",{"2":{"258":1}}],["491042171137685e",{"2":{"258":1}}],["491081f",{"2":{"256":1}}],["491677f",{"2":{"256":1}}],["4914188f",{"2":{"256":1}}],["493830581493888e",{"2":{"258":1}}],["4937",{"2":{"249":1}}],["4936414",{"2":{"232":1}}],["4962437732975195e",{"2":{"258":1}}],["4961199f",{"2":{"256":1}}],["4968",{"2":{"249":1}}],["496063",{"2":{"232":2}}],["49501",{"2":{"244":1}}],["4953s\\ttraining",{"2":{"217":1}}],["4902115095643754e",{"2":{"258":1}}],["49026f",{"2":{"256":1}}],["4907303f",{"2":{"256":1}}],["49001",{"2":{"244":1}}],["4901161f",{"2":{"106":6}}],["49012f",{"2":{"106":2}}],["498685695510351e",{"2":{"258":1}}],["4987402f",{"2":{"256":1}}],["49804282",{"2":{"232":1}}],["4983s\\ttraining",{"2":{"219":1}}],["4981351",{"2":{"106":1}}],["494067f",{"2":{"256":1}}],["4944s\\ttraining",{"2":{"217":1}}],["4947s\\ttraining",{"2":{"217":1}}],["49256",{"2":{"187":1}}],["49998975",{"2":{"106":1}}],["49913f",{"2":{"77":1}}],["49",{"2":{"59":1,"228":2,"233":1,"236":1}}],["4442015057000899e",{"2":{"258":1}}],["444661219911101e",{"2":{"258":1}}],["4440342f",{"2":{"256":1}}],["444494f",{"2":{"256":1}}],["44439f",{"2":{"77":1}}],["4412269882105496e",{"2":{"258":1}}],["441287f",{"2":{"256":1}}],["441650694680668e",{"2":{"258":1}}],["4418456",{"2":{"179":1}}],["442866233796219e",{"2":{"258":1}}],["4427048f",{"2":{"256":1}}],["4420",{"2":{"249":1}}],["449778871947236e",{"2":{"258":1}}],["4497464f",{"2":{"256":1}}],["44911",{"2":{"187":1}}],["4453803196178463e",{"2":{"258":1}}],["445304f",{"2":{"256":1}}],["4456",{"2":{"249":1}}],["44501",{"2":{"244":1}}],["44571686",{"2":{"148":1}}],["44s",{"2":{"196":1}}],["4408877f",{"2":{"256":1}}],["44001",{"2":{"244":1}}],["44096947",{"2":{"235":1}}],["44093",{"2":{"187":1}}],["44078717",{"2":{"232":1}}],["44048917",{"2":{"179":1}}],["4488101521328277",{"2":{"174":1}}],["44807646",{"2":{"107":1}}],["447955671007791e",{"2":{"258":1}}],["4479628f",{"2":{"256":1}}],["4472",{"2":{"249":1}}],["44746",{"2":{"171":1}}],["44735",{"2":{"168":1}}],["446827092743486e",{"2":{"258":1}}],["4467785f",{"2":{"256":1}}],["446424im",{"2":{"168":1}}],["446214",{"2":{"147":1}}],["44397",{"2":{"148":1}}],["4437156",{"2":{"147":1}}],["44",{"2":{"46":1,"196":2,"217":1,"228":2,"249":1}}],["4",{"2":{"2":1,"24":1,"43":1,"44":1,"46":8,"52":2,"53":2,"59":2,"64":1,"66":4,"68":1,"69":5,"77":3,"78":1,"102":1,"106":2,"107":4,"111":8,"114":6,"126":5,"129":8,"131":1,"135":2,"136":2,"139":8,"140":4,"145":9,"146":2,"147":3,"148":11,"152":7,"153":16,"155":2,"171":15,"173":5,"176":2,"179":1,"180":3,"187":17,"189":3,"193":2,"195":1,"196":2,"210":3,"217":5,"219":1,"220":33,"221":2,"224":1,"228":2,"229":2,"232":6,"235":1,"237":3,"244":9,"246":3,"248":9,"249":20,"254":7,"255":1,"256":102,"258":97}}],["0f4",{"2":{"255":1}}],["0f",{"2":{"195":1,"215":2}}],["0f0",{"2":{"59":2,"78":2,"113":2,"114":1,"152":2,"183":2,"206":4,"214":4,"218":2,"232":2,"243":8,"245":6,"248":4,"255":1}}],["0+560",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["0x1911b814c02405e8",{"2":{"174":1}}],["0x12c522b8034ae186",{"2":{"126":1}}],["0x8c49bc52dc8a77ea",{"2":{"174":1}}],["0x8e0c3a65079041bb",{"2":{"126":1}}],["0x48d73dc42d195740",{"2":{"174":1}}],["0x4fa3403dd074e603",{"2":{"126":1}}],["0xdb2fa90498613fdf",{"2":{"174":1}}],["0x22a21880af5dc689",{"2":{"126":1,"174":1}}],["0x21617f7747d97206",{"2":{"126":1}}],["0e",{"2":{"77":7,"234":1,"236":1}}],["0813",{"2":{"249":1}}],["08125",{"2":{"187":1}}],["084503445303279e",{"2":{"258":1}}],["0845636f",{"2":{"256":1}}],["08456781",{"2":{"77":1}}],["0842",{"2":{"249":1}}],["0843",{"2":{"249":1}}],["08347",{"2":{"187":1}}],["08338",{"2":{"187":1}}],["0836414",{"2":{"168":1}}],["0827568614509572e",{"2":{"258":1}}],["0827318f",{"2":{"256":1}}],["08278",{"2":{"187":1}}],["082086",{"2":{"236":1}}],["08202",{"2":{"187":2}}],["0829282",{"2":{"145":1}}],["08",{"2":{"177":1,"228":2}}],["0855234296848334e",{"2":{"258":1}}],["0857597f",{"2":{"256":1}}],["08517",{"2":{"171":1}}],["085170805",{"2":{"171":1}}],["0851708",{"2":{"171":2}}],["08526564",{"2":{"77":1}}],["0884558947925516e",{"2":{"258":1}}],["0884834705765853",{"2":{"153":2}}],["0883958f",{"2":{"256":1}}],["0885",{"2":{"249":1}}],["0887",{"2":{"249":1}}],["08820387",{"2":{"232":1}}],["088205874",{"2":{"106":1}}],["08866",{"2":{"187":1}}],["0862010144555884e",{"2":{"258":1}}],["0863249f",{"2":{"256":1}}],["0867",{"2":{"249":1}}],["0865",{"2":{"249":1}}],["0866141",{"2":{"232":2}}],["08667634",{"2":{"106":2}}],["08694684883050086",{"2":{"174":4}}],["0869468",{"2":{"171":1}}],["086429894",{"2":{"147":1}}],["0876",{"2":{"249":1}}],["08713347",{"2":{"137":1}}],["08791955",{"2":{"106":1}}],["08909951353186e",{"2":{"258":1}}],["0890484f",{"2":{"256":1}}],["08922641",{"2":{"232":1}}],["08922",{"2":{"187":1}}],["089530945",{"2":{"106":1}}],["08953094",{"2":{"106":1}}],["08993f",{"2":{"77":1}}],["080408391826853e",{"2":{"258":1}}],["0804738f",{"2":{"256":1}}],["0809817648757658e",{"2":{"258":1}}],["08097465f",{"2":{"256":1}}],["0803379",{"2":{"232":1}}],["08031",{"2":{"187":1}}],["0801",{"2":{"78":1}}],["08085155",{"2":{"78":1}}],["08022",{"2":{"19":1,"69":1}}],["0367432029938036e",{"2":{"258":1}}],["036692f",{"2":{"256":1}}],["03614",{"2":{"187":1}}],["03f0",{"2":{"234":1}}],["03",{"2":{"208":8,"234":1,"236":1}}],["03s",{"2":{"196":1}}],["034535090281437e",{"2":{"258":1}}],["034535f",{"2":{"256":1}}],["03479",{"2":{"187":1}}],["03437",{"2":{"187":1}}],["034259234",{"2":{"179":1}}],["0377035382311122e",{"2":{"258":1}}],["0379057f",{"2":{"256":1}}],["03704",{"2":{"217":3}}],["03751",{"2":{"187":1}}],["03758",{"2":{"187":1}}],["037500158",{"2":{"106":1}}],["03786",{"2":{"187":1}}],["035556547",{"2":{"244":1}}],["0355555",{"2":{"171":1}}],["03522",{"2":{"187":1}}],["03591",{"2":{"187":1}}],["03516",{"2":{"187":1}}],["03532",{"2":{"187":1}}],["0350433",{"2":{"77":1}}],["0335547560276554e",{"2":{"258":1}}],["0332749183269495e",{"2":{"258":1}}],["03322105",{"2":{"232":1}}],["0336203f",{"2":{"256":1}}],["033320315",{"2":{"244":1}}],["03334",{"2":{"187":1}}],["03385",{"2":{"187":1}}],["0338569",{"2":{"147":1}}],["033022970820454e",{"2":{"258":1}}],["0330944",{"2":{"77":1}}],["03306928",{"2":{"77":1}}],["0310954f",{"2":{"256":1}}],["031768262",{"2":{"244":1}}],["03130",{"2":{"187":1}}],["03118",{"2":{"187":1}}],["03113",{"2":{"187":1}}],["031414255389526885",{"2":{"153":2}}],["031403534",{"2":{"106":1}}],["031403527",{"2":{"106":1}}],["03157",{"2":{"187":1}}],["031503614",{"2":{"106":1}}],["0315817",{"2":{"77":1}}],["032728f",{"2":{"256":1}}],["032705",{"2":{"232":1}}],["03280",{"2":{"187":1}}],["032846544",{"2":{"147":1}}],["03263",{"2":{"187":1}}],["03261588",{"2":{"106":1}}],["032615878",{"2":{"106":1}}],["03224",{"2":{"187":1}}],["03213",{"2":{"187":1}}],["03258145",{"2":{"179":1}}],["032405667",{"2":{"106":1}}],["032405674",{"2":{"106":1}}],["0324213",{"2":{"106":1}}],["03242131",{"2":{"106":1}}],["0320963",{"2":{"106":1}}],["039549f",{"2":{"256":1}}],["03982s\\ttraining",{"2":{"228":1}}],["03930",{"2":{"187":1}}],["03902",{"2":{"187":1}}],["039647065",{"2":{"106":2}}],["039132368",{"2":{"77":1}}],["03081657776027e",{"2":{"258":1}}],["030881835",{"2":{"106":2}}],["030331207",{"2":{"244":1}}],["03022",{"2":{"187":1}}],["03028",{"2":{"187":1}}],["0302464",{"2":{"77":1}}],["030016",{"2":{"236":1}}],["03009",{"2":{"187":1}}],["03002",{"2":{"187":1}}],["03070261",{"2":{"107":1}}],["030926049",{"2":{"106":1}}],["030926052",{"2":{"106":1}}],["03094203",{"2":{"77":1}}],["0301974179265634e",{"2":{"258":1}}],["0301662f",{"2":{"256":1}}],["0301",{"2":{"78":1}}],["038943053642793e",{"2":{"258":1}}],["03832f",{"2":{"256":1}}],["03830",{"2":{"187":1}}],["03842f",{"2":{"256":1}}],["0388885f",{"2":{"256":1}}],["03884",{"2":{"187":1}}],["03873",{"2":{"187":1}}],["03803",{"2":{"187":1}}],["03804328",{"2":{"77":1}}],["038141233765218e",{"2":{"258":1}}],["0381317",{"2":{"148":1}}],["0381727",{"2":{"147":1}}],["038",{"2":{"69":3}}],["02f0",{"2":{"245":2}}],["02",{"2":{"208":12,"228":2}}],["0250840666626814e",{"2":{"258":1}}],["0256",{"2":{"249":1}}],["02568",{"2":{"187":1}}],["025281426",{"2":{"244":1}}],["025583776",{"2":{"244":1}}],["0257s\\ttraining",{"2":{"217":1}}],["025140665",{"2":{"244":1}}],["02511",{"2":{"187":1}}],["02518",{"2":{"187":1}}],["02535537",{"2":{"179":1}}],["02535769",{"2":{"147":1}}],["0212426596476283e",{"2":{"258":1}}],["02101s\\ttraining",{"2":{"228":1}}],["02102",{"2":{"187":1}}],["02182s\\ttraining",{"2":{"228":1}}],["021166507",{"2":{"244":1}}],["02110s\\ttraining",{"2":{"228":1}}],["02118s\\ttraining",{"2":{"228":1}}],["02111",{"2":{"187":1}}],["02135s\\ttraining",{"2":{"228":1}}],["02152s\\ttraining",{"2":{"228":1}}],["02153s\\ttraining",{"2":{"228":1}}],["02156",{"2":{"187":1}}],["02172s\\ttraining",{"2":{"228":1}}],["02170",{"2":{"187":1}}],["02177",{"2":{"187":1}}],["02149",{"2":{"187":1}}],["02143306",{"2":{"107":1}}],["02160",{"2":{"187":1}}],["029164422",{"2":{"244":1}}],["02916s\\ttraining",{"2":{"228":1}}],["02954s\\ttraining",{"2":{"228":1}}],["02951",{"2":{"187":1}}],["02909s\\ttraining",{"2":{"228":1}}],["02999",{"2":{"187":1}}],["02964765308691042",{"2":{"174":4}}],["0296477",{"2":{"171":1}}],["029290024",{"2":{"77":1}}],["028128654",{"2":{"244":1}}],["02813s\\ttraining",{"2":{"228":1}}],["02895s\\ttraining",{"2":{"228":1}}],["0288",{"2":{"249":1}}],["02886s\\ttraining",{"2":{"228":1}}],["02888",{"2":{"187":1}}],["02877s\\ttraining",{"2":{"228":1}}],["02879",{"2":{"187":1}}],["028738942",{"2":{"147":1}}],["02807",{"2":{"187":1}}],["02805",{"2":{"187":1}}],["02854",{"2":{"187":1}}],["0285962",{"2":{"147":1}}],["0283093500712565e",{"2":{"258":1}}],["028305896",{"2":{"244":1}}],["0283695f",{"2":{"256":1}}],["02839",{"2":{"187":1}}],["02831",{"2":{"187":1}}],["028461456",{"2":{"126":1}}],["02821s\\ttraining",{"2":{"228":1}}],["02827",{"2":{"187":1}}],["02827571",{"2":{"106":1}}],["028275706",{"2":{"106":1}}],["02828",{"2":{"187":1}}],["028251555",{"2":{"77":1}}],["027620461",{"2":{"244":1}}],["02764",{"2":{"187":1}}],["0277897",{"2":{"232":1}}],["027702922",{"2":{"77":1}}],["02797s\\ttraining",{"2":{"228":1}}],["02795s\\ttraining",{"2":{"228":1}}],["027273500",{"2":{"244":1}}],["02729s\\ttraining",{"2":{"228":1}}],["02726s\\ttraining",{"2":{"228":1}}],["027226416",{"2":{"106":1}}],["027226413",{"2":{"106":1}}],["02784s\\ttraining",{"2":{"228":1}}],["02753",{"2":{"187":1}}],["02731",{"2":{"187":1}}],["02706",{"2":{"187":1}}],["02716",{"2":{"187":1}}],["0209",{"2":{"249":1}}],["02095s\\ttraining",{"2":{"228":1}}],["02097s\\ttraining",{"2":{"228":1}}],["020389127",{"2":{"244":1}}],["02033s\\ttraining",{"2":{"228":1}}],["02039s\\ttraining",{"2":{"228":1}}],["02035",{"2":{"187":1}}],["02008s\\ttraining",{"2":{"228":1}}],["0204",{"2":{"249":1}}],["020493284",{"2":{"244":1}}],["02045s\\ttraining",{"2":{"228":1}}],["02047s\\ttraining",{"2":{"228":1}}],["02040s\\ttraining",{"2":{"228":1}}],["02042s\\ttraining",{"2":{"228":1}}],["02041",{"2":{"187":1}}],["02057s\\ttraining",{"2":{"228":1}}],["02050s\\ttraining",{"2":{"228":1}}],["02052s\\ttraining",{"2":{"228":1}}],["02055s\\ttraining",{"2":{"228":3}}],["02056s\\ttraining",{"2":{"228":3}}],["02059s\\ttraining",{"2":{"228":2}}],["02054s\\ttraining",{"2":{"228":2}}],["02058s\\ttraining",{"2":{"228":2}}],["020624701",{"2":{"244":1}}],["02066s\\ttraining",{"2":{"228":1}}],["02060s\\ttraining",{"2":{"228":2}}],["02067s\\ttraining",{"2":{"228":1}}],["02065s\\ttraining",{"2":{"228":2}}],["02069s\\ttraining",{"2":{"228":2}}],["02063s\\ttraining",{"2":{"228":4}}],["02061s\\ttraining",{"2":{"228":3}}],["02068s\\ttraining",{"2":{"228":2}}],["02089s\\ttraining",{"2":{"228":1}}],["02084s\\ttraining",{"2":{"228":1}}],["02081s\\ttraining",{"2":{"228":3}}],["02088s\\ttraining",{"2":{"228":1}}],["02080s\\ttraining",{"2":{"228":3}}],["02083s\\ttraining",{"2":{"228":1}}],["02029s\\ttraining",{"2":{"228":1}}],["02024s\\ttraining",{"2":{"228":1}}],["02028s\\ttraining",{"2":{"228":1}}],["02022s\\ttraining",{"2":{"228":1}}],["02021s\\ttraining",{"2":{"228":1}}],["02020",{"2":{"187":1}}],["02027s\\ttraining",{"2":{"228":1}}],["02027",{"2":{"187":1}}],["02079s\\ttraining",{"2":{"228":1}}],["0207926",{"2":{"77":1}}],["02075s\\ttraining",{"2":{"228":2}}],["02076s\\ttraining",{"2":{"228":1}}],["02078s\\ttraining",{"2":{"228":2}}],["02073s\\ttraining",{"2":{"228":1}}],["02071s\\ttraining",{"2":{"228":3}}],["02077s\\ttraining",{"2":{"228":1}}],["02070",{"2":{"187":1}}],["0207279",{"2":{"135":1}}],["02012s\\ttraining",{"2":{"228":1}}],["02016s\\ttraining",{"2":{"228":2}}],["02013s\\ttraining",{"2":{"228":1}}],["02013",{"2":{"187":1}}],["0201",{"2":{"78":1}}],["023622",{"2":{"232":2}}],["02354s\\ttraining",{"2":{"228":1}}],["02352",{"2":{"187":1}}],["02300",{"2":{"187":1}}],["023031702",{"2":{"77":1}}],["0232043f",{"2":{"256":1}}],["02322",{"2":{"187":1}}],["02328",{"2":{"187":2}}],["02313",{"2":{"187":1}}],["02318",{"2":{"187":1}}],["0231775",{"2":{"77":1}}],["022150228",{"2":{"244":1}}],["02210",{"2":{"187":2}}],["022293953",{"2":{"244":1}}],["022215",{"2":{"236":1}}],["02223s\\ttraining",{"2":{"228":1}}],["0223586941926334e",{"2":{"258":1}}],["02239s\\ttraining",{"2":{"228":1}}],["02231s\\ttraining",{"2":{"228":1}}],["02238",{"2":{"187":2}}],["0226212f",{"2":{"256":1}}],["02260s\\ttraining",{"2":{"228":1}}],["022618050",{"2":{"244":1}}],["02261",{"2":{"187":1}}],["02289",{"2":{"187":1}}],["0228109822209213",{"2":{"153":2}}],["022739200",{"2":{"244":1}}],["022735020",{"2":{"244":1}}],["02272",{"2":{"187":1}}],["0227737",{"2":{"77":1}}],["02297",{"2":{"187":1}}],["02290",{"2":{"187":1}}],["022434518",{"2":{"77":1}}],["0268136f",{"2":{"256":1}}],["026851978",{"2":{"77":1}}],["026282774",{"2":{"244":1}}],["026170366",{"2":{"244":1}}],["02617",{"2":{"187":1}}],["02615",{"2":{"187":1}}],["02646",{"2":{"187":1}}],["02648",{"2":{"187":1}}],["02656",{"2":{"187":1}}],["026516732",{"2":{"77":1}}],["0249331915329287e",{"2":{"258":1}}],["0249986f",{"2":{"256":1}}],["02496",{"2":{"187":1}}],["0242",{"2":{"249":1}}],["02429",{"2":{"187":1}}],["0246",{"2":{"249":1}}],["02463",{"2":{"187":1}}],["0240",{"2":{"249":1}}],["02450",{"2":{"187":1}}],["02412",{"2":{"187":1}}],["02471",{"2":{"187":1}}],["0244548f",{"2":{"106":1}}],["0244214",{"2":{"77":1}}],["024",{"2":{"77":1}}],["0946710374296105e",{"2":{"258":1}}],["0948732f",{"2":{"256":1}}],["09454",{"2":{"187":1}}],["09",{"2":{"249":1}}],["093474579812071e",{"2":{"258":1}}],["093289773220052e",{"2":{"258":1}}],["0932027f",{"2":{"256":1}}],["093000583673822e",{"2":{"258":1}}],["09377f",{"2":{"256":1}}],["0935077",{"2":{"254":2}}],["09351656",{"2":{"177":1,"178":1}}],["093925126",{"2":{"179":1}}],["0988385951586975e",{"2":{"258":1}}],["098861866",{"2":{"77":1}}],["0987623f",{"2":{"256":1}}],["09874",{"2":{"187":1}}],["0987538",{"2":{"168":1}}],["09852",{"2":{"171":1}}],["0985227",{"2":{"171":2}}],["0914869426216386e",{"2":{"258":1}}],["0914626",{"2":{"148":1}}],["091659f",{"2":{"256":1}}],["0913593358445652",{"2":{"153":1}}],["091359335844565",{"2":{"153":1}}],["0929265f",{"2":{"256":1}}],["09299411",{"2":{"107":1}}],["09230",{"2":{"187":1}}],["09289699",{"2":{"106":1}}],["0965",{"2":{"249":1}}],["09693718",{"2":{"235":1}}],["09699093",{"2":{"147":1}}],["096988946",{"2":{"126":1}}],["09692798",{"2":{"77":1}}],["09643307",{"2":{"106":2}}],["095133f",{"2":{"256":1}}],["095568806",{"2":{"235":1}}],["09597",{"2":{"187":1}}],["09599f",{"2":{"77":1}}],["0954186",{"2":{"106":1}}],["0991955",{"2":{"106":1}}],["09919551",{"2":{"106":1}}],["099906564",{"2":{"106":1}}],["09926938",{"2":{"106":1}}],["0973",{"2":{"249":1}}],["0971",{"2":{"249":1}}],["097182155",{"2":{"106":1}}],["09760",{"2":{"187":1}}],["09701932",{"2":{"179":1}}],["097253",{"2":{"126":1}}],["09724942",{"2":{"106":1}}],["09724939",{"2":{"106":1}}],["097701035",{"2":{"106":2}}],["09785451",{"2":{"77":1}}],["0903271",{"2":{"106":2}}],["0901",{"2":{"78":1}}],["09098f",{"2":{"77":1}}],["0900306",{"2":{"53":10}}],["076084677854146e",{"2":{"258":1}}],["076992f",{"2":{"256":1}}],["076404385",{"2":{"106":1}}],["07640441",{"2":{"106":1}}],["0758826f",{"2":{"256":1}}],["07565363",{"2":{"235":1}}],["075135306",{"2":{"147":1}}],["07",{"2":{"228":1}}],["0782",{"2":{"249":1}}],["07828",{"2":{"187":1}}],["0787",{"2":{"249":1}}],["07874016",{"2":{"232":2}}],["07872",{"2":{"187":1}}],["078427e",{"2":{"208":1}}],["0733",{"2":{"249":1}}],["07375115",{"2":{"235":1}}],["07395",{"2":{"187":1}}],["073442355",{"2":{"77":1}}],["072294782567414e",{"2":{"258":1}}],["07224",{"2":{"187":1}}],["07259f",{"2":{"256":1}}],["0728675615927385",{"2":{"53":1}}],["07767",{"2":{"187":1}}],["077052306857798e",{"2":{"258":1}}],["07702",{"2":{"171":1}}],["0770206",{"2":{"171":2}}],["07704306",{"2":{"77":1}}],["079728458034808e",{"2":{"258":1}}],["0797285f",{"2":{"256":1}}],["079208925",{"2":{"114":1}}],["07964921",{"2":{"107":1}}],["0742047",{"2":{"232":1}}],["07425846",{"2":{"106":1}}],["07425845",{"2":{"106":1}}],["07412187",{"2":{"77":1}}],["0709",{"2":{"232":1}}],["07093",{"2":{"187":1}}],["0701",{"2":{"78":1}}],["07058401",{"2":{"77":1}}],["07057328",{"2":{"77":1}}],["071601003795665e",{"2":{"258":1}}],["07167f",{"2":{"77":1}}],["0713344f",{"2":{"256":1}}],["07133968",{"2":{"116":1}}],["071482725",{"2":{"179":1}}],["07125676",{"2":{"77":1}}],["0444",{"2":{"249":1}}],["04416",{"2":{"236":1}}],["04479",{"2":{"187":1}}],["0415702711312987e",{"2":{"258":1}}],["041541956",{"2":{"244":1}}],["04159",{"2":{"187":1}}],["04199602",{"2":{"179":1}}],["04107",{"2":{"187":1}}],["041031003",{"2":{"145":1}}],["04108f",{"2":{"77":1}}],["0492632138680406e",{"2":{"258":1}}],["04923",{"2":{"187":1}}],["04915744594797e",{"2":{"258":1}}],["04957875332556e",{"2":{"258":1}}],["0497316f",{"2":{"256":1}}],["049368351281167e",{"2":{"258":1}}],["0493395f",{"2":{"256":1}}],["049317f",{"2":{"256":1}}],["0493766f",{"2":{"256":1}}],["04903387",{"2":{"179":1}}],["049417756",{"2":{"106":1}}],["049417753",{"2":{"106":1}}],["043131f",{"2":{"256":1}}],["0435",{"2":{"249":1}}],["0435721",{"2":{"77":1}}],["04386",{"2":{"187":1}}],["043883",{"2":{"171":1}}],["04349",{"2":{"187":1}}],["0462551189458834e",{"2":{"258":1}}],["0462954",{"2":{"148":1}}],["046534f",{"2":{"256":1}}],["046876e",{"2":{"208":1}}],["0469301",{"2":{"148":1}}],["040918160",{"2":{"244":1}}],["04093",{"2":{"187":1}}],["04004",{"2":{"187":1}}],["0402095",{"2":{"147":1}}],["0401",{"2":{"78":1}}],["04019f",{"2":{"77":1}}],["045774488692488e",{"2":{"258":1}}],["04535",{"2":{"187":1}}],["04534",{"2":{"187":1}}],["04586",{"2":{"187":1}}],["04551310",{"2":{"107":1}}],["04566476",{"2":{"106":1}}],["047244094",{"2":{"232":2}}],["04732",{"2":{"187":1}}],["047356773",{"2":{"77":1}}],["047897488",{"2":{"106":2}}],["048109",{"2":{"148":1}}],["048133414",{"2":{"106":2}}],["048522998",{"2":{"129":1}}],["04853325",{"2":{"106":1}}],["048533253",{"2":{"106":1}}],["048811335",{"2":{"106":2}}],["0480501",{"2":{"77":1}}],["04d",{"2":{"78":1}}],["042072848076068e",{"2":{"258":1}}],["04208f",{"2":{"256":1}}],["0420834",{"2":{"77":1}}],["042499166",{"2":{"244":1}}],["0428307",{"2":{"232":1}}],["0427s\\ttraining",{"2":{"217":1}}],["04256",{"2":{"187":1}}],["0425432",{"2":{"147":1}}],["04295495283931e",{"2":{"258":1}}],["04297",{"2":{"187":1}}],["04294",{"2":{"187":1}}],["04265762",{"2":{"179":1}}],["0422s\\ttraining",{"2":{"217":1}}],["042254835",{"2":{"106":1}}],["04225484",{"2":{"106":1}}],["042256515",{"2":{"106":1}}],["04224006",{"2":{"77":1}}],["04226003",{"2":{"77":1}}],["0614390821541168e",{"2":{"258":1}}],["0614478412968307e",{"2":{"258":1}}],["0614642f",{"2":{"256":1}}],["0618",{"2":{"249":1}}],["0613824f",{"2":{"256":1}}],["0613",{"2":{"249":1}}],["069790764052438e",{"2":{"258":1}}],["0697596f",{"2":{"256":1}}],["0690823",{"2":{"235":1}}],["06942138",{"2":{"77":1}}],["0641140321522063e",{"2":{"258":1}}],["064738696900327e",{"2":{"258":1}}],["064739f",{"2":{"256":1}}],["0642813f",{"2":{"256":1}}],["06433817",{"2":{"232":1}}],["06450",{"2":{"19":1}}],["06843883722068e",{"2":{"258":1}}],["0688757f",{"2":{"256":1}}],["0686756",{"2":{"232":1}}],["06854",{"2":{"187":1}}],["067131212087478e",{"2":{"258":1}}],["0671347",{"2":{"168":1}}],["0672023f",{"2":{"256":1}}],["0672",{"2":{"249":1}}],["06745",{"2":{"187":1}}],["067872316",{"2":{"77":1}}],["062907",{"2":{"236":1}}],["062281e",{"2":{"208":1}}],["06217",{"2":{"187":1}}],["062155924699545",{"2":{"153":1}}],["0621559246995447",{"2":{"153":1}}],["062079933",{"2":{"147":1}}],["0664114f",{"2":{"256":1}}],["06669",{"2":{"131":2}}],["06610983789208e",{"2":{"258":1}}],["0661932f",{"2":{"256":1}}],["066128",{"2":{"148":1}}],["06612328",{"2":{"107":1}}],["06618893",{"2":{"106":1}}],["06675106",{"2":{"106":1}}],["066751055",{"2":{"106":1}}],["0633",{"2":{"249":1}}],["063336e",{"2":{"208":1}}],["06353",{"2":{"187":1}}],["06301335",{"2":{"106":1}}],["06325f",{"2":{"77":1}}],["063480526",{"2":{"77":1}}],["060623652600814e",{"2":{"258":1}}],["06086f",{"2":{"256":1}}],["06027697",{"2":{"106":1}}],["06026962",{"2":{"77":1}}],["06019",{"2":{"187":1}}],["0601",{"2":{"78":1}}],["060364496",{"2":{"77":1}}],["06053336",{"2":{"77":1}}],["065978969163425e",{"2":{"258":1}}],["0659136f",{"2":{"256":1}}],["06519687",{"2":{"107":1}}],["06550143",{"2":{"77":1}}],["065317236",{"2":{"77":1}}],["06",{"2":{"53":1,"196":1}}],["05f0",{"2":{"244":2,"245":1}}],["0559330740240846e",{"2":{"258":1}}],["0559876f",{"2":{"256":1}}],["0551181",{"2":{"232":2}}],["05510",{"2":{"187":1}}],["05531",{"2":{"187":1}}],["0510",{"2":{"249":1}}],["05181",{"2":{"187":1}}],["05138",{"2":{"187":1}}],["05128",{"2":{"187":1}}],["05217",{"2":{"187":1}}],["05218964",{"2":{"179":1}}],["0582605269986785e",{"2":{"258":1}}],["0580584f",{"2":{"256":1}}],["05807",{"2":{"187":1}}],["0584",{"2":{"249":1}}],["05877",{"2":{"187":1}}],["0585f",{"2":{"77":1}}],["0546",{"2":{"249":1}}],["0540",{"2":{"249":1}}],["0540537",{"2":{"77":1}}],["05471",{"2":{"187":1}}],["05991",{"2":{"187":1}}],["059518166",{"2":{"106":2}}],["057664478431455e",{"2":{"258":1}}],["0578545",{"2":{"235":1}}],["05771",{"2":{"187":1}}],["05779423",{"2":{"126":1}}],["057005208",{"2":{"177":1,"178":1}}],["05729505",{"2":{"106":1}}],["0534",{"2":{"249":1}}],["0536",{"2":{"249":1}}],["053627778",{"2":{"106":1}}],["05308",{"2":{"187":1}}],["053747915",{"2":{"106":2}}],["0568",{"2":{"249":1}}],["0569",{"2":{"249":1}}],["0563s\\ttraining",{"2":{"217":1}}],["05646",{"2":{"187":1}}],["0564903986057956",{"2":{"153":2}}],["05657739601024536",{"2":{"174":1}}],["056",{"2":{"78":3,"256":1}}],["0501",{"2":{"78":1}}],["05058f",{"2":{"77":1}}],["050089",{"2":{"77":1}}],["05",{"2":{"53":1,"249":1}}],["00",{"2":{"196":14,"228":40}}],["008026988",{"2":{"244":1}}],["008074892",{"2":{"244":1}}],["008014920",{"2":{"244":1}}],["008724037",{"2":{"244":1}}],["008732248",{"2":{"244":1}}],["008772802",{"2":{"244":1}}],["00877",{"2":{"187":1}}],["008925664",{"2":{"244":1}}],["008556721",{"2":{"244":1}}],["008808360",{"2":{"244":1}}],["00881",{"2":{"187":1}}],["00886",{"2":{"187":1}}],["008870317",{"2":{"77":1}}],["008153505",{"2":{"244":1}}],["008185850",{"2":{"244":1}}],["00816",{"2":{"187":1}}],["00819",{"2":{"187":1}}],["00839",{"2":{"187":1}}],["008405063",{"2":{"244":1}}],["008404830",{"2":{"244":1}}],["00840",{"2":{"187":1}}],["00846",{"2":{"187":1}}],["00845",{"2":{"187":1}}],["00845341",{"2":{"168":1}}],["00849",{"2":{"187":2}}],["00826",{"2":{"187":1}}],["00825",{"2":{"187":1}}],["00820",{"2":{"187":1}}],["00827",{"2":{"187":1}}],["00823197",{"2":{"171":1}}],["00865",{"2":{"187":1}}],["00868",{"2":{"187":1}}],["0074345f",{"2":{"256":1}}],["00742",{"2":{"187":1}}],["007187600",{"2":{"244":1}}],["007129070",{"2":{"244":1}}],["007586177559976e",{"2":{"258":1}}],["007526f",{"2":{"256":1}}],["007501942",{"2":{"244":1}}],["007509941",{"2":{"244":1}}],["007593059",{"2":{"244":1}}],["007535893",{"2":{"244":1}}],["00751",{"2":{"187":1}}],["007325693",{"2":{"244":1}}],["007372384",{"2":{"244":1}}],["007333768",{"2":{"244":1}}],["007363341",{"2":{"244":1}}],["0073489705",{"2":{"106":2}}],["007075997",{"2":{"244":1}}],["00707",{"2":{"187":1}}],["00706411",{"2":{"77":1}}],["007220342",{"2":{"244":1}}],["007273634",{"2":{"244":1}}],["00726",{"2":{"187":1}}],["00724",{"2":{"187":1}}],["007249862",{"2":{"106":1}}],["0072498624",{"2":{"106":1}}],["007885863",{"2":{"244":1}}],["00786",{"2":{"187":1}}],["00781",{"2":{"187":1}}],["00787",{"2":{"187":1}}],["007676640",{"2":{"244":1}}],["007679918",{"2":{"244":1}}],["00765",{"2":{"187":1}}],["00762",{"2":{"187":2}}],["007947534",{"2":{"244":1}}],["007950513",{"2":{"244":1}}],["007905648",{"2":{"244":1}}],["00796",{"2":{"187":1}}],["007984845",{"2":{"147":1}}],["007740238886854e",{"2":{"258":1}}],["00775",{"2":{"187":1}}],["00777",{"2":{"187":1}}],["00770",{"2":{"187":1}}],["0094647f",{"2":{"256":1}}],["00947",{"2":{"187":1}}],["009313912",{"2":{"244":1}}],["00936",{"2":{"187":1}}],["0096590541210817e",{"2":{"258":1}}],["009693242",{"2":{"244":1}}],["00968",{"2":{"187":2}}],["009895653",{"2":{"244":1}}],["009887908",{"2":{"244":1}}],["00987",{"2":{"187":1}}],["009987052",{"2":{"244":1}}],["009963844931984",{"2":{"153":2}}],["009196416",{"2":{"244":1}}],["00914141",{"2":{"179":1}}],["009766437",{"2":{"232":1}}],["00976",{"2":{"187":1}}],["00972",{"2":{"187":1}}],["00971",{"2":{"187":1}}],["00973899",{"2":{"131":2}}],["009083126",{"2":{"244":1}}],["009084041f0",{"2":{"53":1}}],["00903",{"2":{"187":1}}],["00902",{"2":{"187":1}}],["00904",{"2":{"187":1}}],["00907",{"2":{"187":2}}],["009256126",{"2":{"244":1}}],["00925",{"2":{"187":1}}],["0095083823350953e",{"2":{"258":1}}],["0095757f",{"2":{"256":1}}],["009578831",{"2":{"77":1}}],["00955",{"2":{"187":1}}],["009514628",{"2":{"244":1}}],["009516605",{"2":{"244":1}}],["00951",{"2":{"187":1}}],["0059814559306675e",{"2":{"258":1}}],["005978104",{"2":{"244":1}}],["005315070",{"2":{"244":1}}],["005351806",{"2":{"244":1}}],["0053856913",{"2":{"77":1}}],["005200472",{"2":{"244":1}}],["005807751",{"2":{"244":1}}],["005835793",{"2":{"244":1}}],["005824474",{"2":{"244":1}}],["005463609",{"2":{"244":1}}],["005461216",{"2":{"244":1}}],["005433898",{"2":{"244":1}}],["005112350",{"2":{"244":1}}],["005628760",{"2":{"244":1}}],["005615299",{"2":{"244":1}}],["00564f",{"2":{"77":1}}],["005f0",{"2":{"244":1}}],["005",{"2":{"208":1}}],["005065940",{"2":{"244":1}}],["005044522",{"2":{"244":1}}],["005014097",{"2":{"77":1}}],["005000000000000009",{"2":{"53":1}}],["0060764f",{"2":{"256":1}}],["006616294",{"2":{"244":1}}],["006642802",{"2":{"244":1}}],["0066712513",{"2":{"179":1}}],["00633053182541e",{"2":{"258":1}}],["006305961",{"2":{"244":1}}],["00631",{"2":{"187":1}}],["006243715020314e",{"2":{"258":1}}],["006221022",{"2":{"244":1}}],["006288857",{"2":{"106":1}}],["0062888456",{"2":{"106":1}}],["006527f",{"2":{"256":1}}],["006572613",{"2":{"244":1}}],["0065859724",{"2":{"77":1}}],["00682",{"2":{"187":1}}],["006942283",{"2":{"244":1}}],["006962581",{"2":{"244":1}}],["006906096",{"2":{"244":1}}],["00699",{"2":{"187":1}}],["0069942693",{"2":{"106":1}}],["00697",{"2":{"187":1}}],["006432486",{"2":{"77":1}}],["006105572",{"2":{"244":1}}],["0061099795",{"2":{"77":1}}],["006184267",{"2":{"77":1}}],["006736353",{"2":{"77":1}}],["0067729075",{"2":{"77":1}}],["004855067",{"2":{"244":1}}],["004880759",{"2":{"244":1}}],["00488f",{"2":{"77":1}}],["004657542",{"2":{"244":1}}],["004607514",{"2":{"244":1}}],["004590358",{"2":{"244":1}}],["004518208",{"2":{"244":1}}],["004542338",{"2":{"244":1}}],["004978007",{"2":{"244":1}}],["004947479",{"2":{"244":1}}],["004958895",{"2":{"244":1}}],["004958282",{"2":{"244":1}}],["004790282",{"2":{"244":1}}],["004397592",{"2":{"244":1}}],["004335414",{"2":{"244":1}}],["004338491",{"2":{"244":1}}],["004365115",{"2":{"244":1}}],["004326937",{"2":{"244":1}}],["004329988",{"2":{"244":1}}],["004253434",{"2":{"244":1}}],["004287243",{"2":{"106":2}}],["004434146",{"2":{"244":1}}],["004466736",{"2":{"244":1}}],["004491198",{"2":{"244":1}}],["0044189203",{"2":{"77":1}}],["004043682",{"2":{"244":1}}],["004047669",{"2":{"77":1}}],["004061943",{"2":{"244":1}}],["00407581",{"2":{"179":1}}],["004119421",{"2":{"244":1}}],["004116789",{"2":{"244":1}}],["0041674743",{"2":{"106":1}}],["0041674753",{"2":{"106":1}}],["004169049",{"2":{"77":1}}],["004144425",{"2":{"77":1}}],["001398915",{"2":{"244":1}}],["00139126",{"2":{"77":1}}],["001329915",{"2":{"244":1}}],["001325711",{"2":{"244":1}}],["001325290",{"2":{"244":1}}],["001367094",{"2":{"244":1}}],["001362987",{"2":{"244":1}}],["001362466",{"2":{"244":1}}],["001364744",{"2":{"244":1}}],["001380907",{"2":{"244":1}}],["001386037",{"2":{"244":1}}],["001346401",{"2":{"244":1}}],["001356683",{"2":{"244":1}}],["001336156",{"2":{"244":1}}],["001307026",{"2":{"244":1}}],["001303701",{"2":{"244":1}}],["001302390",{"2":{"244":1}}],["001302087",{"2":{"244":1}}],["001300987",{"2":{"244":1}}],["001309370",{"2":{"244":1}}],["001305370",{"2":{"244":1}}],["001636085",{"2":{"244":1}}],["001639404",{"2":{"244":1}}],["001659834",{"2":{"244":1}}],["001658659",{"2":{"244":1}}],["001672024",{"2":{"244":1}}],["001671217",{"2":{"244":1}}],["001649067",{"2":{"244":1}}],["001608545",{"2":{"244":1}}],["001694928",{"2":{"244":1}}],["001690981",{"2":{"244":1}}],["001615586",{"2":{"244":1}}],["001611185",{"2":{"244":1}}],["00154755280313e",{"2":{"258":1}}],["001541014",{"2":{"244":1}}],["001592739",{"2":{"244":1}}],["001594031",{"2":{"244":1}}],["001531350",{"2":{"244":1}}],["0015313211",{"2":{"77":1}}],["001513747",{"2":{"244":1}}],["001510418",{"2":{"244":1}}],["001517879",{"2":{"244":1}}],["001560361",{"2":{"244":1}}],["001566568",{"2":{"244":1}}],["001564218",{"2":{"244":1}}],["001571818",{"2":{"244":1}}],["001",{"2":{"208":1}}],["001093546",{"2":{"244":1}}],["001014936",{"2":{"244":1}}],["001011056",{"2":{"244":1}}],["00101147",{"2":{"179":1}}],["001068081",{"2":{"244":1}}],["001060384",{"2":{"244":1}}],["001060399",{"2":{"244":1}}],["00106574",{"2":{"77":1}}],["001085206",{"2":{"244":1}}],["001085730",{"2":{"244":1}}],["001085884",{"2":{"244":1}}],["001089040",{"2":{"244":1}}],["001008533",{"2":{"244":1}}],["001009728",{"2":{"244":1}}],["001049175",{"2":{"244":1}}],["001044580",{"2":{"244":1}}],["001041825",{"2":{"244":1}}],["001047856",{"2":{"244":1}}],["001053359",{"2":{"244":1}}],["001050717",{"2":{"244":1}}],["001058994",{"2":{"244":1}}],["001059842",{"2":{"244":1}}],["0010350421",{"2":{"232":1}}],["001f0",{"2":{"78":1,"107":1,"217":1,"228":1}}],["0012",{"2":{"249":1}}],["001202180",{"2":{"244":1}}],["001209691",{"2":{"244":1}}],["001203537",{"2":{"244":1}}],["001257007",{"2":{"244":1}}],["001258897",{"2":{"244":1}}],["001272172",{"2":{"244":1}}],["001279382",{"2":{"244":1}}],["001228882",{"2":{"244":1}}],["001224264",{"2":{"244":1}}],["001240770",{"2":{"244":1}}],["001243982",{"2":{"244":1}}],["001243673",{"2":{"244":1}}],["001241080",{"2":{"244":1}}],["001245214",{"2":{"244":1}}],["001218763",{"2":{"244":1}}],["001215764",{"2":{"244":1}}],["001292084",{"2":{"244":1}}],["00126273",{"2":{"77":1}}],["001289588",{"2":{"244":1}}],["00128162",{"2":{"77":1}}],["0012843215",{"2":{"77":1}}],["001886566",{"2":{"244":1}}],["001884541",{"2":{"244":1}}],["001864895",{"2":{"244":1}}],["001825325",{"2":{"244":1}}],["001837625",{"2":{"244":1}}],["001894138",{"2":{"244":1}}],["001845157",{"2":{"244":1}}],["001849154",{"2":{"244":1}}],["001848093",{"2":{"244":1}}],["001879478",{"2":{"244":1}}],["001871931",{"2":{"244":1}}],["00187748",{"2":{"77":1}}],["0018058",{"2":{"77":1}}],["0018510937",{"2":{"77":1}}],["001448896",{"2":{"244":1}}],["001448493",{"2":{"244":1}}],["001439294",{"2":{"244":1}}],["001479216",{"2":{"244":1}}],["001498525",{"2":{"244":1}}],["001494603",{"2":{"244":1}}],["001492234",{"2":{"244":1}}],["001485110",{"2":{"244":1}}],["001454655",{"2":{"244":1}}],["00145586",{"2":{"77":1}}],["001404967",{"2":{"244":1}}],["001409405",{"2":{"244":1}}],["001402911",{"2":{"244":1}}],["00140763051",{"2":{"78":1}}],["0014175134",{"2":{"77":1}}],["001185708",{"2":{"244":1}}],["001182971",{"2":{"244":1}}],["001168622",{"2":{"244":1}}],["001160439",{"2":{"244":1}}],["001101400",{"2":{"244":1}}],["001103305",{"2":{"244":1}}],["001110448",{"2":{"244":1}}],["001113559",{"2":{"244":1}}],["001141232",{"2":{"244":1}}],["001143823",{"2":{"244":1}}],["001148389",{"2":{"244":1}}],["001171167",{"2":{"244":1}}],["001179150",{"2":{"244":1}}],["00117228",{"2":{"77":1}}],["001126162",{"2":{"244":1}}],["001191893",{"2":{"244":1}}],["001197650",{"2":{"244":1}}],["001136703",{"2":{"244":1}}],["001135292",{"2":{"244":1}}],["001135202",{"2":{"244":1}}],["001138862",{"2":{"244":1}}],["001153235",{"2":{"244":1}}],["001154157",{"2":{"244":1}}],["00115296",{"2":{"77":1}}],["0017205f",{"2":{"256":1}}],["001720798",{"2":{"244":1}}],["001728887",{"2":{"244":1}}],["001789054",{"2":{"244":1}}],["001770784",{"2":{"244":1}}],["001747428",{"2":{"244":1}}],["001717428",{"2":{"244":1}}],["001714717",{"2":{"244":1}}],["001715113",{"2":{"244":1}}],["001764537",{"2":{"244":1}}],["001731674",{"2":{"244":1}}],["001730468",{"2":{"244":1}}],["0017369908",{"2":{"77":1}}],["001798752",{"2":{"244":1}}],["001798573",{"2":{"244":1}}],["001792779",{"2":{"244":1}}],["0017900232",{"2":{"77":1}}],["00175152",{"2":{"77":1}}],["00170922",{"2":{"77":1}}],["001930058",{"2":{"244":1}}],["001994215",{"2":{"244":1}}],["001959185",{"2":{"244":1}}],["0019504696",{"2":{"77":1}}],["001972227",{"2":{"244":1}}],["001975382",{"2":{"244":1}}],["001970405",{"2":{"244":1}}],["001973281",{"2":{"77":1}}],["001919697",{"2":{"244":1}}],["001913586",{"2":{"244":1}}],["0019153744",{"2":{"77":1}}],["001940841",{"2":{"244":1}}],["001982885",{"2":{"244":1}}],["00198415",{"2":{"179":1}}],["00198767",{"2":{"77":1}}],["0019262917",{"2":{"77":1}}],["002989073",{"2":{"244":1}}],["002972272",{"2":{"244":1}}],["002935605",{"2":{"244":1}}],["002511334",{"2":{"244":1}}],["002511769",{"2":{"244":1}}],["002553018",{"2":{"244":1}}],["002556076",{"2":{"244":1}}],["002370826366617e",{"2":{"258":1}}],["002371583",{"2":{"244":1}}],["00232f",{"2":{"256":1}}],["002343247",{"2":{"244":1}}],["002318957",{"2":{"244":1}}],["002301548",{"2":{"244":1}}],["002303589",{"2":{"244":1}}],["002304791",{"2":{"244":1}}],["002395016",{"2":{"244":1}}],["002358936",{"2":{"244":1}}],["002834306",{"2":{"244":1}}],["002806861",{"2":{"244":1}}],["002853032",{"2":{"244":1}}],["0028297466",{"2":{"106":1}}],["0028297475",{"2":{"106":1}}],["002670718",{"2":{"244":1}}],["002677995",{"2":{"244":1}}],["002668741",{"2":{"244":1}}],["002668936",{"2":{"244":1}}],["002606506",{"2":{"244":1}}],["002613829",{"2":{"244":1}}],["002696904",{"2":{"244":1}}],["002695386",{"2":{"244":1}}],["0026271285",{"2":{"117":1}}],["002236427",{"2":{"244":1}}],["002234935",{"2":{"244":1}}],["0022308934",{"2":{"77":1}}],["002276814",{"2":{"244":1}}],["002210422",{"2":{"244":1}}],["002218451",{"2":{"244":1}}],["002247394",{"2":{"244":1}}],["0022497",{"2":{"232":1}}],["002262217",{"2":{"244":1}}],["002201537",{"2":{"244":1}}],["002068657",{"2":{"244":1}}],["002089650",{"2":{"244":1}}],["002081697",{"2":{"244":1}}],["002059042",{"2":{"244":1}}],["0020579377",{"2":{"106":1}}],["002057937",{"2":{"106":1}}],["002023818",{"2":{"244":1}}],["002015869",{"2":{"244":1}}],["002019164",{"2":{"244":1}}],["002037087",{"2":{"244":1}}],["00209504",{"2":{"77":1}}],["002752088",{"2":{"244":1}}],["00270753",{"2":{"77":1}}],["0027386106",{"2":{"77":1}}],["0027955552",{"2":{"77":1}}],["002155375",{"2":{"244":1}}],["002129085",{"2":{"244":1}}],["002125610",{"2":{"244":1}}],["002123826",{"2":{"244":1}}],["0021203319",{"2":{"77":1}}],["00219681",{"2":{"77":1}}],["00214946",{"2":{"77":1}}],["002106787",{"2":{"77":1}}],["002417301",{"2":{"244":1}}],["002489866",{"2":{"244":1}}],["002474443",{"2":{"244":1}}],["002448601",{"2":{"244":1}}],["0024971329",{"2":{"77":1}}],["0024323382",{"2":{"77":1}}],["003299790",{"2":{"244":1}}],["003847855",{"2":{"244":1}}],["003813077",{"2":{"244":1}}],["0038745",{"2":{"77":1}}],["003409335099896e",{"2":{"258":1}}],["003463188",{"2":{"244":1}}],["003488734",{"2":{"244":1}}],["0034101289",{"2":{"77":1}}],["003329928",{"2":{"244":1}}],["003180855",{"2":{"244":1}}],["003136263",{"2":{"244":1}}],["003132161",{"2":{"244":1}}],["0031376407",{"2":{"77":1}}],["003162773",{"2":{"244":1}}],["003125851",{"2":{"244":1}}],["003911218",{"2":{"244":1}}],["003920683",{"2":{"244":1}}],["00390526722",{"2":{"78":1}}],["003504427",{"2":{"244":1}}],["003519468",{"2":{"244":1}}],["00351666",{"2":{"77":1}}],["003553676",{"2":{"244":1}}],["003585588",{"2":{"244":1}}],["003658900",{"2":{"244":1}}],["003630795",{"2":{"244":1}}],["0036810287",{"2":{"106":1}}],["0036810301",{"2":{"106":1}}],["00364828",{"2":{"77":1}}],["003095849",{"2":{"244":1}}],["003031955",{"2":{"244":1}}],["003061282",{"2":{"244":1}}],["00306169",{"2":{"77":1}}],["003071612",{"2":{"244":1}}],["003027094",{"2":{"77":1}}],["0037205f",{"2":{"256":1}}],["0037725805",{"2":{"77":1}}],["003750000000000005",{"2":{"53":1}}],["000",{"2":{"250":1}}],["000846881",{"2":{"244":1}}],["000847362",{"2":{"244":1}}],["0008224149504703233",{"2":{"258":1}}],["000826713",{"2":{"244":1}}],["000826687",{"2":{"244":1}}],["000824789",{"2":{"244":1}}],["000868335",{"2":{"244":1}}],["000866074",{"2":{"244":1}}],["000864926",{"2":{"244":1}}],["000835373",{"2":{"244":1}}],["000835131",{"2":{"244":1}}],["000831873",{"2":{"244":1}}],["000887047",{"2":{"244":1}}],["000883626",{"2":{"244":1}}],["000881248",{"2":{"244":1}}],["000873007",{"2":{"244":1}}],["000871024327",{"2":{"78":1}}],["0008989226899086836",{"2":{"258":1}}],["000897480944822825",{"2":{"258":1}}],["0008930617061522622",{"2":{"258":1}}],["000893347",{"2":{"244":1}}],["000892342",{"2":{"244":1}}],["000892691",{"2":{"244":1}}],["000890729",{"2":{"244":1}}],["000894285",{"2":{"244":1}}],["000857553",{"2":{"244":1}}],["0008023907532141756",{"2":{"258":1}}],["000809421",{"2":{"244":1}}],["000805100",{"2":{"244":1}}],["000807529",{"2":{"244":1}}],["0008185306631164176",{"2":{"258":1}}],["000816396",{"2":{"244":1}}],["000814481",{"2":{"244":1}}],["000043311",{"2":{"244":1}}],["000046194",{"2":{"244":1}}],["000044412",{"2":{"244":1}}],["000098905",{"2":{"244":1}}],["000098134",{"2":{"244":1}}],["000092742",{"2":{"244":1}}],["000094398",{"2":{"244":1}}],["000097348",{"2":{"244":1}}],["000097578",{"2":{"244":1}}],["000097049",{"2":{"244":1}}],["000095254",{"2":{"244":1}}],["000058640",{"2":{"244":1}}],["000056836",{"2":{"244":1}}],["000059547",{"2":{"244":1}}],["000052356",{"2":{"244":1}}],["000067247",{"2":{"244":1}}],["000088090",{"2":{"244":1}}],["000086152",{"2":{"244":1}}],["000089840",{"2":{"244":1}}],["000083818",{"2":{"244":1}}],["000083883",{"2":{"244":1}}],["000085040",{"2":{"244":1}}],["000081115",{"2":{"244":1}}],["000087234",{"2":{"244":1}}],["000079111",{"2":{"244":1}}],["000070152",{"2":{"244":1}}],["000075086",{"2":{"244":1}}],["000077215",{"2":{"244":1}}],["000071670",{"2":{"244":1}}],["00000",{"2":{"187":100,"217":18,"219":3}}],["00022404868707168502",{"2":{"258":1}}],["00022406709667843232",{"2":{"258":1}}],["00022406431",{"2":{"256":1}}],["00022491736075073136",{"2":{"258":1}}],["0002268154333976734",{"2":{"258":1}}],["00022681341",{"2":{"256":1}}],["00022788436",{"2":{"256":1}}],["00022923113944772652",{"2":{"258":1}}],["00022923049",{"2":{"256":1}}],["00022985375926535572",{"2":{"258":1}}],["0002298531",{"2":{"256":1}}],["000229884",{"2":{"244":1}}],["0002460896645807945",{"2":{"258":1}}],["00024482527590054617",{"2":{"258":1}}],["00024482232",{"2":{"256":1}}],["0002405210130246256",{"2":{"258":1}}],["00024051977",{"2":{"256":1}}],["00024784683626402274",{"2":{"258":1}}],["00024784482",{"2":{"256":1}}],["0002482421229337115",{"2":{"258":1}}],["00024824127",{"2":{"256":1}}],["00024892094601913496",{"2":{"258":1}}],["0002489197",{"2":{"256":1}}],["00024135885131153673",{"2":{"258":1}}],["00024135718",{"2":{"256":1}}],["00024108002913641536",{"2":{"258":1}}],["00024108063",{"2":{"256":1}}],["000242183",{"2":{"244":1}}],["000242905",{"2":{"244":1}}],["000242692",{"2":{"244":1}}],["0002671154860365163",{"2":{"258":1}}],["00026578090844042306",{"2":{"258":1}}],["00026578025",{"2":{"256":1}}],["0002607891957749194",{"2":{"258":1}}],["0002607908",{"2":{"256":1}}],["00026127356845255187",{"2":{"258":1}}],["00026127408",{"2":{"256":1}}],["0002614535756439225",{"2":{"258":1}}],["0002614541",{"2":{"256":1}}],["0002610481817646952",{"2":{"258":1}}],["0002610488",{"2":{"256":1}}],["000263541",{"2":{"244":1}}],["000264404",{"2":{"244":1}}],["000269295",{"2":{"244":1}}],["0002744319173546264",{"2":{"258":1}}],["00027568385565150753",{"2":{"258":1}}],["000275985",{"2":{"244":1}}],["000271556",{"2":{"244":1}}],["000277635",{"2":{"244":1}}],["000270049",{"2":{"244":1}}],["000270824",{"2":{"244":1}}],["000273393",{"2":{"244":1}}],["00020038443210656232",{"2":{"258":1}}],["00020038383",{"2":{"256":1}}],["00020467261793479338",{"2":{"258":1}}],["00020466995",{"2":{"256":1}}],["0002045495991517601",{"2":{"258":1}}],["00020454697",{"2":{"256":1}}],["000204869",{"2":{"244":1}}],["00020154964761464605",{"2":{"258":1}}],["00020154905",{"2":{"256":1}}],["00020102633",{"2":{"256":1}}],["000201183",{"2":{"244":1}}],["000208557849988628",{"2":{"258":1}}],["00020855725",{"2":{"256":1}}],["00020834956310056393",{"2":{"258":1}}],["00020834891",{"2":{"256":1}}],["0002084776267331782",{"2":{"258":1}}],["000208475",{"2":{"256":1}}],["00020849705",{"2":{"146":1}}],["00020346984936233685",{"2":{"258":1}}],["00020347247",{"2":{"256":1}}],["00020326738959187797",{"2":{"258":1}}],["00020326376",{"2":{"256":1}}],["00020378697243217167",{"2":{"258":1}}],["00020378857",{"2":{"256":1}}],["00020399482",{"2":{"256":1}}],["00020943673717939665",{"2":{"258":1}}],["0002094385",{"2":{"256":1}}],["000209925",{"2":{"244":1}}],["0002051364886438361",{"2":{"258":1}}],["00020513525",{"2":{"256":1}}],["00020576113187342084",{"2":{"258":1}}],["00020575941",{"2":{"256":1}}],["000205284",{"2":{"244":1}}],["00020679663074566048",{"2":{"258":1}}],["0002067958",{"2":{"256":1}}],["00020606276550485159",{"2":{"258":1}}],["00020606014",{"2":{"256":1}}],["000206055",{"2":{"244":1}}],["000206944",{"2":{"244":1}}],["00023729097376672882",{"2":{"258":1}}],["000231588849364864",{"2":{"258":1}}],["00023158824",{"2":{"256":1}}],["00023573439036328377",{"2":{"258":1}}],["00023573374",{"2":{"256":1}}],["00023374535",{"2":{"256":1}}],["00023276656004835546",{"2":{"258":1}}],["00023276742",{"2":{"256":1}}],["00023244778",{"2":{"256":1}}],["00023230375",{"2":{"256":1}}],["00023091105158523368",{"2":{"258":1}}],["00023090826",{"2":{"256":1}}],["000230181",{"2":{"244":1}}],["0002398825145134938",{"2":{"258":1}}],["000239237",{"2":{"244":1}}],["000239130808",{"2":{"78":1}}],["0002559040862068346",{"2":{"258":1}}],["00025590206",{"2":{"256":1}}],["00025069248141661675",{"2":{"258":1}}],["0002506956",{"2":{"256":1}}],["000250679",{"2":{"244":1}}],["000257633",{"2":{"244":1}}],["000253915",{"2":{"244":1}}],["00025326014",{"2":{"146":1}}],["00028777974423444203",{"2":{"258":1}}],["0002871830221809317",{"2":{"258":1}}],["00028718126",{"2":{"256":1}}],["000287677",{"2":{"244":1}}],["00028989690526936494",{"2":{"258":1}}],["00028989412",{"2":{"256":1}}],["000289123",{"2":{"244":1}}],["000288360",{"2":{"244":1}}],["000280093",{"2":{"244":1}}],["000286281",{"2":{"244":1}}],["00021678054810414323",{"2":{"258":1}}],["000216420",{"2":{"244":1}}],["00021033249636811404",{"2":{"258":1}}],["00021033199",{"2":{"256":1}}],["00021257582794474437",{"2":{"258":1}}],["00021257819",{"2":{"256":1}}],["00021554910660425",{"2":{"258":1}}],["00021546796989567625",{"2":{"258":1}}],["00021547108",{"2":{"256":1}}],["000215771",{"2":{"244":1}}],["00021108418576964766",{"2":{"258":1}}],["00021108685",{"2":{"256":1}}],["00021145534359898768",{"2":{"258":1}}],["0002114561",{"2":{"256":1}}],["0002115731164177468",{"2":{"258":1}}],["00021157016",{"2":{"256":1}}],["00021153228722677796",{"2":{"258":1}}],["00021153178",{"2":{"256":1}}],["000211123",{"2":{"244":1}}],["00021722840239233896",{"2":{"258":1}}],["00021723151",{"2":{"256":1}}],["00021714670600684853",{"2":{"258":1}}],["00021714495",{"2":{"256":1}}],["000217459",{"2":{"244":1}}],["0002141998426323734",{"2":{"258":1}}],["00021465863919734152",{"2":{"258":1}}],["00021466143",{"2":{"256":1}}],["000214095",{"2":{"244":1}}],["000214067404",{"2":{"78":1}}],["00021360929369762725",{"2":{"258":1}}],["0002136077",{"2":{"256":1}}],["00021372428879012443",{"2":{"258":1}}],["00021372396",{"2":{"256":1}}],["000213564",{"2":{"244":1}}],["000213855",{"2":{"244":1}}],["000218087",{"2":{"244":1}}],["0002931727880915839",{"2":{"258":1}}],["0002983343139061395",{"2":{"258":1}}],["000298148",{"2":{"244":1}}],["000291289",{"2":{"244":1}}],["000296087",{"2":{"244":1}}],["00029203",{"2":{"179":1}}],["000410463",{"2":{"244":1}}],["000412207",{"2":{"244":1}}],["000412151",{"2":{"244":1}}],["000412629",{"2":{"244":1}}],["000454265",{"2":{"244":1}}],["000454922556",{"2":{"78":1}}],["000455096",{"2":{"244":1}}],["00040863465278771955",{"2":{"258":1}}],["000408040",{"2":{"244":1}}],["000405329",{"2":{"244":1}}],["000401171",{"2":{"244":1}}],["000490567",{"2":{"244":1}}],["000485339",{"2":{"244":1}}],["000489585",{"2":{"244":1}}],["000486189",{"2":{"244":1}}],["000482833",{"2":{"244":1}}],["000482373",{"2":{"244":1}}],["000484853",{"2":{"244":1}}],["000421408",{"2":{"244":1}}],["000421422",{"2":{"244":1}}],["000426714",{"2":{"244":1}}],["000426463",{"2":{"244":1}}],["000425117",{"2":{"244":1}}],["000422994",{"2":{"244":1}}],["000420685",{"2":{"244":1}}],["00046118253009551705",{"2":{"258":1}}],["000461797",{"2":{"244":1}}],["000461412",{"2":{"244":1}}],["000460401",{"2":{"244":1}}],["00046014786",{"2":{"145":1}}],["000477057",{"2":{"244":1}}],["000474958",{"2":{"244":1}}],["000473843",{"2":{"244":1}}],["000435325",{"2":{"244":1}}],["000432069",{"2":{"244":1}}],["000431564",{"2":{"244":1}}],["000438132",{"2":{"77":1}}],["000449657",{"2":{"244":1}}],["000445312",{"2":{"244":1}}],["000444564",{"2":{"244":1}}],["000448536",{"2":{"244":1}}],["000447986",{"2":{"244":1}}],["000442930",{"2":{"244":1}}],["000441076729",{"2":{"78":1}}],["0006285802307338592",{"2":{"258":1}}],["000620959",{"2":{"244":1}}],["000622923",{"2":{"244":1}}],["000624455",{"2":{"244":1}}],["0006001426243246246",{"2":{"258":1}}],["000600817",{"2":{"244":1}}],["000609521",{"2":{"77":1}}],["0006771698666891329",{"2":{"258":1}}],["000673423",{"2":{"244":1}}],["000675569",{"2":{"244":1}}],["0006913845436001116",{"2":{"258":1}}],["000691815",{"2":{"244":1}}],["0006982017109555845",{"2":{"258":1}}],["000694873",{"2":{"244":1}}],["000699265",{"2":{"77":1}}],["000635906",{"2":{"244":1}}],["000637475",{"2":{"244":1}}],["0006482839826710184",{"2":{"258":1}}],["000648518",{"2":{"244":1}}],["000641742644755058",{"2":{"258":1}}],["000641108",{"2":{"244":1}}],["0006477029338901213",{"2":{"258":1}}],["000646517344881e",{"2":{"258":1}}],["000645338",{"2":{"244":1}}],["000649345",{"2":{"244":1}}],["000640270",{"2":{"244":1}}],["0006512250660108998",{"2":{"258":1}}],["0006514905371313426",{"2":{"258":1}}],["0006529859924286357",{"2":{"257":1}}],["000652327",{"2":{"244":1}}],["000652603",{"2":{"244":1}}],["000650020",{"2":{"244":1}}],["000659882",{"2":{"244":1}}],["0006651773484053201",{"2":{"258":1}}],["0006648955565676422",{"2":{"258":1}}],["000664698",{"2":{"244":1}}],["000662921138863801",{"2":{"258":1}}],["000667115",{"2":{"244":1}}],["000667028",{"2":{"77":1}}],["000666251",{"2":{"244":1}}],["000666125",{"2":{"244":1}}],["0006140691446771656",{"2":{"258":1}}],["000613936",{"2":{"244":1}}],["000612554",{"2":{"244":1}}],["000612726",{"2":{"77":1}}],["000619220",{"2":{"244":1}}],["000618138",{"2":{"244":1}}],["000687853",{"2":{"244":1}}],["000685339",{"2":{"244":1}}],["000685782",{"2":{"244":1}}],["000688489",{"2":{"244":1}}],["00068473816",{"2":{"145":1}}],["000689046",{"2":{"77":1}}],["000382126",{"2":{"244":1}}],["000386392",{"2":{"244":1}}],["000381213",{"2":{"244":1}}],["000389668",{"2":{"244":1}}],["000387762",{"2":{"244":1}}],["000314013",{"2":{"244":1}}],["000314716",{"2":{"244":1}}],["000312205",{"2":{"244":1}}],["000317277",{"2":{"244":1}}],["000317301",{"2":{"244":1}}],["000313576",{"2":{"244":1}}],["000311768",{"2":{"244":1}}],["000310122",{"2":{"244":1}}],["0003650959602604641",{"2":{"258":1}}],["000362759",{"2":{"244":1}}],["0003676f",{"2":{"256":1}}],["000367492",{"2":{"244":1}}],["000367264",{"2":{"244":1}}],["000369106",{"2":{"244":1}}],["000363892",{"2":{"244":1}}],["000368751",{"2":{"244":1}}],["000366831",{"2":{"244":1}}],["000366380",{"2":{"244":1}}],["00036629336",{"2":{"78":1}}],["0003237640460638284",{"2":{"258":1}}],["000323999",{"2":{"244":1}}],["000322635",{"2":{"244":1}}],["000324281",{"2":{"244":1}}],["000321921",{"2":{"244":1}}],["000326662",{"2":{"244":1}}],["000326938",{"2":{"244":1}}],["000376061",{"2":{"244":1}}],["000379668",{"2":{"244":1}}],["000379169",{"2":{"244":1}}],["000378937",{"2":{"244":1}}],["000378884",{"2":{"244":1}}],["000374305",{"2":{"244":1}}],["000374875",{"2":{"77":1}}],["000377450",{"2":{"244":1}}],["000372513",{"2":{"244":1}}],["0003513619042523084",{"2":{"258":1}}],["000351864",{"2":{"244":1}}],["0003556723695087773",{"2":{"258":1}}],["000350503",{"2":{"244":1}}],["000350560",{"2":{"244":1}}],["000353724",{"2":{"244":1}}],["000354412",{"2":{"244":1}}],["000359855",{"2":{"244":1}}],["00030288317297621684",{"2":{"258":1}}],["00030288193",{"2":{"256":1}}],["000301691",{"2":{"244":1}}],["000306839",{"2":{"244":1}}],["000306047",{"2":{"244":1}}],["000305183",{"2":{"244":1}}],["000300986",{"2":{"244":1}}],["000300984",{"2":{"244":1}}],["000308547",{"2":{"244":1}}],["000307154",{"2":{"244":2}}],["000335865",{"2":{"244":1}}],["000334467",{"2":{"244":1}}],["000330688",{"2":{"244":1}}],["000330550",{"2":{"244":1}}],["000333685",{"2":{"244":1}}],["0003478199629957018",{"2":{"258":1}}],["00034781685",{"2":{"256":1}}],["000347842",{"2":{"244":1}}],["000346362",{"2":{"244":1}}],["000348117",{"2":{"244":1}}],["000344165",{"2":{"244":1}}],["000344849",{"2":{"244":1}}],["000345872",{"2":{"244":1}}],["0003935834320404575",{"2":{"258":1}}],["00039841043602634435",{"2":{"258":1}}],["000392804",{"2":{"244":1}}],["0003920444",{"2":{"77":1}}],["000395994",{"2":{"244":1}}],["000391877",{"2":{"244":1}}],["000391076",{"2":{"77":1}}],["0005536322785323924",{"2":{"258":1}}],["000559556",{"2":{"244":1}}],["000554417",{"2":{"244":1}}],["000554349",{"2":{"244":1}}],["000554918",{"2":{"77":1}}],["000505657627539614",{"2":{"258":1}}],["000506370",{"2":{"244":1}}],["000503808",{"2":{"244":1}}],["000545968",{"2":{"244":1}}],["000545245",{"2":{"244":1}}],["000549527",{"2":{"244":1}}],["000540564",{"2":{"244":1}}],["000547043",{"2":{"244":1}}],["000547293",{"2":{"244":1}}],["000543516",{"2":{"244":1}}],["0005936614162959661",{"2":{"258":1}}],["000590442",{"2":{"244":1}}],["000595116",{"2":{"244":1}}],["000595709",{"2":{"244":1}}],["000599332",{"2":{"244":1}}],["0005922087",{"2":{"106":1}}],["00059220614",{"2":{"106":1}}],["0005875524598459974",{"2":{"258":1}}],["000583868",{"2":{"244":1}}],["000583094",{"2":{"244":1}}],["000580873",{"2":{"244":1}}],["000580056",{"2":{"244":1}}],["000580705",{"2":{"77":1}}],["000525872",{"2":{"244":1}}],["000525994",{"2":{"244":1}}],["000522715",{"2":{"244":1}}],["000521560",{"2":{"244":1}}],["000512687",{"2":{"244":1}}],["000511645",{"2":{"244":1}}],["000516831",{"2":{"244":1}}],["000516864",{"2":{"244":1}}],["000510494",{"2":{"244":1}}],["000510198",{"2":{"77":1}}],["000575209",{"2":{"244":1}}],["000579479",{"2":{"244":1}}],["000577084",{"2":{"244":1}}],["000574894",{"2":{"77":1}}],["000560744",{"2":{"244":1}}],["000566683",{"2":{"244":1}}],["000561465",{"2":{"244":1}}],["000564703",{"2":{"244":1}}],["000563284",{"2":{"77":1}}],["000539345",{"2":{"244":1}}],["000530431",{"2":{"244":1}}],["000537486",{"2":{"244":1}}],["000536315",{"2":{"244":1}}],["000535839",{"2":{"244":1}}],["00053332",{"2":{"179":1}}],["0005f0",{"2":{"244":1}}],["0007275369507994201",{"2":{"258":1}}],["0007221957419533718",{"2":{"258":1}}],["000726597",{"2":{"244":1}}],["000723986",{"2":{"244":1}}],["000723805",{"2":{"244":1}}],["000710499",{"2":{"244":1}}],["000714949",{"2":{"244":1}}],["0007461286129488532",{"2":{"258":1}}],["000746255",{"2":{"244":1}}],["000740137",{"2":{"244":1}}],["000744279",{"2":{"244":1}}],["000788119",{"2":{"244":1}}],["000780730",{"2":{"244":1}}],["000784697",{"2":{"244":1}}],["000781053",{"2":{"244":1}}],["000734937",{"2":{"244":1}}],["000732894",{"2":{"244":1}}],["000731462",{"2":{"77":1}}],["000764329",{"2":{"244":1}}],["000792289",{"2":{"244":1}}],["000792221",{"2":{"244":1}}],["000798627",{"2":{"244":1}}],["000798481",{"2":{"244":1}}],["000798861",{"2":{"244":1}}],["000796866",{"2":{"244":1}}],["000757746606194418",{"2":{"258":1}}],["000756769",{"2":{"244":1}}],["000754971",{"2":{"244":1}}],["000752965",{"2":{"244":1}}],["000753467",{"2":{"244":1}}],["0007006402569624884",{"2":{"258":1}}],["0007083074698248665",{"2":{"258":1}}],["000708074",{"2":{"244":1}}],["000709375",{"2":{"244":1}}],["000705982",{"2":{"244":1}}],["000707158",{"2":{"244":1}}],["0007713635499856448",{"2":{"258":1}}],["0007798362903145225",{"2":{"258":1}}],["000778712",{"2":{"244":1}}],["000778960",{"2":{"244":1}}],["000776893",{"2":{"244":1}}],["00077649",{"2":{"77":1}}],["0007736237",{"2":{"179":1}}],["00077338",{"2":{"77":1}}],["000925605",{"2":{"244":1}}],["000924496",{"2":{"244":1}}],["000924424",{"2":{"244":1}}],["000963809",{"2":{"244":1}}],["000964463",{"2":{"244":1}}],["000944194",{"2":{"244":1}}],["000942026",{"2":{"77":1}}],["000907963",{"2":{"244":1}}],["000908168",{"2":{"244":1}}],["000901216",{"2":{"244":1}}],["000906830",{"2":{"244":1}}],["000939374",{"2":{"244":1}}],["000938448",{"2":{"244":1}}],["000936727",{"2":{"77":1}}],["000952036",{"2":{"244":1}}],["000994050",{"2":{"244":1}}],["000992391663909964",{"2":{"53":2}}],["000972152",{"2":{"244":1}}],["000971555",{"2":{"244":1}}],["000975201",{"2":{"244":1}}],["000983244",{"2":{"244":1}}],["000987124",{"2":{"244":1}}],["000910802",{"2":{"77":1}}],["00018523919873291076",{"2":{"258":1}}],["00018563338077856912",{"2":{"258":1}}],["00018563253",{"2":{"256":1}}],["0001878828167595362",{"2":{"258":1}}],["0001878797",{"2":{"256":1}}],["00018992414466535698",{"2":{"258":1}}],["00018992538",{"2":{"256":1}}],["00018979269130572327",{"2":{"258":1}}],["00018979193",{"2":{"256":1}}],["0001809469019683818",{"2":{"258":1}}],["00018094428",{"2":{"256":1}}],["00018019285778773963",{"2":{"258":1}}],["00018019084",{"2":{"256":1}}],["00018053753639497483",{"2":{"258":1}}],["00018053921",{"2":{"256":1}}],["0001805462516128645",{"2":{"258":1}}],["00018054691",{"2":{"256":1}}],["00018308068473417853",{"2":{"258":1}}],["00018307757",{"2":{"256":1}}],["00018367340180727918",{"2":{"258":1}}],["00018367275",{"2":{"256":1}}],["0001832826581449361",{"2":{"258":1}}],["00018328545",{"2":{"256":1}}],["0001831156508497774",{"2":{"258":1}}],["00018311598",{"2":{"256":1}}],["00018211911149490352",{"2":{"258":1}}],["00018211977",{"2":{"256":1}}],["00018221662216738526",{"2":{"258":1}}],["00018221501",{"2":{"256":1}}],["000182339",{"2":{"244":1}}],["000186362",{"2":{"244":1}}],["00014672021287855984",{"2":{"258":1}}],["00014671938",{"2":{"256":1}}],["00014176119264001894",{"2":{"258":1}}],["00014176163",{"2":{"256":1}}],["00014260215301932077",{"2":{"258":1}}],["0001424155",{"2":{"256":1}}],["00014298115",{"2":{"256":1}}],["00014259979",{"2":{"256":1}}],["0001438044794446979",{"2":{"258":1}}],["0001438071",{"2":{"256":1}}],["0001432925769623622",{"2":{"258":1}}],["00014329309",{"2":{"256":1}}],["0001435019311767551",{"2":{"258":1}}],["00014350242",{"2":{"256":1}}],["0001406830145914539",{"2":{"258":1}}],["00014068597",{"2":{"256":1}}],["00014088963297254262",{"2":{"258":1}}],["00014088652",{"2":{"256":1}}],["0001404479736172602",{"2":{"258":1}}],["00014044714",{"2":{"256":1}}],["00014050759569733472",{"2":{"258":1}}],["0001405082",{"2":{"256":1}}],["00014051892045319808",{"2":{"258":1}}],["00014052064",{"2":{"256":1}}],["00014706112790922307",{"2":{"258":1}}],["00014706138",{"2":{"256":1}}],["0001474218262319864",{"2":{"258":1}}],["00014742208",{"2":{"256":1}}],["0001473621241532582",{"2":{"258":1}}],["00014736147",{"2":{"256":1}}],["0001476289906461012",{"2":{"258":1}}],["00014763075",{"2":{"256":1}}],["0001490338564274168",{"2":{"258":1}}],["0001490329",{"2":{"256":1}}],["00014910514200039036",{"2":{"258":1}}],["00014910278",{"2":{"256":1}}],["00014493874363065735",{"2":{"258":1}}],["00014493923",{"2":{"256":1}}],["000144504",{"2":{"244":1}}],["000148600576923457",{"2":{"258":1}}],["000148993",{"2":{"244":1}}],["00014859898",{"2":{"256":1}}],["000148591",{"2":{"244":1}}],["000148517",{"2":{"244":1}}],["000148092",{"2":{"244":1}}],["00011310309275915098",{"2":{"258":1}}],["00011310014",{"2":{"256":1}}],["00011397560168059937",{"2":{"258":1}}],["00011397646",{"2":{"256":1}}],["00011346335045607408",{"2":{"258":1}}],["000113464965",{"2":{"256":1}}],["00011307021",{"2":{"256":1}}],["00011825127195725942",{"2":{"258":1}}],["000118252945",{"2":{"256":1}}],["00011879198000920598",{"2":{"258":1}}],["000118789314",{"2":{"256":1}}],["00011893893517623317",{"2":{"258":1}}],["000118937336",{"2":{"256":1}}],["00011042508839943037",{"2":{"258":1}}],["000110427754",{"2":{"256":1}}],["00011024306368558402",{"2":{"258":1}}],["00011023943",{"2":{"256":1}}],["000112948647492945",{"2":{"258":1}}],["00011294925",{"2":{"256":1}}],["00011219705257667261",{"2":{"258":1}}],["000112197886",{"2":{"256":1}}],["000112325",{"2":{"244":1}}],["00011409716751033118",{"2":{"258":1}}],["0001140298720201378",{"2":{"258":1}}],["00011402708",{"2":{"256":1}}],["00011465894",{"2":{"256":1}}],["00011414959328836457",{"2":{"258":1}}],["00011414899",{"2":{"256":1}}],["00011410012",{"2":{"256":1}}],["00011487463569794437",{"2":{"258":1}}],["000114871524",{"2":{"256":1}}],["00011489574601808303",{"2":{"258":1}}],["0001148964",{"2":{"256":1}}],["00011483243185852522",{"2":{"258":1}}],["00011483148",{"2":{"256":1}}],["000111545",{"2":{"256":1}}],["00011144529840375548",{"2":{"258":1}}],["00011144479",{"2":{"256":1}}],["00011149531230776743",{"2":{"258":1}}],["000111494875",{"2":{"256":1}}],["00011113947141571466",{"2":{"258":1}}],["00011113887",{"2":{"256":1}}],["00011116959054188046",{"2":{"258":1}}],["00011116908",{"2":{"256":1}}],["00011115387503581927",{"2":{"258":1}}],["00011115292",{"2":{"256":1}}],["00011138957846918396",{"2":{"258":1}}],["000111388974",{"2":{"256":1}}],["00011133964128162024",{"2":{"258":1}}],["00011133804",{"2":{"256":1}}],["00011534390159821088",{"2":{"258":1}}],["000115343246",{"2":{"256":1}}],["00011540063843515129",{"2":{"258":1}}],["0001154024",{"2":{"256":1}}],["00011575050793055044",{"2":{"258":1}}],["000115748146",{"2":{"256":1}}],["000115742",{"2":{"244":1}}],["00011501850022244405",{"2":{"258":1}}],["00011501487",{"2":{"256":1}}],["00011504296869901559",{"2":{"258":1}}],["00011504457",{"2":{"256":1}}],["00011752773524101682",{"2":{"258":1}}],["00011752606",{"2":{"256":1}}],["00011787550734954773",{"2":{"258":1}}],["00011787723",{"2":{"256":1}}],["000117897",{"2":{"244":1}}],["0001173811674963009",{"2":{"258":1}}],["00011738168",{"2":{"256":1}}],["00011733128181995121",{"2":{"258":1}}],["00011733096",{"2":{"256":1}}],["00011739355553536533",{"2":{"258":1}}],["00011739388",{"2":{"256":1}}],["00011940793950784414",{"2":{"258":1}}],["00011940488",{"2":{"256":1}}],["00011933259684251707",{"2":{"258":1}}],["00011933267",{"2":{"256":1}}],["00011969898237536553",{"2":{"258":1}}],["00011969867",{"2":{"256":1}}],["00011918021179500511",{"2":{"258":1}}],["00011915152",{"2":{"256":1}}],["000119179356",{"2":{"256":1}}],["00011640213568755016",{"2":{"258":1}}],["000116407",{"2":{"244":1}}],["00011627310810244291",{"2":{"258":1}}],["00011627484",{"2":{"256":1}}],["00011688509861339084",{"2":{"258":1}}],["00011688338",{"2":{"256":1}}],["000116399024",{"2":{"256":1}}],["00011615114799592718",{"2":{"258":1}}],["00011615122",{"2":{"256":1}}],["00011611161755373647",{"2":{"258":1}}],["00011611364",{"2":{"256":1}}],["0001792947104920181",{"2":{"258":1}}],["00017377811610154093",{"2":{"258":1}}],["00017377545",{"2":{"256":1}}],["00017366600661280175",{"2":{"258":1}}],["00017366803",{"2":{"256":1}}],["000173648",{"2":{"244":1}}],["00017848797095547202",{"2":{"258":1}}],["0001784863",{"2":{"256":1}}],["0001781363402696047",{"2":{"258":1}}],["00017813583",{"2":{"256":1}}],["00017860869197075727",{"2":{"258":1}}],["00017860696",{"2":{"256":1}}],["00017636516606650494",{"2":{"258":1}}],["00017651922594719507",{"2":{"258":1}}],["00017652047",{"2":{"256":1}}],["00017698574943633054",{"2":{"258":1}}],["00017698524",{"2":{"256":1}}],["00017642804596096446",{"2":{"258":1}}],["00017642805",{"2":{"256":1}}],["000176415",{"2":{"244":1}}],["00017401066424404033",{"2":{"258":1}}],["00017401333",{"2":{"256":1}}],["0001746790737706161",{"2":{"258":1}}],["00017467847",{"2":{"256":1}}],["00017496838717105307",{"2":{"258":1}}],["00017496663",{"2":{"256":1}}],["00017227842482194194",{"2":{"258":1}}],["00017227576",{"2":{"256":1}}],["00017280460099758978",{"2":{"258":1}}],["00017280485",{"2":{"256":1}}],["00017239979561986312",{"2":{"258":1}}],["00017203413372403965",{"2":{"258":1}}],["0001720305",{"2":{"256":1}}],["0001726965907401498",{"2":{"258":1}}],["00017240063",{"2":{"256":1}}],["00017270022",{"2":{"256":1}}],["00017259505154856278",{"2":{"258":1}}],["0001725941",{"2":{"256":1}}],["00017250242643128175",{"2":{"258":1}}],["00017250188",{"2":{"256":1}}],["00017500171135571485",{"2":{"258":1}}],["00017500338",{"2":{"256":1}}],["00017521538422372607",{"2":{"258":1}}],["00017521593",{"2":{"256":1}}],["0001751779540342266",{"2":{"258":1}}],["00017517559",{"2":{"256":1}}],["00017180809112473725",{"2":{"258":1}}],["00017180503",{"2":{"256":1}}],["00017145255272871558",{"2":{"258":1}}],["00017145288",{"2":{"256":1}}],["00017136620426658485",{"2":{"258":1}}],["00017136613",{"2":{"256":1}}],["0001705264202662507",{"2":{"258":1}}],["00017052738",{"2":{"256":1}}],["000170466",{"2":{"244":1}}],["000170909",{"2":{"244":1}}],["00015056999791557195",{"2":{"258":1}}],["000150136063990197",{"2":{"258":1}}],["00015416799487840587",{"2":{"258":1}}],["00015416865",{"2":{"256":1}}],["0001531485357827098",{"2":{"258":1}}],["00015314828",{"2":{"256":1}}],["0001530667696344408",{"2":{"258":1}}],["00015306505",{"2":{"256":1}}],["00015395948",{"2":{"256":1}}],["00015335342",{"2":{"256":1}}],["00015541871864562236",{"2":{"258":1}}],["00015541828",{"2":{"256":1}}],["0001559643459313135",{"2":{"258":1}}],["0001559641",{"2":{"256":1}}],["0001559074115996235",{"2":{"258":1}}],["00015590824",{"2":{"256":1}}],["0001559706923217688",{"2":{"258":1}}],["00015597348",{"2":{"256":1}}],["000155360",{"2":{"244":1}}],["0001561621153872714",{"2":{"258":1}}],["000156159",{"2":{"256":1}}],["00015603045129291171",{"2":{"258":1}}],["00015601891275662826",{"2":{"258":1}}],["00015601957",{"2":{"256":1}}],["000156025118195224",{"2":{"258":1}}],["00015602878",{"2":{"256":1}}],["00015602457",{"2":{"256":1}}],["00015637557789791322",{"2":{"258":1}}],["0001563755",{"2":{"256":1}}],["0001599588262751778",{"2":{"258":1}}],["00015996007",{"2":{"256":1}}],["0001590628689315487",{"2":{"258":1}}],["00015905981",{"2":{"256":1}}],["000159896391835762",{"2":{"258":1}}],["0001598959",{"2":{"256":1}}],["00015951965",{"2":{"256":1}}],["000159178",{"2":{"244":1}}],["00015741255696405715",{"2":{"258":1}}],["00015741195",{"2":{"256":1}}],["00015791237837018728",{"2":{"258":1}}],["00015791062",{"2":{"256":1}}],["00015708922904846402",{"2":{"258":1}}],["00015708971",{"2":{"256":1}}],["00015710228272780086",{"2":{"258":1}}],["00015710465",{"2":{"256":1}}],["00015786205915009716",{"2":{"258":1}}],["00015786408",{"2":{"256":1}}],["00015729013616119967",{"2":{"258":1}}],["00015723762",{"2":{"256":1}}],["0001572884",{"2":{"256":1}}],["00015255711285469946",{"2":{"258":1}}],["00015255544",{"2":{"256":1}}],["00015212040929975444",{"2":{"258":1}}],["0001521188",{"2":{"256":1}}],["0001521374781068033",{"2":{"258":1}}],["0001521374",{"2":{"256":1}}],["000152438",{"2":{"244":1}}],["00015892369475621366",{"2":{"258":1}}],["0001589268",{"2":{"256":1}}],["00015813822764335032",{"2":{"258":1}}],["0001581374",{"2":{"256":1}}],["000158824",{"2":{"244":1}}],["000158011",{"2":{"244":1}}],["00015179132673431847",{"2":{"258":1}}],["00015179072",{"2":{"256":1}}],["00015167921926669987",{"2":{"258":1}}],["00015165226097897555",{"2":{"258":1}}],["00015165182",{"2":{"256":1}}],["00015168184",{"2":{"256":1}}],["000151947",{"2":{"244":1}}],["00015113852",{"2":{"78":1}}],["00012905169870159122",{"2":{"258":1}}],["00012904903",{"2":{"256":1}}],["000129240",{"2":{"244":1}}],["00012806956394731785",{"2":{"258":1}}],["00012806694",{"2":{"256":1}}],["00012864310069319755",{"2":{"258":1}}],["00012864341",{"2":{"256":1}}],["00012810744635639062",{"2":{"258":1}}],["00012810805",{"2":{"256":1}}],["00012816098768577894",{"2":{"258":1}}],["00012815923",{"2":{"256":1}}],["0001213522601021198",{"2":{"258":1}}],["00012135054",{"2":{"256":1}}],["00012181994518101749",{"2":{"258":1}}],["000121818346",{"2":{"256":1}}],["0001215108062068825",{"2":{"258":1}}],["00012151037",{"2":{"256":1}}],["00012066584800554172",{"2":{"258":1}}],["000120665085",{"2":{"256":1}}],["00012047243166020841",{"2":{"258":1}}],["00012044191153103992",{"2":{"258":1}}],["00012043912",{"2":{"256":1}}],["00012083387293833621",{"2":{"258":1}}],["000120833436",{"2":{"256":1}}],["000120209",{"2":{"244":1}}],["00012310168647743032",{"2":{"258":1}}],["00012310114",{"2":{"256":1}}],["0001237932582145024",{"2":{"258":1}}],["00012379164",{"2":{"256":1}}],["00012386192",{"2":{"256":1}}],["00012397852",{"2":{"256":1}}],["00012246462591772832",{"2":{"258":1}}],["00012246286",{"2":{"256":1}}],["00012204590195253912",{"2":{"258":1}}],["00012204645",{"2":{"256":1}}],["000122752",{"2":{"244":1}}],["00012474154726659566",{"2":{"258":1}}],["00012473844",{"2":{"256":1}}],["00012432185625958184",{"2":{"258":1}}],["00012432237",{"2":{"256":1}}],["00012427559540715012",{"2":{"258":1}}],["00012427552",{"2":{"256":1}}],["00012429585",{"2":{"256":1}}],["00012411661436393897",{"2":{"258":1}}],["00012411928",{"2":{"256":1}}],["0001241965704963434",{"2":{"258":1}}],["00012419421",{"2":{"256":1}}],["00012489618154173332",{"2":{"258":1}}],["00012489593",{"2":{"256":1}}],["000124850436689979",{"2":{"258":1}}],["00012485104",{"2":{"256":1}}],["0001248413506463285",{"2":{"258":1}}],["00012484103",{"2":{"256":1}}],["0001245625624515501",{"2":{"258":1}}],["00012456263",{"2":{"256":1}}],["000124517",{"2":{"244":1}}],["00012658564926435023",{"2":{"258":1}}],["00012658827",{"2":{"256":1}}],["00012628203599664346",{"2":{"258":1}}],["00012628127",{"2":{"256":1}}],["00012691746667751962",{"2":{"258":1}}],["00012691545",{"2":{"256":1}}],["00012687509363684538",{"2":{"258":1}}],["00012687575",{"2":{"256":1}}],["00012680853623328007",{"2":{"258":1}}],["0001268077",{"2":{"256":1}}],["00012642852888899184",{"2":{"258":1}}],["00012642943092091197",{"2":{"258":1}}],["00012642707",{"2":{"256":1}}],["00012643216",{"2":{"256":1}}],["0001261118450201706",{"2":{"258":1}}],["00012611185",{"2":{"256":1}}],["000126193",{"2":{"244":1}}],["000126034",{"2":{"244":1}}],["0001259656733464354",{"2":{"258":1}}],["00012596443",{"2":{"256":1}}],["0001256172271838726",{"2":{"258":1}}],["00012561417",{"2":{"256":1}}],["0001258139879830786",{"2":{"258":1}}],["00012581136",{"2":{"256":1}}],["0001258904479410381",{"2":{"258":1}}],["00012589077",{"2":{"256":1}}],["0001252187109120308",{"2":{"258":1}}],["00012521839",{"2":{"256":1}}],["000125214",{"2":{"244":1}}],["000125504",{"2":{"244":1}}],["00012734968447470352",{"2":{"258":1}}],["00012735029",{"2":{"256":1}}],["00012790382673200554",{"2":{"258":1}}],["00012790448",{"2":{"256":1}}],["00012764699914642993",{"2":{"258":1}}],["00012764876",{"2":{"256":1}}],["00012728528059811163",{"2":{"258":1}}],["00012728852743741338",{"2":{"258":1}}],["00012728853",{"2":{"256":1}}],["00012728834",{"2":{"256":1}}],["000127500",{"2":{"244":1}}],["000127040",{"2":{"244":1}}],["00019062613300871163",{"2":{"258":1}}],["0001906262",{"2":{"256":1}}],["00019318277",{"2":{"256":1}}],["000193357",{"2":{"244":1}}],["00019401008040099318",{"2":{"258":1}}],["00019400741",{"2":{"256":1}}],["00019467959179886857",{"2":{"258":1}}],["0001946802",{"2":{"256":1}}],["00019641761669387614",{"2":{"258":1}}],["00019641886",{"2":{"256":1}}],["00019615531127424067",{"2":{"258":1}}],["00019615894",{"2":{"256":1}}],["00019921713118290995",{"2":{"258":1}}],["00019921537",{"2":{"256":1}}],["00019912244346200657",{"2":{"258":1}}],["00019911939",{"2":{"256":1}}],["00019941878969108163",{"2":{"258":1}}],["00019942185",{"2":{"256":1}}],["00019294186671469824",{"2":{"258":1}}],["00019293881",{"2":{"256":1}}],["00019279813673427136",{"2":{"258":1}}],["00019279975",{"2":{"256":1}}],["000192454",{"2":{"244":1}}],["00019555344448494636",{"2":{"258":1}}],["00019555296",{"2":{"256":1}}],["0001953394671397077",{"2":{"258":1}}],["0001953378",{"2":{"256":1}}],["00019539318484410414",{"2":{"258":1}}],["00019539056",{"2":{"256":1}}],["00019522991570188633",{"2":{"258":1}}],["00019522628",{"2":{"256":1}}],["000195276",{"2":{"244":1}}],["000191858",{"2":{"244":1}}],["000191290",{"2":{"244":1}}],["00013413560728644923",{"2":{"258":1}}],["00013413644",{"2":{"256":1}}],["00013420281",{"2":{"256":1}}],["000134948",{"2":{"244":1}}],["00013301863861666608",{"2":{"258":1}}],["00013301839",{"2":{"256":1}}],["00013361183883167066",{"2":{"258":1}}],["0001336082",{"2":{"256":1}}],["00013334190803679445",{"2":{"258":1}}],["00013334364",{"2":{"256":1}}],["00013518759591871196",{"2":{"258":1}}],["00013518587",{"2":{"256":1}}],["0001350471191472717",{"2":{"258":1}}],["00013504991",{"2":{"256":1}}],["00013503155682621117",{"2":{"258":1}}],["00013502994",{"2":{"256":1}}],["00013920795578303128",{"2":{"258":1}}],["00013921101",{"2":{"256":1}}],["00013990668654778032",{"2":{"258":1}}],["00013990636",{"2":{"256":1}}],["0001394896032615441",{"2":{"258":1}}],["000139489",{"2":{"256":1}}],["00013970565392586398",{"2":{"258":1}}],["00013970489",{"2":{"256":1}}],["00013978591434493068",{"2":{"258":1}}],["00013978432",{"2":{"256":1}}],["0001397165213984028",{"2":{"258":1}}],["00013971627",{"2":{"256":1}}],["00013971561548916717",{"2":{"258":1}}],["00013971738",{"2":{"256":1}}],["00013715422903816277",{"2":{"258":1}}],["00013715128",{"2":{"256":1}}],["000137855812796677",{"2":{"258":1}}],["00013785521",{"2":{"256":1}}],["0001379568291471869",{"2":{"258":1}}],["00013795843",{"2":{"256":1}}],["00013721343",{"2":{"256":1}}],["00013740570301504828",{"2":{"258":1}}],["00013740394",{"2":{"256":1}}],["00013748680994462204",{"2":{"258":1}}],["00013748402",{"2":{"256":1}}],["00013746663066478056",{"2":{"258":1}}],["00013746502",{"2":{"256":1}}],["0001308141393625719",{"2":{"258":1}}],["00013081147",{"2":{"256":1}}],["00013092193061534614",{"2":{"258":1}}],["0001309227",{"2":{"256":1}}],["00013066113856700814",{"2":{"258":1}}],["00013066275",{"2":{"256":1}}],["00013041010066658793",{"2":{"258":1}}],["00013041316",{"2":{"256":1}}],["00013049759016069826",{"2":{"258":1}}],["00013049919",{"2":{"256":1}}],["00013644821893345413",{"2":{"258":1}}],["00013644555",{"2":{"256":1}}],["00013615125482120902",{"2":{"258":1}}],["00013615302",{"2":{"256":1}}],["00013626279083912471",{"2":{"258":1}}],["00013626451",{"2":{"256":1}}],["00013652103547214534",{"2":{"258":1}}],["00013651798",{"2":{"256":1}}],["00013603266239701657",{"2":{"258":1}}],["00013603218",{"2":{"256":1}}],["000136768",{"2":{"244":1}}],["00013145073567494362",{"2":{"258":1}}],["00013147806748750233",{"2":{"258":1}}],["00013147807",{"2":{"256":1}}],["00013199541345806668",{"2":{"258":1}}],["00013199625",{"2":{"256":1}}],["00013105472825926016",{"2":{"258":1}}],["00013105830833025017",{"2":{"258":1}}],["00013105836",{"2":{"256":1}}],["00013105926",{"2":{"256":1}}],["00013121",{"2":{"77":1}}],["00013201640836588033",{"2":{"258":1}}],["00013214018879283523",{"2":{"258":1}}],["00013213923",{"2":{"256":1}}],["00013229278005124026",{"2":{"258":1}}],["00013229327",{"2":{"256":1}}],["00013252692440783356",{"2":{"258":1}}],["0001325249",{"2":{"256":1}}],["00013269154266058503",{"2":{"258":1}}],["00013269106",{"2":{"256":1}}],["000132676",{"2":{"244":1}}],["000132906",{"2":{"244":1}}],["00013804465292119772",{"2":{"258":1}}],["00013804293",{"2":{"256":1}}],["0001385447835821711",{"2":{"258":1}}],["00013854646",{"2":{"256":1}}],["00013857145044797106",{"2":{"258":1}}],["00013857176",{"2":{"256":1}}],["00013839693141299363",{"2":{"258":1}}],["00013839726",{"2":{"256":1}}],["000138344",{"2":{"244":1}}],["000138281",{"2":{"244":1}}],["00010003003378708063",{"2":{"258":1}}],["00010003266",{"2":{"256":1}}],["00010026951610920748",{"2":{"258":1}}],["00010027012",{"2":{"256":1}}],["0001027414939987017",{"2":{"258":1}}],["00010273883",{"2":{"256":1}}],["00010256551686684742",{"2":{"258":1}}],["000102561884",{"2":{"256":1}}],["00010215279022568149",{"2":{"258":1}}],["000102152466",{"2":{"256":1}}],["00010269012575972111",{"2":{"258":1}}],["00010269098",{"2":{"256":1}}],["000102658024961822",{"2":{"258":1}}],["000102658276",{"2":{"256":1}}],["00010429555334314332",{"2":{"258":1}}],["0001042926",{"2":{"256":1}}],["00010481063298939619",{"2":{"258":1}}],["00010480752",{"2":{"256":1}}],["0001046846657664584",{"2":{"258":1}}],["00010468527",{"2":{"256":1}}],["00010407246467291496",{"2":{"258":1}}],["00010407074",{"2":{"256":1}}],["0001049144600013644",{"2":{"258":1}}],["000104917126",{"2":{"256":1}}],["00010493927778046945",{"2":{"258":1}}],["00010493953",{"2":{"256":1}}],["00010457094982490636",{"2":{"258":1}}],["000104571554",{"2":{"256":1}}],["00010455398496555676",{"2":{"258":1}}],["000104551196",{"2":{"256":1}}],["00010959616127949386",{"2":{"258":1}}],["00010959305",{"2":{"256":1}}],["00010991657203627335",{"2":{"258":1}}],["00010991485",{"2":{"256":1}}],["00010926800937078438",{"2":{"258":1}}],["000109266395",{"2":{"256":1}}],["00010344155671547738",{"2":{"258":1}}],["000103437924",{"2":{"256":1}}],["00010387897958676388",{"2":{"258":1}}],["000103879465",{"2":{"256":1}}],["00010321408494210545",{"2":{"258":1}}],["000103214596",{"2":{"256":1}}],["0001032820264655795",{"2":{"258":1}}],["000103280305",{"2":{"256":1}}],["000103729",{"2":{"244":1}}],["00010754331837456985",{"2":{"258":1}}],["00010754408",{"2":{"256":1}}],["0001073214201739075",{"2":{"258":1}}],["00010731906",{"2":{"256":1}}],["00010763267701827329",{"2":{"258":1}}],["00010763351",{"2":{"256":1}}],["00010768450501077066",{"2":{"258":1}}],["00010768214",{"2":{"256":1}}],["00010797254071798166",{"2":{"258":1}}],["00010797254",{"2":{"256":1}}],["00010155072447913914",{"2":{"258":1}}],["000101553676",{"2":{"256":1}}],["00010113434720940033",{"2":{"258":1}}],["00010113172",{"2":{"256":1}}],["00010149392838134362",{"2":{"258":1}}],["000101490295",{"2":{"256":1}}],["00010183441237705685",{"2":{"258":1}}],["000101836085",{"2":{"256":1}}],["0001018967840296145",{"2":{"258":1}}],["000101897385",{"2":{"256":1}}],["00010185551309430941",{"2":{"258":1}}],["00010185486",{"2":{"256":1}}],["000101051720408621",{"2":{"258":1}}],["00010105535",{"2":{"256":1}}],["0001010620859318409",{"2":{"258":1}}],["00010106445",{"2":{"256":1}}],["0001010015274014704",{"2":{"258":1}}],["0001010016",{"2":{"256":1}}],["00010135865583188165",{"2":{"258":1}}],["00010135833",{"2":{"256":1}}],["00010137273480986426",{"2":{"258":1}}],["00010137241",{"2":{"256":1}}],["00010137852786915331",{"2":{"258":1}}],["00010137846",{"2":{"256":1}}],["000108091",{"2":{"244":1}}],["00010628525694297254",{"2":{"258":1}}],["000106285006",{"2":{"256":1}}],["00010650963763085304",{"2":{"258":1}}],["00010650995",{"2":{"256":1}}],["00010666924008551315",{"2":{"258":1}}],["00010669620254439335",{"2":{"258":1}}],["00010669653",{"2":{"256":1}}],["00010667203",{"2":{"256":1}}],["00010611892650576301",{"2":{"258":1}}],["0001061864",{"2":{"256":1}}],["00010612095",{"2":{"256":1}}],["000106173",{"2":{"244":1}}],["000106987",{"2":{"244":1}}],["000106323",{"2":{"244":1}}],["0001050108826749296",{"2":{"258":1}}],["00010501139",{"2":{"256":1}}],["00010571498569335307",{"2":{"258":1}}],["000105714986",{"2":{"256":1}}],["0001052690654031679",{"2":{"258":1}}],["000105268555",{"2":{"256":1}}],["000105296",{"2":{"77":1}}],["00010558566166244384",{"2":{"258":1}}],["000105583036",{"2":{"256":1}}],["0001055645689118281",{"2":{"258":1}}],["000105565225",{"2":{"256":1}}],["00010551",{"2":{"179":1}}],["00010583802825229028",{"2":{"258":1}}],["0001058354",{"2":{"256":1}}],["00010581292208365888",{"2":{"258":1}}],["00010581013",{"2":{"256":1}}],["000105879",{"2":{"244":1}}],["000105440",{"2":{"244":1}}],["00010546855",{"2":{"179":1}}],["00016925391737454296",{"2":{"258":1}}],["00016925113",{"2":{"256":1}}],["00016870042384299792",{"2":{"258":1}}],["00016878",{"2":{"179":1}}],["0001685039575871451",{"2":{"258":1}}],["00016850662",{"2":{"256":1}}],["00016895208471980533",{"2":{"258":1}}],["00016894897",{"2":{"256":1}}],["00016848296590046868",{"2":{"258":1}}],["00016848213",{"2":{"256":1}}],["00016860544262218825",{"2":{"258":1}}],["00016860459",{"2":{"256":1}}],["00016869747",{"2":{"256":1}}],["00016730541192379013",{"2":{"258":1}}],["0001673082",{"2":{"256":1}}],["00016762771672355632",{"2":{"258":1}}],["00016762815",{"2":{"256":1}}],["00016759197052963493",{"2":{"258":1}}],["00016759252",{"2":{"256":1}}],["00016794122348050335",{"2":{"258":1}}],["00016794122",{"2":{"256":1}}],["0001642447746086802",{"2":{"258":1}}],["0001642431",{"2":{"256":1}}],["00016439423779959398",{"2":{"258":1}}],["00016439424",{"2":{"256":1}}],["000164069",{"2":{"244":1}}],["00016289136179357017",{"2":{"258":1}}],["00016289309",{"2":{"256":1}}],["00016246069410186855",{"2":{"258":1}}],["0001624591",{"2":{"256":1}}],["000162398370529336",{"2":{"258":1}}],["00016239624528652472",{"2":{"258":1}}],["00016239625",{"2":{"256":1}}],["00016239902",{"2":{"256":1}}],["00016209405597428542",{"2":{"258":1}}],["00016209438",{"2":{"256":1}}],["000162085",{"2":{"244":1}}],["000162970",{"2":{"244":1}}],["00016624314066571072",{"2":{"258":1}}],["00016624035",{"2":{"256":1}}],["000166804",{"2":{"244":1}}],["000166462",{"2":{"244":1}}],["00016136170006947375",{"2":{"258":1}}],["00016125945057890732",{"2":{"258":1}}],["0001612588",{"2":{"256":1}}],["00016199772144678218",{"2":{"258":1}}],["00016199409",{"2":{"256":1}}],["000161607",{"2":{"244":1}}],["000161545",{"2":{"244":1}}],["00016012509054921307",{"2":{"258":1}}],["00016012433",{"2":{"256":1}}],["00016092859308364536",{"2":{"258":1}}],["00016092892",{"2":{"256":1}}],["000160591",{"2":{"244":1}}],["000160725",{"2":{"244":1}}],["00016363867501935345",{"2":{"258":1}}],["00016363562",{"2":{"256":1}}],["00016357305315718972",{"2":{"258":1}}],["00016357354",{"2":{"256":1}}],["000163823",{"2":{"244":1}}],["000163182",{"2":{"244":1}}],["000165743",{"2":{"244":1}}],["000165069",{"2":{"244":1}}],["000165657",{"2":{"77":1}}],["0001",{"2":{"77":7,"78":1}}],["0001f0",{"2":{"77":1}}],["0136942601152516e",{"2":{"258":1}}],["01360",{"2":{"187":1}}],["0137547f",{"2":{"256":1}}],["01376",{"2":{"187":1}}],["013011228",{"2":{"244":1}}],["01301",{"2":{"187":1}}],["01300",{"2":{"187":1}}],["01330",{"2":{"187":1}}],["01337",{"2":{"187":1}}],["01384",{"2":{"187":1}}],["01329",{"2":{"187":1}}],["01322",{"2":{"187":1}}],["01320",{"2":{"187":1}}],["013499700",{"2":{"244":1}}],["01341",{"2":{"187":1}}],["01340",{"2":{"187":1}}],["01343",{"2":{"187":1}}],["01354",{"2":{"187":2}}],["01358",{"2":{"187":2}}],["013992591",{"2":{"106":1}}],["013992601",{"2":{"106":1}}],["0175230970680213e",{"2":{"258":1}}],["0175803f",{"2":{"256":1}}],["0175543f",{"2":{"256":1}}],["01755",{"2":{"187":1}}],["017215939",{"2":{"244":1}}],["017614847",{"2":{"244":1}}],["017605182",{"2":{"244":1}}],["017645847",{"2":{"244":1}}],["01763",{"2":{"187":1}}],["01708",{"2":{"187":1}}],["017197266",{"2":{"244":1}}],["01711",{"2":{"187":1}}],["0171745",{"2":{"168":1}}],["0177563954154805e",{"2":{"258":1}}],["017750096",{"2":{"244":1}}],["017799046",{"2":{"244":1}}],["01776",{"2":{"187":1}}],["01773",{"2":{"187":1}}],["01748",{"2":{"187":1}}],["017432783",{"2":{"106":2}}],["016490150",{"2":{"244":1}}],["016479947",{"2":{"106":2}}],["016228525",{"2":{"244":1}}],["01625",{"2":{"187":1}}],["016118063",{"2":{"244":1}}],["0163",{"2":{"249":1}}],["016386209",{"2":{"244":1}}],["01637",{"2":{"187":1}}],["01635",{"2":{"187":1}}],["01657",{"2":{"187":1}}],["01668",{"2":{"187":1}}],["01663",{"2":{"187":1}}],["01666",{"2":{"187":1}}],["01674",{"2":{"187":1}}],["016987354",{"2":{"244":1}}],["01693",{"2":{"187":1}}],["01699",{"2":{"187":1}}],["01681",{"2":{"187":1}}],["01682",{"2":{"187":1}}],["0168695",{"2":{"145":1}}],["01f0",{"2":{"145":1,"146":1,"179":2,"187":1}}],["019838908326368e",{"2":{"258":1}}],["0198875f",{"2":{"256":1}}],["019859709",{"2":{"244":1}}],["019325165",{"2":{"244":1}}],["01935",{"2":{"187":1}}],["019155467",{"2":{"244":1}}],["01911576",{"2":{"106":1}}],["019115759",{"2":{"106":1}}],["01900",{"2":{"187":1}}],["01908",{"2":{"187":1}}],["01998",{"2":{"187":1}}],["01997",{"2":{"187":1}}],["019567011",{"2":{"244":1}}],["01956",{"2":{"187":1}}],["01950",{"2":{"187":1}}],["01972",{"2":{"187":1}}],["019791335",{"2":{"106":1}}],["019791333",{"2":{"106":1}}],["019427970",{"2":{"244":1}}],["019484218",{"2":{"244":1}}],["01945",{"2":{"187":1}}],["01947",{"2":{"187":1}}],["01964",{"2":{"187":1}}],["01965834",{"2":{"126":1}}],["019236464",{"2":{"77":1}}],["0185274f",{"2":{"256":1}}],["018008159",{"2":{"244":1}}],["018073166",{"2":{"244":1}}],["018483378",{"2":{"244":1}}],["01888",{"2":{"187":1}}],["018884756",{"2":{"106":1}}],["018884761",{"2":{"106":1}}],["01839",{"2":{"187":1}}],["018367462838316e",{"2":{"258":1}}],["01836",{"2":{"187":1}}],["018338893",{"2":{"106":1}}],["018338889",{"2":{"106":1}}],["01812",{"2":{"187":2}}],["01820",{"2":{"187":1}}],["0189805",{"2":{"232":1}}],["01898",{"2":{"187":1}}],["01899",{"2":{"187":2}}],["01868",{"2":{"187":1}}],["018706074",{"2":{"244":1}}],["018701272",{"2":{"77":1}}],["018723227",{"2":{"244":1}}],["01872db4",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["018743665453639813",{"2":{"174":1}}],["0187668",{"2":{"106":1}}],["015748031",{"2":{"232":2}}],["01505",{"2":{"187":1}}],["01508",{"2":{"187":1}}],["015340659",{"2":{"244":1}}],["01531",{"2":{"187":1}}],["01537",{"2":{"187":1}}],["015280124",{"2":{"244":1}}],["015216017",{"2":{"244":1}}],["01522",{"2":{"187":1}}],["01527",{"2":{"187":1}}],["0156773660068764e",{"2":{"258":1}}],["01565",{"2":{"187":1}}],["0156169f",{"2":{"256":1}}],["01561",{"2":{"187":1}}],["01564",{"2":{"187":1}}],["01560",{"2":{"187":1}}],["01557",{"2":{"187":1}}],["01558",{"2":{"187":1}}],["01553",{"2":{"187":2}}],["0155126",{"2":{"106":1}}],["01595",{"2":{"187":1}}],["01594",{"2":{"187":1}}],["015937408",{"2":{"126":1,"129":1}}],["015494239",{"2":{"244":1}}],["01548",{"2":{"187":1}}],["015478307",{"2":{"235":1}}],["01547",{"2":{"187":1}}],["01542492",{"2":{"107":1}}],["0158726",{"2":{"77":1}}],["01027",{"2":{"187":1}}],["01021",{"2":{"187":1}}],["0102189295",{"2":{"106":1}}],["010218928",{"2":{"106":1}}],["010336538",{"2":{"244":1}}],["01036",{"2":{"187":1}}],["01034",{"2":{"187":1}}],["01071",{"2":{"187":1}}],["010755754",{"2":{"244":1}}],["01075",{"2":{"187":1}}],["010920972",{"2":{"244":1}}],["010936455",{"2":{"244":1}}],["01095",{"2":{"187":1}}],["01094",{"2":{"187":1}}],["010004668",{"2":{"244":1}}],["01007",{"2":{"187":1}}],["0100971125",{"2":{"77":1}}],["010616f",{"2":{"256":1}}],["0106557f",{"2":{"256":1}}],["01066",{"2":{"187":1}}],["010629211",{"2":{"244":1}}],["01062",{"2":{"187":1}}],["010637204",{"2":{"106":1}}],["010637202",{"2":{"106":1}}],["010452257",{"2":{"244":1}}],["01042",{"2":{"187":1}}],["01044",{"2":{"187":1}}],["010404",{"2":{"145":1}}],["010518193",{"2":{"244":1}}],["010586374",{"2":{"244":1}}],["01057",{"2":{"187":1}}],["0105465",{"2":{"77":1}}],["010882485695678e",{"2":{"258":1}}],["010876314",{"2":{"244":1}}],["010898398",{"2":{"244":1}}],["01089",{"2":{"187":1}}],["01082",{"2":{"187":1}}],["0108668577150213",{"2":{"171":1}}],["01014",{"2":{"187":1}}],["0101",{"2":{"78":1}}],["0122",{"2":{"249":1}}],["01222",{"2":{"187":1}}],["012140292",{"2":{"244":1}}],["012183795",{"2":{"106":1}}],["012183794",{"2":{"106":1}}],["012508835",{"2":{"244":1}}],["012522168",{"2":{"77":1}}],["012002973",{"2":{"244":1}}],["01203",{"2":{"187":1}}],["012775686",{"2":{"244":1}}],["01273",{"2":{"187":1}}],["01276",{"2":{"187":1}}],["01278",{"2":{"187":1}}],["01284",{"2":{"187":1}}],["01280",{"2":{"187":1}}],["01286",{"2":{"187":1}}],["01289",{"2":{"187":1}}],["01266",{"2":{"187":1}}],["01269",{"2":{"187":1}}],["012910118557003e",{"2":{"258":1}}],["01291",{"2":{"187":1}}],["01298",{"2":{"187":1}}],["01295",{"2":{"187":1}}],["012377445040195e",{"2":{"258":1}}],["012323526",{"2":{"244":1}}],["01239",{"2":{"187":1}}],["01235",{"2":{"187":1}}],["0123139",{"2":{"77":1}}],["0149",{"2":{"249":1}}],["014400685",{"2":{"244":1}}],["014098606",{"2":{"244":1}}],["01405",{"2":{"187":1}}],["014708381",{"2":{"244":1}}],["01475",{"2":{"187":1}}],["014241929",{"2":{"244":1}}],["014243508",{"2":{"244":1}}],["01428",{"2":{"187":1}}],["0142814",{"2":{"77":1}}],["014815190",{"2":{"244":1}}],["01480",{"2":{"187":1}}],["01486",{"2":{"187":1}}],["01484",{"2":{"187":1}}],["014877922",{"2":{"106":1}}],["014340003",{"2":{"244":1}}],["014307282",{"2":{"244":1}}],["01431",{"2":{"187":1}}],["01433",{"2":{"187":1}}],["01438",{"2":{"187":1}}],["01462",{"2":{"187":1}}],["0146587",{"2":{"77":1}}],["014530762",{"2":{"244":1}}],["014555071",{"2":{"244":1}}],["01450",{"2":{"187":1}}],["01456",{"2":{"187":3}}],["01415047",{"2":{"106":1}}],["014150472",{"2":{"106":1}}],["014100174",{"2":{"77":1}}],["011247103",{"2":{"244":1}}],["011284535",{"2":{"244":1}}],["011598296",{"2":{"244":1}}],["011505678",{"2":{"244":1}}],["011906523",{"2":{"244":1}}],["011915382",{"2":{"244":1}}],["011937321",{"2":{"179":1}}],["01186",{"2":{"187":1}}],["01185",{"2":{"187":1}}],["011342090",{"2":{"244":1}}],["011374202",{"2":{"244":1}}],["01137",{"2":{"187":1}}],["01130",{"2":{"187":1}}],["011037273",{"2":{"244":1}}],["011087346",{"2":{"244":1}}],["011073254",{"2":{"244":1}}],["011062279",{"2":{"244":1}}],["01106",{"2":{"187":1}}],["01105",{"2":{"187":2}}],["01169",{"2":{"187":1}}],["01167",{"2":{"187":1}}],["011611394",{"2":{"106":1}}],["011611392",{"2":{"106":1}}],["011457845",{"2":{"244":1}}],["01146",{"2":{"187":1}}],["01148",{"2":{"187":2}}],["01143",{"2":{"187":1}}],["01178",{"2":{"187":1}}],["01179",{"2":{"187":1}}],["01172",{"2":{"187":2}}],["011734666",{"2":{"77":1}}],["011739168",{"2":{"77":1}}],["01110",{"2":{"187":1}}],["011100831f0",{"2":{"53":1}}],["01117",{"2":{"187":1}}],["01",{"2":{"24":3,"53":2,"179":1,"208":5,"258":1}}],["0",{"2":{"2":1,"4":1,"7":3,"11":1,"19":3,"24":44,"46":2,"49":2,"53":122,"54":4,"59":16,"62":3,"64":3,"67":2,"68":10,"69":1,"74":1,"77":204,"78":14,"106":292,"107":11,"111":1,"112":1,"114":2,"116":5,"117":37,"120":1,"126":54,"129":25,"131":8,"135":6,"136":1,"137":7,"145":18,"146":5,"147":35,"148":23,"152":7,"153":358,"155":1,"168":54,"170":1,"171":155,"172":15,"173":3,"174":62,"176":12,"177":9,"178":4,"179":72,"180":11,"183":1,"186":1,"187":452,"189":11,"192":1,"193":2,"194":2,"195":7,"197":3,"206":4,"208":2,"210":11,"213":1,"214":2,"215":1,"216":2,"217":26,"218":1,"219":8,"220":19,"221":11,"224":2,"227":2,"228":105,"229":11,"232":139,"233":1,"234":4,"235":40,"236":11,"237":11,"243":7,"244":800,"245":6,"246":11,"248":4,"249":68,"251":3,"254":4,"255":9,"256":511,"257":1,"258":418,"260":3}}],["1e",{"2":{"208":1,"254":1}}],["1g",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["1×16",{"2":{"147":1,"148":1}}],["1×32",{"2":{"59":1,"78":1}}],["1`",{"2":{"136":3}}],["1st",{"2":{"127":1}}],["1191545595130843e",{"2":{"258":1}}],["1197551f",{"2":{"256":1}}],["11984776",{"2":{"77":1}}],["1143",{"2":{"249":1}}],["114975",{"2":{"147":1}}],["110263503469658e",{"2":{"258":1}}],["11023622",{"2":{"232":2}}],["11023762",{"2":{"179":1}}],["1108167f",{"2":{"256":1}}],["110797f",{"2":{"256":1}}],["110425f",{"2":{"256":1}}],["11001",{"2":{"244":1}}],["11007148",{"2":{"232":1}}],["110",{"2":{"220":6}}],["11123376116672e",{"2":{"258":1}}],["11111",{"2":{"219":1}}],["111552626",{"2":{"106":1}}],["115820917456447e",{"2":{"258":1}}],["11588",{"2":{"187":1}}],["11579f",{"2":{"256":1}}],["11501",{"2":{"244":1}}],["11515346",{"2":{"106":1}}],["11515345",{"2":{"106":1}}],["116989",{"2":{"171":1}}],["116796836",{"2":{"106":1}}],["118333548048649e",{"2":{"258":1}}],["118536f",{"2":{"256":1}}],["1181102",{"2":{"232":2}}],["118",{"2":{"217":1}}],["11842",{"2":{"187":1}}],["1188200246206075",{"2":{"153":2}}],["1187928",{"2":{"106":2}}],["1120056717862873e",{"2":{"258":1}}],["1128604021259633e",{"2":{"258":1}}],["112693467242631e",{"2":{"258":1}}],["1126356",{"2":{"148":1}}],["1129297f",{"2":{"256":1}}],["112973",{"2":{"168":1}}],["112152827835879e",{"2":{"258":1}}],["1121",{"2":{"249":1}}],["1124609",{"2":{"106":1}}],["1131393f",{"2":{"256":1}}],["11399",{"2":{"187":1}}],["1137304",{"2":{"147":1}}],["11365668",{"2":{"106":2}}],["117573674690805e",{"2":{"258":1}}],["1175871f",{"2":{"106":1}}],["117311f",{"2":{"256":1}}],["1173706",{"2":{"147":1}}],["117236",{"2":{"232":1}}],["11702",{"2":{"187":1}}],["11",{"2":{"145":2,"146":2,"147":2,"148":1,"180":2,"187":14,"189":2,"196":1,"210":2,"221":2,"228":2,"229":2,"236":1,"237":2,"246":2,"249":3,"258":1}}],["1477708102315184e",{"2":{"258":1}}],["147705370924899",{"2":{"153":1}}],["1477053709248992",{"2":{"153":1}}],["1476753f",{"2":{"256":1}}],["1407898503261834e",{"2":{"258":1}}],["140736620836752e",{"2":{"258":1}}],["1408754f",{"2":{"256":1}}],["1408605f",{"2":{"256":1}}],["1406",{"2":{"249":2}}],["1406s\\ttraining",{"2":{"217":1}}],["1405",{"2":{"249":1}}],["14001",{"2":{"244":1}}],["149160869887804e",{"2":{"258":1}}],["1492464f",{"2":{"256":1}}],["1498173f",{"2":{"256":1}}],["1490868",{"2":{"235":1}}],["1496500483909854e",{"2":{"258":1}}],["1496063",{"2":{"232":2}}],["14964",{"2":{"187":1}}],["14941669",{"2":{"77":1}}],["1435",{"2":{"249":1}}],["1439",{"2":{"249":1}}],["14377",{"2":{"187":1}}],["1437841",{"2":{"179":1}}],["14340512",{"2":{"107":1}}],["1458",{"2":{"249":1}}],["14501",{"2":{"244":1}}],["145296",{"2":{"168":1}}],["14568081",{"2":{"106":1}}],["142470881033475",{"2":{"153":2}}],["142251",{"2":{"106":1}}],["148195598388567e",{"2":{"258":1}}],["14811f",{"2":{"256":1}}],["14815",{"2":{"217":2}}],["1484191f",{"2":{"256":1}}],["1482518417602913e",{"2":{"258":1}}],["14825",{"2":{"171":1}}],["148248",{"2":{"171":2}}],["1489007",{"2":{"147":1}}],["1488175",{"2":{"113":1}}],["14",{"2":{"145":1,"187":14,"217":2,"228":4,"249":7}}],["1445339866684955e",{"2":{"258":1}}],["144777919157657e",{"2":{"258":1}}],["144778f",{"2":{"256":1}}],["14441f",{"2":{"256":1}}],["14448",{"2":{"187":1}}],["14445141",{"2":{"106":1}}],["14488737",{"2":{"106":1}}],["141835410123147e",{"2":{"258":1}}],["1415296f",{"2":{"256":1}}],["1415927",{"2":{"53":1}}],["14132261",{"2":{"232":1}}],["14173229",{"2":{"232":2}}],["14118214",{"2":{"106":1}}],["175549615013621e",{"2":{"258":1}}],["1758045739991064e",{"2":{"258":1}}],["175846130909918e",{"2":{"258":1}}],["175845f",{"2":{"256":1}}],["17501",{"2":{"244":1}}],["1731386f",{"2":{"256":1}}],["17322835",{"2":{"232":2}}],["1791",{"2":{"249":1}}],["179584495291061",{"2":{"153":2}}],["17111921",{"2":{"235":1}}],["178858288727115e",{"2":{"258":1}}],["17881",{"2":{"187":1}}],["178563f",{"2":{"256":1}}],["17816184",{"2":{"232":1}}],["178438",{"2":{"168":1}}],["17706361",{"2":{"179":1}}],["1706",{"2":{"249":1}}],["17063im",{"2":{"168":1}}],["170017f",{"2":{"256":1}}],["17001",{"2":{"244":1}}],["170",{"2":{"229":1}}],["170342",{"2":{"171":1}}],["17048413",{"2":{"106":1}}],["172325e",{"2":{"148":1}}],["17239982",{"2":{"77":1}}],["176863312430556e",{"2":{"258":1}}],["1764507f",{"2":{"256":1}}],["1764085",{"2":{"126":1}}],["1767678f",{"2":{"256":1}}],["1765",{"2":{"249":1}}],["1763",{"2":{"249":1}}],["176",{"2":{"208":1}}],["1769385",{"2":{"147":1}}],["17",{"2":{"117":1,"148":1,"187":14,"196":10,"228":3,"233":1,"249":4,"258":22}}],["17441",{"2":{"171":1}}],["17449912",{"2":{"77":1}}],["17435724",{"2":{"106":2}}],["1746128",{"2":{"77":1}}],["1th",{"2":{"69":3}}],["134217385318269e",{"2":{"258":1}}],["1345286f",{"2":{"256":1}}],["13475561",{"2":{"232":1}}],["13403",{"2":{"187":1}}],["133634e",{"2":{"208":1}}],["13343152",{"2":{"106":1}}],["132562358475261e",{"2":{"258":1}}],["1325936f",{"2":{"256":1}}],["13269",{"2":{"187":1}}],["13289",{"2":{"187":1}}],["13",{"2":{"187":14,"217":1,"228":3,"244":4,"249":3}}],["136763024996941e",{"2":{"258":1}}],["1364964f",{"2":{"256":1}}],["13646",{"2":{"187":1}}],["13655",{"2":{"187":1}}],["136356",{"2":{"168":1}}],["130167279100299e",{"2":{"258":1}}],["1304785f",{"2":{"256":1}}],["1303",{"2":{"249":1}}],["13001",{"2":{"244":1}}],["13073668",{"2":{"179":1}}],["1306755",{"2":{"148":1}}],["1302",{"2":{"68":1}}],["13819042",{"2":{"126":1}}],["13864",{"2":{"106":1}}],["137157e",{"2":{"208":1}}],["13725637",{"2":{"106":1}}],["13725635",{"2":{"106":1}}],["13760868",{"2":{"106":1}}],["13760866",{"2":{"106":1}}],["1377298",{"2":{"106":2}}],["135184714088322e",{"2":{"258":1}}],["13518013",{"2":{"106":1}}],["13518015",{"2":{"106":1}}],["13572",{"2":{"187":1}}],["13559",{"2":{"171":1}}],["1350125f",{"2":{"256":1}}],["13501",{"2":{"244":1}}],["1350521",{"2":{"147":1}}],["13504265",{"2":{"106":1}}],["1395766",{"2":{"232":1}}],["139433",{"2":{"148":1}}],["13913931",{"2":{"106":1}}],["13988f",{"2":{"77":1}}],["1310",{"2":{"249":1}}],["13151",{"2":{"187":1}}],["131583631",{"2":{"78":1}}],["131726",{"2":{"147":1,"148":1}}],["13168f",{"2":{"77":1}}],["1312",{"2":{"24":1}}],["1c",{"2":{"66":1}}],["1b",{"2":{"66":3}}],["1a",{"2":{"66":3}}],["1828",{"2":{"249":1}}],["1828456",{"2":{"232":1}}],["18297",{"2":{"187":1}}],["18270",{"2":{"187":1}}],["1846",{"2":{"249":1}}],["1846486482317626",{"2":{"153":2}}],["18481395",{"2":{"235":1}}],["18410",{"2":{"187":1}}],["18501",{"2":{"244":1}}],["18535946",{"2":{"235":1}}],["18539",{"2":{"187":1}}],["18564",{"2":{"232":1}}],["18519",{"2":{"217":2}}],["185124",{"2":{"147":1}}],["1802074718741762e",{"2":{"258":1}}],["180175f",{"2":{"256":1}}],["18001",{"2":{"244":1}}],["18056087",{"2":{"179":1}}],["18089005",{"2":{"106":1}}],["1871622",{"2":{"176":4}}],["18794",{"2":{"168":1}}],["187646",{"2":{"168":1}}],["18941",{"2":{"168":1}}],["186",{"2":{"256":1}}],["18695377",{"2":{"179":1}}],["18652338",{"2":{"148":1}}],["186451",{"2":{"148":1}}],["183419939128277e",{"2":{"258":1}}],["1834s\\ttraining",{"2":{"217":1}}],["1833344f",{"2":{"256":1}}],["183307",{"2":{"147":1}}],["1832",{"2":{"249":1}}],["18365",{"2":{"168":1}}],["1836515766351254",{"2":{"153":2}}],["18308182",{"2":{"126":1}}],["188165951972205e",{"2":{"258":1}}],["18811704",{"2":{"77":1}}],["1882493f",{"2":{"256":1}}],["1887817",{"2":{"232":1}}],["18841726",{"2":{"179":1}}],["18860114",{"2":{"126":1}}],["18807876",{"2":{"106":2}}],["1811024",{"2":{"232":2}}],["18102339",{"2":{"77":1}}],["181208",{"2":{"77":1}}],["18",{"2":{"59":1,"187":14,"196":1,"228":2,"249":5,"258":6}}],["1−z",{"2":{"66":1}}],["1−2∑yy^+α∑y2+∑y^2+α",{"2":{"53":1}}],["1−α",{"2":{"53":2}}],["1−y",{"2":{"53":1}}],["1−yy^",{"2":{"53":2}}],["1−y^+ϵ",{"2":{"53":1}}],["1−y~",{"2":{"53":2}}],["1=dense",{"2":{"44":1}}],["10f",{"2":{"257":1}}],["1010683001150635e",{"2":{"258":1}}],["1016347981198496e",{"2":{"258":1}}],["101728390865519e",{"2":{"258":1}}],["1015515f",{"2":{"256":1}}],["1018898f",{"2":{"256":1}}],["101",{"2":{"208":1,"220":6,"236":1}}],["10140238162746013",{"2":{"153":2}}],["108145",{"2":{"232":1}}],["108",{"2":{"196":1}}],["108558",{"2":{"168":1}}],["104",{"2":{"196":2}}],["10458f",{"2":{"77":1}}],["1058",{"2":{"249":1}}],["10501",{"2":{"244":1}}],["105",{"2":{"196":1}}],["10552457",{"2":{"106":1}}],["105524555",{"2":{"106":1}}],["10356369307312e",{"2":{"258":1}}],["103509f",{"2":{"256":1}}],["10316",{"2":{"187":1}}],["10387",{"2":{"187":1}}],["1032465",{"2":{"106":1}}],["107888",{"2":{"232":1}}],["107",{"2":{"196":1}}],["10728",{"2":{"187":1}}],["1079326",{"2":{"147":1}}],["10754459",{"2":{"106":1}}],["1024",{"2":{"153":7}}],["102",{"2":{"116":1,"196":1,"217":1}}],["1090",{"2":{"254":2}}],["10993",{"2":{"187":1}}],["109261915",{"2":{"126":1}}],["109203495",{"2":{"106":1}}],["1095082",{"2":{"106":1}}],["10950818",{"2":{"106":1}}],["106411621077796e",{"2":{"258":1}}],["106787780813939e",{"2":{"258":1}}],["1067367f",{"2":{"256":1}}],["106",{"2":{"196":1}}],["106069",{"2":{"145":1}}],["10688411",{"2":{"106":1}}],["106884085",{"2":{"106":1}}],["1069",{"2":{"106":1}}],["10",{"2":{"34":2,"46":2,"49":1,"62":1,"69":12,"74":2,"76":1,"77":5,"78":1,"106":5,"113":2,"114":1,"116":1,"145":2,"146":2,"148":2,"171":4,"174":2,"179":3,"180":5,"187":14,"189":5,"193":5,"195":1,"196":22,"197":2,"210":5,"215":5,"220":30,"221":5,"226":1,"228":2,"229":5,"237":5,"246":5,"249":4,"250":1,"251":2,"254":2,"258":15,"260":2}}],["1009444f",{"2":{"256":1}}],["1001",{"2":{"179":1,"244":1}}],["1003911",{"2":{"137":1}}],["10001",{"2":{"244":1}}],["10000",{"2":{"179":2,"244":1}}],["1000",{"2":{"59":1,"78":3,"107":14,"179":1}}],["100",{"2":{"24":1,"49":2,"63":6,"78":1,"107":2,"180":2,"189":2,"195":2,"197":1,"210":2,"217":2,"220":6,"221":2,"228":32,"229":2,"237":2,"246":2,"251":1,"255":2,"260":1}}],["1200339870060362e",{"2":{"258":1}}],["12001",{"2":{"244":1}}],["12096125",{"2":{"235":1}}],["1204217935313214",{"2":{"153":2}}],["12s",{"2":{"228":2}}],["1269650811645857e",{"2":{"258":1}}],["1265",{"2":{"249":1}}],["126407e",{"2":{"208":1}}],["126",{"2":{"208":1}}],["1261990389976131",{"2":{"174":1}}],["12214",{"2":{"187":1}}],["12253",{"2":{"187":1}}],["12501",{"2":{"244":1}}],["12589",{"2":{"179":1}}],["12556516",{"2":{"106":2}}],["124913038213655e",{"2":{"258":1}}],["1240454f",{"2":{"256":1}}],["12477568",{"2":{"179":1}}],["12442",{"2":{"168":1}}],["1245008",{"2":{"147":1}}],["12abac4f24f6",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["12920506445426e",{"2":{"258":1}}],["1295477515847536e",{"2":{"258":1}}],["1295f",{"2":{"256":1}}],["1295227f",{"2":{"256":1}}],["12902719",{"2":{"106":2}}],["1293236",{"2":{"106":1}}],["12992674",{"2":{"106":1}}],["129",{"2":{"59":1,"69":1}}],["1277556f0",{"2":{"53":1}}],["12",{"2":{"46":2,"135":1,"147":6,"148":7,"152":9,"171":2,"180":6,"187":14,"189":6,"197":1,"210":6,"221":6,"228":18,"229":6,"237":6,"246":6,"249":5,"251":1,"254":1,"260":1}}],["1286868f",{"2":{"256":1}}],["1286439",{"2":{"232":1}}],["12827216",{"2":{"106":1}}],["128",{"2":{"34":2,"59":12,"69":3,"77":4,"193":6,"195":1,"217":1,"232":4,"251":5,"260":1}}],["1234",{"2":{"248":1}}],["12345",{"2":{"232":1}}],["12336",{"2":{"187":1}}],["1239749674027375",{"2":{"153":2}}],["123",{"2":{"24":3,"62":1,"153":1}}],["1675390001372084e",{"2":{"258":1}}],["16764",{"2":{"187":1}}],["1638469087755e",{"2":{"258":1}}],["1632666343442166e",{"2":{"258":1}}],["163206f",{"2":{"256":1}}],["1633031461512262e",{"2":{"258":1}}],["1633819413955027e",{"2":{"258":1}}],["16316",{"2":{"187":1}}],["1685634679624405e",{"2":{"258":1}}],["168618f",{"2":{"256":1}}],["16827",{"2":{"187":1}}],["16270905436827e",{"2":{"258":1}}],["162709f",{"2":{"256":1}}],["16241",{"2":{"220":12}}],["16240",{"2":{"220":12}}],["1650249f",{"2":{"256":1}}],["16501",{"2":{"244":1}}],["1652",{"2":{"249":1}}],["16566901",{"2":{"145":1}}],["1668s\\ttraining",{"2":{"217":1}}],["1617677f",{"2":{"256":1}}],["1617228180412864",{"2":{"153":2}}],["16187939",{"2":{"106":1}}],["164158f",{"2":{"256":1}}],["1641532f",{"2":{"106":1}}],["16450",{"2":{"220":6}}],["164786",{"2":{"171":1}}],["164379",{"2":{"168":1}}],["1642133",{"2":{"147":1}}],["16407",{"2":{"106":1}}],["16409892",{"2":{"106":1}}],["1699660050313614e",{"2":{"258":1}}],["16946f",{"2":{"77":1}}],["16909766",{"2":{"77":1}}],["16001",{"2":{"244":1}}],["16004",{"2":{"187":1}}],["160148e",{"2":{"208":1}}],["160876",{"2":{"168":1}}],["16083185f0",{"2":{"53":1}}],["160832f0",{"2":{"53":1}}],["16094846",{"2":{"145":1}}],["1607805723239765e",{"2":{"258":1}}],["1607735f",{"2":{"256":1}}],["1607",{"2":{"19":2,"69":1}}],["16",{"2":{"34":1,"59":3,"111":1,"116":13,"117":14,"147":1,"148":1,"171":1,"187":14,"193":3,"217":1,"228":2,"233":4,"236":2,"243":1,"244":8,"249":4,"258":4,"260":4}}],["1d",{"2":{"24":1,"66":1}}],["1f0",{"2":{"19":3,"53":2,"59":1,"69":2,"78":1,"232":1}}],["196243486574254e",{"2":{"258":1}}],["1964456f",{"2":{"256":1}}],["196703",{"2":{"131":2}}],["198412722765804e",{"2":{"258":1}}],["1984668",{"2":{"145":1}}],["198580875235234e",{"2":{"258":1}}],["1985504325127225e",{"2":{"258":1}}],["198646f",{"2":{"256":1}}],["198892f",{"2":{"256":1}}],["1988",{"2":{"254":2}}],["1943",{"2":{"249":1}}],["19422",{"2":{"187":1}}],["197363839830605e",{"2":{"258":1}}],["197443728142482e",{"2":{"258":1}}],["1978849530028129e",{"2":{"258":1}}],["197007f",{"2":{"256":1}}],["197045",{"2":{"171":1}}],["19750",{"2":{"187":1}}],["19501",{"2":{"244":1}}],["195746",{"2":{"232":1}}],["1953009",{"2":{"177":1,"178":1}}],["19597391412112541",{"2":{"174":1}}],["1958431",{"2":{"179":1}}],["1958",{"2":{"17":1,"249":1}}],["191323710835339e",{"2":{"258":1}}],["191238f",{"2":{"256":1}}],["19124654",{"2":{"145":1}}],["1914",{"2":{"249":1}}],["191956",{"2":{"172":1}}],["19364647038794e",{"2":{"258":1}}],["1936812",{"2":{"145":1}}],["1935471893634369e",{"2":{"258":1}}],["1935723f",{"2":{"256":1}}],["193592f",{"2":{"256":1}}],["19340",{"2":{"187":1}}],["19397983",{"2":{"179":1}}],["193291",{"2":{"171":1}}],["190200187285233e",{"2":{"258":1}}],["190244f",{"2":{"256":1}}],["190079147924201e",{"2":{"258":1}}],["190079f",{"2":{"256":1}}],["19001",{"2":{"244":1}}],["1904755",{"2":{"148":1}}],["19084926",{"2":{"106":1}}],["1992588f",{"2":{"256":1}}],["1990",{"2":{"149":1}}],["199387",{"2":{"145":1}}],["19975941",{"2":{"77":1}}],["19",{"2":{"112":1,"187":14,"196":1,"217":1,"228":2,"249":5}}],["192836",{"2":{"168":1}}],["1921258",{"2":{"147":1}}],["19209f",{"2":{"106":2}}],["1929",{"2":{"17":1}}],["159497881946504e",{"2":{"258":1}}],["159809f",{"2":{"256":1}}],["159042358",{"2":{"244":2}}],["152074564813159e",{"2":{"258":1}}],["1522475f",{"2":{"256":1}}],["15276335",{"2":{"135":1}}],["1565",{"2":{"249":1}}],["15681",{"2":{"220":6}}],["15680",{"2":{"220":6}}],["15686738",{"2":{"106":1}}],["156",{"2":{"193":2}}],["1552883805534698e",{"2":{"258":1}}],["1550",{"2":{"249":2}}],["15501",{"2":{"244":1}}],["1557574",{"2":{"148":1}}],["1554613f",{"2":{"256":1}}],["1554476916162734e",{"2":{"258":1}}],["1554477f",{"2":{"256":1}}],["15544322",{"2":{"106":1}}],["15542f",{"2":{"77":1}}],["15701",{"2":{"220":6}}],["15700",{"2":{"220":12}}],["15707001",{"2":{"147":1}}],["1577715",{"2":{"77":1}}],["151377727482514e",{"2":{"258":1}}],["151397503750668e",{"2":{"258":1}}],["15133555",{"2":{"106":1}}],["1515926f",{"2":{"256":1}}],["151072f",{"2":{"256":1}}],["15107603",{"2":{"106":1}}],["15107606",{"2":{"106":1}}],["1510856f",{"2":{"256":1}}],["1511",{"2":{"249":1}}],["15119",{"2":{"187":1}}],["151",{"2":{"208":1,"236":1}}],["15191",{"2":{"187":1}}],["158426286117389e",{"2":{"258":1}}],["158475f",{"2":{"256":1}}],["15836",{"2":{"187":1}}],["1586047939092094",{"2":{"153":2}}],["1585735",{"2":{"106":1}}],["15802455",{"2":{"106":1}}],["15449",{"2":{"187":1}}],["154041",{"2":{"171":1}}],["1545162",{"2":{"148":1}}],["15451166",{"2":{"77":1}}],["154",{"2":{"112":1,"193":2}}],["15493166",{"2":{"106":1}}],["1549292",{"2":{"106":1}}],["150352429637832e",{"2":{"258":1}}],["1501",{"2":{"244":1}}],["15001",{"2":{"244":1}}],["1500",{"2":{"213":2,"250":1}}],["15055",{"2":{"187":1}}],["150",{"2":{"69":1}}],["15",{"2":{"17":1,"53":1,"147":1,"148":3,"171":4,"180":3,"187":14,"189":3,"197":1,"210":3,"221":3,"228":2,"229":3,"237":3,"246":3,"249":5,"251":1,"260":1}}],["1",{"2":{"2":1,"4":1,"5":2,"7":1,"11":4,"17":8,"19":10,"24":48,"34":5,"44":6,"48":6,"49":1,"52":1,"53":110,"54":1,"59":24,"62":27,"63":9,"64":1,"65":24,"67":7,"68":12,"69":13,"70":3,"77":57,"78":13,"85":1,"94":1,"102":2,"106":36,"107":5,"111":6,"113":1,"114":2,"116":35,"117":47,"126":4,"127":2,"128":2,"129":9,"131":4,"135":3,"136":11,"137":1,"140":2,"145":13,"147":23,"148":19,"150":3,"151":3,"152":3,"153":88,"168":6,"171":45,"173":4,"174":5,"176":4,"177":6,"178":1,"179":6,"180":4,"183":11,"185":1,"186":1,"187":116,"189":4,"192":5,"193":8,"194":1,"195":5,"196":2,"197":1,"206":12,"208":17,"209":7,"210":4,"213":5,"214":3,"216":1,"217":6,"218":1,"219":1,"220":118,"221":4,"224":4,"225":1,"226":1,"227":1,"228":7,"229":4,"232":90,"233":5,"234":1,"235":4,"236":17,"237":4,"241":1,"242":6,"243":3,"244":6,"245":5,"246":4,"248":6,"249":67,"250":9,"251":1,"254":37,"255":7,"256":142,"257":2,"258":135,"259":5,"260":1}}],["e^2",{"2":{"255":1}}],["err",{"2":{"245":3}}],["errs",{"2":{"245":3}}],["errored",{"2":{"115":1}}],["errors",{"2":{"45":3,"118":1,"208":1}}],["error",{"2":{"2":3,"3":1,"43":1,"45":6,"50":2,"52":1,"53":3,"55":1,"57":2,"60":1,"69":1,"87":2,"94":1,"116":3,"145":1,"179":1,"245":1,"249":2}}],["ess",{"2":{"249":3}}],["essentially",{"2":{"59":1,"62":1,"136":1,"166":1,"236":1}}],["estimated",{"2":{"149":1}}],["estimate",{"2":{"149":1,"152":1}}],["estimation",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":1,"208":4}}],["established",{"2":{"98":1,"174":1}}],["especially",{"2":{"104":1,"115":1}}],["epyc",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["epochs",{"2":{"208":2,"236":3}}],["epoch",{"2":{"59":1,"187":353,"195":2,"217":2,"228":2,"236":11}}],["eps",{"2":{"19":8,"53":1}}],["epsilon=1f",{"2":{"69":4}}],["epsilon=0",{"2":{"53":2}}],["epsilon=nothing",{"2":{"53":1}}],["epsilon",{"2":{"19":9,"53":9,"69":4}}],["emp",{"2":{"145":2,"146":2}}],["empting",{"2":{"63":1}}],["empty",{"2":{"7":1,"8":2,"52":2,"53":1,"63":2,"65":9,"67":4,"68":5,"69":5,"70":1,"135":1,"179":1,"256":1}}],["embeddings",{"2":{"67":2}}],["embedding",{"2":{"67":2,"96":1,"153":1,"226":1}}],["embed",{"2":{"59":8,"78":8,"153":1}}],["either",{"2":{"53":1,"63":4,"65":6,"254":1}}],["eagerly",{"2":{"160":1}}],["easily",{"2":{"99":1,"145":1,"157":1}}],["easier",{"2":{"52":1,"68":1,"250":1}}],["easy",{"2":{"62":1,"71":1,"100":1,"115":1,"155":1}}],["eachindex",{"2":{"249":1}}],["eachslice",{"2":{"54":2,"184":1,"185":1}}],["each",{"2":{"5":1,"19":2,"24":1,"46":4,"62":15,"63":8,"65":12,"66":4,"67":2,"68":4,"69":2,"146":3,"171":2,"207":1,"249":1,"255":1}}],["evolved",{"2":{"250":1}}],["evalpoly",{"2":{"232":2,"236":1}}],["eval=32",{"2":{"224":1}}],["eval",{"2":{"153":4,"224":5}}],["evaluate",{"2":{"50":1}}],["eventually",{"2":{"153":1}}],["even",{"2":{"3":1,"7":1,"8":1,"55":1,"81":1,"99":1,"100":1,"115":1,"117":1,"156":1,"171":1,"175":1,"196":1}}],["everystep",{"2":{"220":13}}],["everystep=false",{"2":{"215":1}}],["everything",{"2":{"100":2,"140":1}}],["every",{"2":{"48":1,"64":1,"109":1,"110":1,"160":1,"171":1}}],["ever",{"2":{"3":1,"100":1}}],["edges",{"2":{"144":1}}],["edge",{"2":{"42":1,"100":1,"104":1,"105":1,"118":1,"153":1}}],["ecosystem",{"2":{"32":1,"78":1,"101":1}}],["eccv",{"2":{"19":1}}],["effort",{"2":{"106":1}}],["efficient",{"2":{"53":1,"55":1,"74":1,"97":1,"138":1,"145":1,"146":1,"175":2,"240":1}}],["efficiently",{"2":{"27":2,"53":1}}],["effectively",{"2":{"208":2}}],["effect",{"2":{"1":1,"60":1,"70":1}}],["european",{"2":{"19":1}}],["equilibrium",{"2":{"100":1}}],["equivalent",{"2":{"8":1,"34":1,"53":2,"55":4,"59":1,"68":1,"88":1,"171":1}}],["equations",{"2":{"74":1,"257":1}}],["equals",{"2":{"68":1}}],["equal",{"2":{"24":1,"46":1,"53":1,"63":2,"65":3}}],["equally",{"2":{"19":1,"54":1}}],["et",{"2":{"17":2,"24":2,"53":2,"252":1}}],["etc",{"2":{"3":2,"7":1,"20":1,"100":1,"135":1,"138":1,"139":1,"140":1,"142":1,"145":2,"174":1,"204":1}}],["elem",{"2":{"243":2,"245":2}}],["element",{"0":{"57":1},"2":{"17":2,"24":1,"46":1,"53":1,"55":1,"57":10,"59":2,"62":1,"66":6,"69":1,"153":4,"171":4,"176":1,"177":1,"179":1,"184":1,"188":1}}],["elementwise",{"2":{"15":1,"19":1,"68":2,"69":5}}],["elements",{"2":{"8":1,"24":9,"44":1,"63":2,"65":3,"67":1,"98":1,"171":6,"184":1}}],["elman",{"2":{"66":1}}],["eltypes",{"2":{"155":1,"165":1}}],["eltype",{"0":{"165":1},"2":{"19":9,"55":6,"56":3,"57":3,"111":1,"155":5,"165":2,"173":1}}],["elseif",{"2":{"132":1,"137":1,"254":1}}],["else",{"2":{"8":1,"17":1,"28":1,"32":1,"50":1,"52":1,"54":1,"55":1,"62":1,"64":3,"66":1,"68":1,"132":1,"254":3}}],["eg",{"2":{"8":1,"63":2,"65":3}}],["energy",{"2":{"249":2}}],["environment",{"2":{"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["engine",{"2":{"173":1}}],["enforce",{"2":{"173":1}}],["enough",{"2":{"139":1,"160":1,"208":1}}],["encountered",{"2":{"117":1}}],["encounter",{"2":{"105":1,"106":1,"144":1}}],["enhance",{"2":{"104":1}}],["entries",{"2":{"146":1}}],["entry",{"2":{"125":1}}],["entropy",{"2":{"53":3}}],["enter",{"2":{"76":1}}],["entire",{"2":{"64":2,"66":5,"69":1,"110":1,"208":1}}],["entirely",{"2":{"6":1,"63":2,"67":3,"218":1,"220":1}}],["enzymemlir",{"2":{"106":1}}],["enzyme",{"2":{"49":3,"52":1,"85":2,"100":1,"102":1,"103":4,"105":1,"106":10,"144":1,"153":1,"173":1,"175":1}}],["enable",{"2":{"157":1}}],["enables",{"2":{"18":1,"100":1,"247":1}}],["enabled",{"2":{"7":1,"45":2,"94":1,"247":1}}],["ensuring",{"2":{"17":1}}],["ensures",{"2":{"120":1}}],["ensure",{"2":{"2":1,"8":1,"24":1,"100":1,"105":1,"120":1,"149":1,"184":1,"235":1}}],["endpoints",{"2":{"254":2}}],["end",{"2":{"5":2,"11":1,"24":2,"44":2,"59":14,"62":4,"63":2,"65":3,"77":1,"78":7,"106":2,"107":4,"111":3,"112":1,"113":3,"116":2,"117":4,"126":1,"127":2,"128":2,"132":1,"135":4,"136":2,"137":3,"140":3,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"153":1,"160":2,"172":1,"174":2,"179":3,"180":3,"181":1,"183":3,"184":4,"185":3,"186":1,"187":4,"189":3,"192":1,"194":2,"195":3,"197":3,"206":2,"207":1,"208":4,"209":1,"210":3,"213":1,"214":7,"215":1,"216":2,"217":3,"218":3,"221":3,"224":2,"225":3,"226":1,"227":2,"228":4,"229":3,"232":2,"236":4,"237":3,"241":3,"242":3,"243":1,"244":3,"245":2,"246":3,"248":1,"249":4,"250":3,"251":3,"254":26,"255":2,"256":2,"257":2,"259":2,"260":3}}],["enumerate",{"2":{"5":2}}],["execution",{"2":{"174":1}}],["executables",{"2":{"106":1}}],["exhaustive",{"2":{"161":1}}],["existent",{"2":{"123":1}}],["exists",{"2":{"122":1,"214":1}}],["exist",{"2":{"59":2}}],["external",{"2":{"249":1}}],["extensively",{"2":{"100":1,"104":2,"105":1,"175":1}}],["extensive",{"2":{"99":1,"100":1}}],["extensible",{"2":{"99":1}}],["extensions",{"0":{"50":1}}],["extending",{"2":{"173":1,"184":1}}],["extended",{"2":{"43":1,"44":1,"45":1,"53":2,"57":1,"59":1,"62":6,"63":2,"65":3,"66":2,"68":1,"69":4,"70":1}}],["extend",{"2":{"7":1,"43":1,"100":1}}],["ext",{"2":{"112":2}}],["extrema",{"2":{"243":2}}],["extremely",{"2":{"45":1,"52":1,"130":1,"140":1}}],["extract",{"2":{"249":2,"250":1}}],["extra",{"2":{"24":1}}],["exact",{"2":{"24":1,"136":1,"187":1}}],["exactly",{"2":{"16":1,"54":1,"70":1,"88":1,"135":1,"136":1,"142":1}}],["examples",{"0":{"169":1},"2":{"5":1,"24":2,"53":2,"59":4,"81":1,"169":1,"202":1}}],["example",{"0":{"110":1},"1":{"111":1,"112":1,"113":1,"114":1},"2":{"2":1,"3":2,"4":1,"8":1,"19":1,"24":1,"28":1,"32":1,"34":1,"43":3,"44":2,"46":3,"48":1,"49":1,"52":1,"53":14,"58":1,"62":3,"68":5,"69":3,"81":1,"88":1,"91":1,"115":1,"119":2,"126":1,"127":1,"137":1,"138":1,"139":1,"140":1,"146":4,"147":1,"148":1,"155":1,"166":1,"176":1,"177":1,"256":1}}],["excellent",{"2":{"190":1}}],["except",{"2":{"7":1,"66":2,"68":1,"156":1}}],["exceptions",{"2":{"24":1}}],["exception",{"2":{"3":2,"50":1}}],["exciting",{"2":{"82":1}}],["exclusively",{"2":{"44":1,"47":1}}],["excluding",{"2":{"34":1,"43":1,"68":1,"69":1}}],["exclude",{"2":{"10":2,"84":1}}],["exclude=internal",{"2":{"10":1}}],["exp",{"2":{"243":1}}],["exposing",{"2":{"220":1}}],["exported",{"2":{"3":1,"6":1,"42":1,"172":1}}],["exploiting",{"2":{"160":1}}],["explore",{"2":{"144":2}}],["explicit",{"2":{"74":1,"100":1,"168":2,"247":1}}],["explicitly",{"2":{"7":1,"59":2}}],["express",{"2":{"249":1}}],["expressed",{"2":{"107":1}}],["expression",{"2":{"48":1,"59":1}}],["expr",{"2":{"50":4,"59":1}}],["expected",{"2":{"53":4,"62":1,"63":2,"67":1,"100":1,"117":1,"123":1,"174":1}}],["expect",{"2":{"42":1,"100":1,"106":1,"123":1,"153":1,"247":1}}],["expects",{"2":{"28":1,"34":1,"48":1,"63":1,"65":3,"70":1,"137":1}}],["experimental",{"0":{"42":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"2":1,"4":2,"7":1,"42":2,"43":10,"44":2,"45":6,"46":2,"94":5,"95":5,"106":2,"115":1,"116":2,"117":2,"118":1,"124":1,"125":1,"126":3,"127":2,"128":2,"129":1,"130":1,"153":2,"155":1}}],["e",{"2":{"2":1,"11":1,"19":1,"24":1,"32":1,"34":1,"43":1,"53":3,"57":1,"58":1,"63":2,"64":2,"65":9,"69":3,"87":2,"106":1,"112":2,"116":4,"117":4,"120":1,"138":1,"149":1,"254":4,"255":6,"256":3}}],["gw",{"2":{"255":1}}],["gc",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["gnu",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["guess",{"2":{"258":1}}],["guide",{"0":{"120":1,"121":1},"1":{"122":1,"123":1},"2":{"121":1,"135":1}}],["guarantee",{"2":{"95":1,"166":1}}],["guarantees",{"2":{"42":1}}],["guaranteed",{"2":{"16":1,"18":1,"145":2}}],["gdev",{"2":{"92":1,"131":1,"132":4,"153":2,"160":6,"205":1,"207":3,"208":3,"209":1,"239":1,"244":3}}],["gif",{"2":{"245":1,"250":2}}],["gib",{"2":{"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["gigantic",{"2":{"115":1}}],["github",{"2":{"66":1,"71":2,"81":2,"118":1,"144":1,"153":1}}],["give",{"2":{"183":1,"258":2}}],["gives",{"2":{"54":1,"59":1,"135":1}}],["given",{"2":{"2":1,"4":2,"19":1,"20":1,"24":6,"25":24,"32":1,"37":6,"38":2,"39":5,"53":3,"62":2,"67":3,"69":5,"99":1,"100":1,"208":1}}],["g=tanh",{"2":{"66":1}}],["gs",{"2":{"59":2,"77":3,"195":1}}],["globals",{"2":{"256":1}}],["globally",{"2":{"247":1}}],["globallppool",{"2":{"65":1,"97":1}}],["globalmeanpool",{"2":{"65":1}}],["globalmaxpool",{"2":{"65":1}}],["global",{"2":{"48":1,"65":3,"153":1,"155":1,"207":1,"247":1}}],["glorot",{"2":{"24":4,"135":2}}],["gaussian",{"2":{"249":1}}],["gaussadjoint",{"2":{"217":1}}],["gated",{"2":{"66":1}}],["gamma=0",{"2":{"53":1}}],["gamma",{"2":{"53":2}}],["ganguli",{"2":{"24":1}}],["gain=1",{"2":{"24":1,"168":2}}],["gain",{"2":{"24":15,"63":4,"67":2,"96":2}}],["grid",{"2":{"243":5,"245":8}}],["green",{"2":{"236":1}}],["great",{"2":{"204":1}}],["ground",{"2":{"179":3}}],["group",{"2":{"19":3,"69":1,"249":1}}],["groups=1",{"2":{"63":2}}],["groups",{"2":{"19":4,"63":8,"69":4}}],["groupnorm",{"2":{"19":1,"69":7,"142":1}}],["gravitational",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["gravitate",{"2":{"55":1}}],["graph",{"2":{"248":1,"250":1}}],["grads",{"2":{"52":11,"107":1,"179":1}}],["gradient|jacobian",{"2":{"144":2}}],["gradient",{"0":{"49":1,"147":1},"2":{"27":2,"47":1,"49":1,"52":1,"106":8,"117":4,"139":2,"140":2,"145":6,"146":3,"147":6,"148":6,"152":3,"176":9,"179":1,"204":1,"242":3}}],["gradients",{"0":{"176":1},"2":{"27":2,"40":1,"49":10,"52":29,"53":2,"54":1,"62":1,"77":6,"122":1,"144":1,"145":1,"152":1,"176":1,"195":1,"240":1}}],["grucell",{"2":{"66":6,"96":1}}],["gru",{"2":{"7":1,"66":1}}],["günter",{"2":{"17":1}}],["gt",{"2":{"5":1,"11":1,"24":1,"53":1,"54":1,"59":2,"63":2,"65":3,"68":2,"69":2,"70":3,"79":1,"87":2,"92":1,"131":1,"136":2,"144":12,"184":1}}],["geared",{"2":{"204":1}}],["gelu",{"2":{"106":6,"107":2,"155":1}}],["geoffrey",{"2":{"19":1}}],["genericlossfunction",{"2":{"53":2}}],["genericbroadcastop",{"2":{"20":1}}],["generic",{"2":{"13":1,"16":2,"18":2,"78":1,"94":1,"106":1,"117":1,"131":1,"150":1,"151":1,"152":1,"153":1,"183":1,"185":1,"186":1,"192":1,"194":1,"195":1,"205":1,"213":1,"214":2,"215":1,"216":1,"224":1,"225":1,"226":1,"227":1,"232":1,"239":1,"242":2,"249":2,"250":1,"254":5,"256":1,"257":2}}],["generating",{"0":{"248":1},"2":{"70":1}}],["generation",{"2":{"66":3}}],["generator=lux",{"2":{"225":1}}],["generator",{"2":{"17":4,"24":3,"64":3,"135":1,"170":1,"174":2,"225":4,"226":2,"232":1}}],["generates",{"2":{"248":1}}],["generated",{"2":{"17":1,"24":1,"59":1,"64":1,"144":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"230":1,"237":1,"246":1,"250":1,"251":1,"260":1}}],["generate",{"0":{"206":1,"243":1},"2":{"3":1,"9":1,"10":1,"59":1,"66":1,"138":1,"168":2,"179":2,"183":1,"225":1,"232":4,"243":1,"248":2}}],["generalization",{"2":{"92":1}}],["generalized",{"2":{"5":1}}],["general",{"0":{"8":1},"2":{"62":1,"71":1,"79":1,"154":1,"256":1}}],["generally",{"2":{"5":1}}],["getaxes",{"2":{"225":1}}],["getindex",{"2":{"207":1}}],["getproperty",{"2":{"54":3,"137":2,"220":3}}],["getting",{"0":{"75":1},"1":{"76":1,"77":1,"78":1,"79":1,"80":1},"2":{"8":1}}],["getfield",{"2":{"7":1,"111":3,"137":3}}],["get",{"0":{"81":1},"2":{"3":5,"13":1,"14":1,"20":1,"36":2,"37":2,"38":2,"54":1,"77":1,"120":2,"137":1,"139":1,"149":2,"153":2,"165":1,"169":1,"171":2,"174":2,"183":3,"184":1,"187":2,"203":1,"249":2,"254":1}}],["g",{"2":{"2":1,"62":4,"64":2,"69":4,"106":1,"254":1}}],["goal",{"2":{"248":1}}],["goodies",{"2":{"177":1}}],["good",{"2":{"133":1,"145":1,"208":1,"249":1,"258":2}}],["goodfellow",{"2":{"68":1}}],["going",{"2":{"32":1,"147":1}}],["go",{"2":{"2":1,"81":2,"110":1,"117":1,"125":1,"136":1,"137":1,"154":1,"181":1,"238":1}}],["gpuci",{"2":{"116":1,"220":3,"244":4}}],["gpuarray",{"2":{"54":1}}],["gpuarrayscore",{"2":{"156":2}}],["gpuarrays",{"2":{"22":7,"54":1}}],["gpubroadcastop",{"2":{"20":1}}],["gpusintel",{"2":{"72":2}}],["gpusmetal",{"2":{"72":2}}],["gpusamd",{"2":{"72":2}}],["gpus",{"2":{"13":1,"37":1,"72":2,"130":1,"155":1,"156":1,"160":1}}],["gpu",{"0":{"4":1,"72":1,"80":1,"130":1,"153":1,"156":1,"163":1,"164":1},"1":{"131":1,"132":1},"2":{"1":8,"2":7,"3":10,"5":2,"16":1,"20":1,"49":1,"65":3,"72":4,"73":1,"77":2,"78":2,"80":1,"95":2,"102":1,"103":3,"106":2,"111":1,"123":4,"130":4,"131":9,"132":5,"140":1,"153":15,"160":4,"164":3,"168":1,"172":3,"187":1,"205":1,"207":3,"217":3,"220":1,"228":1,"235":3,"236":2,"239":1}}],["nparameters",{"2":{"249":3}}],["nvml",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["nvidia",{"2":{"3":1,"5":1,"72":2,"180":4,"189":4,"197":2,"210":4,"221":4,"229":4,"237":4,"246":4}}],["ndrange=length",{"2":{"153":2}}],["ndims",{"2":{"11":1,"28":4,"63":4,"65":9,"68":4,"69":2,"70":2,"111":2,"150":2,"151":2}}],["nicely",{"2":{"145":1}}],["nilarray",{"2":{"87":1}}],["nitish",{"2":{"17":1}}],["n=tanh⁡",{"2":{"66":1}}],["nn",{"2":{"63":2,"67":1,"153":7,"249":12,"250":11,"256":14,"257":1,"259":9}}],["nnlib",{"2":{"13":2,"14":2,"63":1,"70":6,"94":3,"139":2,"140":2,"158":3,"236":1}}],["ntuple",{"2":{"62":1,"63":2,"69":2,"111":1,"220":13,"244":1}}],["nlayers",{"2":{"59":2,"78":2}}],["nabla",{"2":{"176":1}}],["native",{"2":{"143":1,"204":1}}],["naming",{"2":{"62":9,"68":2}}],["name>=",{"2":{"161":3}}],["namefreezing",{"2":{"127":1}}],["name=",{"2":{"59":1,"62":1}}],["name=nothing",{"2":{"59":1,"62":11}}],["names",{"2":{"43":1,"59":1,"69":1}}],["named",{"2":{"19":2,"43":2,"53":1,"137":1,"215":2,"217":1,"249":1}}],["namedtuples",{"2":{"137":1}}],["namedtuple",{"2":{"7":4,"8":1,"10":3,"43":5,"44":1,"52":1,"54":1,"55":1,"59":3,"62":10,"63":2,"65":9,"66":2,"67":4,"68":9,"69":5,"70":1,"77":24,"78":1,"87":1,"106":7,"109":3,"111":3,"112":7,"113":3,"116":3,"117":6,"126":3,"129":4,"135":4,"136":3,"138":1,"153":2,"184":2,"220":147,"235":2,"236":1,"249":1,"256":6}}],["name",{"0":{"127":1},"2":{"7":2,"8":5,"43":1,"44":1,"59":3,"68":1,"79":1,"94":1,"111":5,"127":5,"128":6,"131":1,"137":1,"228":4}}],["navab",{"2":{"53":1}}],["nassir",{"2":{"53":1}}],["nans",{"0":{"117":1},"2":{"45":3,"117":7,"118":1}}],["nan",{"2":{"45":5,"117":31,"244":1}}],["nccl",{"2":{"36":1,"37":4,"122":1}}],["ncclbackend",{"2":{"36":2,"37":2,"120":4}}],["nₙ",{"2":{"28":2}}],["n₂",{"2":{"28":2}}],["n₁",{"2":{"28":2}}],["n",{"2":{"19":5,"54":2,"59":5,"62":23,"63":27,"65":54,"67":2,"68":11,"69":11,"70":8,"78":5,"107":1,"179":3,"187":2,"192":3,"195":1,"208":1,"213":3,"214":5,"217":1,"218":5,"224":12,"228":2,"236":1,"244":1,"248":2,"249":3,"250":2,"257":1}}],["numer",{"2":{"255":3,"256":3}}],["numeric",{"2":{"24":2}}],["numerically",{"2":{"186":1}}],["numerical",{"2":{"19":4,"53":1,"69":4,"155":1,"249":1}}],["num",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"250":2,"251":1,"260":1}}],["number=0",{"2":{"24":2}}],["number=1",{"2":{"24":2}}],["numbers",{"2":{"24":6,"25":12,"168":1,"171":3}}],["number",{"2":{"9":1,"10":1,"17":4,"19":1,"24":8,"38":1,"52":1,"53":4,"54":3,"55":1,"59":3,"62":3,"63":10,"64":3,"65":3,"67":7,"68":2,"69":3,"70":3,"135":1,"170":1,"171":1,"174":2,"175":1,"232":1,"248":1,"249":4}}],["nepochs",{"2":{"179":3,"195":3,"217":3,"228":3}}],["nest",{"2":{"145":1}}],["nested",{"0":{"29":1,"144":1,"162":1},"1":{"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1},"2":{"3":1,"29":2,"46":2,"55":5,"58":2,"102":1,"144":1,"147":2,"162":3,"238":1,"240":1,"245":1}}],["next",{"2":{"120":1,"135":1,"249":1,"250":1,"254":1,"256":1,"257":1}}],["nearest",{"2":{"70":7}}],["necessary",{"2":{"65":3,"135":1}}],["negative",{"2":{"53":1,"254":1}}],["net",{"2":{"53":1,"242":11,"249":1,"256":1,"259":2}}],["network",{"0":{"233":1,"247":1,"249":1,"256":1,"257":1,"258":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"19":1,"24":1,"78":1,"98":2,"100":1,"110":1,"145":1,"146":2,"153":2,"170":1,"181":2,"225":5,"226":3,"233":1,"248":1,"249":6,"250":4,"256":6}}],["networks",{"0":{"153":1,"241":1},"2":{"17":3,"24":3,"34":1,"53":2,"68":1,"74":1,"100":3,"152":1,"153":1,"190":1,"204":2,"241":2,"249":1}}],["never",{"2":{"32":1,"42":1,"68":1,"256":1}}],["neuralpde",{"2":{"238":2}}],["neuralode",{"2":{"214":7,"217":4,"220":5}}],["neuralodecompact",{"2":{"214":2,"217":1,"220":1}}],["neural",{"0":{"153":1,"211":1,"214":1,"215":1,"219":1,"233":1,"241":1,"247":1,"249":1,"252":1,"256":1,"257":1,"258":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"248":1,"249":1,"250":1,"251":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"17":4,"24":4,"34":1,"53":1,"58":1,"74":2,"78":1,"98":2,"100":4,"110":3,"145":3,"146":2,"152":1,"153":3,"170":1,"181":2,"190":1,"204":5,"211":2,"214":1,"215":1,"233":1,"248":1,"249":7,"250":2,"256":7,"257":1,"259":2}}],["needed",{"2":{"13":2,"27":2,"28":1,"52":1,"94":1,"122":1,"140":1,"144":1,"168":2}}],["need",{"2":{"6":1,"8":1,"32":1,"43":1,"46":1,"59":2,"68":1,"77":1,"82":1,"94":1,"106":3,"107":1,"109":1,"116":1,"120":1,"121":2,"135":3,"136":2,"140":1,"144":1,"153":1,"166":1,"169":1,"171":1,"183":1,"184":1,"193":2,"208":1,"214":1,"226":1,"235":1,"242":1,"254":1,"256":1}}],["needs",{"2":{"3":1,"7":1,"10":1,"13":2,"63":1,"64":1,"111":1,"121":1,"148":1,"164":1}}],["newtonian",{"2":{"254":2,"256":1,"258":1}}],["new",{"0":{"85":1,"88":1,"92":1,"97":1,"219":1},"2":{"1":1,"44":9,"46":5,"52":1,"59":1,"64":1,"66":4,"68":5,"82":1,"84":1,"95":1,"106":1,"109":1,"121":1,"130":1,"133":1,"135":1,"136":1,"144":1,"153":1,"225":2,"249":3}}],["nom",{"2":{"249":1}}],["noisy",{"2":{"183":2}}],["noise",{"2":{"179":1}}],["now",{"2":{"85":1,"92":1,"95":1,"96":4,"97":2,"106":4,"107":1,"116":2,"117":2,"121":1,"123":3,"135":1,"136":1,"140":1,"145":1,"146":2,"147":1,"148":1,"152":2,"153":2,"155":1,"170":2,"175":1,"179":1,"184":2,"186":1,"193":1,"196":1,"208":1,"236":1,"247":1,"249":2,"250":1,"254":1,"256":2,"257":1,"259":1}}],["norm",{"2":{"69":1,"145":7,"146":7,"147":6,"148":6,"152":12}}],["normally",{"2":{"106":1}}],["normal",{"2":{"22":4,"24":12,"25":6,"168":6,"249":1,"256":3}}],["normalises",{"2":{"19":2,"69":2}}],["normalize",{"2":{"69":2}}],["normalized",{"2":{"19":8,"53":3,"69":7,"245":2}}],["normalization",{"0":{"19":1,"69":1},"2":{"19":12,"24":1,"28":1,"69":11,"146":1}}],["normalizing",{"2":{"17":1}}],["nooplayer",{"2":{"62":6,"64":3,"68":3}}],["nonlinear",{"2":{"24":1}}],["non",{"0":{"114":1},"2":{"16":1,"18":1,"24":2,"37":2,"43":1,"45":1,"52":2,"53":1,"59":9,"67":1,"87":1,"90":1,"115":1,"123":1,"140":2,"203":1,"214":2,"254":1}}],["nonetheless",{"2":{"2":1}}],["none",{"2":{"2":1,"42":1,"43":1,"45":1,"57":1,"117":1}}],["node",{"2":{"3":1}}],["notice",{"2":{"135":1,"136":1,"146":2,"217":1}}],["notion",{"2":{"55":1}}],["notangent",{"2":{"49":1,"117":1}}],["not",{"2":{"2":1,"3":4,"4":2,"6":1,"7":1,"8":1,"15":1,"16":1,"17":1,"19":3,"24":2,"28":1,"34":3,"36":2,"45":1,"46":1,"47":1,"50":1,"52":1,"54":1,"57":1,"58":1,"59":1,"62":1,"66":11,"68":1,"69":1,"77":1,"87":2,"94":1,"100":1,"101":1,"104":1,"105":2,"106":1,"116":1,"117":1,"120":1,"123":1,"131":1,"133":1,"140":3,"144":1,"145":5,"149":1,"152":1,"153":1,"160":1,"167":1,"169":1,"196":2,"203":1,"208":1,"214":1,"217":2,"219":1,"220":2,"233":1,"249":1,"250":1,"254":1}}],["notes",{"2":{"16":1,"18":1,"49":1,"177":1,"211":1}}],["note",{"2":{"2":1,"3":4,"13":2,"24":1,"35":1,"43":1,"52":1,"53":1,"55":2,"58":1,"59":1,"68":1,"74":1,"105":1,"106":1,"107":1,"116":1,"120":1,"135":1,"136":1,"143":1,"144":1,"147":1,"156":1,"173":1,"176":1,"188":1,"214":1,"220":1,"247":1}}],["nothing",{"2":{"2":4,"3":5,"4":1,"8":2,"15":1,"16":1,"17":2,"18":1,"19":15,"36":3,"37":1,"43":5,"49":1,"53":12,"54":1,"55":2,"58":1,"62":4,"63":4,"64":1,"66":11,"67":4,"68":10,"69":5,"77":27,"78":1,"106":9,"112":4,"117":1,"145":2,"147":2,"148":2,"206":1,"207":4,"208":2,"220":117,"236":6,"249":1,"256":1}}],["no",{"2":{"2":2,"4":2,"5":1,"34":1,"42":1,"43":3,"48":1,"52":1,"53":2,"57":1,"60":1,"68":1,"77":1,"89":1,"92":1,"94":1,"100":1,"112":1,"116":1,"117":1,"122":3,"131":2,"132":1,"135":1}}],["u=u",{"2":{"242":1}}],["u0=res",{"2":{"208":1}}],["u0",{"2":{"206":2,"208":2,"209":1,"255":2,"256":2,"257":1,"259":2}}],["utc",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["utility",{"0":{"216":1,"227":1,"254":1}}],["utilities",{"0":{"51":1},"1":{"52":1,"53":1,"54":1,"55":1,"56":1,"57":1,"58":1,"59":1,"60":1},"2":{"47":1,"140":1}}],["utils",{"0":{"35":1},"1":{"36":1,"37":1,"38":1,"39":1,"40":1,"41":1},"2":{"43":2,"55":1,"145":4}}],["url",{"2":{"74":1}}],["url=",{"2":{"71":1}}],["u",{"2":{"27":7,"206":4,"208":11,"209":6,"214":7,"218":2,"241":4,"242":11,"244":4,"245":7,"255":3,"256":4,"258":1,"259":2}}],["ulyanov",{"2":{"19":1,"69":1}}],["upto",{"2":{"240":1}}],["upsampled",{"2":{"70":1}}],["upsample",{"2":{"70":5,"96":1}}],["upsampling",{"0":{"70":1},"2":{"70":2}}],["upscaling",{"2":{"70":1}}],["upscale",{"2":{"70":2}}],["up",{"0":{"257":1},"2":{"54":1,"149":1,"152":1,"203":2,"211":1}}],["updating",{"0":{"56":1,"82":1},"1":{"83":1,"84":1,"85":1,"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1},"2":{"19":2,"40":1,"76":1,"179":1}}],["updates",{"2":{"46":1,"52":3,"69":1,"90":1,"120":1}}],["updated",{"2":{"15":1,"17":2,"19":2,"46":1,"52":5,"53":1,"62":6,"64":3,"66":7,"68":1,"69":1,"78":1,"96":1,"99":1,"100":1,"123":1,"135":1,"184":1,"203":1,"235":1}}],["update",{"2":{"10":2,"15":1,"17":2,"52":2,"56":1,"59":1,"64":3,"66":5,"69":3,"76":2,"82":1,"173":1}}],["upon",{"2":{"5":1}}],["untrained",{"2":{"256":2,"259":1}}],["until",{"2":{"64":1}}],["uncertain",{"2":{"250":1}}],["unchanged",{"2":{"62":1}}],["unable",{"2":{"173":1}}],["unnecessary",{"2":{"146":1}}],["unnormalized",{"2":{"69":1}}],["unexpected",{"2":{"100":1}}],["un",{"2":{"94":1}}],["unreleased",{"2":{"71":1}}],["unreasonably",{"2":{"62":1}}],["unrolls",{"2":{"62":1}}],["unroll",{"2":{"13":1}}],["unwrapped",{"2":{"55":1}}],["unwrap",{"2":{"55":2}}],["unwraps",{"2":{"43":1}}],["undone",{"2":{"46":1}}],["undef",{"2":{"174":1}}],["undefined",{"2":{"8":1,"66":1}}],["underlying",{"2":{"214":1}}],["under",{"2":{"140":1}}],["understand",{"2":{"59":1,"133":1,"184":1,"211":1,"214":1}}],["understanding",{"2":{"24":2,"225":1}}],["undesirable",{"2":{"11":1}}],["unfreezes",{"2":{"43":1}}],["unfreeze",{"2":{"43":2}}],["unlike",{"2":{"25":1,"59":1}}],["unless",{"2":{"2":1,"45":1,"225":1}}],["unified",{"2":{"204":1}}],["uniformly",{"2":{"120":1}}],["uniform",{"2":{"24":6,"25":6,"63":5,"66":6,"67":4,"77":1,"135":3}}],["uninitiated",{"0":{"170":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1}}],["universal",{"2":{"68":1}}],["unity",{"2":{"254":1}}],["unitrange",{"2":{"173":1}}],["unit",{"2":{"66":1}}],["union",{"2":{"2":3,"8":1,"17":1,"24":2,"34":1,"45":2,"53":3,"66":2,"69":1}}],["unpack",{"2":{"8":1}}],["unsupported",{"2":{"8":1,"22":1}}],["unsafe",{"2":{"5":1}}],["unknown",{"2":{"3":2}}],["unknowndevice",{"2":{"3":2}}],["usr",{"2":{"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["usacases",{"2":{"103":2}}],["usage",{"2":{"43":1,"81":1}}],["usual",{"2":{"69":3,"148":1,"153":1}}],["usually",{"2":{"53":2,"59":1,"70":1,"171":1,"208":1}}],["us",{"2":{"27":2,"42":1,"115":1,"116":1,"117":6,"136":2,"137":3,"140":1,"144":1,"170":1,"174":1,"177":1,"178":1,"179":3,"183":1,"219":1,"256":1,"257":1,"259":2}}],["usecases",{"2":{"55":1,"58":1,"153":1,"238":1}}],["uses",{"2":{"16":1,"18":2,"32":1,"55":4,"56":1,"58":1,"69":1,"87":1,"144":1,"147":1,"155":1,"158":2,"161":1,"174":1,"204":1,"254":2,"255":1,"256":1,"258":1}}],["userbase",{"2":{"32":1}}],["user",{"2":{"7":2,"8":2,"13":2,"58":1,"62":1,"98":1,"109":1,"140":1,"142":2,"147":1}}],["users",{"2":{"6":1,"7":2,"15":1,"36":2,"37":1,"43":1,"52":1,"55":1,"87":1,"90":1,"98":1,"100":1,"155":1,"173":1,"211":2,"214":1,"220":1}}],["useful",{"2":{"3":1,"20":1,"24":3,"45":2,"53":1,"54":1,"57":1,"59":2,"74":1,"115":1,"116":1}}],["use",{"0":{"100":1,"143":1},"2":{"2":1,"4":2,"5":1,"13":3,"15":1,"17":1,"18":2,"20":1,"24":1,"27":2,"28":1,"36":4,"42":1,"43":2,"45":1,"47":2,"52":4,"59":1,"63":11,"66":18,"67":16,"69":3,"71":1,"74":1,"77":2,"78":1,"81":2,"90":1,"94":2,"95":4,"96":1,"97":1,"100":1,"103":3,"105":1,"106":3,"107":2,"115":1,"116":1,"120":2,"124":1,"126":1,"131":1,"143":1,"144":3,"145":1,"146":1,"147":2,"149":2,"153":4,"155":2,"158":1,"159":1,"163":1,"164":2,"167":2,"169":1,"170":1,"171":1,"174":1,"175":1,"176":1,"177":2,"179":3,"183":2,"184":3,"186":2,"190":2,"192":1,"211":1,"213":1,"214":1,"215":2,"217":2,"234":1,"235":1,"236":1,"238":1,"242":1,"244":4,"247":1,"248":1,"249":5,"250":1,"254":1,"256":5}}],["used",{"2":{"2":2,"3":6,"8":2,"17":3,"24":1,"32":1,"34":2,"41":1,"45":2,"46":2,"52":2,"53":5,"55":1,"58":1,"59":11,"63":3,"64":3,"66":1,"67":1,"68":1,"69":7,"70":1,"87":1,"91":1,"94":1,"95":1,"97":1,"106":1,"123":1,"130":1,"131":2,"144":1,"145":2,"155":1,"166":1,"167":1,"169":1,"171":1,"218":1,"252":1}}],["using",{"0":{"33":1,"48":1,"106":1,"107":1,"114":1,"146":1,"150":1,"151":1,"152":1,"185":1,"204":1,"211":1,"218":1,"230":1},"1":{"34":1,"107":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"2":1,"3":1,"4":2,"5":2,"7":2,"20":3,"24":2,"27":2,"32":3,"34":4,"39":1,"40":1,"41":1,"43":1,"44":1,"48":2,"49":1,"52":7,"53":4,"55":1,"59":9,"62":11,"66":6,"67":1,"68":2,"69":1,"71":2,"76":1,"77":3,"78":4,"87":1,"92":1,"97":1,"98":1,"99":1,"100":2,"106":8,"107":2,"116":1,"119":1,"123":1,"130":3,"132":2,"133":2,"137":2,"139":2,"142":1,"143":1,"144":4,"145":6,"146":1,"147":1,"148":2,"149":5,"152":14,"153":3,"155":1,"156":1,"163":1,"170":1,"171":2,"174":1,"177":2,"178":1,"179":1,"180":1,"181":2,"183":1,"185":1,"188":1,"189":1,"190":1,"191":2,"196":1,"197":1,"204":1,"205":1,"208":1,"210":1,"212":2,"214":1,"218":1,"220":2,"221":1,"229":1,"231":1,"233":1,"237":1,"238":3,"239":1,"240":1,"241":1,"242":1,"246":1,"247":2,"249":1,"250":1,"251":1,"253":1,"255":1,"256":1,"260":1}}],["w=w",{"2":{"242":1}}],["w=gv∥v∥weight",{"2":{"69":1}}],["w=",{"2":{"59":1}}],["w=rand",{"2":{"59":1}}],["w=ones",{"2":{"59":3}}],["w3",{"2":{"59":3,"78":3}}],["w3=dense",{"2":{"59":1,"78":1}}],["w2",{"2":{"59":3,"78":3}}],["w2=",{"2":{"59":1,"78":1}}],["w1",{"2":{"59":3,"78":3}}],["w1=dense",{"2":{"59":1,"78":1}}],["w",{"2":{"59":21,"65":6,"67":1,"70":4,"78":4,"137":2,"158":1,"171":1,"179":5,"192":1,"213":1,"241":4,"242":10}}],["wrote",{"0":{"98":1},"1":{"99":1,"100":1},"2":{"203":1}}],["write",{"2":{"171":2,"173":1,"179":1,"225":1}}],["writing",{"0":{"113":1},"2":{"94":1,"114":1,"133":1}}],["written",{"2":{"32":1}}],["wrapping",{"2":{"87":1}}],["wrappers",{"0":{"27":1}}],["wrapper",{"2":{"14":1,"27":2,"45":1,"58":1,"66":1,"136":2,"148":1,"236":1,"241":1}}],["wrappedlayer",{"2":{"64":1}}],["wrappedfunction",{"2":{"63":1,"68":2,"117":3,"220":9,"256":1}}],["wrapped",{"2":{"7":3,"45":1,"62":8,"68":2,"136":1}}],["wrap",{"2":{"40":1,"115":1,"120":1,"136":1,"207":1}}],["wraps",{"2":{"34":1,"37":2,"66":2,"68":1}}],["wrt",{"2":{"27":2,"49":1,"52":3,"53":2,"54":1,"121":1,"148":2,"242":1}}],["wu",{"2":{"19":1}}],["wide",{"2":{"175":1}}],["wider",{"2":{"85":1,"89":1,"100":1,"177":1}}],["width",{"2":{"63":1}}],["wio",{"2":{"66":1}}],["wio×x+who×hprev+bo",{"2":{"66":1}}],["wig",{"2":{"66":1}}],["wig×x+whg×hprev+bg",{"2":{"66":1}}],["wif",{"2":{"66":1}}],["wif×x+whf×hprev+bf",{"2":{"66":1}}],["wii",{"2":{"66":1}}],["wii×x+whi×hprev+bi",{"2":{"66":1}}],["wiz",{"2":{"66":1}}],["wiz×x+biz+whz×hprev+bhz",{"2":{"66":1}}],["wir",{"2":{"66":1}}],["wir×x+bir+whr×hprev+bhr",{"2":{"66":1}}],["win",{"2":{"66":1}}],["win×x+bin+r⋅",{"2":{"66":1}}],["window",{"2":{"65":18}}],["will",{"2":{"3":2,"5":1,"11":1,"19":1,"24":3,"28":4,"32":4,"34":1,"39":2,"42":1,"45":2,"46":3,"48":2,"49":1,"50":4,"52":1,"55":4,"57":4,"58":1,"59":8,"62":5,"64":2,"66":3,"68":1,"69":3,"71":1,"77":1,"78":1,"87":1,"106":5,"110":1,"115":1,"116":1,"117":2,"125":1,"130":1,"133":1,"136":1,"139":1,"140":2,"144":2,"145":3,"147":1,"149":2,"150":1,"153":8,"155":2,"157":1,"167":1,"169":1,"171":1,"172":1,"173":1,"174":2,"175":4,"176":1,"177":1,"179":2,"181":2,"183":3,"184":6,"186":1,"190":2,"196":2,"203":1,"204":2,"207":4,"208":1,"214":1,"217":1,"230":1,"234":1,"235":1,"236":2,"238":3,"240":1,"241":1,"242":2,"243":3,"247":2,"249":4,"256":1}}],["withgradient",{"2":{"59":1}}],["within",{"2":{"17":2,"19":2,"59":1,"145":2}}],["without",{"2":{"2":1,"6":1,"59":1,"87":2,"100":2,"106":1,"107":1,"133":1,"145":2,"173":1,"184":1,"208":2}}],["with",{"0":{"190":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1},"2":{"1":1,"2":1,"3":1,"5":2,"6":1,"7":2,"8":1,"10":2,"11":1,"13":7,"15":1,"16":2,"18":1,"20":1,"24":5,"28":2,"32":1,"34":1,"37":2,"40":1,"41":2,"43":5,"44":2,"45":3,"46":1,"48":2,"49":1,"52":2,"53":6,"54":2,"55":2,"56":3,"57":1,"59":3,"62":14,"63":11,"64":6,"65":12,"66":10,"67":12,"68":5,"69":3,"70":5,"78":3,"84":1,"85":1,"87":2,"94":3,"96":1,"97":1,"98":1,"100":7,"101":2,"103":1,"104":2,"105":2,"106":2,"107":1,"114":2,"115":3,"116":1,"117":1,"120":2,"122":2,"123":1,"124":1,"131":2,"133":1,"135":1,"136":1,"137":6,"139":1,"144":3,"145":1,"146":4,"147":1,"150":1,"151":1,"152":1,"153":5,"155":1,"157":1,"158":6,"163":1,"167":2,"168":2,"169":2,"170":1,"171":5,"172":2,"173":1,"174":2,"175":1,"177":2,"179":4,"181":1,"183":1,"185":1,"186":1,"187":1,"192":1,"194":1,"195":1,"203":1,"204":2,"205":1,"208":1,"213":1,"214":3,"215":1,"216":1,"224":1,"225":2,"226":1,"227":1,"232":1,"239":1,"240":1,"242":2,"247":2,"248":1,"249":6,"250":2,"254":5,"256":2,"257":2}}],["what",{"2":{"116":1,"117":1,"135":1,"137":1,"142":1,"148":1,"165":1,"171":2}}],["whatever",{"2":{"52":1,"68":1}}],["why",{"0":{"98":1,"100":1},"1":{"99":1,"100":1},"2":{"135":1}}],["whole",{"2":{"69":2}}],["whose",{"2":{"67":1,"69":1}}],["who",{"2":{"66":1}}],["whg",{"2":{"66":1}}],["whf",{"2":{"66":1}}],["whn",{"2":{"66":1}}],["whn×hprev+bhn",{"2":{"66":1}}],["whz",{"2":{"66":1}}],["whr",{"2":{"66":1}}],["whcn",{"2":{"63":1,"64":2,"69":3}}],["whi",{"2":{"66":1}}],["while",{"2":{"3":1,"34":1,"44":1,"58":1,"62":1,"70":1,"100":1,"107":1,"128":1,"133":1,"135":1,"155":1,"177":1}}],["which",{"0":{"109":1},"2":{"2":1,"3":1,"7":1,"19":1,"24":1,"42":1,"43":14,"46":1,"52":1,"53":5,"58":1,"59":5,"62":3,"63":1,"65":3,"66":1,"68":3,"69":4,"81":1,"96":1,"97":1,"100":2,"115":2,"133":1,"135":1,"136":2,"137":1,"142":1,"149":1,"155":1,"170":1,"171":2,"173":1,"179":1,"185":1,"208":1,"214":1,"218":1,"236":1,"247":1,"249":1,"252":1,"255":1,"256":1}}],["whether",{"2":{"34":1,"45":2,"64":1,"183":1}}],["whereas",{"2":{"63":1}}],["where",{"2":{"3":1,"13":2,"16":1,"18":1,"19":1,"20":1,"24":7,"44":1,"45":1,"53":4,"58":1,"63":5,"65":6,"66":6,"67":4,"68":3,"69":3,"70":1,"105":1,"117":2,"140":1,"159":1,"176":1,"184":1,"185":1,"224":1,"249":2,"250":2,"254":3}}],["when",{"2":{"3":2,"16":1,"24":2,"34":2,"48":1,"53":2,"54":1,"59":2,"63":1,"67":1,"68":1,"70":1,"104":1,"122":1,"127":1,"135":2,"144":1,"156":1,"165":1,"175":2,"184":1,"196":1,"214":1,"254":1,"257":1}}],["world",{"2":{"208":1}}],["worthwhile",{"2":{"100":1}}],["word",{"2":{"67":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["words",{"2":{"63":1}}],["workaround",{"2":{"153":1}}],["worker",{"2":{"160":1}}],["workers",{"2":{"38":3,"39":5}}],["worked",{"2":{"85":1,"147":1}}],["working",{"2":{"45":1,"101":1,"103":2,"105":1,"106":1,"123":1,"124":1,"131":1,"169":1,"248":1}}],["work",{"2":{"2":1,"5":1,"43":1,"68":1,"70":1,"74":1,"104":1,"105":1,"109":1,"118":1,"120":1,"123":1,"133":1,"140":1,"144":1,"167":1,"171":2,"176":1}}],["works",{"2":{"2":1,"32":1,"45":1,"55":1,"58":1,"70":2,"92":1,"100":1,"105":1,"114":1,"122":1,"135":2,"137":1,"144":1,"145":1,"153":1,"160":1,"171":1,"214":1}}],["would",{"2":{"7":2,"8":1,"63":2,"68":3,"87":1,"98":1,"100":1,"106":2,"116":1,"127":1,"140":1}}],["wondered",{"2":{"100":1}}],["won",{"2":{"2":1,"8":1,"59":1,"90":1,"110":1,"184":1,"204":1}}],["waveform",{"2":{"254":5,"255":7,"256":9,"257":4,"259":12}}],["waveforms",{"0":{"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1}}],["wall",{"2":{"249":1}}],["wan",{"2":{"144":1}}],["wants",{"2":{"142":1}}],["wanted",{"2":{"100":1}}],["want",{"0":{"72":1,"73":1},"2":{"8":2,"52":2,"71":1,"78":1,"107":1,"110":1,"127":1,"133":1,"145":1,"148":1,"171":1,"177":1,"204":1,"207":1,"243":1}}],["warmup",{"2":{"195":1,"257":1}}],["warde",{"2":{"68":1}}],["warntype",{"2":{"220":3}}],["warn",{"2":{"57":3,"60":1}}],["warning",{"2":{"2":3,"3":1,"4":2,"8":1,"16":1,"27":2,"32":3,"34":2,"47":1,"52":1,"53":1,"57":5,"58":1,"63":1,"69":1,"125":1,"131":1,"144":1,"145":2,"153":1,"202":1,"203":1,"244":4}}],["way",{"2":{"17":2,"32":3,"34":2,"62":1,"100":1,"115":1,"133":2,"150":1,"161":1,"185":2,"249":1}}],["wasteful",{"2":{"135":1}}],["was",{"2":{"2":1,"5":3,"7":1,"11":1,"24":1,"32":1,"46":1,"63":1,"87":3,"89":1,"90":1,"96":1,"106":1,"122":2,"155":2,"180":1,"189":1,"197":1,"208":1,"210":1,"214":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["welcome",{"2":{"144":1}}],["well",{"2":{"34":1,"53":1,"98":1,"101":1,"103":1,"106":1,"145":1,"146":1,"170":1}}],["weird",{"2":{"135":1,"137":2}}],["weightnorm",{"2":{"69":4,"100":1,"135":1}}],["weightih×x+biasih+weighthh×hprev+biashh",{"2":{"66":1}}],["weighting",{"2":{"53":1}}],["weightinitializers",{"0":{"21":1,"89":1},"1":{"22":1,"23":1,"24":1,"25":1},"2":{"3":1,"24":8,"25":24,"135":3,"168":2,"256":1}}],["weight=truncated",{"2":{"256":3}}],["weight=l",{"2":{"135":1}}],["weight=randn32",{"2":{"96":1}}],["weight=rand32",{"2":{"67":1,"96":1}}],["weight=ones32",{"2":{"67":1}}],["weight=glorot",{"2":{"63":1,"135":1}}],["weight=nothing",{"2":{"63":1,"66":3,"67":3}}],["weight=ps",{"2":{"59":1}}],["weight=zero",{"2":{"44":1}}],["weights",{"0":{"168":1},"1":{"169":1},"2":{"53":1,"66":4,"168":10,"169":5,"225":1,"249":2,"250":3}}],["weight",{"2":{"16":4,"18":4,"21":1,"24":3,"43":3,"44":4,"46":4,"63":6,"66":12,"67":24,"69":5,"77":15,"106":12,"112":2,"117":4,"126":4,"127":2,"128":4,"135":6,"137":4,"145":2,"147":4,"148":4,"168":1,"179":1,"220":30,"225":5,"226":2,"235":2,"249":1,"256":3,"258":3}}],["weak",{"2":{"47":1}}],["weren",{"2":{"94":1}}],["were",{"2":{"11":1,"82":1,"89":1,"94":2,"144":1}}],["we",{"0":{"98":1,"143":1},"1":{"99":1,"100":1},"2":{"2":4,"3":3,"7":2,"8":4,"13":2,"16":2,"18":3,"32":1,"37":1,"42":2,"47":1,"52":2,"53":2,"55":5,"57":1,"59":3,"62":3,"63":4,"66":6,"67":4,"77":4,"78":2,"82":2,"87":1,"89":1,"90":2,"94":1,"95":1,"96":1,"100":4,"101":1,"103":2,"104":3,"105":3,"106":13,"107":2,"109":1,"110":3,"114":1,"116":3,"117":6,"118":2,"121":1,"123":3,"124":2,"125":1,"126":1,"128":1,"133":1,"135":5,"136":4,"137":4,"138":1,"139":2,"140":2,"143":1,"144":3,"145":2,"146":3,"147":3,"148":3,"149":3,"152":2,"153":11,"155":3,"159":1,"160":1,"170":1,"171":11,"173":4,"174":9,"175":6,"177":2,"179":3,"181":1,"183":3,"184":11,"185":2,"186":1,"187":1,"188":3,"190":2,"193":2,"196":3,"203":1,"204":4,"207":4,"208":5,"211":1,"214":3,"217":1,"219":1,"220":2,"225":2,"230":1,"234":1,"235":2,"236":2,"238":3,"240":2,"241":2,"242":3,"243":3,"247":3,"248":1,"249":8,"250":7,"254":3,"256":9,"257":1}}],[">randn32",{"2":{"59":2}}],[">",{"2":{"2":3,"3":8,"8":2,"24":8,"25":24,"53":3,"59":2,"66":1,"106":1,"111":1,"117":2,"139":2,"140":4,"145":2,"146":2,"147":2,"148":2,"152":1,"186":1,"232":1,"236":1,"244":2,"254":1,"258":1}}],["cycle",{"2":{"145":2,"244":2}}],["cdev",{"2":{"131":1,"132":1,"205":1,"209":1,"239":1,"244":2}}],["cdims",{"2":{"16":3}}],["circle",{"2":{"255":1,"256":2,"259":4}}],["circumvented",{"2":{"132":1}}],["circular",{"2":{"24":1,"63":1}}],["ci",{"2":{"100":1,"202":1}}],["cite",{"2":{"74":2}}],["citation",{"0":{"74":1}}],["cnew",{"2":{"66":2}}],["cnew=f⋅cprev+i⋅ghnew=o⋅tanh",{"2":{"66":1}}],["cvpr",{"2":{"53":1}}],["cluster",{"2":{"250":1}}],["clockwise",{"2":{"181":1,"183":4}}],["closest",{"2":{"112":1}}],["closures",{"2":{"59":1}}],["cl",{"2":{"136":3,"168":4}}],["classifying",{"2":{"250":1}}],["classify",{"2":{"181":1,"248":1}}],["classifier=st",{"2":{"184":1}}],["classifier",{"0":{"184":1},"2":{"184":11,"185":3,"249":1}}],["classifiers",{"2":{"53":2}}],["classified",{"2":{"53":1}}],["classification",{"0":{"190":1,"211":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1},"2":{"24":2,"53":2,"247":1}}],["classes",{"2":{"53":1}}],["class",{"0":{"104":1},"2":{"53":1,"102":1,"105":1,"194":5,"216":5,"227":5,"247":1}}],["clear",{"2":{"46":1,"117":1}}],["c",{"2":{"44":3,"63":4,"65":21,"66":1,"70":5,"184":2,"192":1,"213":1,"220":9,"250":2,"254":4}}],["cell=st",{"2":{"184":1}}],["cell`",{"2":{"184":1}}],["cells",{"2":{"66":1,"96":1}}],["cell",{"2":{"66":45,"184":10,"185":4}}],["central",{"2":{"24":1,"220":13}}],["certain",{"0":{"141":1},"1":{"142":1},"2":{"7":1,"8":2,"13":1,"54":1,"64":2,"87":1,"94":1,"98":1,"106":1,"120":1,"124":1,"125":1,"155":1,"167":1,"214":1}}],["ca",{"2":{"208":2,"225":3}}],["cairomakie",{"2":{"205":1,"206":1,"209":1,"231":1,"232":1,"236":1,"239":1,"245":2,"247":1,"248":1,"253":1,"255":1,"256":1,"259":2}}],["capabilities",{"2":{"172":1,"238":1}}],["capture",{"2":{"144":1}}],["causing",{"2":{"155":1}}],["causes",{"2":{"124":1}}],["cause",{"2":{"8":1,"13":1,"46":1,"155":1}}],["cat",{"2":{"111":1,"183":1}}],["catch",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"52":1,"112":1,"116":2,"117":2,"157":2}}],["care",{"2":{"174":1}}],["cartesian",{"2":{"67":1}}],["carry",{"2":{"66":4,"184":3,"185":3}}],["caching",{"2":{"52":1}}],["cached",{"2":{"52":2}}],["cache",{"2":{"52":2,"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["case",{"2":{"43":1,"57":1,"66":19,"122":1,"135":1,"143":1,"145":1,"179":1,"208":1,"225":1,"250":1,"256":1}}],["cases",{"2":{"8":3,"24":1,"32":1,"42":1,"52":2,"55":6,"57":1,"58":1,"59":1,"71":1,"100":1,"104":1,"105":1,"118":2,"139":1,"146":1,"153":1,"159":1,"173":1}}],["caveats",{"2":{"24":1}}],["calculation",{"2":{"145":1,"146":3}}],["calculations",{"2":{"34":1}}],["calculates",{"2":{"65":3}}],["calculate",{"2":{"63":4,"65":3,"67":1}}],["calculated",{"2":{"7":2,"17":1,"53":7}}],["caller",{"2":{"244":4}}],["called",{"2":{"3":1,"17":2,"19":2,"37":1,"62":2,"64":3,"127":1,"135":1,"140":2,"142":1,"169":1,"171":2,"184":1}}],["callback",{"2":{"208":4,"257":3,"258":1}}],["callable",{"2":{"62":2}}],["calls",{"2":{"7":1,"8":2,"144":2}}],["call",{"2":{"7":1,"8":1,"17":1,"44":1,"48":6,"58":1,"64":5,"69":3,"77":1,"100":1,"144":1,"145":2,"153":1,"168":4,"174":2,"184":1}}],["calling",{"2":{"5":1,"7":1,"8":1,"37":1,"48":1,"107":1,"120":1,"123":1,"249":1}}],["candidates",{"2":{"112":1}}],["cannot",{"2":{"24":1,"59":1,"99":1,"135":1,"136":1,"140":1,"153":1,"202":1}}],["can",{"0":{"143":1},"2":{"2":2,"3":4,"4":2,"7":3,"8":3,"11":1,"13":1,"15":1,"16":1,"17":2,"18":1,"19":14,"20":1,"37":3,"43":1,"45":1,"48":1,"49":1,"52":1,"53":2,"55":1,"58":1,"59":5,"60":1,"62":6,"63":6,"65":3,"66":7,"67":3,"68":3,"71":2,"76":2,"77":3,"78":1,"79":1,"95":1,"106":2,"114":1,"115":2,"116":1,"117":2,"120":1,"126":1,"128":1,"130":1,"132":1,"136":1,"137":3,"140":1,"142":1,"143":1,"144":2,"145":2,"146":3,"148":1,"149":2,"153":1,"155":2,"156":1,"157":1,"160":1,"163":1,"166":1,"167":1,"168":1,"169":1,"171":11,"172":2,"174":1,"176":1,"179":1,"184":1,"185":1,"187":1,"188":1,"208":1,"214":2,"217":1,"218":1,"226":1,"240":1,"241":1,"249":2,"250":3,"254":1,"256":1}}],["coordinates",{"2":{"250":1}}],["cos",{"2":{"147":1,"243":1,"254":2,"255":3,"256":8}}],["cover",{"2":{"139":1}}],["covered",{"2":{"118":1}}],["covariate",{"2":{"19":1}}],["could",{"2":{"135":1,"136":1,"249":1}}],["course",{"2":{"114":1,"145":1}}],["courville",{"2":{"68":1}}],["coupled",{"2":{"98":1,"167":1}}],["counterpart",{"2":{"143":1}}],["count",{"2":{"59":1}}],["colormap=",{"2":{"250":3}}],["colorbar",{"2":{"245":1}}],["color=",{"2":{"206":2,"209":4,"232":2,"236":2,"248":2}}],["columns",{"2":{"171":1}}],["column",{"2":{"67":1,"171":4}}],["collects",{"2":{"62":1}}],["collect",{"2":{"59":1,"78":1,"183":2,"192":2,"213":2,"232":1,"243":4,"245":1,"250":4}}],["cols",{"2":{"24":1}}],["coefficient",{"2":{"53":2}}],["corner",{"2":{"70":1}}],["corners",{"2":{"70":1,"96":1}}],["corners=false",{"2":{"70":1}}],["cores",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["core",{"2":{"32":1,"48":3,"100":1,"139":1,"173":1,"180":1,"189":1,"197":1,"210":1,"220":2,"221":1,"225":5,"226":3,"229":1,"237":1,"244":4,"246":1,"251":1,"260":1}}],["correlation=true",{"2":{"63":2,"95":1,"97":1}}],["correlation=false",{"2":{"63":2}}],["correlation",{"2":{"63":7}}],["corresponding",{"2":{"4":3,"44":2,"46":2,"53":3,"67":1,"70":2,"91":1,"94":1,"131":1,"146":1}}],["corresponds",{"2":{"2":1,"208":1}}],["corrections",{"2":{"114":1}}],["correctness",{"0":{"49":1},"2":{"47":1,"55":1}}],["correct",{"0":{"113":1},"2":{"3":2,"49":1,"155":2,"194":3,"216":3,"227":3}}],["correctly",{"2":{"2":1,"45":1,"118":1,"120":1}}],["codebases",{"2":{"58":1}}],["code",{"0":{"120":1},"2":{"32":1,"34":1,"42":2,"55":4,"59":4,"78":1,"82":1,"103":1,"106":1,"120":1,"130":1,"138":1,"142":1,"143":1,"145":2,"155":1,"156":1,"157":1,"161":1,"170":1,"185":1,"187":1,"220":3,"248":1,"252":2}}],["combination",{"2":{"144":1}}],["combined",{"2":{"66":2,"77":1}}],["combines",{"2":{"62":1}}],["come",{"2":{"116":1,"155":1,"177":1}}],["comes",{"2":{"62":1,"98":1,"100":2,"145":1,"147":1}}],["coming",{"2":{"78":1,"135":1,"155":1}}],["com",{"2":{"66":1,"71":1}}],["community",{"2":{"203":1}}],["communication",{"0":{"39":1},"2":{"122":1,"123":1}}],["communications",{"2":{"37":2}}],["commit",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["command",{"2":{"71":2,"161":2}}],["comm",{"2":{"36":2}}],["common",{"2":{"21":1,"46":1,"55":4,"136":1,"160":1,"166":1,"168":1}}],["comprises",{"2":{"136":1}}],["compilable",{"2":{"106":1}}],["compilation",{"2":{"106":1}}],["compiling",{"0":{"106":1},"1":{"107":1},"2":{"100":1}}],["compiled",{"2":{"103":1,"106":7}}],["compiler",{"2":{"98":2,"106":1}}],["compile=true",{"2":{"52":1}}],["compile",{"2":{"3":1,"48":1,"52":1,"62":1,"106":8}}],["completeness",{"2":{"152":1,"217":1}}],["completely",{"2":{"66":1,"94":1,"98":1}}],["complete",{"2":{"65":3,"66":1,"168":1}}],["complexity",{"2":{"81":1,"175":1}}],["complexf64",{"2":{"25":8}}],["complexf32",{"2":{"25":8,"168":2}}],["complexf16",{"2":{"25":8}}],["complex",{"2":{"5":1,"52":1,"59":1,"168":1}}],["compare",{"2":{"121":1,"152":1}}],["comparison",{"2":{"62":1}}],["compactmacroimpl",{"2":{"220":3}}],["compactluxlayer",{"2":{"59":2,"220":3,"225":2}}],["compact",{"0":{"59":1,"185":1},"2":{"58":1,"59":20,"78":3,"95":2,"133":2,"144":2,"185":2,"187":1,"214":4,"218":1,"220":7,"225":1}}],["compatibility",{"2":{"32":1,"105":1,"133":1}}],["compatible",{"2":{"6":1,"16":1,"18":1,"40":1,"41":2,"45":2,"53":2,"133":1,"144":1,"153":2,"217":1}}],["components",{"2":{"254":4}}],["component",{"2":{"254":7}}],["componentvector",{"2":{"179":1,"220":6,"258":1}}],["componentarray",{"2":{"46":1,"145":2,"146":2,"147":2,"148":3,"152":4,"179":1,"208":2,"215":1,"217":1,"225":2,"256":1}}],["componentarrays",{"2":{"46":2,"137":1,"144":1,"175":1,"205":1,"212":1,"220":12,"223":1,"249":1,"253":1}}],["composability",{"2":{"247":1}}],["composes",{"2":{"101":1}}],["composedlinear",{"2":{"136":3}}],["composed",{"2":{"52":1,"66":1}}],["compose",{"2":{"34":1,"249":1}}],["composition",{"2":{"19":1,"136":1}}],["computing",{"0":{"148":1,"150":1,"151":1,"152":1},"2":{"27":2,"28":1,"146":1,"150":1,"176":1}}],["computed",{"2":{"17":1,"19":1,"52":3,"53":3,"62":4,"63":2,"67":1,"68":1,"69":2,"176":4,"235":1}}],["computes",{"2":{"14":1,"16":1,"19":3,"24":1,"28":1,"52":2,"69":4,"140":1,"149":1}}],["compute",{"2":{"13":2,"18":1,"27":6,"28":1,"52":10,"53":1,"77":3,"87":1,"96":2,"106":1,"117":1,"148":3,"149":5,"152":1,"175":1,"176":1,"177":1,"178":1,"186":1,"195":1,"208":1,"240":1,"242":2,"249":1,"254":4,"255":1,"256":1,"257":1,"259":1}}],["computer",{"2":{"5":1,"19":1,"24":2,"53":4}}],["computationally",{"2":{"175":1}}],["computational",{"2":{"175":1}}],["computation",{"0":{"145":1,"147":1},"1":{"146":1},"2":{"3":1,"13":1,"49":1,"53":1,"62":2,"69":2,"87":1,"160":2}}],["copying",{"2":{"184":1}}],["copyto",{"2":{"55":3}}],["copy",{"2":{"8":1,"52":2,"55":1,"140":2,"173":5}}],["copied",{"2":{"5":1,"66":4}}],["conjunction",{"2":{"100":1}}],["concise",{"2":{"133":1,"185":1}}],["conclusion",{"0":{"118":1}}],["concatenate",{"2":{"111":1}}],["concatenated",{"2":{"66":7}}],["concrete",{"0":{"110":1},"1":{"111":1,"112":1,"113":1,"114":1},"2":{"82":1,"137":1,"140":1}}],["connection",{"2":{"62":26,"66":4}}],["connected",{"0":{"18":1},"2":{"24":1,"67":3}}],["confusing",{"2":{"208":1}}],["confusion",{"2":{"87":1}}],["conform",{"2":{"62":1}}],["conference",{"2":{"19":2,"24":6,"53":4}}],["convtranspose",{"2":{"63":1,"96":1,"97":2}}],["convolutions",{"2":{"63":5}}],["convolution",{"2":{"24":1,"63":9}}],["convolutional",{"0":{"16":1,"63":1},"2":{"24":2,"53":1,"63":4}}],["convdims",{"2":{"16":2}}],["conv",{"2":{"16":3,"34":2,"63":4,"95":1,"96":1,"139":1,"158":2,"193":6}}],["convention",{"2":{"173":1}}],["conveniently",{"2":{"171":1}}],["convenience",{"0":{"25":1},"2":{"7":1,"58":1,"101":1,"236":1}}],["conversely",{"2":{"175":1}}],["converse",{"2":{"63":1}}],["conversions",{"2":{"57":2}}],["conversion",{"0":{"165":1},"2":{"5":1}}],["converts",{"2":{"56":3,"70":1}}],["converted",{"2":{"34":2,"57":1}}],["converting",{"2":{"34":3,"46":1}}],["convert",{"2":{"5":1,"8":1,"32":2,"34":4,"57":3,"140":1,"143":1,"153":1,"193":1,"208":1,"209":1,"214":1}}],["cond",{"2":{"8":3}}],["conditions",{"2":{"196":1,"242":1,"255":1,"256":1}}],["condition",{"2":{"8":2,"24":1}}],["continua",{"2":{"254":1}}],["continue",{"2":{"123":1}}],["contour",{"2":{"245":1,"250":6}}],["contrib",{"2":{"116":1}}],["control",{"2":{"70":2,"106":2,"155":1,"170":1}}],["controlled",{"2":{"57":1,"97":1,"99":1}}],["controlling",{"2":{"57":1,"165":1,"166":1}}],["controls",{"2":{"24":1,"53":1,"63":4,"66":3,"69":8}}],["contrastive",{"2":{"53":1}}],["contrast",{"2":{"2":1}}],["context",{"2":{"17":1,"19":2,"54":1}}],["context`",{"2":{"17":1}}],["contained",{"2":{"62":1}}],["containerlayer",{"2":{"214":1}}],["containers",{"0":{"62":1},"2":{"87":2,"220":3}}],["container",{"0":{"136":1},"2":{"7":2,"8":1,"41":1,"45":1,"88":1,"120":1,"136":2,"184":2}}],["contains",{"0":{"147":1},"2":{"8":1,"28":1,"54":2,"64":1,"68":1}}],["contain",{"2":{"7":1,"57":1,"59":1,"100":1,"135":2,"184":1}}],["containing",{"0":{"145":1},"1":{"146":1},"2":{"3":1,"7":3,"19":2,"24":4,"25":24,"52":2,"53":2,"66":14,"67":1,"99":1}}],["consensus",{"2":{"147":1}}],["consecutive",{"2":{"62":1}}],["consequence",{"2":{"44":1}}],["consult",{"2":{"29":1}}],["considering",{"2":{"175":1}}],["consider",{"2":{"110":1,"135":1,"137":1,"155":1,"176":1,"179":1}}],["considered",{"2":{"3":1,"4":2,"49":1,"52":1,"125":1,"130":1}}],["consistent",{"2":{"66":1}}],["consistency",{"2":{"24":1}}],["consists",{"2":{"62":1,"146":1}}],["constrained",{"2":{"138":1}}],["constructured",{"2":{"140":1}}],["constructing",{"2":{"52":1,"120":1}}],["construction",{"2":{"7":1,"70":1,"94":1}}],["constructor",{"2":{"43":1}}],["construct",{"2":{"24":1,"43":1,"45":2,"59":2,"77":2,"99":1,"107":1,"115":1,"136":2,"208":1,"215":1,"249":2}}],["constructed",{"2":{"17":1,"59":1,"135":1}}],["constructs",{"2":{"7":1,"24":2,"43":1}}],["const",{"2":{"72":4,"73":3,"106":5,"117":1,"205":2,"206":2,"220":2,"235":2,"239":2,"254":6,"255":1,"256":2}}],["constants",{"2":{"17":1,"255":1,"256":1}}],["constant",{"2":{"3":1}}],["cupti",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["cusparse",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["cusolver",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["customparamtype",{"2":{"43":2}}],["customabstractluxlayer",{"2":{"7":4}}],["customize",{"2":{"3":1}}],["custom",{"0":{"78":1,"108":1,"140":1},"1":{"109":1,"110":1,"111":1,"112":1,"113":1,"114":1},"2":{"3":1,"7":2,"8":1,"11":1,"20":1,"43":1,"59":1,"78":1,"100":1,"113":1,"117":2,"133":1,"137":1,"144":1,"163":1,"173":1,"181":1,"184":1,"185":1,"214":1,"238":1,"242":1}}],["cufft",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["curand",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["currently",{"2":{"4":4,"18":2,"20":1,"45":1,"65":3,"70":1,"104":2,"105":1,"106":1,"107":1,"124":1,"130":1,"144":1,"153":2,"177":1}}],["current",{"2":{"2":1,"5":1,"7":1,"43":2,"52":1,"64":1,"101":1,"110":1,"173":1,"249":1,"250":1}}],["cublas",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["cublaslt",{"2":{"18":1}}],["cu",{"2":{"172":4}}],["cuarray",{"2":{"5":3,"22":3,"77":18,"131":1,"153":2,"168":2,"172":1,"220":18}}],["cuiterator",{"2":{"5":2,"92":1,"160":1}}],["cudevice",{"2":{"4":1}}],["cudnn",{"2":{"2":2,"3":1,"16":1,"72":1}}],["cudadevice",{"2":{"2":2,"4":2,"5":1,"78":1,"91":1,"123":1,"132":2,"180":1,"189":1,"197":1,"207":2,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["cuda",{"0":{"172":1},"2":{"2":6,"3":2,"4":1,"5":5,"16":1,"18":1,"22":1,"37":7,"72":1,"77":19,"78":1,"80":1,"85":2,"92":1,"122":1,"123":1,"124":1,"130":1,"131":1,"153":2,"160":1,"163":3,"164":1,"168":5,"172":7,"180":10,"189":10,"197":3,"210":10,"212":1,"220":36,"221":10,"223":1,"229":10,"237":10,"239":1,"246":10,"251":3,"260":3}}],["ch",{"2":{"249":2,"250":1}}],["cheat",{"2":{"147":1}}],["checking",{"2":{"135":1,"220":1}}],["check=",{"2":{"117":2}}],["check",{"2":{"8":4,"37":1,"45":11,"49":1,"59":1,"64":3,"69":3,"106":1,"117":3,"152":1,"157":2,"166":3}}],["checked",{"2":{"3":1}}],["checks",{"2":{"3":2,"43":2,"45":1,"59":2}}],["chs",{"2":{"63":17,"69":19}}],["chunks",{"2":{"54":1}}],["choice",{"2":{"188":1}}],["chosen",{"2":{"169":1}}],["chopra",{"2":{"53":1}}],["choose",{"2":{"2":1}}],["christian",{"2":{"19":1}}],["chain=chain",{"2":{"44":1}}],["chains",{"0":{"34":1},"2":{"34":4,"193":1,"196":1,"249":2}}],["chain",{"0":{"111":1,"129":1},"2":{"32":1,"34":2,"44":10,"46":3,"59":1,"62":7,"66":2,"68":4,"69":6,"77":9,"94":1,"106":2,"107":1,"110":1,"111":2,"114":3,"116":8,"117":6,"126":2,"127":1,"129":2,"139":2,"145":1,"146":1,"147":1,"148":1,"152":1,"153":1,"193":6,"208":1,"215":2,"220":18,"226":2,"233":2,"236":1,"241":1,"249":5,"250":2,"256":2}}],["chainrulescore",{"2":{"117":2}}],["chainrules",{"2":{"16":1,"18":1,"45":2,"53":1,"102":1,"105":1,"144":1}}],["channel",{"2":{"65":3,"69":12,"70":3}}],["channels",{"2":{"19":1,"63":6,"64":2,"69":1,"70":1}}],["changed",{"2":{"84":1,"121":1,"142":1}}],["changes",{"0":{"84":1,"87":1,"91":1,"94":1,"95":1,"96":2},"2":{"7":1,"11":1,"60":1,"62":9,"68":2,"82":1,"87":1,"89":1,"98":1,"116":1,"121":1,"139":1,"144":1}}],["change",{"2":{"1":1,"2":1,"58":1,"87":1,"90":1,"120":1,"135":1,"217":1,"254":1}}],["crude",{"2":{"254":1}}],["crc",{"2":{"117":2}}],["critical",{"2":{"104":1,"130":1}}],["criteria",{"2":{"2":1,"45":1}}],["creating",{"0":{"184":1},"2":{"59":1,"214":1}}],["created",{"2":{"59":1}}],["create",{"0":{"215":1,"226":1},"2":{"2":1,"5":1,"32":1,"36":2,"52":1,"55":2,"59":1,"62":2,"67":3,"106":2,"107":1,"113":1,"117":1,"171":1,"174":1,"181":1,"183":3,"184":2,"187":1,"215":2,"217":1,"220":3,"226":2,"228":1,"236":1,"241":4,"249":2}}],["creates",{"2":{"1":1,"8":1,"24":1,"59":1,"66":5,"174":1,"249":1}}],["crosscor",{"2":{"63":1,"95":1}}],["crossentropyloss",{"2":{"53":6,"194":1,"216":1,"227":1}}],["cross",{"2":{"53":3,"63":11,"95":1,"97":1,"133":1}}],["cpu=true",{"2":{"217":2}}],["cpus",{"2":{"13":1,"14":2,"34":1,"155":1,"159":1}}],["cpudevice",{"2":{"2":4,"4":2,"131":3,"132":1,"205":1,"239":1}}],["cpu",{"2":{"2":4,"5":1,"18":1,"20":1,"49":1,"65":3,"73":2,"95":2,"100":1,"102":1,"103":3,"105":1,"106":1,"130":1,"131":6,"132":7,"140":1,"153":3,"167":1,"180":2,"187":1,"188":1,"189":2,"196":1,"197":2,"205":1,"210":2,"217":4,"221":2,"229":2,"235":2,"236":1,"237":2,"239":1,"246":2,"251":2,"260":2}}],["rk4",{"2":{"255":1,"256":1,"257":1,"259":1}}],["r₂",{"2":{"254":2}}],["r₁",{"2":{"254":2}}],["r2",{"2":{"254":1}}],["r1",{"2":{"254":1}}],["r=r1−r2",{"2":{"254":1}}],["r=σ",{"2":{"66":1}}],["rhat",{"2":{"249":1}}],["risk",{"2":{"124":1}}],["right",{"2":{"68":1}}],["rrule",{"2":{"117":1}}],["rrules",{"2":{"105":1}}],["rᴰ",{"2":{"70":1}}],["r²",{"2":{"70":1}}],["rgb",{"2":{"63":1}}],["r",{"2":{"59":9,"70":9,"254":3}}],["rnns",{"2":{"66":1}}],["rnncell",{"2":{"66":9,"96":1,"97":1}}],["rnn",{"2":{"54":1,"66":3}}],["rng=xoshiro",{"2":{"195":1}}],["rngs",{"2":{"99":1}}],["rng",{"0":{"22":1},"2":{"3":3,"7":2,"8":6,"9":1,"10":1,"11":2,"17":6,"22":10,"24":11,"25":24,"32":1,"34":1,"44":3,"59":5,"62":4,"64":6,"66":3,"67":4,"68":18,"77":7,"78":3,"89":2,"94":1,"106":1,"107":1,"112":2,"114":1,"116":3,"117":2,"126":4,"129":6,"135":9,"136":2,"137":5,"139":7,"140":9,"155":4,"168":13,"169":6,"170":2,"171":1,"174":5,"176":1,"177":1,"179":8,"187":2,"195":2,"207":1,"208":1,"215":4,"225":3,"228":2,"232":4,"235":1,"244":4,"248":11,"249":1,"256":1}}],["row",{"2":{"171":3,"250":1}}],["rows",{"2":{"24":1,"171":1}}],["routines",{"2":{"144":1}}],["route",{"2":{"136":1}}],["rough",{"2":{"144":1}}],["rounded",{"2":{"63":2,"65":3}}],["round",{"2":{"37":1,"228":4,"248":1}}],["root",{"2":{"39":7,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["robin",{"2":{"37":1}}],["rocarray",{"2":{"22":4}}],["rocrand",{"2":{"22":1}}],["rocm",{"2":{"3":1,"72":2,"123":1,"124":1,"163":3,"164":1}}],["ryan",{"2":{"19":1}}],["raw",{"2":{"192":2,"213":2}}],["rademacher",{"2":{"152":1}}],["ra",{"2":{"106":18,"107":9}}],["ratio≥0",{"2":{"254":1}}],["ratio≤1",{"2":{"254":1}}],["ratio",{"2":{"254":6,"255":2,"256":1,"257":1,"259":1}}],["rationale",{"2":{"87":1}}],["rate",{"2":{"179":1,"249":1}}],["rather",{"2":{"3":1,"16":1,"66":1,"99":1,"184":1}}],["raia",{"2":{"53":1}}],["ran",{"2":{"116":1}}],["randc64",{"2":{"25":1}}],["randc32",{"2":{"25":1}}],["randc16",{"2":{"25":1}}],["rand64",{"2":{"25":1}}],["rand32",{"2":{"25":1}}],["rand16",{"2":{"25":1}}],["randnc64",{"2":{"25":1}}],["randnc32",{"2":{"25":1}}],["randnc16",{"2":{"25":1}}],["randn64",{"2":{"25":1}}],["randn32",{"2":{"25":1}}],["randn16",{"2":{"25":1}}],["randn",{"2":{"24":1,"32":1,"34":1,"59":1,"62":2,"68":2,"106":1,"107":1,"112":1,"116":1,"126":1,"129":1,"131":1,"132":1,"135":1,"136":1,"137":1,"139":2,"140":6,"145":2,"146":2,"171":2,"174":1,"176":1,"177":1,"179":4,"195":1,"232":1}}],["randomness",{"0":{"174":1},"2":{"66":3,"77":1,"99":1,"149":1,"170":1,"174":1}}],["randomly",{"2":{"64":2}}],["random",{"2":{"17":4,"22":1,"24":8,"25":12,"32":2,"34":2,"44":2,"59":3,"62":2,"64":3,"68":8,"77":4,"78":3,"106":3,"107":1,"111":1,"116":1,"126":2,"129":4,"135":4,"137":3,"139":4,"140":4,"144":1,"149":1,"153":1,"155":2,"168":3,"170":5,"171":1,"174":8,"179":4,"182":1,"191":1,"205":1,"207":1,"208":1,"212":1,"215":2,"223":1,"225":1,"231":1,"232":2,"239":1,"243":1,"244":2,"247":1,"248":2,"253":1,"256":1}}],["rand",{"2":{"5":1,"59":1,"77":3,"78":1,"147":1,"148":1,"152":2,"153":1,"155":2,"171":5,"172":1,"174":3,"248":8}}],["rank",{"2":{"4":3,"38":3,"120":1,"121":1}}],["ranges",{"2":{"62":1}}],["range",{"2":{"3":2,"89":1,"177":1,"206":1,"232":1,"243":4,"250":23,"255":1}}],["rule",{"2":{"70":1,"236":2}}],["rules",{"0":{"113":1},"2":{"0":1,"55":1,"104":1}}],["runtime",{"2":{"49":2,"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["running",{"0":{"112":1},"2":{"17":2,"19":18,"69":16,"87":1,"116":5,"117":11,"126":2,"129":2,"142":1,"156":1,"184":1}}],["run",{"2":{"2":1,"3":1,"5":1,"48":1,"64":1,"71":2,"72":1,"73":1,"77":1,"94":1,"106":3,"113":1,"117":3,"124":1,"135":1,"140":1,"153":3,"155":1,"161":3,"174":1,"184":1,"202":1,"250":1}}],["red",{"2":{"206":1,"209":2,"248":1}}],["reduction",{"2":{"53":1,"65":3}}],["reduce",{"2":{"39":3,"47":1,"68":1,"249":1}}],["reducing",{"2":{"19":1,"62":1}}],["reiterate",{"2":{"140":1}}],["refresher",{"2":{"135":1}}],["referred",{"2":{"171":1}}],["refer",{"2":{"78":1,"203":1,"204":1,"238":1}}],["reference",{"0":{"23":1},"1":{"24":1,"25":1},"2":{"190":1}}],["references",{"2":{"17":2,"19":4,"24":6,"53":4,"68":1,"69":1}}],["rev=true",{"2":{"244":4}}],["revising",{"2":{"100":1}}],["reverses",{"2":{"68":1}}],["reversesequence",{"2":{"68":3}}],["reversed",{"2":{"68":1}}],["reversediffadjoint",{"2":{"214":1}}],["reversediff",{"2":{"8":2,"16":1,"18":1,"20":1,"49":1,"52":1,"55":1,"102":1,"103":1,"112":1,"217":1}}],["reverse",{"2":{"27":2,"45":2,"49":1,"59":1,"68":2,"102":6,"106":1,"117":1,"175":2,"176":2}}],["reexport",{"2":{"94":2}}],["renamed",{"2":{"87":2,"90":1,"91":2}}],["renormalize",{"2":{"69":2}}],["request",{"2":{"81":1,"87":1}}],["requisites",{"2":{"77":1}}],["requirements",{"2":{"62":1,"236":1}}],["require",{"2":{"19":1,"32":1,"34":1,"59":1,"123":1,"208":1}}],["requires",{"2":{"8":1,"10":1,"37":4,"54":1,"80":1,"109":1,"208":1,"225":1}}],["retcode",{"2":{"258":1}}],["retrieve",{"2":{"67":1}}],["retained",{"2":{"64":1}}],["retuened",{"2":{"3":2}}],["returned",{"2":{"17":1,"19":1,"22":1,"24":1,"32":1,"34":1,"41":1,"46":1,"52":6,"54":1,"64":4,"66":3,"67":1,"68":1}}],["returning",{"2":{"8":1,"135":1}}],["returns",{"2":{"3":3,"7":2,"8":3,"13":2,"17":2,"19":4,"20":1,"24":6,"27":2,"28":1,"46":1,"50":1,"52":2,"53":6,"54":2,"57":1,"62":7,"63":2,"64":3,"65":9,"66":8,"67":5,"68":6,"69":5,"70":2,"92":1,"131":3,"146":1,"147":1,"148":1,"208":1,"220":6,"256":1}}],["return",{"2":{"2":3,"3":5,"7":1,"9":1,"10":1,"11":2,"24":6,"25":24,"43":1,"44":2,"52":4,"53":9,"54":2,"55":2,"59":19,"62":3,"66":4,"68":1,"78":3,"87":1,"99":1,"106":2,"107":1,"111":1,"113":2,"117":2,"126":1,"127":3,"128":4,"135":5,"136":2,"137":3,"140":1,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"169":1,"179":1,"183":1,"184":2,"185":2,"186":1,"187":1,"192":1,"194":1,"195":1,"206":1,"208":3,"213":1,"214":5,"215":1,"216":1,"218":2,"220":3,"224":2,"225":3,"226":1,"227":1,"228":1,"232":1,"235":1,"236":1,"241":2,"242":3,"244":1,"245":1,"248":1,"249":2,"250":2,"254":11,"255":1,"256":1,"257":2}}],["remove",{"2":{"174":1}}],["removed",{"0":{"94":1,"122":1},"2":{"63":1,"84":1,"87":2,"91":1,"94":10,"95":4,"106":1,"122":1,"130":1,"145":2,"146":1}}],["remake",{"2":{"208":1}}],["remark",{"2":{"146":1}}],["remains",{"2":{"17":1}}],["remember",{"2":{"107":1,"115":1,"118":1,"135":1,"144":1,"153":1,"184":1,"217":1}}],["regions",{"2":{"250":1}}],["registry",{"2":{"71":1,"79":1}}],["registered",{"2":{"59":1,"71":1}}],["regression",{"0":{"179":1},"2":{"155":1,"179":1}}],["regressions",{"2":{"8":1,"130":1,"196":1}}],["reg",{"2":{"145":2,"146":2}}],["regarding",{"2":{"140":1,"175":1}}],["regularization",{"2":{"145":1,"152":1,"249":1}}],["regular",{"2":{"34":2,"59":2,"153":1}}],["reuse",{"2":{"16":1,"18":1}}],["reusing",{"2":{"16":1,"18":1}}],["reltol",{"2":{"220":13}}],["reltol=1",{"2":{"215":1}}],["reliance",{"2":{"145":2}}],["reliable",{"2":{"103":2}}],["reliability",{"2":{"100":1}}],["relies",{"2":{"140":2,"161":1,"174":1}}],["relevant",{"2":{"106":1,"247":1,"254":1}}],["release",{"2":{"82":2,"89":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["released",{"2":{"71":1}}],["relativisticorbitmodel",{"2":{"255":3}}],["relatively",{"2":{"144":1,"153":1}}],["relation",{"2":{"66":1}}],["related",{"2":{"81":1}}],["relu",{"2":{"16":1,"32":1,"34":4,"59":3,"62":4,"66":1,"68":3,"69":8,"78":1,"116":5,"117":5,"139":2,"153":1,"193":12,"226":2,"233":2,"236":2}}],["rely",{"2":{"15":1,"59":1,"87":1,"174":1}}],["readme",{"2":{"106":1}}],["read",{"2":{"81":1,"173":1}}],["reactant",{"0":{"106":1},"1":{"107":1},"2":{"73":7,"100":1,"106":9,"107":1}}],["reason",{"2":{"103":1,"155":1}}],["reasonable",{"2":{"62":1}}],["reasons",{"2":{"8":1,"214":1}}],["really",{"2":{"34":1,"59":1,"105":1,"135":1,"171":1}}],["reallocations",{"2":{"16":1,"18":1}}],["real",{"2":{"24":1,"53":4,"64":1,"113":1,"245":2}}],["real=eps",{"2":{"19":1}}],["rewrite",{"2":{"13":1,"32":1,"240":1}}],["receives",{"2":{"68":1}}],["reconstruction",{"2":{"135":1}}],["recognition",{"2":{"53":1}}],["record",{"2":{"50":1,"245":1,"250":1}}],["recorded",{"2":{"49":1,"50":4}}],["recomputing",{"2":{"17":1}}],["recommendations",{"0":{"103":1},"2":{"154":1,"155":1}}],["recommendation",{"2":{"100":1}}],["recommend",{"2":{"7":1,"100":1,"123":1,"133":1,"135":1,"173":1,"177":1,"185":1,"188":1,"211":1,"214":1,"220":1,"225":1}}],["recommended",{"0":{"131":1},"2":{"3":1,"7":2,"8":1,"15":1,"32":1,"43":1,"47":1,"55":1,"59":2,"100":1,"133":1,"135":2,"140":1,"144":1,"145":2,"150":1,"152":1,"153":1,"156":1,"164":1,"186":1,"256":1}}],["recvbuf",{"2":{"39":6}}],["rectifiers",{"2":{"24":2}}],["recur",{"2":{"66":3}}],["recurrence",{"2":{"66":4,"184":1}}],["recurrent",{"0":{"66":1},"2":{"24":1,"66":10,"96":1,"181":2}}],["recurse",{"2":{"55":3}}],["recurses",{"2":{"45":1,"56":3}}],["recursion",{"2":{"55":1}}],["recursive",{"0":{"55":1},"2":{"55":9,"153":2,"155":1}}],["recursively",{"2":{"10":1,"55":5}}],["repository",{"2":{"144":1}}],["report",{"2":{"42":1,"81":1,"105":1}}],["reproducer",{"2":{"124":1}}],["represents",{"2":{"249":1}}],["representing",{"2":{"70":1}}],["representation",{"2":{"59":1}}],["represent",{"2":{"24":1}}],["reparameterized",{"2":{"69":2}}],["reparameterization",{"2":{"69":1}}],["repack",{"2":{"8":1}}],["repeat",{"2":{"183":2}}],["repeatedly",{"2":{"62":1}}],["repeatedlayer",{"2":{"62":1}}],["repeats",{"2":{"62":6,"66":5}}],["replacing",{"2":{"139":1,"171":1}}],["replacement",{"2":{"87":1}}],["replaces",{"2":{"45":1,"65":3}}],["replaced",{"2":{"13":1}}],["replace",{"2":{"13":2,"114":1,"158":5}}],["repl",{"2":{"48":1,"71":1,"76":1,"156":2,"161":1}}],["replicated",{"2":{"174":1}}],["replicate",{"2":{"7":1,"8":1,"62":1,"174":3}}],["resolve",{"2":{"148":1}}],["resolution",{"2":{"70":1}}],["resources",{"0":{"81":1}}],["rescale",{"2":{"69":4}}],["resnet",{"2":{"62":1}}],["res",{"2":{"59":2,"62":4,"208":5,"258":1,"259":1}}],["respective",{"2":{"98":1,"120":2,"140":1,"173":1}}],["respect",{"2":{"28":2,"49":1,"146":2}}],["responsibility",{"2":{"8":1,"137":1}}],["reshapes",{"2":{"68":1}}],["reshapelayer",{"2":{"68":3}}],["reshape",{"2":{"53":4,"78":1,"150":2,"151":2,"173":2,"183":2,"192":1,"213":1,"214":4,"224":2,"232":1,"243":2,"245":2,"249":1}}],["reshaped",{"2":{"16":1,"68":1}}],["reshaping",{"2":{"24":1}}],["restricted",{"2":{"55":1}}],["rest",{"2":{"24":1,"184":2,"185":2}}],["restarted",{"2":{"1":1,"60":1}}],["reserved",{"2":{"59":1}}],["research",{"2":{"17":1}}],["resets",{"2":{"3":1}}],["reset",{"2":{"3":1}}],["results",{"0":{"209":1,"245":1,"259":1},"2":{"106":1,"152":2,"236":1,"250":2,"255":1,"256":1,"258":1,"259":1}}],["result",{"2":{"15":1,"24":1,"28":2,"39":4,"50":4,"69":1,"99":1,"137":4}}],["mtensor",{"2":{"254":4}}],["mtlarray",{"2":{"22":3}}],["mcse",{"2":{"249":1}}],["mcmcchains",{"2":{"249":1}}],["mcmc",{"2":{"249":1,"250":1}}],["mcclelland",{"2":{"24":1}}],["mvnormal",{"2":{"249":1}}],["md",{"2":{"244":8}}],["mnist\\ttraining",{"2":{"228":1}}],["mnist\\ttime",{"2":{"228":50}}],["mnist",{"0":{"190":1,"192":1,"211":1,"213":1,"222":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1,"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"191":1,"192":2,"212":1,"213":2,"224":1,"228":2}}],["my",{"2":{"196":1}}],["mybias",{"2":{"137":3}}],["myinputtype",{"2":{"109":3}}],["myfancychain",{"2":{"62":2}}],["myweight",{"2":{"43":1,"137":4}}],["mse",{"2":{"242":3}}],["mseloss",{"2":{"53":4,"77":2,"78":1,"106":1,"107":1,"179":1,"208":1,"235":1,"242":1,"257":3}}],["msleloss",{"2":{"53":2}}],["mpi",{"0":{"163":1},"2":{"36":2,"37":8,"123":2,"124":1,"163":5}}],["mpibackend",{"2":{"36":2,"37":2}}],["m2",{"2":{"32":3}}],["mₘ",{"2":{"28":2}}],["m₂",{"2":{"28":2,"254":7}}],["m₁",{"2":{"28":2,"254":6}}],["m",{"2":{"19":2,"32":3,"56":6,"62":2,"72":2,"214":2,"218":2,"248":15,"254":4,"255":5,"256":3}}],["mkl",{"2":{"18":1}}],["mldatasets",{"2":{"191":1,"212":1,"223":1}}],["mldatadevices",{"0":{"0":1,"90":1},"1":{"1":1,"2":1,"3":1,"4":1,"5":1,"91":1,"92":1},"2":{"0":1,"1":1,"2":3,"3":12,"4":1,"5":2,"90":2,"132":2,"164":2,"180":3,"189":3,"197":3,"205":1,"207":3,"210":3,"221":3,"229":3,"237":3,"239":1,"246":3,"251":3,"260":3}}],["ml",{"2":{"171":1,"208":2}}],["mlir",{"2":{"106":1}}],["mlp",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"59":1,"78":1,"106":1,"226":1,"230":1,"241":4}}],["mlutils",{"0":{"41":1},"2":{"5":3,"41":4,"92":1,"120":1,"160":2,"182":1,"183":3,"191":1,"205":1,"207":1,"212":1,"223":1,"239":1}}],["motion",{"2":{"254":1,"255":1,"256":1}}],["motivating",{"2":{"8":1,"58":1}}],["moment",{"2":{"147":1,"254":1}}],["momentum=0",{"2":{"69":2}}],["momentum",{"2":{"19":6,"69":2}}],["mooncake",{"2":{"102":1}}],["monolithic",{"2":{"100":1}}],["month",{"2":{"74":1}}],["mouthful",{"2":{"46":1}}],["moved",{"0":{"95":1},"2":{"87":1,"95":2,"106":1}}],["move",{"2":{"42":1,"106":1,"107":2,"172":1,"173":1,"207":3}}],["movement",{"2":{"3":2}}],["modivations",{"2":{"94":1}}],["modify",{"2":{"69":1}}],["modified",{"2":{"55":2,"111":1}}],["modulating",{"2":{"53":1}}],["modules=",{"2":{"48":2}}],["modules",{"2":{"48":8}}],["module",{"0":{"54":1},"2":{"35":1,"42":1,"48":1,"54":1,"94":1,"95":1}}],["modes",{"2":{"20":1,"60":1,"70":1}}],["mode",{"0":{"142":1},"2":{"17":2,"19":2,"20":5,"27":2,"45":7,"49":1,"60":2,"64":6,"66":2,"69":3,"70":5,"95":1,"102":1,"103":1,"115":3,"116":2,"117":2,"142":4,"145":2,"155":1,"156":1,"161":1,"175":1,"176":4,"177":2,"254":1}}],["model`",{"2":{"214":1}}],["models",{"0":{"32":2,"34":1,"106":1,"114":1,"115":1,"202":1,"204":1},"1":{"107":1,"116":1,"117":1,"118":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"8":1,"21":1,"32":2,"34":2,"47":1,"52":2,"59":2,"77":1,"100":6,"106":1,"110":1,"115":2,"118":1,"153":1,"168":1,"171":1,"177":1,"181":1,"185":1,"202":2,"204":1}}],["model",{"0":{"116":1,"125":1,"187":1,"188":1,"193":1,"196":1,"208":1,"252":1,"255":1,"256":1},"1":{"126":1,"127":1,"128":1,"129":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"8":8,"32":6,"34":11,"44":1,"45":2,"46":2,"52":5,"53":26,"58":3,"59":14,"62":16,"66":3,"68":12,"69":4,"77":6,"78":10,"98":1,"106":18,"107":8,"110":1,"111":1,"112":2,"113":2,"114":2,"115":3,"116":17,"117":25,"118":1,"120":2,"125":1,"126":6,"129":7,"135":1,"136":5,"139":9,"140":11,"142":1,"143":1,"145":10,"146":7,"147":7,"148":7,"150":2,"151":2,"152":9,"153":4,"155":6,"179":11,"184":2,"185":1,"186":2,"187":10,"188":6,"190":1,"193":4,"194":2,"195":7,"196":4,"202":1,"208":9,"209":2,"214":11,"215":8,"216":2,"217":10,"218":8,"220":17,"226":4,"227":2,"228":9,"235":2,"236":3,"242":4,"243":1,"244":7,"249":7,"250":1,"254":7,"255":4,"256":11,"257":1,"258":1,"259":2}}],["mostly",{"2":{"14":1,"82":1,"94":1,"100":1,"124":1,"214":1}}],["most",{"2":{"7":1,"8":1,"20":1,"24":1,"45":1,"52":2,"55":1,"71":1,"90":1,"100":1,"101":1,"103":2,"107":1,"133":1,"139":1,"148":1,"150":1,"153":1,"203":1,"256":1}}],["more",{"2":{"5":1,"7":1,"8":1,"11":1,"14":1,"24":2,"32":1,"34":2,"49":1,"54":1,"55":1,"58":1,"59":3,"62":1,"70":1,"81":1,"84":1,"87":1,"89":1,"92":1,"94":1,"96":2,"100":1,"123":1,"145":1,"146":1,"155":1,"157":1,"162":1,"165":1,"166":1,"171":3,"175":1,"184":1,"185":1,"186":1,"204":1,"250":1}}],["mechanics",{"2":{"254":1}}],["mechanism",{"2":{"100":1}}],["message",{"2":{"144":1}}],["mersennetwister",{"2":{"112":1,"168":2,"232":1}}],["merged",{"2":{"66":1}}],["merge",{"2":{"43":1,"44":1,"66":2,"111":1,"184":1}}],["mentioned",{"2":{"79":1,"135":1}}],["measure",{"2":{"53":1}}],["meanpool",{"2":{"65":1}}],["meant",{"2":{"45":1,"52":1,"58":1,"169":1}}],["means",{"2":{"24":1,"42":1,"59":1,"114":1,"135":1,"137":1,"155":1,"171":1,"173":1,"208":2}}],["meaningless",{"2":{"17":1}}],["mean",{"2":{"17":2,"19":13,"24":2,"53":17,"65":7,"69":13,"116":1,"117":2,"126":1,"129":1,"137":1,"179":1,"242":3,"244":12,"249":1,"250":1}}],["medical",{"2":{"53":1}}],["medium",{"2":{"18":1}}],["met",{"2":{"24":1,"45":1}}],["methodinstance",{"2":{"220":3}}],["methoderror",{"2":{"112":1}}],["methods",{"2":{"70":1,"78":1,"131":1,"149":1,"153":1,"169":2,"204":2,"205":1,"208":1,"214":1,"215":1,"224":1,"239":1,"249":1,"254":2}}],["method",{"2":{"24":3,"106":1,"109":1,"112":1,"117":1,"150":1,"151":1,"152":1,"183":1,"185":1,"186":1,"192":1,"194":1,"195":1,"213":1,"214":1,"216":1,"225":1,"226":1,"227":1,"232":1,"242":2,"249":1,"250":2,"254":3,"256":1,"257":2}}],["metaldevice",{"2":{"4":2}}],["metal",{"2":{"2":1,"3":2,"22":1,"72":2,"77":1,"78":1,"80":2,"130":4,"164":1}}],["memory=false",{"2":{"66":1}}],["memory=zeros32",{"2":{"66":1}}],["memory",{"2":{"5":1,"16":1,"18":1,"66":20,"180":2,"184":1,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["mutability",{"0":{"173":1}}],["mutables",{"2":{"140":1}}],["mutable",{"2":{"135":1,"140":1}}],["mutations",{"2":{"98":1,"100":2,"103":1}}],["mutation",{"2":{"16":1,"18":1,"173":2}}],["mutating",{"2":{"16":1,"18":1}}],["mutate",{"2":{"173":1}}],["mutated",{"2":{"15":1,"55":1,"173":2,"174":1}}],["mutates",{"2":{"8":1}}],["much",{"2":{"53":2,"100":1,"121":1,"250":1}}],["mul",{"2":{"14":2,"158":1}}],["multilayer",{"2":{"230":1}}],["multigate",{"2":{"54":1}}],["multiply",{"2":{"171":1}}],["multiplied",{"2":{"17":1}}],["multiplication",{"2":{"14":3}}],["multiple",{"0":{"146":1},"2":{"16":1,"18":1,"62":1,"66":1,"69":1,"130":1,"153":2,"250":1}}],["multi",{"0":{"4":1},"2":{"53":1,"67":1}}],["must",{"2":{"1":1,"3":2,"4":1,"7":1,"19":1,"24":4,"28":2,"32":1,"34":2,"37":1,"41":2,"44":2,"46":2,"52":3,"60":1,"62":3,"63":2,"64":3,"66":7,"67":4,"69":1,"70":1,"99":4,"100":1,"120":1,"135":1,"138":1,"235":1,"254":4}}],["major",{"0":{"85":1,"88":1,"92":1},"2":{"82":1,"89":1}}],["masses",{"2":{"254":1}}],["mass2",{"2":{"254":5}}],["mass1",{"2":{"254":6}}],["mass",{"2":{"254":14,"255":2,"256":1,"257":1,"259":1}}],["mass=1",{"2":{"254":3}}],["massachusetts",{"2":{"74":1}}],["master",{"2":{"120":1}}],["mask",{"2":{"17":9,"64":10}}],["magnitude",{"2":{"69":2}}],["maeloss",{"2":{"53":2}}],["made",{"2":{"52":1,"82":1,"87":1,"90":1,"135":1,"144":1}}],["making",{"2":{"52":1,"90":1,"98":1,"100":1,"106":1,"153":1}}],["makes",{"2":{"44":1,"53":1,"66":1,"97":1,"100":1,"133":1}}],["make",{"2":{"10":2,"46":1,"52":1,"55":6,"66":1,"98":2,"100":1,"116":1,"121":1,"135":1,"137":1,"139":1,"145":1,"146":1,"153":1,"161":1,"171":1,"175":1,"183":1,"249":1,"256":2}}],["macro",{"2":{"48":1,"49":1,"59":4,"78":2,"106":1,"133":2,"185":1,"214":2}}],["machines",{"2":{"53":2}}],["machine",{"2":{"17":1,"19":1,"24":1,"149":1,"171":1,"196":1,"254":1}}],["maxiters",{"2":{"244":4}}],["maxiters=1000",{"2":{"258":1}}],["maxiters=500",{"2":{"208":1}}],["maxiters=epochs",{"2":{"208":1}}],["maximum",{"2":{"16":1,"18":1,"68":1,"243":2,"245":1}}],["maxout",{"2":{"68":5}}],["max",{"2":{"53":2,"65":4,"68":1,"243":8,"245":1,"250":1}}],["maxpool",{"2":{"34":2,"65":1,"193":6}}],["margin=2",{"2":{"53":1}}],["margin−y^",{"2":{"53":1}}],["margin",{"2":{"53":2}}],["marker=",{"2":{"255":1,"256":2,"259":4}}],["markersize=16",{"2":{"248":2}}],["markersize=12",{"2":{"232":1,"236":2,"255":1,"256":2,"259":4}}],["marked",{"2":{"5":1,"56":3,"59":1}}],["mark",{"2":{"59":2,"256":1}}],["marks",{"2":{"48":2,"90":1}}],["martens",{"2":{"24":1}}],["main",{"0":{"24":1},"2":{"47":1,"113":1,"114":1,"135":1,"160":1,"184":1,"187":3,"207":2,"214":1,"220":18,"236":2,"241":1}}],["maintaining",{"2":{"19":1}}],["matches",{"2":{"186":2}}],["matched",{"2":{"112":5}}],["matching",{"0":{"57":1},"2":{"112":1}}],["match",{"2":{"44":1,"57":4,"66":5,"138":2,"155":1,"165":1}}],["matrices",{"2":{"18":1,"67":1}}],["matrix",{"2":{"5":3,"14":3,"15":1,"18":2,"24":13,"53":6,"59":4,"67":6,"68":1,"78":1,"106":2,"112":3,"113":1,"114":1,"116":5,"117":13,"131":1,"146":1,"147":1,"148":1,"149":5,"153":1,"168":4,"171":13,"175":3,"207":2}}],["matmul",{"2":{"14":2,"150":1,"151":1,"158":1}}],["maps",{"2":{"24":2,"53":1,"65":3,"66":2}}],["mapping",{"2":{"24":7,"53":1}}],["map",{"0":{"44":1},"2":{"7":1,"44":4,"55":2,"66":4,"94":2,"95":1,"126":2,"127":1,"153":2,"250":1}}],["managing",{"0":{"174":1},"2":{"174":1}}],["management",{"0":{"130":1,"131":1,"132":1},"1":{"131":1,"132":1},"2":{"90":1,"130":1,"131":1}}],["manager",{"2":{"76":1}}],["manage",{"2":{"59":1}}],["many",{"2":{"146":1,"171":1,"175":1}}],["mandatorily",{"2":{"135":1}}],["manual",{"0":{"132":1},"2":{"8":1,"29":1,"43":1,"58":1,"118":1,"125":1,"144":1,"162":1,"172":1}}],["manually",{"2":{"2":1,"32":1,"34":1,"120":1,"133":1,"172":1}}],["manipulation",{"2":{"7":2,"100":1}}],["maybe",{"2":{"57":1,"175":1}}],["may",{"2":{"3":1,"59":3,"70":2,"196":1,"203":1}}],["min",{"2":{"243":10,"245":2}}],["minibatching",{"2":{"204":1,"208":1}}],["minibatch",{"2":{"183":1,"192":1,"213":1}}],["mini",{"2":{"177":1}}],["minimum",{"2":{"243":4,"245":2}}],["minimally",{"2":{"252":1}}],["minimal",{"2":{"124":1}}],["minimized",{"2":{"257":1}}],["minimize",{"2":{"18":1}}],["minimizes",{"2":{"16":1,"179":1}}],["minor",{"2":{"144":1,"153":1}}],["mig",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["migrating",{"0":{"139":1},"1":{"140":1,"141":1,"142":1,"143":1}}],["migration",{"0":{"121":1},"1":{"122":1,"123":1},"2":{"135":1}}],["might",{"2":{"2":1,"8":1,"13":1,"15":1,"32":1,"34":1,"52":3,"58":2,"59":1,"62":1,"68":1,"105":1,"106":1,"107":1,"118":1,"130":1,"135":2,"144":1,"145":2,"155":1,"159":1,"167":2,"171":1,"173":1,"196":1,"208":1,"219":1}}],["miopen",{"2":{"97":1}}],["mirza",{"2":{"68":1}}],["milletari",{"2":{"53":1}}],["mixing",{"2":{"28":1}}],["mixed",{"2":{"16":1,"27":2}}],["mistakes",{"2":{"100":1}}],["misc",{"0":{"68":1}}],["miscellaneous",{"0":{"3":1,"60":1},"2":{"62":1}}],["mismatch",{"0":{"116":1},"2":{"57":3,"123":1,"155":1,"165":1}}],["missing",{"2":{"19":1,"37":1,"69":1}}],["s3",{"2":{"259":2}}],["s0025",{"2":{"254":2}}],["s2",{"2":{"236":2,"256":2,"259":2}}],["s1",{"2":{"236":2,"256":2,"259":2}}],["sgd",{"2":{"179":1}}],["src",{"2":{"112":1,"116":1,"145":2,"220":3,"244":4}}],["srivastava",{"2":{"17":1}}],["sz",{"2":{"111":2}}],["szegedy",{"2":{"19":1}}],["switch",{"2":{"64":3,"90":1,"144":1,"175":1}}],["switching",{"0":{"31":1},"1":{"32":1},"2":{"144":4,"162":2}}],["square",{"2":{"171":2,"243":1}}],["squaredhingeloss",{"2":{"53":2}}],["squared",{"2":{"53":3,"179":1}}],["sqrt",{"2":{"24":4,"63":2,"66":6,"67":3,"249":1,"255":2}}],["sm",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["smarter",{"2":{"243":1}}],["smatrix",{"2":{"153":4}}],["smaller",{"2":{"100":1}}],["small",{"2":{"18":1,"34":1,"100":1,"106":1,"155":1,"171":1,"190":1,"208":1}}],["smodel",{"2":{"145":3,"146":4,"147":2,"148":2,"150":2,"151":2,"152":2,"208":3}}],["smooth",{"2":{"53":2}}],["smoothing=0",{"2":{"53":3}}],["smoothing",{"2":{"53":16}}],["skipped",{"2":{"62":1,"254":1}}],["skipconnection",{"2":{"62":4}}],["skip",{"2":{"49":3,"62":1,"106":1,"243":1}}],["speedup",{"2":{"196":2}}],["spectralnorm",{"2":{"100":1}}],["specify",{"2":{"53":1,"59":1,"64":2,"70":1,"128":1,"249":1}}],["specifying",{"2":{"24":1,"48":1,"59":1,"63":2,"65":3,"69":2}}],["specifies",{"2":{"63":4,"65":3,"68":1,"177":1}}],["specified",{"2":{"2":1,"24":2,"49":1,"62":8,"67":1,"68":3,"70":1,"131":1,"169":1}}],["specifically",{"2":{"142":1,"171":1}}],["specifications",{"2":{"118":1}}],["specification",{"0":{"116":1},"2":{"249":1}}],["specific",{"2":{"4":2,"8":1,"55":1,"67":1,"104":1,"164":1}}],["specialized",{"2":{"13":1,"20":1,"59":1,"153":1,"173":1}}],["special",{"2":{"3":2,"18":2,"39":2,"53":1,"59":1,"66":1,"104":1,"220":3}}],["spiralclassifiercompact",{"2":{"185":2,"187":1}}],["spiralclassifier",{"2":{"184":5,"187":1}}],["spiral",{"2":{"183":1}}],["spirals",{"2":{"181":1,"183":6}}],["spiritual",{"2":{"119":1}}],["spurious",{"0":{"155":1}}],["space",{"2":{"66":6}}],["spatial",{"2":{"63":5,"65":6,"243":1}}],["sparsity=0",{"2":{"24":1}}],["sparsity",{"2":{"24":4}}],["sparsely",{"2":{"24":2,"67":1}}],["sparse",{"2":{"24":2}}],["spliced",{"2":{"66":2}}],["split=",{"2":{"192":1,"213":1}}],["splitobs",{"2":{"183":1,"192":1,"212":1,"213":1}}],["split",{"2":{"54":1,"120":2,"183":1,"184":1,"192":2,"213":2}}],["splatted",{"2":{"59":1}}],["splatting",{"2":{"59":1}}],["scientific",{"2":{"254":1}}],["scientificmachinelearning",{"2":{"252":1}}],["scimlsensitivity",{"2":{"205":1,"212":1,"220":26,"253":1}}],["sciml",{"2":{"58":1,"59":1,"78":1,"100":2,"177":1}}],["scatter",{"2":{"232":1,"236":2,"248":2,"255":1,"256":2,"259":4}}],["scalable",{"2":{"150":1}}],["scalar",{"0":{"156":1},"2":{"3":2,"20":1,"107":1,"156":1,"176":1,"241":1}}],["scale=ones32",{"2":{"69":4}}],["scaled",{"2":{"24":3}}],["scale",{"2":{"19":15,"67":2,"69":20,"70":6,"117":1,"129":1,"145":1}}],["scaling",{"2":{"17":2,"24":2,"254":6}}],["scratch",{"2":{"179":1,"184":1,"211":1}}],["schwarzschild",{"2":{"255":1}}],["school",{"2":{"74":1}}],["schemes",{"2":{"21":1,"168":1}}],["score",{"2":{"53":1}}],["slow",{"2":{"145":2}}],["slower",{"2":{"115":1}}],["slices",{"2":{"54":1}}],["slice",{"2":{"19":2,"24":1,"66":2,"69":2}}],["slightly",{"2":{"62":1}}],["slight",{"2":{"13":1,"175":1}}],["sleefpirates",{"2":{"13":2}}],["s",{"2":{"10":1,"20":1,"57":3,"58":1,"59":1,"62":1,"64":2,"67":1,"69":3,"70":1,"72":1,"77":1,"100":1,"106":2,"107":1,"116":1,"117":1,"121":2,"126":1,"135":1,"136":1,"140":1,"144":2,"145":3,"147":1,"148":1,"149":1,"152":3,"153":1,"155":1,"170":4,"171":5,"176":1,"179":2,"184":5,"186":1,"188":1,"204":1,"208":2,"217":1,"232":3,"233":1,"236":1,"249":2,"255":3}}],["syntax",{"2":{"59":1,"62":1,"171":2,"172":1}}],["synchronized",{"2":{"120":2}}],["synchronize",{"2":{"39":2,"120":6,"121":2,"123":2,"153":2}}],["symbol",{"2":{"10":1,"45":1,"59":1,"69":1,"106":1,"137":1,"188":1,"220":29,"249":2}}],["systems",{"2":{"17":1,"77":1,"100":2,"167":1}}],["system",{"2":{"2":1,"3":1,"106":1,"130":2,"137":1,"175":1,"238":1,"255":1,"256":1}}],["safely",{"2":{"256":1}}],["saveat=tsteps",{"2":{"255":1,"256":1,"257":1,"259":1}}],["saveat=t",{"2":{"206":1,"208":1,"209":1}}],["save",{"2":{"188":4,"215":2,"220":26}}],["saving",{"0":{"188":1},"2":{"188":1}}],["sake",{"2":{"184":1,"186":1,"243":1}}],["sarray",{"2":{"153":7}}],["sarrays",{"2":{"153":1}}],["sanity",{"2":{"152":1}}],["sampling",{"2":{"243":1,"247":1}}],["sampler",{"2":{"249":1}}],["sampled",{"2":{"249":2}}],["samples",{"2":{"179":15,"249":2,"250":2}}],["sample",{"2":{"149":3,"152":1,"249":3,"250":1}}],["samepad",{"2":{"63":2,"65":3}}],["same",{"0":{"114":1},"2":{"3":1,"5":2,"7":3,"13":2,"15":1,"17":1,"19":3,"24":1,"27":2,"44":1,"45":1,"46":1,"52":2,"54":2,"57":2,"62":1,"63":2,"65":3,"66":3,"68":2,"69":1,"71":1,"78":1,"99":2,"106":1,"120":1,"136":1,"146":1,"152":2,"153":1,"172":1,"174":1,"178":1,"187":1,"190":1,"214":1}}],["saw",{"2":{"107":1}}],["say",{"2":{"100":1,"126":1}}],["satisfactory",{"2":{"250":1}}],["satisfying",{"2":{"63":2,"65":6,"135":1}}],["satisfies",{"2":{"34":2,"68":1}}],["satisfied",{"2":{"8":1,"235":1}}],["said",{"2":{"42":1}}],["saxe",{"2":{"24":1}}],["soln",{"2":{"254":13,"255":2,"256":2,"259":2}}],["soln2orbit",{"2":{"254":3}}],["solution",{"2":{"243":5,"245":1}}],["solutions",{"2":{"24":1}}],["sol",{"2":{"209":2}}],["solver",{"2":{"214":6,"218":3,"220":7}}],["solver=tsit5",{"2":{"214":2,"218":1}}],["solvers",{"2":{"100":1}}],["solves",{"2":{"208":1}}],["solve",{"2":{"206":1,"208":4,"209":1,"214":2,"218":1,"238":1,"255":1,"256":1,"257":1,"258":1,"259":1}}],["sorted",{"2":{"81":1}}],["society",{"2":{"53":1}}],["software",{"2":{"74":2}}],["softmax",{"2":{"53":5}}],["softfail",{"2":{"50":2}}],["soft",{"2":{"49":4}}],["sooner",{"2":{"42":1}}],["so",{"2":{"13":1,"77":1,"140":1,"144":3,"145":1,"146":1,"207":1,"214":2,"218":2,"235":1,"240":1,"241":1}}],["sometimes",{"2":{"171":2}}],["somewhere",{"2":{"32":1,"34":1}}],["some",{"0":{"206":1,"254":1},"2":{"8":1,"32":2,"45":1,"59":1,"66":1,"68":1,"82":1,"94":1,"100":1,"101":1,"103":1,"104":1,"105":1,"115":1,"117":1,"119":1,"123":1,"128":1,"136":1,"140":1,"144":7,"145":1,"155":1,"158":1,"160":2,"171":1,"217":1,"225":1,"243":1}}],["source",{"2":{"1":1,"2":3,"3":8,"4":2,"5":1,"7":3,"8":7,"9":2,"10":5,"11":1,"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"20":1,"24":8,"25":24,"27":2,"28":1,"32":3,"34":3,"36":2,"37":3,"38":2,"39":4,"40":1,"41":1,"43":5,"44":1,"45":2,"46":1,"48":2,"49":2,"50":1,"52":6,"53":15,"54":8,"55":6,"56":3,"57":1,"58":1,"59":3,"60":1,"62":6,"63":2,"64":3,"65":9,"66":6,"67":4,"68":7,"69":5,"70":2,"87":1,"170":2}}],["situations",{"2":{"124":1}}],["si+1",{"2":{"63":1,"65":3}}],["siamese",{"2":{"53":1}}],["siamesecontrastiveloss",{"2":{"53":3}}],["sig",{"2":{"249":2}}],["sigmoid",{"2":{"53":3,"184":1,"185":1,"249":2}}],["signify",{"2":{"82":1,"89":1}}],["significantly",{"2":{"133":1,"217":1}}],["significant",{"2":{"8":1,"196":1,"219":1}}],["signature",{"2":{"17":1,"44":1}}],["silently",{"2":{"46":1}}],["sided",{"2":{"254":2}}],["side",{"2":{"16":1,"59":1}}],["size=1000",{"2":{"183":1}}],["sizes",{"2":{"18":1,"24":2,"87":1}}],["sized",{"2":{"18":1,"19":1,"54":1}}],["size",{"0":{"11":1},"2":{"11":2,"13":2,"19":3,"24":24,"25":72,"28":2,"32":1,"34":1,"59":6,"62":1,"63":16,"65":48,"66":11,"67":16,"68":8,"69":7,"70":8,"97":1,"111":1,"150":2,"151":2,"153":2,"171":1,"179":2,"180":1,"183":7,"189":1,"192":3,"197":1,"210":1,"213":3,"214":2,"217":1,"221":1,"229":1,"237":1,"246":1,"249":4,"250":1,"251":1,"254":4,"260":1}}],["sin",{"2":{"254":1}}],["singular",{"0":{"135":1},"2":{"135":1,"136":2}}],["singleton",{"2":{"7":3,"63":2,"65":3,"87":2,"88":1}}],["single",{"2":{"7":2,"8":1,"15":1,"16":1,"52":5,"53":1,"59":1,"63":6,"65":9,"66":13,"77":2,"78":1,"87":1,"107":3,"136":1,"140":1,"146":1,"149":1,"153":4,"171":1,"179":1,"187":1,"195":1,"217":1,"228":1,"236":1,"244":1,"249":1}}],["since",{"2":{"8":1,"24":1,"44":1,"59":1,"71":1,"87":1,"100":1,"107":1,"109":1,"120":1,"137":1,"177":1,"184":1,"186":1,"208":2,"240":1}}],["simultaneously",{"2":{"137":1}}],["simulate",{"2":{"117":1,"254":1,"255":1,"256":2}}],["simulating",{"0":{"255":1},"2":{"24":1}}],["simd",{"2":{"20":1}}],["simplicity",{"2":{"149":1,"186":1,"243":1}}],["simplified",{"2":{"135":1}}],["simplest",{"2":{"62":1}}],["simplechainslayer",{"2":{"34":2,"193":1}}],["simplechains",{"0":{"190":1},"1":{"191":1,"192":1,"193":1,"194":1,"195":1,"196":1,"197":1},"2":{"34":14,"100":1,"190":3,"191":2,"193":1,"196":2}}],["simple",{"0":{"34":1,"181":1},"1":{"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1},"2":{"17":2,"20":1,"34":4,"52":1,"59":1,"66":1,"106":1,"131":2,"140":1,"142":1,"170":1,"176":1,"193":1,"196":1,"204":1}}],["simply",{"2":{"8":1,"13":1,"52":1,"56":1,"57":1,"59":1,"71":1,"107":1,"114":1,"115":1,"128":1,"137":1,"225":1,"249":1}}],["similarly",{"2":{"136":1}}],["similarity",{"2":{"5":1}}],["similar",{"2":{"2":1,"3":1,"7":1,"19":1,"53":1,"54":1,"55":1,"66":1,"67":1,"106":1,"107":1,"137":1,"139":1,"160":1,"175":1,"256":1}}],["ship",{"2":{"115":1}}],["shifted",{"2":{"24":1}}],["shift",{"2":{"19":1,"24":4,"69":4}}],["shuffle=true",{"2":{"183":2,"192":1,"213":1,"224":1,"244":2}}],["shuffle=false",{"2":{"5":1,"183":1,"192":1,"213":1,"224":1}}],["shuffle",{"2":{"70":1,"183":2,"192":2,"213":2}}],["shuffling",{"2":{"70":1}}],["shapedaxis",{"2":{"220":30}}],["shaped",{"2":{"65":9}}],["shape",{"2":{"63":1,"66":16,"68":1,"69":17,"116":4,"171":1,"179":4}}],["shate",{"2":{"46":1}}],["sharing",{"2":{"46":9}}],["share",{"2":{"46":4}}],["shared",{"2":{"19":2,"46":1,"90":1}}],["shooting",{"2":{"208":1}}],["shortcomings",{"0":{"124":1}}],["shortcut",{"2":{"8":1,"62":1}}],["short",{"2":{"66":1,"175":1}}],["shorthand",{"2":{"8":1}}],["shown",{"2":{"249":1}}],["showcasing",{"2":{"203":1}}],["showing",{"2":{"152":1}}],["shows",{"2":{"137":1,"148":1,"149":1,"152":1,"250":1}}],["showerror",{"2":{"112":1}}],["show",{"2":{"5":2,"77":1,"145":1,"172":1,"204":1,"247":1}}],["shouldn",{"2":{"52":1,"87":1,"137":1,"147":1}}],["should",{"0":{"109":1},"2":{"4":2,"15":1,"20":1,"34":1,"36":2,"43":1,"46":1,"52":2,"53":1,"55":1,"59":1,"63":5,"65":6,"66":1,"69":1,"70":1,"90":1,"91":1,"125":1,"130":1,"135":2,"136":2,"137":1,"139":1,"140":2,"155":1,"173":1,"174":1,"211":1,"233":1,"256":1}}],["stencils",{"2":{"254":2}}],["stepnorm=0",{"2":{"258":1}}],["steprangelen",{"2":{"207":2}}],["steps",{"2":{"52":1,"77":1,"107":1,"249":1}}],["step",{"2":{"52":10,"77":1,"78":1,"107":3,"179":1,"187":1,"195":1,"217":1,"228":1,"236":2,"244":1,"249":3}}],["stime",{"2":{"195":2,"217":2,"228":2}}],["stick",{"2":{"175":1}}],["still",{"0":{"143":1},"2":{"2":1,"8":1,"34":1,"114":1,"116":1,"130":1,"135":1,"136":1,"184":1,"220":1,"233":1,"250":1}}],["stop=6",{"2":{"250":4}}],["stochastic",{"2":{"99":1,"100":1,"179":1}}],["stores",{"2":{"58":1,"64":1,"67":1,"123":1,"140":1,"171":1,"214":1}}],["stored",{"2":{"52":3,"63":1}}],["store",{"2":{"39":2,"67":1,"99":2,"140":1,"208":1,"248":1,"256":1,"257":1}}],["storing",{"2":{"19":1,"113":1,"140":1}}],["std=1e",{"2":{"256":3}}],["std=0",{"2":{"24":1}}],["stdlib",{"2":{"174":1}}],["stdout",{"2":{"112":1}}],["std",{"2":{"19":1,"24":5,"249":1}}],["stylization",{"2":{"19":1,"69":1}}],["stage",{"2":{"144":1}}],["start=false",{"2":{"215":1}}],["start",{"2":{"106":1,"170":1,"220":13,"247":1}}],["starting",{"2":{"85":1,"130":1,"171":1,"218":1}}],["started",{"0":{"75":1,"81":1},"1":{"76":1,"77":1,"78":1,"79":1,"80":1},"2":{"139":1}}],["stacktraces",{"2":{"115":1}}],["stacktrace",{"2":{"94":2,"116":1}}],["stack",{"2":{"66":2,"243":6,"245":2}}],["stackedrnncells",{"2":{"66":1}}],["standard",{"2":{"24":3,"25":6,"63":2,"69":1,"77":1,"145":1,"256":1}}],["stabilities",{"2":{"219":1}}],["stability",{"0":{"220":1},"2":{"8":1,"19":4,"53":1,"59":1,"69":4,"82":1,"89":1,"166":1}}],["stablerng",{"2":{"145":3,"146":3,"147":2,"148":2,"152":3}}],["stablerngs",{"2":{"22":1,"144":1}}],["stable",{"2":{"52":1,"55":5,"59":2,"166":1,"186":1,"214":1,"220":1}}],["stats=false",{"2":{"69":5}}],["stats=true",{"2":{"62":2,"69":7,"116":1,"117":2}}],["stats",{"2":{"52":2,"77":2,"107":1,"244":7}}],["staticint",{"2":{"193":2}}],["staticarrayscore",{"2":{"153":4}}],["staticarrays",{"2":{"153":1}}],["staticsymbol",{"2":{"45":1,"54":2}}],["static",{"2":{"20":1,"34":1,"54":1,"58":1,"77":8,"106":3,"112":2,"153":7,"193":4,"220":58,"236":2}}],["staticbool",{"2":{"17":1,"45":1,"54":1}}],["statistics",{"2":{"19":2,"24":2,"52":2,"69":7,"97":1,"182":1,"191":1,"212":1,"223":1,"231":1,"235":1,"239":1,"249":1}}],["state=false",{"2":{"66":4}}],["state=zeros32",{"2":{"66":3}}],["statements",{"2":{"59":1}}],["statefulneuralode",{"2":{"218":4,"219":1,"220":5}}],["statefulrecurrentcell",{"2":{"66":2}}],["statefulluxlayers",{"2":{"144":1}}],["statefulluxlayer",{"2":{"58":1,"95":4,"144":7,"145":1,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"208":2,"218":2,"242":7,"244":2,"249":2,"256":2}}],["stateful",{"0":{"58":1,"218":1,"219":1},"2":{"66":2,"145":1,"146":1,"148":2,"220":6}}],["stateless",{"2":{"8":1,"68":1}}],["statelength",{"2":{"7":1,"10":1,"135":4,"136":1}}],["state",{"0":{"138":1},"2":{"7":1,"8":3,"10":5,"17":2,"19":1,"32":1,"43":1,"52":7,"58":4,"59":7,"62":6,"64":4,"66":64,"68":1,"69":4,"77":8,"78":4,"99":3,"100":3,"107":4,"120":5,"135":5,"136":3,"138":4,"140":1,"184":3,"187":7,"195":8,"208":6,"228":11,"235":1,"244":6,"249":2,"256":1}}],["states",{"0":{"10":1},"2":{"3":1,"7":4,"8":1,"10":2,"32":1,"34":1,"43":4,"44":6,"45":1,"46":1,"52":2,"53":2,"56":2,"57":4,"58":1,"59":5,"62":16,"64":3,"66":5,"68":2,"69":7,"77":2,"78":2,"87":2,"107":1,"111":1,"116":2,"117":1,"120":2,"126":3,"129":1,"133":1,"135":5,"136":3,"138":1,"140":3,"179":2,"184":2,"187":2,"188":1,"193":2,"195":2,"214":1,"217":2,"228":4,"233":1,"235":1,"236":4,"244":1,"249":1,"256":1}}],["st",{"2":{"7":2,"8":4,"10":6,"32":4,"34":2,"43":2,"44":6,"46":1,"52":1,"53":2,"54":2,"57":1,"58":4,"59":19,"62":6,"66":1,"68":14,"77":4,"78":6,"95":2,"106":14,"107":6,"109":3,"111":7,"112":2,"113":10,"114":2,"116":5,"117":7,"120":2,"126":9,"127":7,"128":6,"129":2,"135":7,"136":11,"137":3,"138":2,"139":3,"140":6,"145":7,"146":7,"147":6,"148":6,"150":2,"151":2,"152":9,"153":8,"155":5,"179":6,"184":10,"186":4,"187":7,"188":3,"194":4,"195":2,"208":5,"214":4,"215":4,"216":4,"217":2,"218":7,"220":16,"227":4,"228":2,"235":1,"236":1,"242":7,"244":3,"249":2,"256":2}}],["strain",{"2":{"254":5}}],["strokewidth=2",{"2":{"232":1,"236":2,"248":2,"256":2,"259":4}}],["strokecolor=",{"2":{"232":1,"236":2,"248":2}}],["strongly",{"2":{"173":1}}],["struct",{"2":{"111":1,"133":1,"135":1,"140":3,"188":1,"207":1}}],["structs",{"2":{"56":3}}],["structured",{"2":{"66":3,"137":1,"225":1}}],["structures",{"2":{"55":2,"100":1,"155":1}}],["structure",{"2":{"3":5,"7":2,"8":3,"27":2,"39":1,"44":4,"46":1,"55":3,"67":1,"116":5,"117":13,"135":2,"136":2,"138":1,"140":1,"217":1}}],["strength",{"2":{"53":1}}],["stride=window",{"2":{"65":3}}],["stride=1",{"2":{"63":2}}],["stride",{"2":{"63":5,"65":6}}],["stridearray",{"2":{"34":1}}],["stridearrayscore",{"2":{"34":1}}],["strings",{"2":{"95":1}}],["string=",{"2":{"60":2}}],["string",{"2":{"1":2,"3":1,"48":1,"60":1}}],["surprise",{"2":{"155":1}}],["surpassing",{"2":{"24":2}}],["sure",{"2":{"137":1,"256":1}}],["super",{"2":{"254":1}}],["supertype",{"2":{"136":1}}],["suppose",{"2":{"250":1}}],["supposed",{"2":{"94":1,"100":1,"166":1}}],["supporting",{"2":{"103":1}}],["support",{"0":{"4":1,"72":1,"73":1,"80":1,"104":1,"163":1},"2":{"3":4,"4":4,"7":2,"16":2,"18":2,"43":1,"53":2,"55":1,"58":1,"65":3,"77":2,"78":2,"80":5,"85":2,"89":1,"100":3,"101":1,"102":1,"103":1,"105":1,"124":1,"130":2,"137":1,"142":1}}],["supports",{"2":{"3":1,"59":1,"97":1,"142":1,"163":1,"171":1,"172":1,"240":1}}],["supported",{"0":{"22":1},"2":{"2":1,"3":2,"20":1,"27":2,"28":1,"32":1,"52":6,"57":1,"65":3,"70":1,"77":1,"89":1,"99":2,"104":2,"105":1,"130":2,"160":1,"168":2}}],["supplied",{"2":{"37":1,"62":1}}],["supply",{"2":{"37":1}}],["suggests",{"2":{"68":1}}],["subtracts",{"2":{"171":1}}],["subtract",{"2":{"171":1}}],["subtyping",{"2":{"136":1}}],["subtype",{"2":{"135":1,"136":2}}],["subarray",{"2":{"54":1}}],["suboptimal",{"2":{"52":1,"153":1}}],["sumit",{"2":{"53":1}}],["sum",{"2":{"48":3,"49":2,"59":5,"117":2,"139":2,"140":2,"145":1,"146":1,"147":2,"148":1,"150":1,"151":1,"186":1,"194":1,"216":1,"227":1,"242":3,"254":1}}],["summary",{"2":{"5":8,"140":1,"249":1}}],["success",{"2":{"258":1}}],["successor",{"2":{"119":1}}],["successfully",{"2":{"1":1}}],["such",{"2":{"4":2,"6":1,"8":1,"59":1,"63":2,"65":6,"100":1,"106":1,"118":1,"169":1,"185":1,"225":1}}],["seaborn",{"2":{"250":3}}],["seamlessly",{"2":{"52":1}}],["seq",{"2":{"183":1}}],["sequentially",{"2":{"62":2,"66":5}}],["sequences",{"2":{"183":1}}],["sequence=true",{"2":{"66":2}}],["sequence",{"2":{"62":1,"66":9,"183":7,"184":3}}],["separately",{"2":{"172":1}}],["separation",{"2":{"100":1}}],["separating",{"2":{"100":1}}],["several",{"2":{"161":1,"175":1}}],["sensealg=reversediffadjoint",{"2":{"217":1}}],["sensealg=gaussadjoint",{"2":{"217":1}}],["sensealg=interpolatingadjoint",{"2":{"215":1,"217":1}}],["sensealg",{"2":{"215":1,"217":1,"220":13}}],["sensitivities",{"2":{"214":1,"217":1}}],["sensible",{"2":{"100":1}}],["sensibly",{"2":{"54":1}}],["sensical",{"2":{"59":1}}],["send",{"2":{"81":1}}],["sendbuf",{"2":{"39":6}}],["sendrecvbuf",{"2":{"39":6}}],["seyed",{"2":{"53":1}}],["segmentation",{"2":{"53":2}}],["sec",{"2":{"249":1}}],["section",{"2":{"34":1,"62":2,"76":1,"118":1,"146":1,"162":1,"172":1,"254":1}}],["seconds",{"2":{"249":2}}],["second",{"2":{"17":1,"62":1,"171":1,"177":1,"254":2}}],["server",{"2":{"180":1,"189":1,"196":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["serves",{"2":{"32":1}}],["serious",{"2":{"238":1}}],["seriously",{"2":{"77":1}}],["serializes",{"2":{"120":1}}],["serialization",{"2":{"120":1,"188":1}}],["series",{"2":{"72":2}}],["sergey",{"2":{"19":1}}],["semvar",{"2":{"95":1}}],["semver",{"2":{"42":1}}],["semantic",{"2":{"145":2}}],["semantically",{"2":{"62":1}}],["semantics",{"2":{"5":2,"155":1}}],["semi",{"2":{"24":2}}],["self",{"2":{"17":1}}],["selu",{"2":{"17":1}}],["selecting",{"2":{"154":1}}],["selection",{"0":{"164":1},"2":{"2":1,"3":1,"131":1,"132":1,"164":1}}],["selectdim",{"2":{"68":1}}],["select",{"2":{"2":4}}],["selected",{"0":{"203":1},"2":{"2":3,"3":1,"94":1}}],["selects",{"2":{"2":1}}],["seems",{"2":{"258":2}}],["seeding",{"2":{"77":1}}],["seed",{"2":{"62":1,"68":4,"77":1,"78":1,"100":1,"129":1,"135":1,"137":1,"170":1,"174":2,"179":1,"215":1,"232":1,"244":3,"248":1}}],["see",{"2":{"3":1,"7":3,"8":3,"11":1,"14":1,"15":2,"17":2,"19":4,"32":1,"34":2,"43":1,"45":2,"46":1,"49":1,"52":1,"53":1,"55":2,"57":1,"58":2,"59":2,"60":1,"62":4,"64":3,"66":5,"68":2,"69":4,"70":1,"76":1,"84":1,"87":3,"89":1,"94":2,"96":2,"100":2,"106":1,"107":1,"115":1,"116":3,"117":2,"119":1,"123":1,"136":1,"155":2,"159":1,"162":1,"165":1,"166":1,"168":1,"171":2,"172":1,"184":1,"196":1,"219":1,"250":2,"254":2}}],["session",{"2":{"1":1,"60":1}}],["setprogress",{"2":{"247":1}}],["sets",{"2":{"48":1,"166":2}}],["setindexing",{"2":{"16":1,"18":1}}],["setups",{"2":{"148":1}}],["setup",{"2":{"8":1,"32":1,"34":1,"44":1,"46":1,"59":6,"62":1,"68":4,"77":1,"78":1,"106":1,"107":1,"112":1,"114":1,"116":2,"117":2,"120":1,"126":1,"129":1,"135":2,"136":1,"137":1,"139":1,"140":1,"145":1,"146":1,"147":1,"148":1,"152":1,"153":1,"155":1,"179":1,"187":1,"195":1,"208":1,"215":1,"228":1,"235":1,"244":1,"249":1,"256":1}}],["setfield",{"2":{"7":1,"87":1,"223":1}}],["setting",{"0":{"257":1},"2":{"4":2,"63":2,"66":4,"67":3,"117":1,"144":1,"157":1,"166":2,"167":1}}],["set",{"2":{"1":2,"2":1,"4":6,"17":4,"19":4,"20":1,"24":1,"32":2,"34":1,"37":2,"43":2,"46":1,"48":1,"49":1,"53":2,"55":1,"58":1,"59":3,"60":5,"63":2,"64":1,"66":18,"70":2,"73":3,"82":1,"117":2,"123":1,"131":1,"144":1,"145":4,"157":2,"161":2,"162":1,"163":3,"164":3,"166":2,"179":3,"247":1}}],["os",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["oops",{"2":{"155":1}}],["ok",{"2":{"147":1}}],["own",{"2":{"124":1,"137":1,"171":1}}],["oh",{"2":{"117":1}}],["our",{"2":{"77":1,"100":1,"104":1,"117":3,"142":1,"145":1,"149":1,"155":2,"170":1,"171":1,"173":1,"174":1,"176":1,"179":2,"183":1,"184":1,"204":1,"208":3,"235":1,"236":1,"238":1,"248":1,"249":3,"250":4}}],["outperforms",{"2":{"105":1}}],["outpad",{"2":{"63":3,"97":1}}],["outpad=0",{"2":{"63":1}}],["outputs",{"2":{"11":1,"34":1,"45":1,"58":1,"62":2,"66":6,"68":2,"99":1,"107":1,"146":1,"147":1,"174":1,"176":1,"256":1}}],["outputsize",{"2":{"11":3,"87":1}}],["output",{"2":{"11":1,"13":2,"16":1,"17":3,"18":1,"24":6,"34":2,"45":1,"53":6,"58":1,"59":2,"62":7,"63":8,"65":33,"66":13,"67":7,"68":4,"69":1,"70":4,"78":1,"97":1,"106":2,"116":4,"117":14,"135":1,"138":2,"153":13,"155":1,"174":1,"184":1,"214":1,"241":1,"249":2}}],["outside",{"2":{"55":1,"94":2}}],["out",{"2":{"17":2,"24":2,"42":1,"55":1,"59":10,"62":1,"63":8,"64":2,"65":3,"66":15,"67":19,"78":6,"82":2,"115":1,"117":1,"133":1,"135":8,"138":1,"157":1,"184":3,"185":2,"217":1,"247":1}}],["o=σ",{"2":{"66":1}}],["oi=",{"2":{"63":1,"65":3}}],["o",{"2":{"63":4,"65":6}}],["odefunction",{"2":{"214":2,"218":1}}],["odeproblem",{"2":{"206":1,"208":1,"209":1,"214":2,"218":1,"255":1,"256":1,"259":1}}],["odesolution",{"2":{"214":1}}],["odes",{"0":{"211":1},"1":{"212":1,"213":1,"214":1,"215":1,"216":1,"217":1,"218":1,"219":1,"220":1,"221":1},"2":{"100":1,"110":1,"211":2,"255":1,"256":1}}],["ode",{"0":{"214":1,"215":1,"219":1,"252":1},"1":{"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"58":1,"110":1,"204":3,"206":3,"207":1,"208":2,"209":2,"214":2,"215":1,"255":3,"256":6,"257":1,"259":2}}],["odd",{"2":{"24":1}}],["old",{"2":{"52":1,"84":1,"88":2,"130":2}}],["older",{"0":{"31":1},"1":{"32":1},"2":{"94":1,"123":1}}],["observe",{"2":{"249":1}}],["observations",{"2":{"63":1}}],["obtain",{"2":{"120":1}}],["obtained",{"2":{"20":1}}],["obviously",{"2":{"115":1}}],["obj",{"2":{"52":2,"136":2,"137":1}}],["objects",{"2":{"53":1,"91":3,"106":1,"132":1,"155":1,"207":1}}],["objective",{"2":{"52":8,"257":1}}],["object",{"2":{"2":3,"3":4,"16":1,"27":2,"52":6,"53":2,"120":1,"131":3}}],["occurs",{"2":{"117":1}}],["occurred",{"2":{"117":3}}],["occurrences",{"2":{"10":3}}],["octavian",{"0":{"167":1},"2":{"18":2,"159":1,"167":1}}],["other",{"0":{"25":1,"30":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"5":1,"8":1,"16":1,"24":1,"25":1,"34":1,"39":2,"49":1,"53":1,"55":2,"62":1,"63":1,"68":2,"70":1,"77":1,"78":1,"85":1,"94":1,"105":1,"107":1,"133":1,"135":1,"136":1,"171":1,"175":1,"188":1,"204":1}}],["otherwisewhere",{"2":{"53":1}}],["otherwise",{"2":{"1":1,"2":2,"3":1,"60":1,"67":1}}],["opening",{"2":{"144":1}}],["open",{"2":{"103":1,"106":1,"118":1,"124":1,"153":1,"202":1,"203":2}}],["operating",{"2":{"66":1}}],["operation",{"2":{"16":2,"18":1,"20":4,"39":4,"65":3,"70":1,"175":1}}],["operations",{"0":{"14":1,"55":1},"2":{"2":1,"16":3,"18":1,"28":1,"34":1,"54":2,"55":1,"85":1,"144":2,"153":2,"167":1,"172":1}}],["operates",{"2":{"106":1}}],["operate",{"2":{"66":1,"140":1}}],["operators",{"2":{"68":1}}],["operator",{"2":{"55":1}}],["optprob",{"2":{"258":2}}],["optf",{"2":{"258":2}}],["opt",{"2":{"48":6,"120":5,"179":2,"208":10,"236":1}}],["optimal",{"2":{"208":1}}],["optim",{"2":{"59":2}}],["optimiser",{"0":{"186":1}}],["optimisers",{"2":{"32":2,"40":1,"52":2,"59":3,"77":10,"78":1,"106":1,"120":1,"140":2,"143":1,"179":2,"181":1,"182":1,"191":1,"204":1,"208":2,"212":1,"223":1,"231":1,"234":1,"239":1,"244":1}}],["optimizer",{"0":{"234":1},"2":{"40":4,"52":4,"120":3,"236":2}}],["optimizers",{"0":{"40":1},"2":{"258":1}}],["optimized",{"2":{"20":1,"159":1,"167":1,"208":1}}],["optimizationproblem",{"2":{"208":2,"258":1}}],["optimizationfunction",{"2":{"208":1,"258":1}}],["optimizationoptimjl",{"2":{"205":1,"253":1}}],["optimizationoptimisers",{"2":{"205":1}}],["optimizations",{"2":{"94":1,"106":1}}],["optimization",{"0":{"204":1},"1":{"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"24":1,"77":1,"78":3,"106":1,"204":4,"205":1,"208":8,"253":1,"258":4}}],["options",{"2":{"57":1,"164":1}}],["option",{"2":{"20":1,"57":2,"70":2,"103":2}}],["optional",{"0":{"159":1},"2":{"16":1,"18":1,"24":1,"43":3,"53":1,"58":1,"59":2,"66":1,"87":1,"167":1,"169":1}}],["optionally",{"2":{"7":1,"24":1,"69":1,"135":1,"169":1}}],["op",{"2":{"4":2,"5":1,"19":1,"39":6,"48":1,"54":3,"57":1,"92":1}}],["orbit₂",{"2":{"254":2}}],["orbit₁",{"2":{"254":2}}],["orbits",{"2":{"254":1}}],["orbit2",{"2":{"254":4}}],["orbit2tensor",{"2":{"254":2}}],["orbit1",{"2":{"254":4}}],["orbit",{"2":{"254":17}}],["orange",{"2":{"232":1,"236":1}}],["ordinarydiffeqloworderrk",{"2":{"253":1}}],["ordinarydiffeqcore",{"2":{"220":26}}],["ordinarydiffeq",{"2":{"214":1,"255":1}}],["ordinarydiffeqtsit5",{"2":{"205":1,"212":1,"220":13}}],["ordering",{"2":{"66":6}}],["order",{"0":{"29":1},"2":{"2":1,"43":1,"63":1,"102":1,"145":1,"147":1,"177":1,"240":3,"242":1,"254":2}}],["orcjit",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["originates",{"2":{"45":1}}],["originally",{"2":{"252":1}}],["original",{"2":{"24":1,"32":1,"43":1,"46":1,"106":1,"116":1,"173":2}}],["org",{"2":{"24":1,"68":1,"74":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"254":2,"260":1}}],["orthogonal",{"2":{"22":4,"24":6}}],["or",{"2":{"2":3,"3":2,"4":2,"8":1,"16":2,"17":3,"18":1,"19":2,"20":2,"24":2,"32":1,"34":2,"39":3,"43":1,"45":3,"52":1,"53":5,"54":2,"59":3,"60":1,"62":7,"63":6,"65":9,"66":3,"67":7,"70":4,"72":1,"76":1,"77":1,"81":1,"100":2,"103":1,"107":1,"117":2,"123":2,"124":1,"135":2,"136":1,"140":1,"142":1,"144":1,"147":1,"155":1,"158":2,"163":1,"169":1,"171":4,"172":1,"183":1,"202":1,"203":2,"254":1}}],["overview",{"0":{"102":1}}],["overloaded",{"2":{"94":1}}],["overloading",{"2":{"55":1}}],["overcome",{"2":{"32":1}}],["overridden",{"2":{"18":1}}],["overrides",{"2":{"49":1}}],["override",{"2":{"11":1,"32":1}}],["overfitting",{"2":{"17":2}}],["over",{"0":{"44":1,"100":1},"2":{"2":1,"5":2,"44":1,"45":1,"58":1,"66":1,"68":1,"69":3,"77":1,"125":1,"181":1,"207":1,"236":1,"238":1,"241":1,"257":1,"259":1}}],["onlinestats",{"2":{"239":1,"244":4}}],["online",{"2":{"203":1}}],["only",{"2":{"2":1,"8":1,"45":2,"49":3,"52":1,"53":3,"57":2,"59":1,"63":1,"64":1,"65":3,"66":9,"67":1,"70":1,"106":1,"120":1,"128":1,"136":1,"137":1,"144":2,"147":1,"148":1,"153":2,"176":2,"188":1,"213":1,"242":3}}],["once",{"2":{"11":1,"57":1,"66":2,"117":1,"135":1,"172":1,"173":1,"175":1}}],["onwards",{"2":{"7":1,"94":1}}],["on",{"0":{"108":1,"156":1,"222":1,"238":1},"1":{"109":1,"110":1,"111":1,"112":1,"113":1,"114":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"2":2,"3":6,"5":1,"6":2,"7":1,"8":2,"13":2,"14":3,"15":1,"16":3,"18":3,"19":4,"24":11,"29":2,"34":1,"39":2,"42":1,"43":1,"44":2,"48":1,"53":5,"55":3,"58":2,"59":2,"60":1,"62":2,"63":2,"64":2,"65":6,"66":2,"67":1,"71":1,"74":1,"76":1,"77":1,"81":2,"85":1,"87":2,"95":1,"96":2,"97":1,"100":1,"103":2,"105":1,"106":2,"109":2,"113":1,"118":1,"120":1,"124":1,"133":2,"135":2,"138":1,"140":4,"144":4,"145":4,"146":1,"147":2,"153":3,"155":2,"156":1,"157":1,"159":1,"160":2,"161":2,"162":1,"165":1,"166":1,"168":1,"170":1,"171":1,"172":1,"174":3,"177":2,"180":1,"189":1,"196":2,"197":1,"202":1,"204":1,"207":1,"208":2,"210":1,"217":2,"221":1,"225":1,"229":1,"230":1,"237":1,"243":2,"246":1,"249":1,"251":1,"260":1}}],["one2two",{"2":{"254":3}}],["onecold",{"2":{"194":2,"216":2,"227":2}}],["onehotbatch",{"2":{"192":1,"195":1,"213":1,"224":2}}],["onehotarrays",{"2":{"191":1,"212":1,"223":1}}],["onearray",{"2":{"22":3}}],["oneapidevice",{"2":{"4":2}}],["oneapi",{"2":{"2":1,"3":2,"22":1,"72":2,"77":1,"78":1,"80":2,"130":2,"164":1}}],["onesc64",{"2":{"25":1}}],["onesc32",{"2":{"25":1}}],["onesc16",{"2":{"25":1}}],["ones64",{"2":{"25":1}}],["ones32",{"2":{"25":1}}],["ones16",{"2":{"25":1}}],["ones",{"2":{"8":1,"25":7,"59":1,"111":1,"171":2,"177":1,"220":1,"243":2,"248":1,"249":1}}],["one",{"2":{"1":1,"45":1,"53":1,"56":1,"62":2,"66":1,"69":2,"70":2,"120":1,"144":1,"158":2,"169":1,"171":1,"202":1,"203":1,"249":1,"254":4}}],["official",{"2":{"154":1,"180":1,"189":1,"197":1,"203":1,"204":1,"210":1,"221":1,"229":1,"237":1,"246":1,"247":2,"251":1,"260":1}}],["off",{"2":{"149":1}}],["offending",{"2":{"117":9}}],["ofcourse",{"2":{"116":1}}],["often",{"2":{"3":1,"67":1,"104":2,"110":1,"138":1,"149":1,"152":1,"155":1,"158":2,"171":2,"204":1,"249":1}}],["of",{"0":{"126":1,"128":1,"129":1,"148":1},"2":{"1":1,"3":7,"4":3,"6":1,"7":2,"8":17,"9":3,"10":6,"11":4,"14":2,"15":2,"16":1,"17":4,"18":1,"19":8,"24":31,"25":36,"27":6,"28":6,"32":7,"34":3,"37":2,"38":1,"42":2,"43":6,"44":11,"45":2,"46":9,"47":1,"48":1,"49":7,"52":11,"53":13,"54":1,"55":10,"56":4,"57":10,"58":8,"59":16,"62":32,"63":38,"64":3,"65":42,"66":37,"67":32,"68":23,"69":30,"70":13,"71":1,"74":3,"76":1,"78":2,"82":3,"87":2,"88":1,"89":2,"90":2,"92":2,"94":6,"96":2,"97":1,"100":5,"101":2,"105":1,"106":6,"107":1,"110":2,"114":1,"116":1,"117":6,"120":4,"123":2,"126":2,"127":1,"128":3,"133":3,"135":3,"136":4,"137":1,"138":3,"140":1,"145":1,"146":6,"147":1,"148":1,"149":6,"153":1,"155":1,"158":4,"159":1,"161":1,"162":1,"163":1,"166":4,"167":1,"168":1,"171":9,"172":1,"173":1,"175":4,"177":1,"179":3,"181":1,"183":1,"184":3,"185":2,"186":1,"188":1,"196":1,"202":2,"203":2,"204":1,"211":1,"214":5,"218":1,"219":1,"220":1,"225":2,"236":2,"238":3,"240":1,"243":1,"247":2,"248":2,"249":6,"250":3,"254":5,"255":2,"256":3}}],["bfgs",{"2":{"204":1,"208":1,"258":2}}],["bs",{"2":{"192":1,"213":1}}],["bh2",{"2":{"254":1}}],["bh",{"2":{"254":1}}],["bhn",{"2":{"66":2}}],["bhz",{"2":{"66":2}}],["bhr",{"2":{"66":2}}],["bright",{"2":{"250":3}}],["break",{"2":{"244":1}}],["breaking",{"0":{"84":1,"87":1,"91":1,"94":1,"95":1,"96":1},"2":{"89":1}}],["branched",{"2":{"62":1}}],["branchlayer",{"2":{"62":5}}],["broken=true",{"2":{"48":2}}],["broken=false",{"2":{"48":2}}],["broken",{"2":{"42":1,"48":5,"49":4,"50":1}}],["broadcastfunction",{"2":{"106":1}}],["broadcastable",{"2":{"69":1}}],["broadcast",{"2":{"39":1,"64":1,"68":1,"106":1,"249":1}}],["broadcasted",{"2":{"15":1,"16":2,"39":2,"53":1}}],["broadcasting",{"2":{"13":1,"20":1,"171":1}}],["b=f",{"2":{"140":1}}],["b=layer",{"2":{"140":1}}],["b=",{"2":{"59":1}}],["b=zeros",{"2":{"59":1}}],["bc",{"2":{"242":7,"243":21,"244":120}}],["bce",{"2":{"53":5}}],["bcast",{"2":{"39":2}}],["bigger",{"2":{"171":1}}],["bigfloat",{"2":{"171":3}}],["bit",{"2":{"147":1,"208":1}}],["bibtex",{"2":{"74":2}}],["bilinear",{"2":{"67":3,"70":6,"96":1}}],["bidirectional",{"2":{"66":1}}],["bidirectionalrnn",{"2":{"66":1}}],["biz",{"2":{"66":1}}],["bir",{"2":{"66":1}}],["bin",{"2":{"53":7,"66":1}}],["binarycrossentropy",{"2":{"186":2}}],["binarycrossentropyloss",{"2":{"53":7,"186":1}}],["binaryfocalloss",{"2":{"53":3}}],["binary",{"2":{"53":3,"183":1}}],["bias=l",{"2":{"135":1}}],["bias=false",{"2":{"66":6,"67":3}}],["bias=true",{"2":{"63":4,"66":2,"67":6}}],["bias=nothing",{"2":{"63":1,"66":3,"67":3}}],["bias=ps",{"2":{"59":1}}],["bias=zeros32",{"2":{"63":1,"67":1,"69":4,"135":1,"256":3}}],["bias=zero",{"2":{"44":1}}],["biases",{"2":{"24":1,"249":1}}],["bias",{"0":{"15":1},"2":{"15":12,"16":3,"18":3,"19":15,"43":1,"44":4,"46":4,"59":1,"63":12,"66":25,"67":22,"69":20,"77":15,"97":4,"106":12,"112":2,"117":3,"126":4,"127":2,"128":2,"129":1,"135":6,"137":4,"145":3,"147":4,"148":4,"158":5,"166":1,"179":1,"220":30,"235":2,"249":1,"256":3,"258":3}}],["bn=batchnorm",{"2":{"44":1}}],["black",{"2":{"232":1,"236":2,"248":2}}],["blas",{"2":{"8":1,"18":1}}],["blue",{"2":{"206":1,"209":2,"232":1,"248":1}}],["block",{"2":{"59":3,"62":1,"65":3,"184":1}}],["blocks",{"2":{"52":1,"66":1,"184":1}}],["blog",{"2":{"34":1}}],["blisblas",{"2":{"18":1}}],["b",{"2":{"16":4,"17":2,"18":3,"28":3,"59":6,"65":6,"67":4,"116":2,"131":1,"137":2,"140":9,"146":8,"158":2,"179":8,"244":8,"254":4}}],["bulk",{"2":{"249":1}}],["builds",{"2":{"116":1,"220":3,"244":4}}],["buildkite",{"2":{"112":3,"116":1,"145":2,"180":1,"189":1,"197":1,"210":1,"220":3,"221":1,"229":1,"237":1,"244":4,"246":1,"251":1,"260":1}}],["build",{"2":{"77":1,"163":3,"170":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["building",{"0":{"249":1},"2":{"52":1}}],["built",{"0":{"61":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"98":1,"133":2,"173":1,"196":1}}],["buffer",{"2":{"16":1,"18":1,"39":3}}],["bugs",{"2":{"81":1,"90":1,"137":1}}],["bug",{"2":{"11":1,"59":1,"153":1,"202":1}}],["but",{"2":{"3":2,"7":2,"8":1,"13":1,"14":1,"15":1,"19":1,"24":1,"27":2,"32":1,"34":3,"54":3,"55":5,"57":2,"59":4,"68":1,"70":2,"87":1,"99":1,"100":2,"101":1,"104":3,"105":2,"106":1,"116":3,"117":2,"119":1,"130":1,"135":2,"136":1,"137":1,"145":3,"146":1,"147":1,"152":1,"153":1,"155":2,"160":1,"167":1,"179":1,"184":2,"186":1,"207":1,"217":1,"219":1,"233":1,"243":1,"249":1,"256":1}}],["box",{"2":{"248":1}}],["boxing",{"2":{"58":1,"218":1,"220":1}}],["borrow",{"2":{"247":1}}],["borders",{"2":{"63":2,"65":3}}],["book",{"2":{"177":1}}],["boom",{"2":{"147":1}}],["bool=true",{"2":{"66":1}}],["bool=false",{"2":{"2":2,"32":2,"34":1,"48":1,"66":4,"215":1,"217":1}}],["boolean",{"2":{"8":1,"34":1,"50":1}}],["bool",{"2":{"2":1,"3":5,"8":3,"34":1,"45":1,"53":2,"54":1,"55":1,"220":26}}],["boilerplate",{"2":{"106":1,"107":1,"185":1}}],["boundaries",{"2":{"250":1}}],["boundary",{"2":{"242":1}}],["bound",{"2":{"63":6,"66":18,"67":9}}],["bounds",{"2":{"63":2,"66":6,"67":3}}],["bottom",{"2":{"59":1,"106":1}}],["both",{"2":{"2":1,"45":2,"53":2,"77":1,"97":1,"98":1,"117":3,"140":1,"163":1,"166":1,"203":1,"249":1}}],["body",{"2":{"59":2,"220":3,"254":9}}],["bypasses",{"2":{"59":2}}],["bypass",{"2":{"11":1,"136":1,"164":1}}],["by",{"0":{"127":1},"2":{"2":1,"7":1,"8":2,"11":1,"13":2,"16":1,"18":1,"19":1,"24":5,"34":1,"52":3,"53":2,"54":1,"56":1,"59":2,"62":2,"63":5,"65":3,"66":5,"67":5,"68":1,"69":1,"70":1,"76":1,"77":1,"87":1,"94":1,"100":2,"106":2,"117":1,"120":3,"127":3,"128":2,"131":1,"132":1,"133":1,"144":3,"145":2,"148":1,"149":1,"155":1,"156":1,"157":1,"163":1,"167":1,"171":2,"172":2,"173":1,"175":1,"181":1,"193":1,"203":1,"207":1,"220":1,"235":2,"247":1,"249":1,"250":1}}],["bayes",{"2":{"249":3}}],["bayesian",{"0":{"247":1},"1":{"248":1,"249":1,"250":1,"251":1},"2":{"248":1,"250":1}}],["battery",{"2":{"100":1}}],["batches",{"2":{"192":1,"213":1}}],["batched",{"0":{"14":1,"28":1,"146":1},"2":{"11":1,"14":6,"28":1,"144":1,"146":8,"150":1,"151":1,"158":2}}],["batchsize=32",{"2":{"244":2}}],["batchsize=256",{"2":{"224":1}}],["batchsize=min",{"2":{"224":2}}],["batchsize=8",{"2":{"207":2}}],["batchsize=128",{"2":{"183":2}}],["batchsize=12",{"2":{"160":2}}],["batchsize=13",{"2":{"5":1}}],["batchsize",{"2":{"192":3,"213":3,"224":4}}],["batching",{"2":{"153":3}}],["batchlastindex",{"2":{"54":1,"66":2}}],["batchnorm",{"2":{"7":1,"19":1,"62":4,"69":8,"116":4,"117":4,"126":1,"129":1,"135":1,"138":1,"139":2,"142":1,"145":2,"146":1}}],["batch",{"2":{"5":1,"19":6,"28":3,"34":1,"63":3,"65":3,"66":13,"68":1,"69":6,"70":3,"84":1,"146":1,"153":6,"183":1,"207":1,"208":9,"244":8}}],["basis",{"2":{"63":2,"67":1}}],["basically",{"2":{"136":1}}],["basic",{"2":{"52":2,"170":1}}],["baseline",{"2":{"53":1}}],["base",{"2":{"48":3,"54":3,"64":1,"68":1,"94":1,"112":1,"137":1,"144":3,"147":1,"148":2,"204":1,"207":2,"215":1,"220":25,"256":2}}],["based",{"2":{"2":1,"3":2,"18":1,"19":1,"55":1,"62":1,"81":1,"96":2,"97":1,"145":1,"149":1,"170":1,"204":1}}],["bad",{"2":{"45":1,"155":1,"250":1}}],["ba",{"2":{"19":1}}],["backtracking",{"2":{"258":1}}],["background",{"2":{"135":1,"155":1,"255":1}}],["backward",{"2":{"45":2,"54":1,"66":11,"117":3,"118":1,"173":1}}],["back",{"2":{"2":1,"3":1,"13":1,"137":1}}],["backendtpu",{"2":{"73":1}}],["backendgpu",{"2":{"73":1}}],["backends=",{"2":{"49":2}}],["backends",{"0":{"36":1},"2":{"1":1,"2":4,"3":5,"5":1,"16":3,"18":4,"20":1,"27":6,"28":2,"49":7,"52":5,"92":1,"99":2,"104":3,"130":3,"153":1,"160":1,"162":1,"172":1,"177":1,"217":1}}],["backend",{"0":{"33":1,"131":1,"132":1,"164":1},"1":{"34":1},"2":{"1":17,"2":3,"3":1,"12":1,"27":6,"28":3,"36":5,"37":20,"38":4,"39":11,"40":1,"41":1,"49":1,"52":4,"73":4,"85":1,"120":14,"121":2,"123":1,"131":5,"144":1,"153":6,"164":4,"175":1,"204":1}}],["bernoulli",{"2":{"249":1}}],["became",{"2":{"155":1}}],["because",{"2":{"137":1,"258":1}}],["become",{"2":{"135":1,"181":1}}],["benchmarking",{"2":{"196":1}}],["benefits",{"2":{"106":1}}],["bengio",{"2":{"24":2,"68":1}}],["behind",{"2":{"87":1}}],["behaving",{"2":{"169":1}}],["behaviour",{"2":{"87":1}}],["behavior",{"2":{"3":2,"7":2,"8":1,"11":2,"24":1,"57":3,"66":1,"145":2,"155":1,"184":1}}],["behaves",{"2":{"62":1,"88":1,"160":1}}],["behave",{"2":{"54":1}}],["better",{"2":{"81":1,"100":1}}],["between",{"0":{"30":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"24":1,"32":1,"46":1,"53":1,"67":1,"100":1,"106":1,"123":1,"140":3,"250":3}}],["beginner",{"0":{"199":1}}],["begin",{"2":{"63":2,"65":3,"206":1,"209":1,"243":1,"245":1,"255":1,"256":1,"259":1}}],["beyond",{"2":{"62":1}}],["below",{"2":{"44":1,"57":2,"62":1,"74":1,"214":1,"248":1,"249":2,"250":1}}],["been",{"2":{"32":2,"84":2,"87":6,"91":3,"94":10,"95":5,"97":1,"100":1,"104":1,"122":1,"148":1,"170":1,"252":1}}],["beware",{"2":{"25":1,"137":1}}],["being",{"2":{"15":1,"17":2,"19":2,"55":1,"69":4,"85":1,"92":1,"103":2,"124":1,"144":1,"145":2,"155":1,"196":1,"208":1}}],["best",{"2":{"13":2,"16":1,"18":1,"32":1,"59":2,"103":2,"104":1}}],["before",{"2":{"8":2,"24":2,"37":1,"40":1,"41":1,"62":1,"78":1,"107":1,"120":1,"135":1,"174":1,"188":1}}],["be",{"2":{"1":2,"2":1,"3":10,"4":5,"6":1,"7":4,"8":2,"10":1,"13":2,"14":1,"15":3,"16":1,"17":4,"18":1,"19":20,"20":1,"24":8,"28":5,"32":5,"34":5,"37":7,"39":2,"41":2,"43":2,"45":3,"46":5,"48":3,"49":3,"50":4,"52":9,"53":5,"55":3,"58":2,"59":16,"60":2,"62":13,"63":16,"64":4,"65":9,"66":17,"67":8,"68":8,"69":4,"70":3,"71":1,"74":1,"77":1,"81":1,"82":1,"90":1,"91":1,"94":1,"95":1,"98":1,"99":3,"100":2,"106":3,"111":1,"115":1,"118":1,"120":2,"121":1,"125":1,"127":1,"130":2,"132":1,"133":1,"135":1,"137":2,"138":1,"139":1,"140":6,"142":1,"144":2,"145":5,"146":2,"148":1,"150":1,"155":1,"159":1,"164":1,"166":3,"167":2,"169":3,"171":1,"173":1,"174":1,"175":1,"176":1,"181":1,"184":1,"196":2,"202":1,"203":1,"208":1,"218":1,"226":1,"233":1,"238":2,"243":1,"248":1,"249":1,"254":4,"257":1}}],["ixy",{"2":{"254":2}}],["ixx",{"2":{"254":3}}],["iyy",{"2":{"254":3}}],["i∈",{"2":{"179":1}}],["i9rhw",{"2":{"145":2}}],["ibkvy",{"2":{"112":3}}],["iid",{"2":{"249":1}}],["iii",{"2":{"102":2,"104":1}}],["ii",{"2":{"102":2,"104":1}}],["ii+pi+p",{"2":{"63":1,"65":3}}],["i=σ",{"2":{"66":1}}],["ih",{"2":{"66":6,"97":1}}],["i+n",{"2":{"63":1,"65":3}}],["ignore",{"2":{"225":1,"256":1}}],["ignores",{"2":{"32":1}}],["ignored",{"2":{"2":1,"24":1,"67":3}}],["iclr",{"2":{"24":1}}],["ieee",{"2":{"24":2,"53":5}}],["ioffe",{"2":{"19":1}}],["imgs",{"2":{"192":5,"213":5,"224":4}}],["im",{"0":{"173":1}}],["immutability",{"2":{"174":1}}],["immutable",{"2":{"13":1,"16":1,"18":1,"55":1,"99":1,"100":1,"173":1}}],["immediately",{"2":{"106":1}}],["imbalanced",{"2":{"53":1}}],["images",{"2":{"69":3,"70":2,"192":1,"213":1}}],["image",{"2":{"53":1,"63":2}}],["imagenet",{"2":{"24":2,"119":1}}],["imply",{"2":{"138":1}}],["implements",{"2":{"47":1}}],["implemented",{"2":{"32":1,"100":1,"140":1,"144":1}}],["implementations",{"2":{"13":1,"14":1,"18":1,"20":1,"52":1,"55":1,"123":1,"167":1}}],["implementation",{"0":{"111":1,"141":1,"218":1},"1":{"142":1},"2":{"11":4,"13":3,"16":4,"18":6,"43":1,"52":1,"55":5,"58":2,"62":1,"66":1,"87":1,"123":1,"149":1,"214":1}}],["implement",{"0":{"225":1},"2":{"7":1,"43":1,"135":2,"140":1,"247":1}}],["implementing",{"0":{"140":1},"2":{"7":2,"100":1,"110":1,"211":1,"247":1}}],["improving",{"2":{"105":1}}],["imports",{"0":{"182":1,"191":1,"205":1,"212":1,"223":1,"231":1,"239":1,"253":1}}],["importing",{"2":{"135":1,"170":1,"247":1}}],["important",{"0":{"141":1},"1":{"142":1},"2":{"120":2,"130":1,"135":1,"140":1,"145":1,"146":1}}],["importantly",{"2":{"107":1,"171":1,"250":1}}],["imported",{"2":{"59":1}}],["import",{"2":{"32":1,"34":1,"79":1,"247":1}}],["i",{"2":{"5":10,"11":1,"24":1,"32":1,"34":1,"43":1,"52":2,"53":3,"57":1,"58":1,"59":1,"62":9,"63":6,"65":21,"67":4,"68":8,"70":2,"78":1,"87":2,"102":4,"104":1,"111":3,"120":1,"138":1,"147":1,"153":6,"174":4,"177":1,"179":4,"207":2,"244":3,"245":6,"248":12,"249":7,"250":10}}],["ith",{"2":{"62":1}}],["its",{"2":{"8":1,"65":3,"67":1,"68":1,"69":1,"71":1,"100":1,"250":1}}],["itself",{"2":{"3":1,"68":1,"116":1}}],["iter",{"2":{"78":4,"107":12,"208":2,"244":7,"257":1}}],["iteratively",{"2":{"62":1}}],["iterations",{"2":{"179":12,"249":2}}],["iteration",{"0":{"5":1},"2":{"5":2,"78":12,"107":4,"110":1,"160":1,"174":8,"207":1,"208":26,"244":101,"250":1,"257":1,"259":1}}],["iterate",{"2":{"5":1,"42":1}}],["iterates",{"2":{"5":1}}],["iterators",{"2":{"5":1,"184":1,"185":1,"243":1,"244":2,"245":1}}],["iterator",{"2":{"5":3,"107":1}}],["item",{"2":{"5":1}}],["it",{"2":{"2":1,"3":2,"4":5,"7":3,"8":6,"11":2,"13":3,"15":2,"16":2,"17":1,"24":3,"32":3,"42":1,"43":1,"44":1,"46":1,"47":2,"50":2,"52":1,"53":5,"54":1,"55":3,"57":3,"58":1,"59":5,"62":2,"63":4,"65":3,"66":8,"68":1,"69":7,"70":3,"74":1,"76":1,"78":1,"87":1,"95":1,"97":1,"98":1,"100":4,"101":1,"105":3,"106":5,"109":1,"115":2,"116":1,"117":3,"120":2,"121":1,"131":1,"133":2,"135":2,"136":5,"137":4,"144":3,"145":6,"146":2,"152":1,"153":2,"155":1,"156":2,"157":1,"160":1,"164":1,"167":1,"170":1,"171":2,"172":1,"173":2,"174":2,"177":1,"179":1,"184":4,"186":2,"202":1,"203":1,"204":1,"207":2,"208":2,"214":1,"225":2,"249":2,"254":1,"256":2}}],["inlined",{"2":{"244":4}}],["in2",{"2":{"67":6}}],["in12",{"2":{"67":3}}],["in1",{"2":{"67":8}}],["injection",{"2":{"62":5}}],["independent",{"2":{"146":1}}],["index",{"2":{"62":1,"67":1,"68":2,"153":1,"250":1}}],["indexing",{"0":{"156":1},"2":{"20":1,"62":1,"68":1,"156":1}}],["indexed",{"2":{"2":2,"4":2,"68":1,"81":1}}],["individual",{"2":{"68":1,"208":1}}],["individually",{"2":{"62":1,"70":2,"146":1}}],["indices",{"2":{"67":2}}],["inplace",{"2":{"52":2}}],["input",{"0":{"108":1,"114":1},"1":{"109":1,"110":1,"111":1,"112":1,"113":1,"114":1},"2":{"8":2,"13":2,"15":1,"16":1,"17":3,"18":2,"19":9,"24":4,"27":2,"28":1,"34":3,"45":2,"53":4,"58":1,"59":1,"62":30,"63":7,"64":2,"65":6,"66":20,"67":12,"68":7,"69":14,"70":6,"77":1,"106":1,"110":1,"116":6,"117":14,"120":1,"121":1,"123":1,"135":1,"138":2,"146":3,"148":1,"153":8,"155":2,"165":1,"166":1,"169":1,"184":1,"193":1,"214":1,"241":1,"256":1}}],["inputsize",{"2":{"66":1,"87":1}}],["inputs",{"0":{"146":1},"2":{"8":2,"11":1,"16":3,"18":1,"28":1,"44":1,"45":1,"52":1,"53":4,"55":1,"58":1,"62":9,"63":2,"64":3,"65":9,"66":11,"67":1,"68":6,"69":5,"70":2,"95":1,"99":1,"106":1,"123":1,"144":1,"146":2,"147":1,"148":1,"178":1,"214":2,"235":1,"249":1,"256":1}}],["inner",{"2":{"43":2,"45":1,"127":1}}],["incase",{"2":{"256":1}}],["including",{"2":{"90":1,"106":1}}],["included",{"2":{"24":1,"100":1}}],["include",{"2":{"7":1,"8":1,"24":1}}],["inconvenient",{"2":{"109":1}}],["inconsistent",{"2":{"87":1}}],["incorrectly",{"2":{"155":1}}],["incorrect",{"0":{"116":1},"2":{"28":2,"59":1,"115":1,"118":1}}],["increase",{"2":{"63":1,"97":1}}],["initilly",{"2":{"64":1}}],["initializing",{"0":{"168":1},"1":{"169":1},"2":{"59":1,"120":1}}],["initialize",{"0":{"215":1,"226":1},"2":{"37":5,"59":3,"77":1,"78":1,"120":3,"121":1,"123":1,"153":1,"179":1,"232":1,"249":1}}],["initialized",{"2":{"24":4,"37":2,"69":8}}],["initializers",{"2":{"24":1}}],["initializer",{"2":{"24":1,"59":3,"66":10,"67":7}}],["initializations",{"2":{"96":2}}],["initialization",{"0":{"37":1},"2":{"21":1,"24":2,"34":1,"37":1,"63":4,"96":1,"140":1,"168":1,"184":1,"225":1,"256":1}}],["initial",{"2":{"9":1,"10":1,"59":2,"62":1,"66":11,"87":1,"100":1,"140":1,"184":1,"255":1,"256":1,"258":2}}],["initialstates",{"2":{"7":3,"8":1,"10":1,"135":2,"140":2,"184":1}}],["initialparameters",{"2":{"7":3,"8":1,"9":1,"135":2,"140":2,"184":1,"225":3}}],["init",{"2":{"24":7,"54":6,"59":6,"63":10,"66":26,"67":21,"69":16,"96":2,"97":1,"121":1,"135":10,"140":4,"169":8,"184":2,"185":2,"256":6}}],["ingredient",{"2":{"19":1,"69":1}}],["investigate",{"2":{"219":1}}],["investigated",{"2":{"124":1}}],["inversability",{"2":{"63":1}}],["inverse",{"2":{"17":1}}],["inv",{"2":{"63":2,"66":6,"67":3,"254":1}}],["involving",{"2":{"58":1}}],["invokes",{"2":{"44":1}}],["invoked",{"2":{"2":1,"184":1}}],["invariant",{"2":{"53":1}}],["invp",{"2":{"17":4}}],["informed",{"2":{"242":3}}],["informs",{"2":{"136":1}}],["information",{"2":{"8":1,"11":1,"17":1,"29":1,"45":1,"49":1,"57":1,"99":1,"100":1,"123":1,"157":1,"203":1,"204":1}}],["info",{"2":{"116":8,"117":20,"130":1,"132":1,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"247":2,"251":2,"260":2}}],["inf",{"2":{"53":1,"145":2,"146":2,"147":2,"148":2,"152":4}}],["inference",{"0":{"142":1},"2":{"45":1,"64":3,"69":6,"74":1,"142":1,"145":2,"153":2,"249":2}}],["inferred",{"2":{"19":1}}],["infinity",{"2":{"17":1}}],["int=50000",{"2":{"244":1}}],["int=32",{"2":{"241":1,"244":1}}],["int=0",{"2":{"39":5,"244":1}}],["intro",{"2":{"170":1}}],["introduction",{"2":{"88":1}}],["introductory",{"2":{"81":1}}],["introducing",{"2":{"32":1}}],["introduces",{"2":{"170":1}}],["introduced",{"2":{"24":1}}],["introduce",{"2":{"11":1,"58":1,"117":2}}],["int64",{"2":{"53":3,"77":16,"106":6,"112":4,"171":3,"173":2,"207":2,"220":99,"236":4}}],["int",{"2":{"19":1,"63":2,"66":2,"68":1,"69":1,"70":1,"135":4,"179":1,"214":2,"224":3,"248":1}}],["into",{"0":{"120":1},"2":{"8":1,"16":1,"24":2,"34":1,"39":1,"45":1,"53":1,"54":1,"56":4,"59":2,"62":2,"63":2,"65":3,"66":4,"68":1,"70":1,"77":1,"87":1,"106":1,"133":1,"135":1,"153":2,"171":2,"184":1,"192":1,"213":1}}],["intentionally",{"2":{"139":1}}],["interpolatingadjoint",{"2":{"220":13}}],["intermediate",{"0":{"200":1},"2":{"244":4}}],["interactiveutils",{"2":{"180":2,"189":2,"197":2,"210":2,"212":1,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["interactive",{"2":{"161":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["interested",{"2":{"146":1,"250":1}}],["interoperability",{"0":{"30":1},"1":{"31":1,"32":1,"33":1,"34":1}}],["inter",{"2":{"28":1}}],["interval",{"2":{"24":2}}],["international",{"2":{"19":1,"24":6,"53":3}}],["internals",{"2":{"225":1,"249":1}}],["internally",{"2":{"32":1,"58":1,"87":1,"144":1,"153":1,"220":1}}],["internal",{"2":{"19":2,"20":3,"34":1,"43":1,"52":1,"58":2,"62":1,"68":2,"94":1,"98":1,"100":1,"120":1}}],["interfacem",{"2":{"34":1}}],["interface",{"0":{"133":1,"134":1,"137":1,"138":1},"1":{"134":1,"135":2,"136":2,"137":1,"138":1},"2":{"7":1,"34":2,"41":2,"133":2,"135":1,"204":1,"214":1,"249":1}}],["integral",{"2":{"53":1}}],["integrating",{"0":{"120":1}}],["integration",{"0":{"40":1,"41":1},"2":{"121":1}}],["integrated",{"2":{"8":1,"157":1}}],["integers",{"2":{"37":1,"63":11,"65":15,"67":5}}],["integer",{"2":{"2":3,"4":3,"24":11,"57":3,"63":12,"65":9,"67":1,"68":1,"69":5}}],["intelligence",{"2":{"24":2}}],["intel",{"2":{"3":1}}],["inside",{"0":{"153":1},"2":{"44":1,"57":1,"59":1,"127":1,"130":1,"140":3,"144":3,"153":3}}],["instructions",{"2":{"76":1}}],["institute",{"2":{"74":1}}],["installation",{"0":{"76":1},"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["install",{"0":{"71":1},"2":{"71":1,"72":1,"73":1,"76":1,"77":1,"79":1}}],["installed",{"2":{"37":4,"41":1,"120":1,"172":1}}],["instability",{"2":{"58":1,"157":1,"166":1}}],["instabilities",{"0":{"157":1},"2":{"32":1,"106":1,"157":2}}],["instance",{"2":{"19":3,"66":1,"69":3,"249":1}}],["instancenorm",{"2":{"19":2,"69":7,"96":1,"97":1}}],["instead",{"2":{"3":2,"4":2,"8":1,"32":1,"36":2,"43":2,"48":1,"52":1,"57":1,"59":1,"63":3,"90":1,"91":1,"92":1,"94":1,"95":4,"96":2,"109":1,"110":1,"120":1,"128":1,"133":1,"136":1,"137":1,"140":2,"147":1,"153":2,"155":1,"158":1,"160":1,"167":1,"171":1,"184":1,"211":1,"244":4,"247":1}}],["inspiration",{"2":{"5":1}}],["in",{"0":{"61":1,"96":1,"109":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"2":2,"3":1,"4":2,"5":2,"7":4,"8":5,"10":3,"13":3,"15":2,"17":3,"19":2,"24":15,"27":2,"32":5,"34":1,"37":1,"39":2,"40":1,"42":1,"43":2,"44":3,"45":3,"46":6,"47":1,"48":1,"52":7,"53":3,"54":4,"55":7,"57":1,"58":4,"59":18,"62":20,"63":22,"64":2,"65":3,"66":23,"67":13,"68":6,"69":10,"71":3,"74":2,"76":1,"78":7,"79":1,"81":2,"82":1,"84":1,"87":1,"96":1,"97":1,"98":2,"99":1,"100":3,"101":2,"103":1,"105":1,"107":4,"110":1,"111":1,"115":1,"116":1,"117":11,"118":3,"120":4,"123":2,"124":1,"125":2,"126":1,"127":1,"130":2,"131":1,"135":10,"138":1,"139":2,"140":1,"143":2,"144":3,"145":3,"146":8,"147":1,"148":1,"149":2,"152":1,"153":5,"155":2,"156":2,"157":1,"160":5,"161":2,"167":1,"168":1,"169":1,"170":2,"171":2,"173":1,"174":3,"175":1,"179":2,"181":1,"183":3,"184":6,"185":3,"187":3,"190":1,"194":1,"195":2,"202":1,"204":2,"207":1,"208":4,"216":1,"217":2,"219":1,"225":1,"227":1,"228":4,"230":1,"236":1,"238":1,"243":1,"244":1,"245":2,"247":1,"248":6,"249":2,"250":13,"255":1,"256":3}}],["idx",{"2":{"227":2,"228":12}}],["ideal",{"2":{"196":1}}],["ideally",{"2":{"20":1,"203":1}}],["identity",{"2":{"16":1,"19":5,"24":13,"62":1,"77":3,"106":1,"112":2,"220":9,"236":1}}],["id",{"2":{"2":7,"4":3}}],["if",{"2":{"1":2,"2":12,"3":9,"4":3,"6":1,"7":1,"8":5,"11":2,"13":4,"15":2,"16":2,"17":7,"18":2,"19":6,"28":2,"32":2,"34":1,"37":2,"43":3,"44":1,"45":5,"46":2,"48":1,"49":2,"50":3,"52":2,"53":8,"54":4,"55":3,"57":1,"58":3,"59":6,"60":1,"62":17,"63":8,"64":10,"66":28,"67":12,"68":5,"69":14,"70":3,"71":1,"74":2,"76":1,"77":1,"78":2,"87":2,"94":1,"100":1,"103":2,"104":1,"106":3,"107":1,"116":1,"117":1,"118":1,"123":1,"124":1,"126":1,"127":1,"130":1,"131":4,"132":1,"133":1,"135":3,"136":3,"137":3,"142":1,"143":1,"144":3,"145":2,"148":1,"153":1,"159":1,"163":3,"167":1,"168":2,"169":2,"172":2,"174":1,"175":1,"179":1,"180":3,"189":3,"197":3,"202":1,"203":2,"208":2,"210":3,"221":3,"229":3,"236":1,"237":3,"244":1,"246":3,"251":3,"254":3,"260":3}}],["isdefined",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"229":3,"237":3,"246":3,"251":3,"260":3}}],["isnan",{"2":{"244":1}}],["isn",{"2":{"87":1,"143":1,"254":1}}],["issuing",{"2":{"57":1}}],["issue",{"2":{"57":1,"103":1,"106":1,"118":1,"124":1,"144":1,"145":2,"148":1,"159":1,"202":1,"203":2}}],["issues",{"2":{"52":1,"59":1,"66":1,"81":1,"100":1,"104":2,"105":1,"144":1,"153":1}}],["isbitstype",{"2":{"55":1}}],["istraining",{"2":{"54":4}}],["iszero",{"2":{"44":1}}],["isa",{"2":{"24":1,"44":1,"59":1}}],["isleaf",{"2":{"3":5,"10":1}}],["is",{"2":{"0":1,"1":4,"2":19,"3":18,"4":10,"5":3,"6":1,"7":6,"8":12,"10":2,"11":4,"13":2,"14":1,"15":3,"16":4,"17":6,"18":2,"19":7,"20":4,"21":1,"24":9,"27":4,"32":3,"34":7,"37":2,"41":2,"43":5,"44":4,"45":6,"47":2,"50":2,"52":4,"53":36,"54":5,"55":9,"57":5,"58":3,"59":9,"60":2,"62":23,"63":3,"64":7,"65":3,"66":38,"67":9,"68":10,"69":18,"70":3,"71":1,"76":1,"82":1,"85":1,"87":1,"88":1,"90":1,"94":3,"96":1,"97":1,"100":6,"101":1,"103":2,"105":2,"106":4,"109":1,"115":2,"116":4,"117":5,"118":1,"119":1,"120":4,"121":1,"122":1,"123":2,"124":2,"127":2,"130":1,"131":9,"132":1,"133":4,"135":3,"136":6,"137":2,"140":6,"142":3,"144":3,"145":12,"146":2,"149":1,"150":1,"152":1,"153":3,"155":2,"156":2,"157":1,"158":2,"160":3,"161":1,"163":2,"164":1,"169":3,"170":1,"171":4,"172":3,"173":4,"175":3,"179":1,"183":1,"184":1,"185":1,"186":2,"190":1,"196":2,"204":2,"208":4,"214":1,"217":2,"220":1,"225":1,"235":1,"236":1,"238":1,"244":4,"247":3,"248":1,"249":8,"250":2,"252":1,"254":1,"256":2}}],["fθ",{"2":{"250":1}}],["fw",{"2":{"179":1}}],["fwiw",{"2":{"136":1}}],["fd",{"2":{"145":8,"146":8,"147":8,"148":8}}],["f2",{"2":{"135":2}}],["f=σ",{"2":{"66":1}}],["f=identity",{"2":{"19":1}}],["ft",{"2":{"58":3}}],["f64",{"2":{"56":1,"256":1}}],["f32",{"2":{"56":1}}],["f16",{"2":{"56":1}}],["f1",{"2":{"53":1,"135":2}}],["fn=neuralode",{"2":{"215":1}}],["fn",{"2":{"52":2,"53":2,"59":8,"215":1}}],["fetch",{"2":{"137":1,"232":1}}],["feel",{"2":{"135":1}}],["feedforward",{"2":{"24":2,"249":1}}],["few",{"2":{"98":1,"171":1,"175":1}}],["fed",{"2":{"66":4}}],["features",{"0":{"42":1,"85":1,"88":1,"92":1,"97":1},"1":{"43":1,"44":1,"45":1,"46":1},"2":{"42":3,"47":1,"82":1,"94":1,"99":1,"124":2,"144":1,"192":1,"213":1}}],["feature",{"2":{"2":1,"24":2,"58":1,"63":1,"65":6,"69":3,"105":1,"106":1,"118":1,"144":2,"153":2,"171":1}}],["func",{"2":{"208":3}}],["functor",{"2":{"55":2,"56":3,"140":2}}],["functors",{"2":{"3":1,"5":1,"7":2,"10":3,"44":2,"55":2,"56":3,"87":1,"92":1,"95":1,"106":1,"126":1,"140":1,"247":1}}],["function3",{"2":{"148":4}}],["function2",{"2":{"147":4}}],["function1",{"2":{"145":5}}],["functions",{"0":{"20":1,"24":1,"25":1,"38":1,"53":1,"194":1,"216":1,"227":1,"242":1,"254":1},"2":{"7":1,"13":1,"22":1,"25":1,"32":1,"34":1,"52":1,"53":2,"54":1,"56":1,"62":1,"66":4,"87":1,"94":2,"98":1,"99":2,"100":1,"101":1,"106":1,"107":1,"123":1,"130":2,"131":1,"135":3,"140":1,"144":1,"148":1,"166":1,"167":1,"171":1,"173":1,"177":2,"225":1,"235":1,"254":1}}],["function",{"0":{"109":1,"145":1,"147":1,"148":1,"235":1},"1":{"146":1},"2":{"3":1,"4":4,"8":4,"10":2,"11":3,"13":4,"14":1,"15":2,"16":1,"17":2,"18":1,"19":7,"24":2,"27":4,"28":3,"34":1,"36":2,"37":1,"43":1,"44":4,"48":1,"49":1,"52":15,"53":2,"55":1,"57":3,"58":1,"59":11,"62":2,"63":4,"66":3,"67":4,"68":5,"70":1,"78":3,"87":1,"94":2,"96":3,"100":1,"106":9,"107":1,"111":1,"113":1,"117":2,"120":1,"126":1,"127":3,"128":2,"131":3,"135":3,"136":1,"137":1,"140":2,"143":1,"144":5,"145":1,"146":6,"148":2,"150":1,"151":1,"152":1,"153":2,"169":1,"172":2,"174":1,"176":1,"178":1,"179":1,"183":1,"185":1,"186":2,"192":1,"194":2,"195":1,"205":1,"208":4,"213":1,"214":4,"215":1,"216":2,"217":2,"218":2,"224":2,"225":1,"226":1,"227":2,"232":2,"235":3,"236":2,"238":1,"239":1,"241":2,"242":14,"244":1,"248":1,"249":5,"250":3,"254":16,"256":2,"257":7}}],["functionalities",{"2":{"35":1,"45":1}}],["functionality",{"0":{"45":1,"94":1,"95":1,"122":1},"2":{"6":1,"52":1,"66":1,"79":1,"87":2,"94":1,"133":2,"172":1}}],["functional",{"2":{"2":5,"3":5,"37":1,"119":1,"132":2,"166":1,"172":1,"180":2,"189":2,"197":2,"203":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["full",{"0":{"152":1},"2":{"146":1,"152":16}}],["fully",{"0":{"18":1},"2":{"24":1,"53":1,"55":1,"67":2,"104":1,"119":1}}],["future",{"2":{"144":1}}],["further",{"2":{"52":1}}],["fusion",{"2":{"16":1,"18":1}}],["fuse",{"2":{"16":1}}],["fuses",{"2":{"16":1}}],["fused",{"2":{"16":2,"18":2,"53":1,"158":3,"166":1}}],["fp32",{"2":{"13":1}}],["f",{"2":{"13":4,"16":2,"18":2,"27":5,"28":7,"44":3,"48":2,"49":6,"52":2,"55":4,"59":1,"62":4,"68":6,"140":1,"169":1,"176":3,"177":1,"178":1,"179":1,"208":1}}],["flexibility",{"2":{"137":1}}],["flexible",{"2":{"7":1}}],["flow",{"2":{"106":2}}],["float16",{"2":{"25":8,"56":1,"57":6}}],["floating",{"0":{"56":1},"2":{"13":1,"56":3}}],["float32",{"2":{"5":3,"24":6,"25":8,"32":1,"34":1,"53":6,"56":2,"57":5,"62":2,"68":2,"77":57,"78":1,"106":28,"107":1,"112":6,"113":3,"114":3,"116":9,"117":27,"126":10,"129":6,"131":3,"132":1,"135":2,"136":2,"137":3,"139":2,"145":10,"146":2,"147":11,"148":11,"152":2,"155":5,"168":5,"171":8,"172":1,"176":5,"177":4,"178":1,"179":8,"183":1,"192":1,"195":1,"207":4,"213":1,"220":77,"232":3,"235":4,"243":5,"244":1,"248":8,"256":7}}],["float64",{"2":{"5":4,"25":8,"53":1,"56":1,"57":4,"59":4,"153":7,"155":3,"171":5,"174":1,"207":4,"249":13,"255":1,"256":1,"257":1,"258":2}}],["flipkernel=true",{"2":{"97":1}}],["flat",{"2":{"217":1}}],["flattening",{"2":{"94":1}}],["flatten",{"2":{"68":1}}],["flattens",{"2":{"68":1}}],["flattenlayer",{"2":{"34":1,"68":3,"193":3,"215":1,"220":9,"226":1}}],["flattened",{"2":{"32":1,"68":3,"247":1}}],["flaky",{"2":{"122":1}}],["flag",{"2":{"34":1}}],["fluxlinear",{"2":{"140":5}}],["fluxlayer",{"2":{"32":2}}],["fluxmpifluxmodel",{"2":{"122":1}}],["fluxmpi",{"0":{"121":1},"1":{"122":1,"123":1},"2":{"119":1,"121":3,"122":1,"123":2}}],["flux",{"0":{"32":1,"100":1,"139":1,"143":1},"1":{"140":1,"141":1,"142":1,"143":1},"2":{"32":14,"66":3,"98":1,"100":4,"122":1,"135":2,"139":3,"140":5,"142":4,"143":1,"170":1,"247":1,"252":1}}],["fmaps",{"2":{"8":1}}],["fmap",{"2":{"7":3,"8":2,"10":1,"44":1,"55":7,"249":1}}],["framerate=10",{"2":{"245":1}}],["framework",{"2":{"98":1,"100":1,"133":1,"170":1,"173":1,"190":1}}],["frameworks",{"0":{"31":1},"1":{"32":1},"2":{"63":1,"66":1,"98":3,"133":1,"171":1,"256":1}}],["frequently",{"2":{"202":1}}],["frees",{"2":{"160":1}}],["freeze",{"2":{"43":5,"125":1,"126":6,"127":6,"128":5,"129":1}}],["freezing",{"0":{"43":1,"125":1,"126":1,"127":1,"128":1,"129":1},"1":{"126":1,"127":1,"128":1,"129":1},"2":{"125":1,"127":1,"128":2}}],["free",{"2":{"5":1,"24":1,"52":1,"254":1}}],["freeable",{"2":{"5":1}}],["friendly",{"2":{"98":2}}],["frozen",{"2":{"43":8,"126":9,"129":4}}],["frozenlayer",{"2":{"43":8}}],["fromfluxadaptor",{"2":{"32":5,"143":1}}],["from",{"0":{"31":1,"121":1,"139":1},"1":{"32":1,"122":1,"123":1,"140":1,"141":1,"142":1,"143":1},"2":{"4":1,"5":3,"7":2,"11":1,"13":1,"17":1,"20":1,"24":6,"25":12,"32":1,"34":1,"44":1,"52":5,"53":2,"54":1,"59":1,"62":1,"63":2,"64":1,"66":10,"67":1,"69":2,"77":1,"78":1,"94":2,"95":1,"96":2,"100":1,"130":1,"135":1,"136":1,"137":1,"145":2,"146":2,"147":1,"148":1,"155":1,"158":1,"160":3,"171":1,"179":2,"184":1,"190":1,"204":1,"208":1,"211":1,"220":3,"230":1,"232":1,"234":1,"238":1,"247":1,"249":3,"250":4,"252":2,"254":3,"256":1}}],["familiar",{"2":{"181":1,"225":1}}],["far",{"2":{"135":1}}],["farley",{"2":{"68":1}}],["fake",{"2":{"117":1}}],["favor",{"2":{"63":1}}],["facusapienza",{"2":{"145":1}}],["fact",{"2":{"123":1}}],["factor",{"2":{"17":2,"19":8,"24":2,"53":2,"70":2}}],["facilitates",{"2":{"58":1}}],["fausto",{"2":{"53":1}}],["fashionmnist",{"0":{"222":1},"1":{"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"224":1,"228":2}}],["fashion",{"2":{"37":1}}],["fastest",{"2":{"103":2,"150":1}}],["faster",{"0":{"158":1},"2":{"3":1,"13":2,"14":3,"103":1,"144":1,"158":2,"159":1}}],["fast",{"2":{"13":4,"15":2,"19":1,"20":1,"34":1,"42":1,"69":1,"94":1,"149":1,"158":2,"160":1,"177":1,"256":3}}],["failed",{"2":{"116":1}}],["failing",{"2":{"49":1}}],["fails",{"2":{"45":1,"48":1,"103":2}}],["failures",{"2":{"32":1}}],["fail",{"2":{"32":1,"49":4,"104":1}}],["fancy",{"2":{"106":1}}],["fan",{"2":{"24":6,"63":2}}],["fallback",{"2":{"7":1,"11":3,"16":2,"18":2,"20":1,"87":1,"143":1}}],["fall",{"2":{"3":1}}],["falls",{"2":{"2":1,"13":1}}],["false",{"2":{"2":2,"10":1,"17":1,"20":1,"32":1,"34":2,"45":1,"50":1,"53":5,"54":1,"55":1,"58":1,"62":2,"63":2,"66":10,"67":3,"96":1,"144":1,"156":1,"162":1,"163":1,"212":1,"214":3,"218":2,"220":13,"223":1,"239":1,"257":1}}],["fitting",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1}}],["fit",{"2":{"208":1,"230":1,"244":4,"250":1}}],["figure",{"2":{"206":1,"209":1,"232":1,"236":1,"245":1,"248":1,"255":1,"256":1,"259":2}}],["figured",{"2":{"117":1}}],["fig",{"2":{"206":3,"209":3,"232":3,"236":3,"245":6,"248":3,"250":6,"255":3,"256":3,"259":6}}],["fix",{"2":{"104":1,"116":1,"117":2,"145":2,"149":1,"155":2,"202":1}}],["fixing",{"2":{"90":1,"148":1,"168":1}}],["fix1",{"2":{"64":1,"68":1,"144":3,"147":1,"148":2,"215":1,"220":9,"256":2}}],["fixed=",{"2":{"59":1}}],["fixed",{"2":{"17":1,"58":1,"59":4,"95":2,"116":4,"173":1}}],["finetune",{"2":{"208":1}}],["fine",{"2":{"136":1}}],["finite",{"2":{"145":4,"146":2,"149":1}}],["finitediff",{"2":{"49":1,"144":1,"145":2,"146":2}}],["finish",{"2":{"115":1}}],["findmax",{"2":{"250":1}}],["find",{"2":{"59":1,"118":1,"153":1,"179":1,"202":1,"250":1}}],["final",{"2":{"59":2,"228":3}}],["finally",{"0":{"196":1},"2":{"37":1,"59":1,"120":1,"135":1,"136":1,"171":1,"179":1,"184":1,"220":1,"236":1,"259":1}}],["fill",{"2":{"53":1,"171":1}}],["filter",{"2":{"24":1}}],["file",{"2":{"1":1,"131":3}}],["fields",{"2":{"52":1,"55":1,"62":10,"68":2,"137":1}}],["fieldnames",{"2":{"7":1,"184":1}}],["field",{"2":{"7":3,"8":1,"32":2,"54":1,"62":1,"184":1}}],["first",{"2":{"2":1,"8":1,"32":1,"34":1,"44":1,"46":1,"59":6,"62":2,"65":3,"68":2,"77":1,"105":1,"106":2,"117":4,"121":1,"131":1,"135":2,"137":2,"139":1,"140":1,"144":1,"145":1,"153":2,"155":2,"161":1,"173":1,"174":2,"176":1,"179":1,"184":3,"194":1,"196":1,"214":1,"216":1,"227":1,"236":1,"240":1,"247":1,"248":2,"250":3,"255":1,"256":2,"257":1,"259":1}}],["footnotes",{"0":{"105":1}}],["foldl",{"2":{"54":3}}],["follow",{"2":{"107":1}}],["follows",{"2":{"3":1,"5":1,"62":1}}],["following",{"2":{"2":1,"5":1,"20":1,"28":1,"44":1,"55":1,"56":1,"57":2,"71":2,"72":2,"73":2,"107":1,"126":1,"130":1,"133":1,"139":1,"144":1,"149":1,"154":1,"155":1,"159":1,"161":2,"163":1,"184":1,"242":1,"250":1}}],["focuses",{"2":{"53":1}}],["focalloss",{"2":{"53":2}}],["focal",{"2":{"53":4}}],["four",{"2":{"171":1}}],["fourth",{"2":{"53":1}}],["found",{"2":{"2":1,"74":1,"203":1}}],["forum",{"2":{"144":1}}],["forget",{"2":{"115":1}}],["formulas",{"2":{"254":1}}],["format",{"2":{"171":1}}],["formats",{"2":{"62":4,"68":1}}],["form",{"2":{"59":1,"67":1,"153":1,"167":1,"173":2}}],["forms",{"2":{"53":1}}],["forwarded",{"2":{"48":1}}],["forward",{"2":{"45":1,"54":1,"59":5,"66":1,"67":2,"78":1,"102":2,"117":1,"118":1,"145":1,"147":1,"148":1,"175":1,"176":2,"177":2,"249":1,"250":5}}],["forwarddiff",{"2":{"20":1,"49":1,"55":1,"102":1,"144":3,"145":2,"147":4,"148":2,"152":1,"175":2,"176":4,"177":1}}],["forbidden",{"2":{"24":1}}],["forces",{"2":{"100":1}}],["force",{"2":{"2":6,"32":4,"48":1}}],["for",{"0":{"33":1,"114":1,"146":1,"159":1,"161":1,"170":1,"257":1},"1":{"34":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"0":1,"1":1,"2":3,"3":14,"4":9,"5":3,"6":1,"7":11,"8":7,"11":1,"12":1,"13":1,"14":1,"16":5,"17":8,"18":5,"19":15,"20":3,"21":1,"24":11,"27":2,"28":3,"29":1,"32":3,"34":5,"36":2,"37":8,"38":2,"43":5,"44":3,"45":8,"46":2,"47":2,"48":1,"49":5,"52":5,"53":9,"54":5,"55":15,"56":1,"57":4,"58":2,"59":16,"60":3,"62":11,"63":10,"65":15,"66":24,"67":12,"68":6,"69":18,"70":8,"76":1,"77":2,"78":7,"79":1,"80":5,"81":1,"84":2,"85":2,"87":2,"88":1,"89":1,"90":1,"91":1,"92":1,"94":1,"96":2,"97":1,"98":1,"99":1,"100":9,"101":2,"103":11,"104":2,"105":1,"106":5,"107":4,"109":1,"111":2,"115":1,"116":1,"117":1,"119":1,"120":1,"123":1,"124":2,"125":1,"127":2,"130":1,"133":2,"135":6,"136":3,"137":3,"138":1,"139":1,"142":1,"144":6,"146":1,"147":2,"148":1,"149":1,"150":1,"152":2,"153":4,"154":2,"155":3,"157":1,"159":1,"160":3,"162":2,"164":1,"165":1,"166":5,"167":1,"168":2,"169":1,"170":1,"171":3,"172":1,"173":2,"174":2,"175":5,"176":2,"177":2,"179":3,"183":3,"184":4,"185":2,"186":1,"187":3,"190":1,"194":1,"195":2,"196":1,"203":1,"204":2,"208":3,"213":1,"214":2,"216":1,"217":5,"220":3,"227":1,"228":3,"233":1,"236":2,"238":1,"243":2,"244":1,"245":2,"248":5,"249":2,"250":5,"255":1}}],["tmp",{"2":{"254":4}}],["tmatch",{"2":{"8":3}}],["t∈",{"2":{"243":1}}],["ttest",{"2":{"217":1,"228":2}}],["ttraining",{"2":{"217":1,"228":2}}],["ttime",{"2":{"195":2,"217":3,"228":3}}],["tl",{"2":{"115":1}}],["tloss",{"2":{"107":1}}],["td",{"0":{"114":1}}],["tdchain",{"0":{"112":1},"2":{"111":4,"113":1,"114":1}}],["tpu",{"2":{"73":1,"106":1}}],["tsteps",{"2":{"255":5,"256":4,"259":6}}],["tstate",{"2":{"179":6,"217":7,"236":9}}],["tsit5",{"2":{"206":1,"208":1,"209":1,"220":13}}],["tspan=",{"2":{"214":2,"218":1}}],["tspan",{"2":{"206":5,"209":2,"214":5,"218":3,"220":7,"255":4,"256":1,"259":1}}],["tsung",{"2":{"53":2}}],["ts",{"2":{"52":16,"245":6,"248":1,"249":4}}],["t=rand",{"2":{"49":1}}],["t=float32",{"2":{"24":8}}],["tail",{"2":{"249":1}}],["tab>`",{"2":{"176":1}}],["table",{"2":{"67":1,"171":1}}],["tall",{"2":{"175":1}}],["tasklocalrng",{"2":{"77":1,"129":1,"170":1,"179":1}}],["tasks",{"2":{"53":3}}],["taking",{"2":{"53":1,"54":1,"68":1}}],["takeaway",{"2":{"137":1}}],["takes",{"2":{"8":1,"53":2,"59":1,"62":4,"78":1,"106":1,"208":1,"235":1,"250":1,"256":2}}],["taken",{"2":{"5":1,"24":1,"60":1,"63":2,"67":1,"96":1,"106":1,"148":1}}],["take",{"2":{"1":1,"25":1,"44":1,"52":1,"53":2,"60":1,"63":1,"77":1,"104":1,"110":1,"145":1,"169":1,"241":1,"243":1,"249":1}}],["targets",{"2":{"53":1,"192":1,"213":1}}],["target",{"2":{"48":10,"53":2,"194":3,"216":3,"227":3,"242":6,"243":12,"244":10}}],["tanh",{"2":{"46":2,"66":1,"77":11,"145":1,"146":1,"147":3,"148":3,"152":3,"208":2,"215":4,"220":36,"241":3,"249":3}}],["turing",{"2":{"247":6}}],["tutorials",{"0":{"198":1,"199":1,"200":1,"201":1,"203":1},"1":{"199":1,"200":1,"201":1,"202":1,"203":1},"2":{"59":1,"81":1,"203":1,"238":1,"244":4}}],["tutorial",{"2":{"58":1,"78":1,"170":3,"175":1,"181":2,"190":2,"203":1,"204":6,"230":1,"238":1,"247":2}}],["tu",{"2":{"27":1}}],["tuple=true",{"2":{"217":1}}],["tuples",{"2":{"7":2}}],["tuple",{"2":{"3":2,"7":2,"19":2,"20":1,"24":3,"34":1,"43":4,"44":1,"52":1,"53":1,"55":1,"62":7,"63":11,"65":15,"66":22,"67":5,"69":1,"70":3,"77":12,"99":1,"111":1,"147":1,"148":1,"153":1,"207":2,"215":2,"220":36,"249":1}}],["two",{"2":{"24":3,"55":2,"62":4,"67":1,"69":1,"70":3,"131":1,"135":1,"136":1,"137":1,"149":1,"169":1,"249":1,"254":4,"256":1}}],["temporal",{"2":{"243":1}}],["te",{"2":{"195":4,"196":1,"217":2}}],["technology",{"2":{"74":1}}],["terminate",{"2":{"208":1}}],["terminology",{"2":{"208":1}}],["terms",{"2":{"105":1}}],["term",{"2":{"66":1,"145":1,"152":1,"249":2}}],["terrible",{"2":{"52":1}}],["tested",{"2":{"99":1,"100":1,"104":2,"175":1,"202":1}}],["test",{"0":{"50":1},"2":{"48":7,"49":10,"50":8,"64":3,"69":2,"100":1,"105":1,"106":1,"145":2,"153":1,"192":5,"195":3,"196":20,"213":5,"217":3,"224":5,"228":13,"255":1}}],["tests",{"2":{"48":2,"49":2,"104":1,"124":2,"149":1}}],["testing",{"0":{"48":1},"2":{"47":3,"99":1}}],["testmode`",{"2":{"145":2}}],["testmode",{"2":{"10":1,"64":3,"69":3,"106":3,"142":1,"187":1,"194":1,"216":1,"227":1,"244":1}}],["tends",{"2":{"17":1,"145":1,"146":1}}],["tensors",{"2":{"70":1}}],["tensorflow",{"2":{"66":1}}],["tensor",{"2":{"16":3,"24":1,"69":1,"171":1,"254":1}}],["tile",{"2":{"171":2}}],["tiles",{"2":{"171":1}}],["tightly",{"2":{"167":1}}],["tier",{"2":{"102":8,"104":3}}],["tied",{"0":{"46":1},"2":{"100":1}}],["title",{"2":{"74":2,"245":1,"250":1}}],["tips",{"2":{"154":2}}],["tip",{"2":{"8":1,"18":1,"32":1,"34":1,"48":1,"59":1,"66":1,"119":1,"135":1,"144":2,"203":1,"254":1}}],["timewrapper",{"2":{"207":7,"208":1}}],["timelastindex",{"2":{"54":1,"66":2}}],["timestep",{"2":{"255":1}}],["timespace",{"2":{"255":1}}],["times",{"2":{"47":1,"62":3,"171":1,"174":1,"204":1}}],["time",{"0":{"111":1},"2":{"3":1,"32":1,"66":2,"103":2,"106":1,"110":3,"113":5,"195":3,"196":20,"207":1,"217":3,"219":1,"228":2,"245":1,"255":2,"256":3,"257":1,"259":2}}],["typing",{"2":{"76":1}}],["typical",{"2":{"59":1}}],["typically",{"2":{"7":1,"66":1,"100":1,"147":1,"160":1,"186":1,"243":1,"256":1}}],["typed",{"2":{"176":1}}],["typeof",{"2":{"58":1,"77":8,"106":3,"112":2,"117":1,"135":4,"220":80,"236":2,"256":2}}],["types",{"0":{"7":1,"22":1,"108":1},"1":{"109":1,"110":1,"111":1,"112":1,"113":1,"114":1},"2":{"3":1,"8":2,"20":2,"43":1,"55":6,"89":2,"92":1,"137":1,"138":1,"166":1,"168":2}}],["type",{"0":{"57":1,"155":1,"157":1,"220":1},"2":{"3":13,"4":8,"7":6,"8":5,"16":1,"20":1,"22":2,"24":7,"25":1,"32":1,"34":1,"37":5,"53":1,"55":6,"57":12,"58":3,"59":4,"62":2,"69":1,"106":1,"109":1,"116":5,"117":13,"120":3,"126":1,"127":1,"136":1,"137":3,"138":1,"155":2,"157":2,"166":2,"169":1,"184":2,"187":2,"219":1,"220":1,"224":1}}],["tr",{"2":{"149":2,"152":11,"195":4,"217":2}}],["trying",{"2":{"135":1,"208":1}}],["try",{"2":{"103":1,"106":1,"112":1,"116":2,"117":1,"137":2,"152":1,"171":1,"179":1,"188":1,"203":1}}],["treat",{"2":{"49":1,"136":1}}],["treated",{"2":{"15":1,"59":1}}],["truth",{"2":{"147":1,"179":1}}],["truly",{"2":{"100":1}}],["truncation",{"2":{"94":2}}],["truncated",{"2":{"22":4,"24":2}}],["true",{"0":{"255":1},"2":{"2":2,"3":3,"10":1,"17":6,"19":4,"24":2,"32":2,"34":1,"44":1,"45":2,"46":1,"49":2,"53":40,"54":3,"55":1,"58":1,"59":4,"62":3,"63":2,"64":4,"66":11,"70":1,"77":8,"96":1,"99":1,"106":3,"112":2,"116":1,"117":2,"126":2,"129":2,"131":1,"132":1,"145":3,"146":1,"147":1,"148":1,"150":1,"151":1,"152":1,"163":2,"167":1,"179":2,"186":4,"194":1,"206":2,"208":2,"216":1,"218":1,"220":58,"227":1,"232":1,"236":3,"242":3,"244":2,"247":2,"249":1,"255":1,"256":2}}],["trivial",{"2":{"100":1,"133":1,"173":1,"208":1,"220":26}}],["trilinear",{"2":{"70":6}}],["tries",{"2":{"16":1}}],["trigger",{"2":{"3":4,"4":2,"120":1,"130":1,"131":2}}],["traditional",{"2":{"67":1,"249":1}}],["transitioned",{"2":{"130":1}}],["transposed",{"2":{"63":2}}],["transpose",{"2":{"63":3,"171":1}}],["transformation",{"2":{"69":1}}],["transformations",{"2":{"32":1}}],["transforms",{"2":{"65":3}}],["transform",{"2":{"32":1}}],["transformed",{"2":{"15":1}}],["transferred",{"2":{"3":1,"140":1}}],["transferring",{"2":{"0":1,"5":1,"160":1}}],["transfer",{"0":{"2":1,"160":1},"2":{"2":1,"140":1,"160":2,"188":1}}],["trace",{"0":{"149":1},"1":{"150":1,"151":1,"152":1},"2":{"149":7,"150":3,"151":2,"152":10,"254":4}}],["tracing",{"2":{"55":1,"106":1}}],["tracking",{"0":{"117":1},"2":{"97":1,"118":2}}],["track",{"2":{"45":1,"57":1,"62":2,"69":12,"116":1,"117":5,"155":1}}],["tracker",{"2":{"16":1,"18":1,"20":1,"49":1,"52":1,"55":1,"102":1,"112":1,"244":12,"247":1}}],["tracked",{"2":{"116":1}}],["trackedarray",{"2":{"8":1,"112":2}}],["trackedreals",{"2":{"8":1}}],["train=1024",{"2":{"224":1}}],["trainstate",{"0":{"107":1},"2":{"52":11,"77":3,"78":1,"94":1,"106":2,"107":3,"179":1,"187":1,"195":1,"204":1,"217":1,"228":1,"236":4,"241":1,"244":1}}],["trained2",{"2":{"187":2}}],["trained",{"2":{"34":1,"187":2,"188":8,"208":2,"209":2,"244":5,"245":1,"259":3}}],["train",{"0":{"219":1},"2":{"34":1,"52":6,"59":1,"66":24,"77":8,"78":7,"106":1,"107":10,"179":4,"183":4,"187":12,"190":1,"192":6,"195":15,"196":3,"204":2,"208":3,"213":6,"217":8,"224":10,"228":24,"236":1,"241":1,"243":1,"244":9}}],["trainmode",{"2":{"10":1}}],["trainable",{"2":{"7":1,"43":1,"52":2,"59":9,"63":2,"66":4,"67":3,"69":1,"140":9}}],["training=val",{"2":{"64":3}}],["training",{"0":{"52":1,"119":1,"142":1,"181":1,"187":1,"195":1,"196":1,"204":1,"206":1,"208":1,"217":1,"222":1,"228":1,"236":1,"238":1,"244":1,"252":1,"257":1,"258":1},"1":{"120":1,"121":1,"122":1,"123":1,"124":1,"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"4":3,"10":2,"17":9,"19":8,"24":2,"36":2,"37":4,"45":1,"52":14,"53":2,"54":3,"64":6,"69":9,"74":1,"77":8,"78":3,"94":1,"95":2,"100":2,"106":1,"107":5,"116":1,"117":3,"119":2,"126":2,"128":1,"129":2,"142":2,"179":5,"181":1,"187":2,"190":1,"195":4,"196":20,"204":1,"217":5,"219":1,"228":2,"235":1,"236":5,"241":1,"244":2,"257":2,"258":1}}],["t",{"2":{"2":1,"3":5,"4":6,"8":1,"13":2,"16":1,"18":1,"24":20,"25":1,"32":1,"44":1,"46":1,"47":1,"49":1,"52":1,"54":1,"59":6,"66":1,"68":1,"77":1,"78":1,"87":2,"90":1,"94":1,"100":1,"104":2,"105":1,"109":1,"110":1,"111":4,"113":2,"115":1,"118":1,"123":1,"124":1,"135":4,"137":3,"138":1,"140":1,"142":1,"143":1,"144":2,"145":1,"146":1,"147":10,"149":1,"153":1,"171":2,"173":2,"177":1,"179":1,"183":1,"184":4,"185":2,"188":1,"192":1,"195":3,"202":1,"204":1,"206":6,"207":11,"208":8,"209":9,"213":1,"214":5,"218":3,"225":1,"226":1,"228":2,"236":1,"243":6,"244":12,"254":16,"255":1,"256":1,"257":2}}],["thousands",{"2":{"171":1}}],["though",{"2":{"135":1,"159":1,"175":1}}],["those",{"2":{"7":1,"8":1,"49":1,"70":1,"79":1,"117":1,"124":1,"136":1,"171":1,"214":1,"250":1}}],["thunk",{"2":{"106":1}}],["thumb",{"2":{"70":1}}],["thus",{"2":{"19":1,"62":1,"70":1}}],["threads",{"2":{"180":3,"189":3,"197":3,"210":3,"221":3,"229":3,"237":3,"246":3,"251":3,"260":3}}],["three",{"2":{"68":1,"171":1}}],["through",{"2":{"5":1,"8":1,"62":4,"69":4,"76":1,"81":2,"110":1,"137":1,"140":1,"154":1,"173":1,"184":3,"207":1,"250":1}}],["throws",{"2":{"50":1}}],["throw",{"2":{"2":1,"3":1,"45":1,"52":1,"57":1,"87":1,"244":1}}],["thrown",{"2":{"2":2,"43":1,"45":2}}],["than",{"2":{"24":2,"175":1,"217":1}}],["that",{"2":{"2":3,"3":8,"5":1,"7":4,"8":5,"10":1,"11":1,"13":2,"16":1,"17":1,"18":2,"20":1,"24":3,"34":2,"39":2,"42":1,"43":1,"45":1,"46":2,"47":1,"52":1,"53":1,"54":1,"55":3,"58":1,"59":5,"62":5,"63":2,"64":2,"65":6,"67":1,"68":4,"69":6,"77":1,"79":1,"82":2,"87":2,"90":2,"100":2,"105":3,"106":1,"107":2,"115":2,"116":3,"117":1,"118":2,"120":5,"121":1,"123":1,"135":3,"136":4,"137":1,"138":2,"139":1,"140":2,"143":1,"144":3,"145":2,"146":3,"147":2,"149":2,"152":3,"153":2,"155":3,"156":1,"161":1,"163":1,"167":1,"171":1,"173":3,"174":1,"176":1,"177":1,"184":3,"188":2,"202":1,"204":2,"208":4,"217":1,"220":1,"233":1,"235":1,"240":1,"241":1,"243":1,"249":1,"250":3,"256":3}}],["third",{"2":{"171":2}}],["thirteenth",{"2":{"24":2}}],["thing",{"2":{"68":2,"153":1,"155":1}}],["things",{"2":{"59":1,"135":1,"137":1,"171":1}}],["think",{"2":{"59":1,"135":1}}],["this",{"2":{"2":3,"3":8,"4":10,"5":1,"7":5,"8":6,"10":1,"11":6,"13":3,"14":1,"16":3,"18":3,"19":2,"20":4,"21":1,"24":5,"27":2,"28":2,"32":6,"34":3,"36":2,"37":3,"40":1,"41":1,"42":2,"44":2,"45":5,"47":2,"48":1,"49":2,"52":6,"53":4,"54":1,"55":12,"57":7,"58":6,"59":11,"62":3,"63":5,"65":3,"66":6,"67":5,"68":1,"69":5,"70":3,"74":2,"79":1,"82":2,"87":6,"88":1,"89":1,"90":3,"95":1,"96":1,"97":1,"99":1,"100":2,"101":1,"103":2,"105":1,"106":5,"107":3,"115":2,"116":3,"118":2,"120":3,"122":2,"124":2,"125":2,"127":1,"130":1,"131":2,"133":1,"135":8,"137":6,"138":1,"140":2,"142":3,"144":6,"145":9,"146":5,"147":1,"148":1,"149":3,"150":1,"152":2,"153":5,"155":7,"156":1,"157":1,"159":1,"160":4,"162":1,"163":2,"164":3,"166":1,"167":1,"170":1,"171":5,"173":4,"175":1,"179":1,"180":1,"181":2,"183":1,"185":2,"189":1,"190":1,"193":1,"196":1,"197":1,"204":3,"208":5,"210":1,"217":1,"218":1,"220":2,"221":1,"225":1,"229":1,"230":1,"233":1,"235":1,"237":1,"238":2,"246":1,"247":1,"249":4,"251":1,"252":1,"254":3,"256":2,"258":1,"260":1}}],["theoretical",{"2":{"254":1}}],["theoretically",{"2":{"175":1}}],["theorem",{"2":{"68":1}}],["thesis",{"2":{"74":1}}],["these",{"2":{"7":5,"8":1,"25":1,"32":3,"34":3,"35":1,"37":1,"42":1,"52":1,"69":1,"77":1,"87":2,"94":3,"98":2,"100":1,"104":7,"110":1,"124":1,"135":2,"136":2,"144":1,"148":1,"159":1,"163":1,"166":1,"167":3,"170":1,"173":1,"177":1,"202":3,"203":1,"211":1,"249":1}}],["they",{"2":{"52":1,"59":2,"117":1,"126":1,"140":1,"171":1,"175":2}}],["them",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"42":1,"44":1,"53":1,"59":1,"62":1,"67":1,"70":1,"77":1,"87":1,"94":1,"98":1,"99":1,"104":1,"153":1,"155":1,"166":1,"171":2,"174":1,"177":1,"203":2,"241":1,"249":1,"256":2}}],["there",{"2":{"34":1,"43":1,"52":1,"77":1,"89":1,"103":1,"105":1,"117":1,"118":1,"120":1,"144":1,"147":2,"155":1,"159":1,"169":1,"171":1,"196":1}}],["their",{"2":{"7":2,"63":1,"81":1,"158":1,"171":1,"173":2}}],["then",{"2":{"1":2,"2":6,"3":1,"8":1,"17":2,"19":1,"28":2,"43":3,"45":2,"48":1,"49":2,"50":1,"53":6,"54":1,"55":2,"58":2,"60":1,"62":5,"63":4,"64":4,"66":9,"67":7,"68":1,"74":1,"76":1,"100":1,"106":1,"131":3,"135":1,"136":3,"149":2,"161":1}}],["the",{"0":{"107":1,"112":1,"113":1,"114":1,"128":1,"148":2,"150":1,"151":1,"152":1,"170":1,"185":1,"187":1,"188":1,"193":1,"195":1,"196":1,"207":1,"208":1,"209":1,"214":1,"215":1,"219":1,"226":1,"241":1,"242":1,"243":1,"245":1,"249":1,"255":1,"257":1,"258":1,"259":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"1":6,"2":12,"3":31,"4":14,"5":13,"6":3,"7":19,"8":22,"9":4,"10":7,"11":6,"13":10,"14":2,"15":3,"16":11,"17":16,"18":6,"19":21,"20":5,"24":60,"25":25,"27":20,"28":13,"29":1,"32":12,"34":9,"35":1,"37":10,"38":4,"39":11,"40":5,"41":3,"42":1,"43":14,"44":22,"45":15,"46":14,"48":6,"49":9,"50":4,"52":35,"53":55,"54":4,"55":13,"56":6,"57":19,"58":18,"59":49,"60":6,"62":46,"63":38,"64":6,"65":45,"66":69,"67":19,"68":33,"69":58,"70":14,"71":7,"72":2,"73":2,"76":3,"77":6,"78":8,"79":1,"81":4,"82":2,"84":2,"87":2,"88":2,"89":2,"90":4,"91":1,"94":11,"96":9,"97":4,"98":1,"99":5,"100":6,"101":3,"103":7,"104":2,"105":1,"106":23,"107":7,"109":2,"110":2,"111":1,"113":3,"114":2,"115":3,"116":6,"117":19,"119":1,"120":22,"121":3,"122":1,"123":6,"126":1,"127":7,"128":3,"130":3,"131":5,"133":6,"135":20,"136":10,"137":4,"138":5,"139":4,"140":6,"142":4,"143":1,"144":9,"145":5,"146":16,"147":5,"148":13,"149":11,"150":2,"152":7,"153":10,"154":1,"155":9,"157":1,"158":3,"159":2,"160":8,"161":4,"162":1,"163":1,"164":1,"165":1,"166":10,"167":4,"169":11,"170":2,"171":12,"172":7,"173":2,"174":5,"175":6,"176":1,"177":4,"178":2,"179":6,"181":2,"183":6,"184":24,"185":3,"186":2,"187":6,"188":5,"190":2,"192":2,"193":3,"195":2,"196":5,"202":1,"203":6,"204":5,"207":8,"208":11,"213":2,"214":13,"215":1,"217":3,"218":2,"219":3,"220":5,"225":6,"228":1,"232":4,"235":3,"236":2,"238":2,"240":3,"241":2,"242":4,"243":3,"247":5,"248":3,"249":16,"250":22,"252":1,"254":7,"255":2,"256":10,"257":4,"258":2,"259":2}}],["top",{"2":{"106":1,"133":1}}],["topic",{"2":{"66":1}}],["toolchain",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["tools",{"2":{"45":2,"59":1,"115":1,"170":1,"175":1}}],["too",{"2":{"98":1,"175":1,"250":1}}],["towards",{"2":{"55":1,"173":1,"204":1}}],["together",{"2":{"44":1}}],["toarray",{"2":{"34":1}}],["tosimplechainsadaptor",{"2":{"34":4,"100":1,"193":2}}],["total",{"2":{"9":1,"10":1,"38":2,"46":1,"59":4,"62":4,"69":3,"77":1,"78":1,"111":1,"116":1,"117":1,"193":2,"194":6,"216":6,"227":6,"233":1,"244":3,"256":1}}],["to",{"0":{"32":1,"34":1,"50":1,"71":1,"81":1,"82":1,"120":1,"139":1,"154":1,"252":1},"1":{"83":1,"84":1,"85":1,"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":1,"95":1,"96":1,"97":1,"140":1,"141":1,"142":1,"143":1,"155":1,"156":1,"157":1,"158":1,"159":1,"160":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"1":3,"2":7,"3":10,"4":5,"5":5,"6":2,"7":4,"8":10,"10":2,"11":2,"13":7,"14":2,"15":4,"16":5,"17":13,"18":3,"19":16,"20":2,"24":14,"27":8,"28":5,"32":14,"34":17,"37":9,"39":6,"41":1,"42":1,"43":7,"44":2,"45":10,"46":5,"47":2,"48":3,"49":10,"52":8,"53":14,"54":3,"55":9,"56":4,"57":4,"58":4,"59":21,"60":1,"62":18,"63":19,"64":13,"65":9,"66":59,"67":7,"68":9,"69":22,"70":8,"71":2,"72":1,"73":1,"74":1,"76":3,"77":2,"78":2,"81":2,"82":5,"84":1,"87":5,"88":1,"89":1,"90":5,"91":2,"94":6,"95":3,"96":6,"97":3,"98":2,"99":1,"100":9,"104":3,"105":1,"106":14,"107":8,"109":2,"110":2,"111":1,"115":4,"116":1,"117":7,"119":1,"120":5,"121":3,"123":4,"125":1,"126":1,"127":1,"128":1,"130":3,"131":2,"133":5,"135":7,"136":5,"137":4,"138":2,"139":3,"140":8,"142":1,"143":3,"144":7,"145":10,"146":6,"147":1,"148":4,"149":6,"152":1,"153":9,"155":7,"156":1,"157":2,"158":1,"160":3,"161":4,"162":2,"163":3,"164":4,"166":4,"167":3,"168":1,"169":3,"170":4,"171":7,"172":1,"173":3,"175":1,"176":1,"177":3,"179":2,"181":2,"183":3,"184":10,"186":1,"188":1,"190":2,"192":1,"193":3,"202":1,"203":6,"204":4,"207":6,"208":7,"211":2,"213":1,"214":6,"215":1,"217":1,"218":1,"220":11,"225":1,"226":1,"235":1,"238":2,"242":2,"243":2,"247":1,"248":3,"249":12,"250":5,"254":5,"256":6,"257":2,"258":2}}],["toml",{"2":{"1":1,"47":1}}],["l3",{"2":{"259":2}}],["lb",{"2":{"256":1,"259":1}}],["lbfgs",{"2":{"204":1,"208":3}}],["lr",{"2":{"244":2}}],["ld",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["llvm",{"2":{"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["ll",{"2":{"170":1,"248":1,"249":1,"256":2}}],["lpnorm",{"2":{"97":1}}],["lppool",{"2":{"65":1,"97":1}}],["lp",{"2":{"65":4,"249":1,"250":1}}],["l=",{"2":{"53":1}}],["ls",{"2":{"53":4}}],["lstmcell",{"2":{"66":6,"96":1,"97":1,"184":3,"185":1}}],["lstm",{"0":{"181":1},"1":{"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1},"2":{"7":1,"66":1,"184":17,"185":4}}],["l2",{"2":{"46":5,"136":4,"256":2,"259":2}}],["l2=dense",{"2":{"46":1}}],["l1",{"2":{"46":5,"136":4,"256":2,"259":2}}],["l1=dense",{"2":{"46":1}}],["lecture",{"2":{"211":1}}],["lecun",{"2":{"53":1}}],["leveraging",{"2":{"153":1}}],["levels=10",{"2":{"245":1}}],["level",{"2":{"24":2,"101":1,"107":1,"153":1,"166":2}}],["less",{"2":{"68":1,"98":1,"104":1,"175":1}}],["lets",{"2":{"137":1,"195":1,"217":1,"228":1,"247":1}}],["let",{"2":{"59":1,"106":2,"107":1,"115":1,"116":2,"117":7,"121":1,"126":1,"135":1,"136":2,"137":2,"140":2,"144":3,"145":2,"147":1,"148":1,"152":3,"153":1,"155":1,"170":2,"171":1,"174":1,"176":1,"177":1,"178":1,"179":4,"184":1,"186":1,"188":1,"208":2,"217":1,"219":1,"232":1,"233":1,"236":1,"249":1,"255":1,"256":1,"257":1,"259":2}}],["len",{"2":{"49":2,"183":1,"243":11}}],["length=25",{"2":{"250":4}}],["length=bc",{"2":{"243":3}}],["length=grid",{"2":{"243":1}}],["length=datasize",{"2":{"206":1,"255":1}}],["length=50",{"2":{"183":1}}],["length",{"2":{"24":7,"25":24,"46":2,"62":2,"63":2,"65":6,"135":8,"136":2,"183":5,"186":1,"194":1,"207":2,"216":1,"227":1,"245":8,"249":3,"254":2,"259":1}}],["leibler",{"2":{"53":1}}],["lei",{"2":{"19":1}}],["lempitsky",{"2":{"19":1,"69":1}}],["leakyrelu",{"2":{"96":1}}],["learn",{"2":{"171":1}}],["learned",{"2":{"100":1}}],["learnable",{"2":{"67":1,"69":4}}],["learning",{"2":{"2":1,"17":1,"19":1,"21":1,"24":3,"53":1,"149":1,"155":1,"158":1,"168":1,"169":1,"171":1,"179":1,"204":1,"254":1}}],["leads",{"2":{"137":1}}],["leading",{"2":{"70":1}}],["lead",{"2":{"13":1,"32":1,"45":1,"55":1,"59":2,"130":1,"153":1,"208":1}}],["leaf",{"2":{"3":1,"8":1,"55":2,"77":12}}],["leaves",{"2":{"3":1,"55":4}}],["l",{"2":{"8":5,"9":2,"10":2,"32":2,"34":1,"43":13,"44":5,"62":2,"64":1,"111":5,"126":2,"127":2,"135":16,"140":4,"179":1,"184":2,"208":4,"214":4,"232":2,"236":2,"255":2,"257":3}}],["lag",{"2":{"244":1}}],["latter",{"2":{"158":2}}],["later",{"2":{"248":1}}],["latest",{"2":{"71":1,"203":1}}],["latentsize",{"2":{"66":3}}],["lazy",{"2":{"140":1}}],["language",{"2":{"55":2}}],["label=l",{"2":{"206":2,"209":4}}],["labels",{"2":{"53":2,"183":3,"192":2,"213":2,"224":4}}],["label",{"2":{"53":18}}],["larger",{"0":{"202":1},"2":{"100":1,"171":1,"202":1}}],["large",{"2":{"32":1,"100":3,"175":1}}],["layerfreezing",{"2":{"128":1}}],["layer2",{"2":{"62":1}}],["layer1",{"2":{"62":1}}],["layernorm",{"2":{"19":1,"69":4,"84":1}}],["layer",{"0":{"11":1,"44":1,"58":1,"59":1,"127":1,"134":1,"135":1,"136":1,"214":1,"215":1,"218":1,"225":1},"1":{"135":1,"136":1},"2":{"7":13,"8":9,"9":4,"10":4,"11":4,"19":3,"32":10,"34":11,"43":7,"44":6,"45":13,"57":5,"58":7,"59":11,"62":67,"63":5,"64":3,"65":12,"66":2,"67":5,"68":8,"69":24,"70":3,"77":47,"94":2,"95":1,"96":1,"100":2,"106":24,"109":2,"111":3,"113":5,"115":2,"116":30,"117":69,"126":9,"127":9,"128":5,"129":9,"135":8,"136":12,"140":5,"144":1,"145":4,"146":1,"147":4,"148":6,"153":2,"155":1,"166":1,"179":1,"184":1,"193":18,"214":2,"220":211,"225":1,"226":1,"233":2,"235":4,"236":4,"249":1,"256":12,"258":4}}],["layers",{"0":{"16":1,"18":1,"61":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"78":1,"126":1,"140":1,"143":1},"1":{"62":1,"63":1,"64":1,"65":1,"66":1,"67":1,"68":1,"69":1,"70":1},"2":{"6":1,"7":9,"8":2,"24":8,"32":2,"44":3,"45":2,"57":1,"58":2,"62":39,"63":1,"68":12,"78":1,"85":1,"94":2,"97":1,"99":4,"100":4,"111":4,"116":9,"117":19,"126":1,"133":2,"135":1,"136":3,"139":1,"140":3,"144":1,"155":1,"159":1,"165":1,"166":1,"184":1,"185":1,"214":1,"219":1,"220":6,"225":1,"249":2}}],["last",{"2":{"2":1,"15":1,"24":1,"58":1,"59":1,"66":1,"68":2,"70":2,"111":1,"120":1,"146":1,"214":1,"248":2}}],["luxlinear",{"2":{"140":7}}],["luxlib",{"0":{"12":1,"83":1},"1":{"13":1,"14":1,"15":1,"16":1,"17":1,"18":1,"19":1,"20":1,"84":1,"85":1},"2":{"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"20":1,"48":2,"60":2,"84":1,"145":4,"158":8,"161":1,"166":2,"167":2,"256":2}}],["luxflux",{"2":{"139":1,"140":2}}],["luxdeviceutils",{"2":{"90":3}}],["luxdl",{"2":{"66":1,"71":1,"79":1}}],["luxops",{"0":{"54":1},"2":{"54":8,"184":1,"185":1}}],["luxtestutils",{"0":{"47":1},"1":{"48":1,"49":1,"50":1},"2":{"48":3,"49":2,"50":1}}],["luxcorearrayinterfacereversediffext",{"2":{"112":2}}],["luxcorearrayinterfacetrackerext",{"2":{"112":2}}],["luxcore",{"0":{"6":1,"86":1},"1":{"7":1,"8":1,"9":1,"10":1,"11":1,"87":1,"88":1},"2":{"6":2,"7":3,"8":7,"9":2,"10":5,"11":1,"60":2,"87":2,"112":5,"133":3,"135":14,"136":8,"137":1,"161":1,"166":3}}],["luxcudadevice",{"2":{"91":1}}],["luxcuda",{"2":{"3":1,"72":2,"77":1,"78":1,"80":1,"130":1,"153":1,"168":1,"172":3,"182":1,"205":1,"212":1,"223":1,"231":1,"239":1}}],["lux",{"0":{"30":1,"32":1,"33":1,"34":1,"71":1,"82":1,"93":1,"98":1,"100":1,"106":1,"115":1,"133":1,"139":1,"161":1,"170":1,"204":1},"1":{"31":1,"32":1,"33":1,"34":2,"83":1,"84":1,"85":1,"86":1,"87":1,"88":1,"89":1,"90":1,"91":1,"92":1,"93":1,"94":2,"95":2,"96":2,"97":2,"99":1,"100":1,"107":1,"116":1,"117":1,"118":1,"134":1,"135":1,"136":1,"137":1,"138":1,"140":1,"141":1,"142":1,"143":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"4":4,"6":4,"7":3,"8":4,"11":1,"12":1,"27":2,"28":1,"32":11,"34":15,"35":1,"36":2,"37":3,"38":2,"39":4,"40":1,"41":1,"43":13,"44":4,"45":8,"46":3,"47":1,"48":2,"52":9,"53":16,"54":9,"55":9,"56":4,"57":2,"58":5,"59":19,"60":1,"62":11,"63":3,"64":7,"65":9,"66":9,"67":4,"68":12,"69":8,"70":2,"71":5,"72":4,"73":3,"74":1,"76":4,"77":8,"78":4,"79":1,"80":1,"81":1,"82":2,"87":3,"90":1,"91":2,"94":6,"95":8,"98":1,"100":9,"101":2,"104":1,"105":2,"106":7,"107":1,"109":1,"111":3,"112":1,"113":4,"114":1,"115":3,"116":6,"117":4,"118":1,"119":1,"123":1,"126":6,"127":2,"128":2,"129":3,"130":3,"131":1,"133":7,"135":3,"136":2,"137":1,"139":3,"140":13,"142":2,"143":2,"144":4,"145":1,"146":1,"147":1,"148":1,"152":1,"153":8,"155":6,"157":2,"161":3,"166":3,"169":1,"170":3,"172":1,"173":1,"174":2,"175":1,"177":1,"179":2,"181":2,"182":1,"184":7,"185":1,"187":2,"190":1,"191":1,"193":4,"194":1,"195":1,"196":2,"202":1,"203":3,"204":2,"205":1,"208":1,"212":1,"214":5,"215":1,"216":1,"218":4,"220":96,"223":1,"225":4,"226":2,"227":1,"228":1,"231":1,"235":2,"236":4,"238":2,"239":1,"240":1,"241":1,"244":6,"247":4,"249":7,"253":1,"256":2}}],["lt",{"2":{"4":2,"5":1,"24":3,"53":1,"59":2,"79":1,"87":2,"131":1,"136":2,"144":12,"206":1,"209":1}}],["literate",{"2":{"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["literature",{"2":{"149":1}}],["little",{"2":{"106":1,"117":1}}],["lib64",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["libllvm",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["libopenlibm",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["libm",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["libraries",{"2":{"144":1,"169":1,"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1,"247":2,"249":1}}],["library",{"2":{"74":1,"139":1,"173":1,"180":1,"188":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["lib",{"2":{"116":1,"180":1,"189":1,"197":1,"210":1,"220":3,"221":1,"229":1,"237":1,"244":4,"246":1}}],["linux",{"2":{"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["linked",{"2":{"203":1}}],["link",{"2":{"170":1}}],["linking",{"2":{"62":1}}],["lin",{"2":{"53":2}}],["liner",{"2":{"249":1}}],["linewidth=2",{"2":{"245":1,"255":1,"256":2,"259":3}}],["linewidth=3",{"2":{"232":1,"236":1,"250":3}}],["linewidth=4",{"2":{"206":2,"209":4,"259":1}}],["linesearch=linesearches",{"2":{"258":1}}],["linesearches",{"2":{"253":1}}],["linestyle=",{"2":{"206":2,"209":2}}],["lines",{"2":{"206":2,"209":4,"232":1,"236":1,"255":1,"256":2,"259":4}}],["line",{"2":{"49":1}}],["linearalgebra",{"2":{"144":1,"247":1}}],["linear",{"0":{"67":1,"179":1},"2":{"24":1,"59":4,"68":1,"70":1,"135":12,"136":23,"153":1,"179":1}}],["lighter",{"2":{"133":1}}],["light",{"2":{"21":1}}],["lightweight",{"2":{"0":1}}],["like",{"2":{"15":1,"16":2,"18":1,"20":1,"24":1,"28":1,"47":1,"52":1,"53":3,"55":1,"59":3,"63":2,"66":3,"88":1,"100":2,"133":1,"135":2,"137":1,"139":1,"140":2,"160":1,"169":1,"171":3,"177":1,"204":1,"214":1,"217":1,"248":1,"255":1,"256":1}}],["limits=extrema",{"2":{"245":1}}],["limiter",{"2":{"220":26}}],["limitations",{"2":{"32":1,"52":1,"100":1}}],["limit",{"2":{"13":1,"17":1,"54":1,"180":2,"189":2,"197":1,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"260":1}}],["lists",{"2":{"46":1,"101":1}}],["listed",{"2":{"42":1,"57":1}}],["list",{"2":{"3":1,"37":1,"46":5,"48":1,"49":3,"62":4,"68":1,"78":1,"82":2,"161":1,"168":1,"171":1,"203":1,"228":4}}],["lost",{"2":{"170":1}}],["lossfn",{"2":{"179":3,"186":2,"187":2}}],["lossfunctions",{"2":{"53":1}}],["losses",{"2":{"53":1,"257":2,"259":3}}],["loss",{"0":{"53":1,"145":1,"147":1,"148":1,"186":1,"235":1,"242":1},"1":{"146":1},"2":{"52":5,"53":35,"59":5,"77":2,"78":14,"106":5,"107":4,"144":1,"145":10,"146":9,"147":4,"148":6,"179":19,"186":4,"187":456,"194":1,"195":2,"208":30,"227":1,"228":1,"235":4,"236":10,"238":1,"240":1,"242":23,"244":436,"257":6,"258":1,"259":2}}],["lower",{"2":{"217":1}}],["low",{"2":{"107":1}}],["love",{"2":{"104":1}}],["lotka",{"2":{"206":2}}],["lot",{"2":{"100":1,"174":1}}],["logs",{"2":{"120":1}}],["logging",{"2":{"120":1,"247":1}}],["logarithm",{"2":{"53":1}}],["logarithmic",{"2":{"53":1}}],["log⁡",{"2":{"53":1}}],["log",{"2":{"53":2,"54":2,"249":1,"250":1}}],["logitcrossentropy",{"2":{"216":1,"217":1}}],["logitbinarycrossentropy",{"2":{"186":1}}],["logitbce",{"2":{"53":5}}],["logit",{"2":{"53":1}}],["logits=val",{"2":{"53":2,"194":1,"216":1,"227":1}}],["logits",{"2":{"53":4}}],["location",{"2":{"45":3,"116":4,"117":10}}],["local",{"2":{"4":1,"38":2,"59":1,"120":1,"121":1,"180":2,"189":2,"196":1,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["localpreferences",{"2":{"1":1,"131":2}}],["loosely",{"2":{"170":1}}],["looks",{"2":{"160":1}}],["lookup",{"2":{"67":1}}],["look",{"2":{"43":1,"70":1,"177":1,"184":1,"211":1}}],["loops",{"2":{"20":1,"55":1}}],["loopedarrayop",{"2":{"20":1}}],["loopvectorization",{"2":{"13":1,"14":2,"18":1,"20":1,"159":1,"167":1}}],["loop",{"0":{"167":1,"195":1},"2":{"13":1,"20":1,"159":1,"167":1,"236":1}}],["looping",{"2":{"5":1}}],["longer",{"2":{"94":2,"122":3}}],["long",{"2":{"32":1,"66":1,"137":1}}],["lo",{"2":{"24":2}}],["loadmnist",{"2":{"192":2,"195":1,"213":2,"217":1}}],["loader",{"2":{"107":1,"187":4}}],["loaded",{"2":{"2":2,"3":7,"4":2,"10":1,"11":1,"18":1,"41":1,"87":2,"120":1,"131":2,"143":1,"167":1,"172":1}}],["load",{"2":{"13":1,"14":1,"18":1,"41":1,"47":1,"94":1,"107":1,"130":1,"159":1,"188":1,"192":1,"213":1,"224":4,"228":1}}],["loading",{"0":{"160":1,"192":1,"213":1,"224":1},"2":{"7":1,"14":1,"18":1,"32":1,"34":1,"80":1,"160":1,"188":1}}],["=ϕwhere",{"2":{"255":1,"256":1}}],["=χu",{"2":{"255":1,"256":1}}],["=∫θp",{"2":{"250":1}}],["=wx+b",{"2":{"179":1}}],["=x",{"2":{"176":1}}],["=12xtx",{"2":{"176":1}}],["=1v∑i=1vvitjvinote",{"2":{"149":1}}],["=1v∑i=1vvitaviwe",{"2":{"149":1}}],["=e",{"2":{"149":1}}],["=i",{"2":{"149":1}}],["=vcat",{"2":{"66":1}}],["=val",{"2":{"34":1,"53":1}}],["=true",{"2":{"45":1}}],["=static",{"2":{"45":1}}],["=>",{"2":{"32":2,"34":5,"43":2,"44":3,"46":8,"59":8,"62":6,"63":2,"66":4,"67":4,"69":12,"77":6,"78":8,"106":6,"107":3,"111":4,"116":17,"117":14,"139":4,"145":2,"146":2,"147":4,"148":4,"152":4,"155":1,"179":1,"184":2,"185":2,"193":15,"215":5,"226":1,"233":4,"236":2,"241":4,"249":3,"256":6}}],["=0",{"2":{"24":2}}],["=nothing",{"2":{"2":1,"53":2,"66":1,"69":1}}],["==3",{"2":{"254":1}}],["===",{"2":{"46":4}}],["==",{"2":{"1":1,"3":1,"17":1,"24":4,"28":1,"54":2,"58":1,"59":1,"63":10,"65":15,"67":3,"70":2,"78":2,"107":2,"111":1,"120":1,"127":1,"128":2,"137":2,"138":1,"179":2,"186":1,"194":1,"208":1,"216":1,"227":1,"228":2,"236":2,"244":2,"249":1,"254":5}}],["=",{"2":{"1":2,"2":2,"3":1,"5":8,"15":1,"17":3,"19":6,"24":14,"28":1,"32":4,"34":5,"36":3,"43":2,"44":5,"46":8,"48":3,"49":2,"53":59,"55":1,"58":2,"59":46,"62":40,"63":7,"64":6,"65":9,"66":6,"67":14,"68":25,"69":15,"70":3,"72":4,"73":3,"74":13,"77":52,"78":26,"87":2,"106":61,"107":10,"111":10,"112":3,"113":3,"114":2,"116":25,"117":47,"120":7,"121":1,"126":29,"127":1,"129":23,"131":4,"132":7,"135":8,"136":9,"137":5,"139":7,"140":12,"145":21,"146":12,"147":20,"148":21,"150":2,"151":2,"152":12,"153":19,"155":5,"160":6,"168":9,"169":6,"170":1,"171":8,"172":1,"173":3,"174":3,"176":3,"177":4,"178":1,"179":17,"180":8,"183":6,"184":6,"185":5,"186":5,"187":13,"189":8,"192":7,"193":21,"194":5,"195":12,"196":1,"197":8,"205":2,"206":12,"207":3,"208":19,"209":6,"210":8,"213":7,"214":6,"215":5,"216":5,"217":10,"218":3,"220":136,"221":8,"224":4,"225":2,"226":3,"227":5,"228":21,"229":8,"232":8,"233":3,"234":1,"235":12,"236":13,"237":8,"239":2,"242":13,"243":19,"244":20,"245":11,"246":8,"248":21,"249":19,"250":19,"251":7,"254":55,"255":20,"256":38,"257":4,"258":14,"259":13,"260":7}}],["jointly",{"2":{"184":1}}],["journal",{"2":{"17":1}}],["jvi",{"2":{"149":1}}],["jvp",{"0":{"27":1},"2":{"27":1,"151":6,"152":10,"175":4,"177":4}}],["j∈rd×d",{"2":{"149":1}}],["jet`",{"2":{"48":1}}],["jet",{"0":{"48":1},"2":{"48":14}}],["jax",{"2":{"170":1}}],["jacobian`",{"2":{"145":1}}],["jacobian",{"0":{"145":1,"146":1,"148":1,"150":1,"151":1,"152":1,"177":1,"178":1},"1":{"146":1},"2":{"27":11,"28":6,"144":3,"145":3,"146":9,"148":5,"149":4,"150":1,"151":1,"152":17,"175":6,"177":2,"178":1}}],["jamie",{"2":{"19":1}}],["j",{"2":{"24":1,"28":2,"145":2,"146":3,"148":2,"149":1,"152":8}}],["jimmy",{"2":{"19":1}}],["just",{"2":{"3":1,"59":1,"62":1,"64":3,"69":3,"77":1,"123":1,"131":1,"133":2,"169":1,"171":1,"177":1,"217":1}}],["julia∂t",{"2":{"147":1}}],["julia∂x",{"2":{"145":1,"146":1,"148":1}}],["juliazygote",{"2":{"117":1}}],["juliazeros",{"2":{"171":2}}],["juliazerosc64",{"2":{"25":1}}],["juliazerosc32",{"2":{"25":1}}],["juliazerosc16",{"2":{"25":1}}],["juliazeros64",{"2":{"25":1}}],["juliazeros32",{"2":{"25":1}}],["juliazeros16",{"2":{"25":1}}],["juliaxt",{"2":{"113":1}}],["juliax",{"2":{"106":1,"131":1,"155":2,"171":11,"173":2,"179":1}}],["juliaxlogy",{"2":{"54":1}}],["juliaxlogx",{"2":{"54":1}}],["juliaxla",{"2":{"2":1}}],["juliaw",{"2":{"171":1,"179":1}}],["juliaweights",{"2":{"168":1,"169":3}}],["juliaweightnorm",{"2":{"69":1}}],["juliawrappedfunction",{"2":{"68":1}}],["juliann",{"2":{"153":1}}],["julian",{"2":{"78":1,"179":1}}],["julianooplayer",{"2":{"68":1}}],["juliancclbackend",{"2":{"36":1}}],["juliaembedding",{"2":{"67":1}}],["juliaeachslice",{"2":{"54":1}}],["juliavjp",{"2":{"178":1,"236":1}}],["juliavariationalhiddendropout",{"2":{"64":1}}],["juliavector",{"2":{"27":1}}],["juliay",{"2":{"62":2}}],["juliapkg>",{"2":{"161":1}}],["juliaprob",{"2":{"255":1,"256":1,"259":1}}],["juliaprintln",{"2":{"152":1,"176":1}}],["juliapred",{"2":{"106":2}}],["juliaps",{"2":{"117":1,"120":1,"179":1,"187":1}}],["juliapixelshuffle",{"2":{"70":1}}],["juliaparallel",{"2":{"62":1}}],["juliaparameterlength",{"2":{"9":1}}],["juliapairwisefusion",{"2":{"62":1}}],["juliapoissonloss",{"2":{"53":1}}],["juliamodel",{"2":{"106":2,"107":1,"114":1,"116":2,"117":4,"136":1,"146":1,"152":1,"179":1,"220":2,"233":1}}],["juliameanpool",{"2":{"65":1}}],["juliamaxout",{"2":{"68":1}}],["juliamaxpool",{"2":{"65":1}}],["juliamatch",{"2":{"57":1}}],["juliamaeloss",{"2":{"53":1}}],["juliamultigate",{"2":{"54":1}}],["juliamsleloss",{"2":{"53":1}}],["juliamseloss",{"2":{"53":1}}],["juliampibackend",{"2":{"36":1}}],["juliakldivergenceloss",{"2":{"53":1}}],["juliakaiming",{"2":{"24":2}}],["juliahuberloss",{"2":{"53":1}}],["juliahingeloss",{"2":{"53":1}}],["juliaupsample",{"2":{"70":1}}],["juliaupdate",{"2":{"10":1}}],["juliausing",{"2":{"48":1,"72":8,"73":4,"77":1,"78":1,"106":1,"111":1,"116":1,"117":1,"126":1,"129":1,"135":1,"137":1,"139":2,"140":2,"144":1,"153":1,"155":1,"156":1,"161":1,"168":2,"170":1,"172":1,"175":1,"179":1,"180":1,"182":1,"189":1,"191":1,"197":1,"205":1,"210":1,"212":1,"221":1,"223":1,"229":1,"231":1,"237":1,"239":1,"246":1,"251":1,"253":1,"260":1}}],["juliaunfreeze",{"2":{"43":2}}],["juliajvp",{"2":{"177":1}}],["juliajet",{"2":{"48":1}}],["juliajacobian",{"2":{"27":1}}],["juliajulia>",{"2":{"5":1,"24":2,"32":1,"34":1,"43":1,"44":1,"46":1,"48":1,"49":1,"53":15,"59":6,"62":3,"68":4,"69":3,"71":2}}],["juliarng",{"2":{"78":1,"112":1,"116":1,"135":1,"140":2,"170":1,"174":1,"232":1}}],["juliarnncell",{"2":{"66":1}}],["juliarandom",{"2":{"174":1}}],["juliarandc64",{"2":{"25":1}}],["juliarandc32",{"2":{"25":1}}],["juliarandc16",{"2":{"25":1}}],["juliarand64",{"2":{"25":1}}],["juliarandnc64",{"2":{"25":1}}],["juliarandnc32",{"2":{"25":1}}],["juliarandnc16",{"2":{"25":1}}],["juliarandn64",{"2":{"25":1}}],["juliarandn32",{"2":{"25":1}}],["juliarandn16",{"2":{"25":1}}],["juliarand32",{"2":{"25":1}}],["juliarand16",{"2":{"25":1}}],["juliareversesequence",{"2":{"68":1}}],["juliarecurrence",{"2":{"66":1}}],["juliarecursive",{"2":{"55":6}}],["juliareshapelayer",{"2":{"68":1}}],["juliares",{"2":{"62":2}}],["juliareset",{"2":{"3":1}}],["juliarepeatedlayer",{"2":{"62":1}}],["juliareplicate",{"2":{"8":1}}],["juliareduce",{"2":{"39":1}}],["juliaopt",{"2":{"120":1,"234":1}}],["juliaonesc64",{"2":{"25":1}}],["juliaonesc32",{"2":{"25":1}}],["juliaonesc16",{"2":{"25":1}}],["juliaones64",{"2":{"25":1}}],["juliaones32",{"2":{"25":1}}],["juliaones16",{"2":{"25":1}}],["juliaorthogonal",{"2":{"24":1}}],["juliaoutputsize",{"2":{"11":1}}],["julialux",{"2":{"193":1}}],["julialuxops",{"2":{"54":1}}],["julialength",{"2":{"171":1}}],["julialang",{"2":{"116":1,"180":1,"189":1,"197":1,"210":1,"220":3,"221":1,"229":1,"237":1,"244":4,"246":1,"251":1,"260":1}}],["julialayer",{"2":{"44":1}}],["julialayernorm",{"2":{"19":1,"69":1}}],["julialstmcell",{"2":{"66":1}}],["julialppool",{"2":{"65":1}}],["julialossfn",{"2":{"179":1}}],["julialoss",{"2":{"106":1,"257":1}}],["julialocal",{"2":{"38":1}}],["julialoaded",{"2":{"3":1}}],["juliabegin",{"2":{"232":1,"236":1,"259":1}}],["juliabackend",{"2":{"120":1,"153":2}}],["juliabatchnorm",{"2":{"19":1,"69":1}}],["juliabatched",{"2":{"14":1,"28":1}}],["juliabranchlayer",{"2":{"62":1}}],["juliabilinear",{"2":{"67":1}}],["juliabidirectionalrnn",{"2":{"66":1}}],["juliabinaryfocalloss",{"2":{"53":1}}],["juliabinarycrossentropyloss",{"2":{"53":1}}],["juliabias",{"2":{"15":2}}],["juliabcast",{"2":{"39":1}}],["juliafig",{"2":{"250":2}}],["juliaf",{"2":{"176":1,"177":1}}],["juliafmap",{"2":{"106":1}}],["juliaflattenlayer",{"2":{"68":1}}],["juliafluxlayer",{"2":{"32":1}}],["juliaf64",{"2":{"56":1}}],["juliaf32",{"2":{"56":1}}],["juliaf16",{"2":{"56":1}}],["juliafor",{"2":{"174":1}}],["juliafoldl",{"2":{"54":1}}],["juliafocalloss",{"2":{"53":1}}],["juliafreeze",{"2":{"43":2}}],["juliafrozenlayer",{"2":{"43":1}}],["juliafromfluxadaptor",{"2":{"32":1}}],["juliafunction",{"2":{"106":2,"113":1,"117":1,"135":2,"145":1,"147":1,"148":1,"150":1,"151":1,"152":1,"179":1,"183":1,"184":2,"185":1,"187":1,"192":1,"195":1,"206":1,"208":1,"213":1,"214":2,"215":1,"217":1,"224":1,"225":2,"226":1,"228":1,"232":1,"236":1,"242":1,"244":1,"249":1,"254":4,"255":1,"256":1}}],["juliafunctional",{"2":{"3":1}}],["juliafused",{"2":{"16":1,"18":1}}],["juliafast",{"2":{"13":2}}],["juliats",{"2":{"245":1}}],["juliatstate",{"2":{"236":1}}],["juliatest",{"2":{"49":1}}],["juliatestmode",{"2":{"10":1}}],["juliatotal",{"2":{"38":1}}],["juliatosimplechainsadaptor",{"2":{"34":1}}],["juliatr",{"2":{"152":1,"196":1}}],["juliatry",{"2":{"116":1,"117":1}}],["juliatrain",{"2":{"196":1,"217":4,"219":1}}],["juliatrainstate",{"2":{"52":1}}],["juliatrainmode",{"2":{"10":1}}],["juliatruncated",{"2":{"24":1}}],["juliaimport",{"2":{"76":1}}],["juliaistraining",{"2":{"54":1}}],["juliaisleaf",{"2":{"3":1}}],["juliaidentity",{"2":{"24":1}}],["juliainput",{"2":{"153":1}}],["juliainternal",{"2":{"20":1}}],["juliainstancenorm",{"2":{"19":1,"69":1}}],["juliainitialized",{"2":{"37":1}}],["juliainitialize",{"2":{"37":1}}],["juliainitialstates",{"2":{"10":1}}],["juliainitialparameters",{"2":{"9":1}}],["juliadudt",{"2":{"209":1}}],["juliadataloader",{"2":{"160":2}}],["juliadata",{"2":{"120":1}}],["juliadicecoeffloss",{"2":{"53":1}}],["juliadistributedutils",{"2":{"120":1}}],["juliadistributeddatacontainer",{"2":{"41":1}}],["juliadistributedoptimizer",{"2":{"40":1}}],["juliadisplay",{"2":{"8":1}}],["juliadropout",{"2":{"17":1,"64":1}}],["juliadense",{"2":{"67":1}}],["juliadebuglayer",{"2":{"45":1}}],["juliadeviceiterator",{"2":{"5":1}}],["juliadefault",{"2":{"3":1}}],["juliacdev",{"2":{"131":1,"132":1}}],["juliachain",{"2":{"62":1}}],["juliacheck",{"2":{"8":1}}],["juliacrossentropyloss",{"2":{"53":1}}],["juliaconst",{"2":{"106":1,"186":1,"194":1,"216":1,"227":1,"235":1,"249":1,"256":2,"257":2}}],["juliaconvtranspose",{"2":{"63":1}}],["juliaconv",{"2":{"63":1}}],["juliacontains",{"2":{"8":1}}],["juliacompute",{"2":{"52":1}}],["juliacpu",{"2":{"2":1}}],["juliaadtype",{"2":{"258":1}}],["juliaadaptor",{"2":{"193":1}}],["juliaadaptivemeanpool",{"2":{"65":1}}],["juliaadaptivemaxpool",{"2":{"65":1}}],["juliaadaptivelppool",{"2":{"65":1}}],["juliaadapt",{"2":{"32":1,"34":1}}],["juliaanalytical",{"2":{"243":1}}],["juliaallreduce",{"2":{"39":1}}],["juliaalphadropout",{"2":{"64":1}}],["juliaalpha",{"2":{"17":1}}],["juliaapply",{"2":{"8":1,"52":2}}],["juliaabstract",{"2":{"7":3}}],["julia>",{"2":{"5":4,"24":3,"32":6,"34":7,"44":6,"46":3,"48":1,"49":2,"53":45,"59":34,"62":3,"68":8,"71":2}}],["juliastruct",{"2":{"113":1,"136":1,"137":1,"184":1,"207":1,"214":1,"218":1,"241":1}}],["juliastatefulrecurrentcell",{"2":{"66":1}}],["juliastatefulluxlayer",{"2":{"58":1}}],["juliastatelength",{"2":{"10":1}}],["juliastateless",{"2":{"8":1}}],["juliaselectdim",{"2":{"68":1}}],["juliasetup",{"2":{"8":1}}],["juliaset",{"2":{"4":2,"60":1}}],["juliascale",{"2":{"67":1}}],["juliaskipconnection",{"2":{"62":1}}],["juliasquaredhingeloss",{"2":{"53":1}}],["juliasize",{"2":{"171":1}}],["juliasiamesecontrastiveloss",{"2":{"53":1}}],["juliasingle",{"2":{"52":2}}],["juliasimplechainslayer",{"2":{"34":1}}],["juliashare",{"2":{"46":1}}],["juliasynchronize",{"2":{"39":1}}],["juliasparse",{"2":{"24":1}}],["juliasupported",{"2":{"3":1}}],["juliagdev",{"2":{"131":1,"153":1}}],["juliagrucell",{"2":{"66":1}}],["juliagroupnorm",{"2":{"19":1,"69":1}}],["juliaglobalmeanpool",{"2":{"65":1}}],["juliaglobalmaxpool",{"2":{"65":1}}],["juliagloballppool",{"2":{"65":1}}],["juliaglorot",{"2":{"24":2}}],["juliagenericlossfunction",{"2":{"53":1}}],["juliagetproperty",{"2":{"54":1}}],["juliaget",{"2":{"3":2,"37":1}}],["juliagpu",{"2":{"1":1,"2":1}}],["julia",{"0":{"170":1},"1":{"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"1":1,"16":1,"20":1,"32":1,"45":1,"48":1,"49":1,"50":1,"59":3,"60":1,"71":2,"74":1,"76":2,"77":3,"79":1,"81":1,"94":2,"98":1,"100":1,"101":1,"106":1,"112":3,"127":2,"128":2,"130":1,"131":1,"135":1,"144":1,"145":3,"147":1,"148":1,"152":1,"153":2,"154":1,"155":1,"167":2,"168":4,"170":2,"171":7,"174":1,"175":1,"180":12,"188":2,"189":12,"197":9,"210":12,"220":2,"221":12,"229":12,"237":12,"242":1,"246":12,"247":1,"248":1,"249":4,"250":2,"251":9,"254":1,"260":9}}],["jld2",{"2":{"182":1,"188":3}}],["jll",{"2":{"180":2,"189":2,"210":2,"221":2,"229":2,"237":2,"246":2}}],["jl`",{"2":{"18":1,"145":2}}],["jl",{"0":{"40":1,"41":1,"48":1,"71":1,"83":1,"86":1,"89":1,"90":1,"93":1,"106":1,"121":1,"161":1,"204":1},"1":{"84":1,"85":1,"87":1,"88":1,"91":1,"92":1,"94":1,"95":1,"96":1,"97":1,"107":1,"122":1,"123":1,"162":1,"163":1,"164":1,"165":1,"166":1,"167":1,"205":1,"206":1,"207":1,"208":1,"209":1,"210":1},"2":{"0":1,"2":3,"3":8,"6":5,"7":2,"8":1,"10":2,"11":1,"12":1,"13":3,"14":1,"18":2,"20":2,"22":2,"27":1,"28":1,"32":1,"34":4,"37":8,"40":1,"41":2,"47":1,"48":1,"49":6,"52":7,"53":2,"54":1,"60":1,"66":1,"71":4,"76":2,"78":3,"79":1,"80":5,"87":3,"90":5,"92":1,"94":1,"100":2,"102":8,"103":7,"105":1,"106":3,"107":1,"112":3,"116":2,"119":1,"120":1,"121":1,"122":1,"123":1,"130":1,"133":5,"135":1,"145":2,"153":3,"157":2,"158":2,"159":2,"160":3,"161":3,"163":1,"167":2,"168":1,"170":1,"172":1,"173":1,"175":4,"177":1,"179":1,"180":1,"181":2,"189":1,"190":4,"193":1,"196":1,"197":1,"203":3,"204":3,"208":3,"210":1,"211":1,"214":1,"220":6,"221":1,"229":1,"234":1,"237":1,"238":3,"244":4,"246":1,"247":2,"251":1,"252":1,"255":1,"256":1,"260":1}}],["p=res",{"2":{"259":1}}],["p=θ",{"2":{"257":1}}],["p=params",{"2":{"256":1}}],["p=2",{"2":{"65":3}}],["p^",{"2":{"255":1,"256":1}}],["p^2",{"2":{"255":1}}],["pdes",{"2":{"238":1}}],["pde",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"243":8,"244":2,"245":3}}],["pcie",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["physics",{"2":{"242":6,"244":108,"256":1}}],["philosophies",{"2":{"140":1}}],["phase",{"2":{"69":4}}],["pptional",{"2":{"77":1,"78":1}}],["pkg>",{"2":{"161":2}}],["pkg",{"2":{"71":4,"72":9,"73":2,"76":2,"77":1,"79":2,"161":1,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["pytorch",{"2":{"63":3,"67":1,"96":3,"97":1,"100":1,"170":1}}],["p2",{"2":{"53":6}}],["p1",{"2":{"53":4}}],["pipe",{"2":{"207":1}}],["pipeline",{"2":{"114":1}}],["pipelines",{"2":{"52":1}}],["pitfalls",{"0":{"154":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1}}],["pinn2dpde",{"2":{"244":8}}],["pinn",{"0":{"238":1},"1":{"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1},"2":{"238":1,"241":4,"244":5,"245":1}}],["pinns",{"2":{"147":1,"238":1}}],["pin",{"2":{"115":2}}],["pixel",{"2":{"70":2}}],["pixelshuffle",{"2":{"70":1}}],["pixels",{"2":{"65":3,"70":2}}],["pi",{"2":{"53":1}}],["pmlr",{"2":{"19":1}}],["p",{"2":{"17":7,"32":1,"59":2,"64":12,"206":2,"208":4,"209":2,"214":6,"218":2,"250":2,"254":3,"255":6,"256":2,"258":1}}],["peel",{"2":{"184":1,"185":1}}],["peel`",{"2":{"184":1}}],["people",{"2":{"135":1}}],["pesky",{"2":{"117":1}}],["pended",{"2":{"94":1}}],["penalties",{"2":{"59":1}}],["penultimate",{"2":{"15":1,"16":1,"66":2}}],["perspective",{"2":{"254":1}}],["perceptron",{"2":{"230":1}}],["per",{"2":{"69":4,"146":1,"249":2}}],["periodic",{"2":{"63":1}}],["period",{"2":{"42":1}}],["perfect",{"2":{"24":1}}],["performs",{"2":{"131":1,"208":1}}],["performing",{"2":{"65":3}}],["performed",{"2":{"34":2,"63":1,"249":1}}],["performance",{"0":{"154":1,"159":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1},"2":{"13":1,"24":2,"45":1,"52":1,"59":3,"103":2,"104":2,"105":1,"130":2,"153":1,"154":2,"155":2,"159":1}}],["perform",{"2":{"8":1,"39":2,"52":2,"63":3,"140":1,"249":1,"254":1}}],["plot",{"2":{"236":1,"248":3,"250":9,"255":1,"256":1,"259":1}}],["plotting",{"0":{"209":1}}],["platform",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["plan",{"2":{"144":1}}],["places",{"2":{"59":2}}],["place",{"2":{"13":1,"15":2,"55":5,"123":1,"130":1}}],["plugin",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["plus",{"2":{"43":1,"46":1,"59":3,"62":6,"69":7,"77":1,"78":1,"111":1,"116":2,"117":2,"193":2,"233":1,"256":1}}],["please",{"2":{"4":2,"74":2,"76":1,"78":1,"81":1,"105":1,"118":1,"124":1,"144":1,"184":1,"202":1,"203":3,"204":1,"238":1}}],["push",{"2":{"257":1}}],["pullback",{"2":{"117":2}}],["pull",{"2":{"81":1,"87":1}}],["publisher",{"2":{"74":1}}],["public",{"2":{"3":1,"53":1}}],["purpose",{"2":{"175":1}}],["purposes",{"2":{"57":1,"137":1,"208":1,"213":1}}],["pure",{"2":{"8":1,"98":1,"99":1,"100":1,"173":1}}],["pseudorandom",{"2":{"174":1}}],["pseudo",{"2":{"64":3,"170":1}}],["ps",{"2":{"7":2,"8":5,"32":4,"34":2,"39":2,"43":2,"44":18,"46":17,"53":2,"57":1,"58":5,"59":22,"62":4,"68":9,"77":3,"78":6,"106":16,"107":6,"109":3,"111":2,"112":2,"113":6,"114":2,"116":5,"117":12,"120":2,"126":9,"127":7,"128":6,"129":2,"135":7,"136":7,"137":10,"139":5,"140":7,"144":3,"145":9,"146":9,"147":8,"148":11,"150":2,"151":2,"152":9,"153":7,"155":5,"179":7,"184":4,"186":2,"187":3,"188":3,"194":2,"195":2,"208":4,"209":1,"214":2,"215":5,"216":2,"217":2,"218":3,"220":15,"225":2,"227":2,"228":2,"235":1,"236":1,"242":4,"244":3,"249":10,"250":1,"256":2}}],["power",{"2":{"250":1}}],["powerful",{"2":{"171":1}}],["pooling",{"0":{"65":1},"2":{"65":21,"97":1}}],["poisson",{"2":{"53":1}}],["poissonloss",{"2":{"53":2}}],["points",{"2":{"140":1,"179":1,"232":1,"248":4,"249":1,"250":1}}],["point",{"0":{"56":1},"2":{"4":2,"13":1,"56":3,"115":2,"120":1,"125":1,"130":1,"171":1,"255":1,"256":1}}],["populated",{"2":{"184":2}}],["populate",{"2":{"45":1}}],["polynomial",{"0":{"230":1},"1":{"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1},"2":{"230":1,"232":1}}],["polyester",{"2":{"20":1}}],["polyalgorithm",{"2":{"18":1}}],["position=",{"2":{"206":1,"209":1,"256":1,"259":1}}],["position",{"2":{"68":2,"254":1}}],["possibly",{"2":{"63":2,"65":3,"103":1}}],["possible",{"2":{"1":1,"2":1,"7":1,"11":1,"13":3,"15":1,"16":2,"18":1,"32":2,"37":2,"104":1,"145":1}}],["posterior",{"2":{"250":2}}],["posted",{"2":{"144":1}}],["post",{"2":{"7":1,"34":1}}],["potentially",{"2":{"7":1,"11":1,"34":1}}],["painful",{"2":{"115":1}}],["pairwisefusion",{"2":{"62":3}}],["pairs",{"2":{"53":1,"220":13}}],["pair",{"2":{"53":1,"63":2,"66":1}}],["past",{"2":{"100":1}}],["passes",{"2":{"62":2,"118":1,"173":1}}],["passed",{"2":{"10":1,"19":1,"46":2,"48":1,"49":1,"59":1,"62":6,"66":5,"68":6,"96":1,"99":1,"136":1}}],["passing",{"2":{"8":1,"59":2,"62":1,"69":1,"120":2,"168":2}}],["pass",{"2":{"8":1,"27":2,"49":1,"50":1,"54":2,"56":1,"59":3,"62":1,"63":1,"67":2,"69":1,"78":1,"94":1,"100":1,"117":3,"120":1,"135":1,"168":1,"184":3}}],["pal2023efficient",{"2":{"74":1}}],["pal2023lux",{"2":{"74":1}}],["pal",{"2":{"74":2}}],["pattern",{"2":{"53":1,"106":1,"160":1,"248":1}}],["path",{"2":{"10":1,"62":1,"95":2,"126":2,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":1,"254":4,"260":1}}],["page",{"2":{"29":1,"42":1,"43":1,"58":1,"82":1,"106":1,"115":1,"153":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["pad",{"2":{"63":6,"65":3}}],["pad=0",{"2":{"63":2,"65":3}}],["padding",{"2":{"24":1,"63":9,"65":12}}],["pads",{"2":{"24":1}}],["parse",{"2":{"146":1}}],["parallelism",{"2":{"160":1}}],["parallel=true",{"2":{"160":2}}],["parallel",{"0":{"119":1},"1":{"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"44":1,"62":12,"68":1}}],["params=nothing",{"2":{"254":2}}],["params",{"2":{"43":14,"44":6,"52":2,"69":3,"126":3,"129":1,"254":4,"255":3,"256":6,"257":2,"258":1,"259":1}}],["parameterized",{"2":{"250":1}}],["parameterization",{"2":{"74":1,"247":1}}],["parameter",{"0":{"43":1,"137":1},"2":{"24":3,"43":2,"46":2,"53":1,"59":6,"63":4,"68":1,"69":2,"77":1,"99":1,"100":4,"117":1,"120":1,"128":2,"135":5,"136":2,"137":5,"140":1,"208":2,"217":1,"247":1,"249":2}}],["parameterlength",{"2":{"7":1,"9":1,"135":4,"136":1,"226":1,"249":3}}],["parameters",{"0":{"9":1,"46":1,"125":1,"128":1,"148":1},"1":{"126":1,"127":1,"128":1,"129":1},"2":{"3":1,"7":5,"8":1,"9":2,"24":1,"32":4,"34":1,"40":1,"43":13,"44":7,"45":1,"46":23,"52":9,"53":1,"56":2,"57":4,"58":5,"59":17,"62":24,"63":2,"66":9,"67":4,"68":2,"69":31,"77":5,"78":7,"87":2,"97":1,"98":1,"100":2,"107":1,"111":5,"116":6,"117":7,"120":1,"125":2,"128":3,"133":1,"135":5,"136":3,"137":7,"140":6,"144":1,"148":1,"175":1,"179":6,"184":2,"187":2,"188":1,"193":12,"195":2,"208":5,"214":1,"217":2,"225":1,"228":4,"233":3,"235":1,"236":3,"244":1,"249":79,"250":2,"256":6}}],["parts",{"2":{"208":1}}],["party",{"0":{"203":1}}],["particle",{"2":{"255":2,"256":1}}],["particles",{"2":{"254":1}}],["particular",{"0":{"126":1},"2":{"115":1,"126":1}}],["participate",{"0":{"109":1}}],["partition",{"2":{"41":1}}],["part",{"0":{"128":1,"129":1},"2":{"3":1,"54":1,"82":1,"145":1,"202":1,"204":1,"214":1}}],["packages",{"0":{"30":1,"79":1,"205":1},"1":{"31":1,"32":1,"33":1,"34":1},"2":{"3":3,"27":4,"28":2,"34":1,"52":2,"60":1,"77":1,"78":1,"79":3,"80":1,"87":1,"90":1,"101":2,"104":3,"105":2,"112":3,"130":1,"145":2,"154":1,"159":1,"166":1,"167":3,"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1,"247":1}}],["package",{"0":{"182":1,"191":1,"212":1,"223":1,"231":1,"239":1,"253":1},"2":{"0":1,"3":1,"4":3,"21":1,"22":1,"40":1,"47":3,"72":1,"73":1,"76":1,"79":1,"90":3,"94":1,"101":1,"102":1,"105":1,"131":2,"164":1,"169":1,"172":1,"203":1,"238":1}}],["practitioners",{"2":{"208":1}}],["prngs",{"2":{"174":1}}],["prng",{"2":{"170":1,"174":2}}],["prs",{"2":{"144":1}}],["primarily",{"2":{"249":1}}],["primitives",{"0":{"39":1,"158":1},"2":{"158":1,"249":1}}],["principles",{"0":{"99":1}}],["printf",{"2":{"78":2,"106":1,"107":1,"179":2,"182":1,"187":2,"191":1,"195":1,"205":1,"208":1,"212":1,"217":1,"223":1,"228":2,"231":1,"236":1,"239":1,"244":1,"253":1,"257":1}}],["printouts",{"2":{"100":1}}],["printout",{"2":{"59":1}}],["print",{"2":{"57":1,"59":1}}],["println",{"2":{"44":1,"116":2,"117":2,"135":2,"136":3,"137":2,"145":2,"146":2,"147":2,"148":2,"152":6,"173":2,"174":2,"176":2,"177":1,"178":1,"179":3,"180":2,"189":2,"197":2,"210":2,"221":2,"228":1,"229":2,"237":2,"246":2,"251":2,"260":2}}],["printing",{"2":{"34":1,"68":1,"78":1,"100":1}}],["prints",{"2":{"4":2,"117":1}}],["printed",{"2":{"2":1,"8":1,"32":1,"59":1}}],["priority",{"2":{"104":2}}],["prior",{"2":{"63":1,"249":1}}],["pr",{"2":{"11":1,"202":1,"203":1}}],["precompile",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["precision",{"0":{"56":1},"2":{"16":2,"56":1,"155":3,"171":1}}],["press",{"2":{"161":1}}],["pressing",{"2":{"76":1}}],["presence",{"2":{"59":1}}],["presented",{"2":{"177":1}}],["presently",{"2":{"106":1}}],["present",{"2":{"32":1,"34":1,"48":1,"54":1,"63":2,"66":10,"67":2,"94":1,"116":1,"117":1,"131":2}}],["preserving",{"2":{"34":1,"70":1}}],["preservation",{"2":{"32":1}}],["preserved",{"2":{"34":1}}],["preserve",{"2":{"32":7,"34":1}}],["pretty",{"2":{"68":1,"78":1,"100":1,"121":1,"145":1}}],["preds",{"2":{"249":2}}],["pred=ŷ",{"2":{"186":1}}],["predictive",{"2":{"250":1}}],["predictions",{"2":{"236":1,"249":2,"250":2}}],["prediction",{"0":{"250":1},"2":{"53":2,"249":1,"250":1}}],["predictor",{"2":{"245":1}}],["predict",{"2":{"183":1,"250":6}}],["predicted",{"2":{"53":3,"194":2,"216":2,"227":2,"250":2}}],["pred",{"2":{"53":8,"106":4,"186":5,"208":2,"209":3,"236":2,"245":4,"257":4}}],["preprint",{"2":{"19":2,"69":1}}],["preprocessing",{"2":{"8":1}}],["prevents",{"2":{"94":1}}],["prevent",{"2":{"17":2,"37":1,"52":1,"53":1,"94":1,"135":1}}],["previously",{"2":{"7":1,"11":1,"87":1,"96":1,"100":1,"122":1}}],["previous",{"2":{"5":1,"7":1,"100":1,"146":1}}],["pre",{"2":{"7":1,"11":1,"63":1,"76":1,"77":1,"94":1}}],["prefer",{"2":{"158":1}}],["preferably",{"2":{"20":1}}],["preferred",{"2":{"5":1,"20":1}}],["preferencetools",{"2":{"161":2}}],["preference",{"2":{"1":2,"2":1,"57":1,"60":2,"144":1,"155":1,"157":1,"161":6,"164":2,"165":1,"166":3,"167":1}}],["preferences",{"0":{"1":1,"161":1},"1":{"162":1,"163":1,"164":1,"165":1,"166":1,"167":1},"2":{"57":1,"60":4,"94":1,"157":2,"161":4,"163":3,"166":2}}],["progress",{"2":{"247":3}}],["programming",{"2":{"170":1}}],["proto",{"2":{"195":4}}],["probabilistic",{"2":{"249":2}}],["probability",{"2":{"17":3,"53":1,"64":3}}],["prob",{"2":{"206":2,"208":9,"209":2,"214":4,"218":2,"255":1,"256":1,"257":1,"259":1}}],["problem",{"0":{"240":1},"2":{"116":1,"117":1,"145":1,"148":1,"173":1,"179":2,"208":1,"218":1,"220":1,"233":1,"240":1}}],["problematic",{"2":{"115":1,"116":1}}],["problems",{"0":{"116":1},"2":{"98":1,"106":1}}],["promotion",{"0":{"155":1},"2":{"155":1}}],["promotions",{"2":{"57":1,"155":1}}],["promote",{"2":{"16":1,"155":1}}],["project",{"2":{"47":1}}],["produce",{"2":{"54":1,"87":1}}],["products",{"2":{"27":2}}],["product",{"0":{"150":1,"151":1,"177":1,"178":1},"2":{"27":8,"144":2,"149":4,"150":1,"151":1,"175":5,"177":2,"178":1,"243":1,"245":1}}],["prod",{"2":{"24":1}}],["propagating",{"2":{"58":1}}],["propagated",{"2":{"62":1}}],["propagate",{"2":{"8":1}}],["proper",{"2":{"114":1,"170":1}}],["properly",{"2":{"45":1}}],["properties",{"2":{"28":1,"62":1}}],["proportion",{"2":{"24":3}}],["providing",{"2":{"21":1,"193":1}}],["provides",{"2":{"34":1,"49":1,"58":1,"77":1,"92":1,"133":1,"161":1,"168":1,"177":1,"204":1,"218":1}}],["provide",{"2":{"11":1,"24":1,"32":1,"52":2,"59":1,"79":1,"101":1,"149":1,"172":1}}],["provided",{"2":{"4":2,"5":1,"17":1,"42":1,"44":2,"54":1,"66":3,"120":1,"133":1,"172":1,"235":1,"250":1}}],["proceeding",{"2":{"135":1}}],["proceedings",{"2":{"19":1,"24":5,"53":2}}],["proceeds",{"2":{"66":8}}],["processor",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["processes",{"2":{"40":1,"41":1,"120":3,"160":1}}],["processing",{"2":{"17":1,"66":1}}],["process",{"2":{"4":1,"120":1,"160":1,"192":1,"213":1,"243":1}}],["ddot",{"2":{"254":2}}],["ddp",{"2":{"119":1}}],["dt^2",{"2":{"254":1}}],["dt2",{"2":{"254":3}}],["dt",{"2":{"254":21,"255":4,"256":2,"257":2,"259":2}}],["dset",{"2":{"224":4}}],["dnns",{"2":{"115":1}}],["dl",{"2":{"54":1,"256":1}}],["dynamic",{"2":{"47":1}}],["dynamics",{"2":{"24":1}}],["d2",{"2":{"46":5,"111":1,"112":3,"254":3}}],["d2=dense",{"2":{"46":1,"111":1,"114":1}}],["d3",{"2":{"46":9}}],["d3=chain",{"2":{"46":1}}],["d1",{"2":{"46":5,"111":1,"112":3}}],["d1=dense",{"2":{"46":1,"111":1,"114":1}}],["d1×",{"2":{"19":2,"69":2}}],["driver",{"2":{"180":3,"189":3,"210":3,"221":3,"229":3,"237":3,"246":3}}],["dr",{"2":{"115":1}}],["drawn",{"2":{"24":5,"250":1}}],["drop",{"2":{"53":1,"144":1}}],["dropped",{"2":{"17":2,"27":2}}],["dropout",{"0":{"17":1,"64":1},"2":{"17":14,"64":17,"100":1,"126":1,"129":1,"138":1,"142":1}}],["duration",{"2":{"249":2}}],["during",{"2":{"3":1,"69":4}}],["dudt",{"2":{"208":2,"209":1,"214":4,"218":2}}],["du",{"2":{"206":3}}],["due",{"2":{"146":1,"153":1,"220":1}}],["dummy",{"2":{"77":1}}],["dual",{"2":{"20":1}}],["dmitry",{"2":{"19":1,"69":1}}],["d",{"2":{"8":1,"59":11,"63":4,"65":6,"68":2,"70":4,"104":1,"126":2,"127":5,"128":6,"137":4,"183":4,"217":2,"254":2}}],["domain",{"2":{"243":1}}],["domainerror",{"2":{"45":1,"117":2}}],["doubt",{"2":{"135":1}}],["dot",{"2":{"116":1,"206":2,"209":2,"220":3,"244":4}}],["doi",{"2":{"74":2,"254":2}}],["doing",{"2":{"55":4,"107":1,"225":1,"249":1}}],["document",{"2":{"101":1}}],["documented",{"2":{"94":1}}],["documentations",{"2":{"154":1}}],["documentation",{"2":{"7":1,"8":1,"14":1,"49":1,"60":1,"70":1,"79":1,"81":1,"96":2,"135":1,"165":1,"166":1,"196":1,"203":1}}],["doctor",{"0":{"166":1},"2":{"60":4,"157":1,"166":3}}],["docstrings",{"2":{"135":1}}],["docstring",{"2":{"94":1}}],["docs",{"2":{"59":2,"84":1,"204":1,"244":4,"247":2}}],["downstream",{"2":{"100":1,"105":1,"203":1}}],["downright",{"2":{"100":1,"133":1}}],["down",{"0":{"117":1},"2":{"53":2,"57":1,"116":1,"117":1,"118":1,"136":1,"155":1,"171":2}}],["does",{"2":{"13":1,"19":2,"57":1,"68":1}}],["doesn",{"2":{"13":2,"44":1,"46":1,"54":1,"59":3,"66":1,"109":1,"135":2,"137":1,"138":1,"142":1,"145":1,"226":1}}],["do",{"2":{"6":1,"59":14,"62":1,"68":2,"76":1,"78":2,"89":1,"106":2,"107":1,"135":1,"136":1,"137":1,"138":1,"142":1,"144":1,"149":2,"153":1,"155":1,"160":3,"171":2,"173":1,"184":1,"185":1,"193":1,"208":2,"214":1,"225":1,"233":1,"245":1,"249":1,"250":1,"254":1,"256":1}}],["done",{"2":{"13":2,"44":1,"57":1,"77":1,"120":1,"131":1,"146":1,"167":1}}],["don",{"2":{"4":2,"16":1,"18":1,"25":1,"47":1,"59":2,"77":1,"100":1,"104":2,"105":1,"115":1,"118":1,"123":1,"124":1,"135":2,"137":1,"140":1,"144":1,"146":1,"171":2,"173":2,"177":1,"183":1,"184":1,"188":1,"192":1,"207":1,"213":1,"225":1}}],["digits=2",{"2":{"228":4}}],["digits",{"2":{"171":2}}],["direactly",{"2":{"94":1}}],["direct",{"2":{"94":1}}],["direction",{"2":{"69":2}}],["directly",{"2":{"3":1,"5":1,"6":1,"8":1,"17":1,"36":2,"43":1,"52":1,"55":2,"62":2,"68":1,"70":1,"91":1,"95":1,"120":1,"132":1,"144":1,"168":1,"211":1}}],["diagonal",{"2":{"67":1,"249":1}}],["divisible",{"2":{"63":2}}],["divides",{"2":{"70":1}}],["divide",{"2":{"63":2}}],["diverges",{"2":{"53":1}}],["divergence",{"2":{"53":2}}],["dilation",{"2":{"63":2,"65":3}}],["dilation=1",{"2":{"63":2,"65":3}}],["dice",{"2":{"53":2}}],["dicecoeffloss",{"2":{"53":4}}],["diffeqsol",{"2":{"214":3,"215":1,"220":9}}],["diffeqflux",{"2":{"211":1}}],["differ",{"2":{"175":1}}],["differern",{"2":{"146":1}}],["differences",{"0":{"123":1},"2":{"119":1,"140":1,"145":2}}],["difference",{"2":{"64":1,"87":1,"106":3,"136":1,"145":2,"146":2,"149":1,"219":1,"254":2}}],["differentiate",{"2":{"106":1,"152":1,"173":1}}],["differentiationinterface",{"2":{"144":1,"177":2}}],["differentiation",{"0":{"26":1,"101":1,"144":1,"162":1,"175":1},"1":{"27":1,"28":1,"29":1,"102":1,"103":1,"104":1,"105":1,"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1,"176":1,"177":1,"178":1},"2":{"29":1,"106":1,"137":1,"144":1,"147":1,"153":1,"162":2,"170":1,"177":1}}],["differential",{"2":{"74":1,"257":1}}],["differently",{"2":{"62":1}}],["different",{"0":{"33":1},"1":{"34":1},"2":{"18":1,"20":1,"44":1,"53":1,"60":1,"62":1,"96":1,"140":1,"146":1,"165":1,"174":1,"208":1}}],["diff",{"2":{"145":1,"147":1,"148":1,"175":1}}],["diffractor",{"2":{"102":1}}],["difficulty",{"2":{"24":2}}],["dim",{"2":{"68":6,"107":1,"179":7}}],["dimwhere",{"2":{"53":1}}],["dims=3",{"2":{"183":1}}],["dims=ndims",{"2":{"111":1}}],["dims=colon",{"2":{"69":1}}],["dims=2",{"2":{"66":1}}],["dims=",{"2":{"64":2}}],["dims=pad",{"2":{"63":1}}],["dims=1",{"2":{"19":1,"53":1,"243":5}}],["dims",{"2":{"17":2,"19":3,"24":11,"34":2,"53":2,"54":1,"63":1,"64":4,"66":31,"67":55,"68":7,"69":3,"84":1,"135":12,"169":4,"184":9,"185":7,"241":11,"244":2}}],["dimensionmismatch",{"2":{"116":2}}],["dimensionality",{"2":{"53":1}}],["dimensional",{"2":{"19":2,"67":3,"70":2,"177":1}}],["dimensions",{"2":{"19":2,"24":11,"28":2,"34":1,"63":3,"65":9,"66":2,"67":10,"68":5,"69":4,"70":8,"97":1,"179":1,"193":1}}],["dimension",{"0":{"116":1},"2":{"15":2,"16":1,"19":1,"34":1,"54":1,"63":4,"64":2,"65":6,"66":8,"67":1,"68":9,"69":14,"84":1,"111":1,"114":1,"115":1,"169":1}}],["disabling",{"0":{"167":1},"2":{"159":1}}],["disabled",{"2":{"8":1,"63":2,"67":3,"156":1}}],["disable",{"2":{"8":1,"45":1,"60":3,"94":2,"117":1,"144":1,"156":1,"162":1,"166":1,"167":2}}],["disallow",{"2":{"156":1}}],["disaster",{"2":{"155":1}}],["discrete",{"2":{"214":1,"217":1}}],["discouraged",{"2":{"145":2}}],["discourse",{"2":{"144":1,"145":1,"147":1}}],["discuss",{"2":{"204":1}}],["discussed",{"2":{"118":1}}],["discussions",{"2":{"81":1}}],["discussion",{"2":{"66":1,"147":1,"177":1}}],["disruptive",{"2":{"98":1}}],["dissimilar",{"2":{"53":1}}],["distinguish",{"2":{"140":2}}],["distinction",{"2":{"100":1}}],["distinct",{"2":{"66":1}}],["distance",{"2":{"53":1}}],["distributions",{"2":{"53":1,"249":2}}],["distribution",{"2":{"24":8,"25":12,"53":5,"63":2,"66":6,"67":3,"250":1}}],["distributeddatacontainer",{"2":{"41":1,"120":2,"121":1}}],["distributedoptimizer",{"2":{"40":2,"120":2,"121":1}}],["distributedutils",{"0":{"120":1},"2":{"4":2,"35":1,"36":2,"37":3,"38":2,"39":6,"40":1,"41":1,"119":1,"120":11,"121":5,"122":1,"123":3}}],["distributed",{"0":{"35":1,"119":1},"1":{"36":1,"37":1,"38":1,"39":1,"40":1,"41":1,"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"4":3,"24":2,"36":4,"37":7,"100":1,"120":2}}],["disjoint",{"2":{"46":1}}],["displays",{"2":{"248":1,"250":1}}],["displayed",{"2":{"131":1}}],["display",{"2":{"8":1,"100":1,"136":1}}],["dispatch=",{"2":{"225":1}}],["dispatch=nothing",{"2":{"59":1}}],["dispatch₋₋₋",{"2":{"220":3}}],["dispatching",{"0":{"108":1},"1":{"109":1,"110":1,"111":1,"112":1,"113":1,"114":1}}],["dispatchdoctor",{"2":{"8":1,"60":1,"157":1}}],["dispatches",{"2":{"3":1,"8":1,"20":2,"54":1,"59":1,"133":1}}],["dispatch",{"0":{"109":1,"113":1,"166":1},"2":{"3":1,"47":1,"59":2,"60":4,"109":2,"113":1,"114":1,"138":1,"153":1,"157":1,"166":3}}],["date",{"2":{"203":2}}],["datapoints",{"2":{"232":1}}],["datasize",{"2":{"206":1,"255":1}}],["datasets",{"0":{"224":1},"2":{"183":1,"224":2,"228":1}}],["dataset",{"0":{"183":1,"232":1},"2":{"41":1,"160":2,"183":7,"192":3,"208":1,"213":3,"224":2,"232":2,"248":2}}],["datatype",{"2":{"8":1}}],["dataloaders",{"2":{"183":3,"187":2,"228":3}}],["dataloader",{"0":{"207":1},"2":{"5":8,"92":1,"107":4,"160":5,"183":5,"192":3,"194":2,"195":5,"207":5,"208":4,"212":1,"213":3,"216":2,"217":5,"224":2,"227":2,"228":9,"244":6}}],["data",{"0":{"2":1,"119":1,"160":1,"206":1,"243":1,"248":1},"1":{"120":1,"121":1,"122":1,"123":1,"124":1},"2":{"0":1,"2":1,"3":3,"5":1,"41":2,"52":6,"53":1,"59":10,"63":5,"65":9,"66":1,"68":1,"69":6,"70":2,"78":10,"92":1,"100":1,"106":1,"107":2,"120":3,"135":1,"153":1,"155":1,"160":5,"179":1,"183":8,"192":6,"206":3,"207":3,"208":3,"209":2,"213":6,"227":2,"228":16,"230":1,"232":4,"235":1,"236":5,"242":5,"243":10,"244":113,"248":6,"249":1,"250":7,"255":3,"256":2,"257":1,"259":2}}],["danger",{"2":{"2":1,"4":2,"28":1,"37":1}}],["derivative",{"2":{"147":1}}],["derivatives",{"2":{"145":1,"146":1,"177":1,"240":1,"256":1}}],["de",{"2":{"145":2}}],["demonstration",{"2":{"208":1,"213":1,"238":1}}],["demonstrative",{"2":{"137":1}}],["demonstrate",{"2":{"140":1,"145":1,"179":1,"190":1}}],["deal",{"2":{"147":1,"214":1}}],["deadlocks",{"2":{"124":1}}],["deactivate",{"2":{"66":3}}],["decision",{"2":{"161":1}}],["decides",{"2":{"142":1}}],["decouples",{"2":{"69":1}}],["declared",{"2":{"59":1}}],["decrease",{"2":{"13":1}}],["debuglayer",{"2":{"45":3,"116":4,"117":4}}],["debug",{"2":{"45":5,"95":1,"115":4,"116":6,"117":10,"155":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["debugging",{"0":{"45":1,"115":1},"1":{"116":1,"117":1,"118":1},"2":{"45":3,"57":1,"115":2}}],["delta=0",{"2":{"53":1}}],["delta",{"2":{"53":2}}],["delving",{"2":{"24":2}}],["deleted",{"2":{"1":1}}],["descent",{"2":{"179":2}}],["describe",{"2":{"153":1}}],["describes",{"2":{"34":1,"115":1,"255":1,"256":1}}],["described",{"2":{"24":3,"62":2}}],["despite",{"2":{"52":1}}],["destructure",{"2":{"32":3,"100":2,"143":1}}],["desirable",{"2":{"167":1,"249":1}}],["desired",{"2":{"1":1,"63":1,"97":1}}],["design",{"0":{"99":1},"2":{"5":1,"98":1,"140":1}}],["density",{"2":{"249":1}}],["denselayerparameters",{"2":{"137":3}}],["dense",{"2":{"18":2,"32":2,"34":3,"43":2,"44":18,"46":4,"53":2,"59":10,"62":12,"67":2,"68":1,"69":12,"77":17,"78":6,"85":1,"96":1,"106":9,"107":3,"111":6,"112":2,"114":2,"116":17,"117":14,"126":9,"127":3,"129":3,"135":1,"137":2,"139":5,"145":2,"146":2,"147":4,"148":4,"152":4,"153":2,"155":1,"158":1,"166":1,"179":2,"184":2,"185":1,"193":9,"208":3,"215":5,"220":45,"226":4,"233":4,"236":4,"241":4,"249":4,"256":6}}],["denom",{"2":{"255":3,"256":3}}],["denominator",{"2":{"19":4,"69":4}}],["denotes",{"2":{"3":4}}],["detour",{"2":{"175":1}}],["detection",{"2":{"53":2}}],["detected",{"2":{"45":1,"117":2,"244":1}}],["deterministic",{"2":{"24":1,"100":1}}],["determined",{"2":{"77":1}}],["determines",{"2":{"34":1}}],["determine",{"2":{"3":1,"17":2,"19":2,"55":1,"123":1,"249":1}}],["details",{"0":{"141":1},"1":{"142":1},"2":{"14":1,"17":2,"19":4,"32":1,"34":2,"45":1,"46":1,"49":1,"58":1,"59":2,"60":1,"69":1,"70":1,"84":1,"87":1,"89":1,"94":1,"96":2,"135":1,"162":1,"165":1,"166":1,"172":1}}],["detailed",{"2":{"7":1,"43":1,"116":1}}],["depot",{"2":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["depots",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["depwarn",{"2":{"94":1}}],["deprecated",{"2":{"84":1,"244":4}}],["deprecation",{"2":{"42":1,"90":2,"145":2}}],["deprecations",{"2":{"2":1,"94":1}}],["depthwise",{"2":{"63":2}}],["depths",{"2":{"44":1}}],["dependent",{"0":{"111":1},"2":{"110":1}}],["dependencies",{"0":{"159":1},"2":{"47":1,"87":1,"167":1,"249":1}}],["dependency",{"2":{"6":1,"21":1,"47":1,"133":1,"146":1}}],["depend",{"2":{"6":1,"19":1,"133":1,"135":1}}],["depending",{"2":{"6":1,"8":1,"24":1}}],["defiing",{"0":{"256":1}}],["definition",{"0":{"240":1}}],["definitions",{"2":{"136":1}}],["definitely",{"2":{"152":1}}],["defining",{"0":{"78":1,"186":1},"2":{"0":1,"3":2,"11":1,"59":1,"78":1,"106":1,"109":1,"133":1,"135":1,"136":1,"137":1,"184":1,"185":2,"193":1,"225":1}}],["define",{"0":{"193":1,"195":1,"207":1,"214":1,"216":1,"227":1,"241":1,"242":1,"254":1},"2":{"7":1,"20":1,"78":1,"109":2,"110":1,"113":1,"117":2,"133":2,"135":3,"136":1,"137":2,"140":2,"153":1,"179":1,"184":3,"185":1,"186":1,"207":1,"208":1,"214":1,"241":1,"242":1,"249":3,"254":2,"256":2,"257":2}}],["defines",{"2":{"6":1,"7":2,"55":1,"254":1,"255":1}}],["defined",{"2":{"3":1,"8":1,"11":1}}],["defer",{"2":{"177":1}}],["defaults",{"0":{"96":1},"2":{"58":1,"66":3,"96":4,"256":1}}],["default",{"2":{"3":2,"8":1,"19":10,"22":5,"24":8,"25":24,"32":1,"34":3,"44":1,"56":1,"57":1,"59":1,"66":1,"68":4,"69":1,"73":4,"77":1,"78":1,"84":1,"87":2,"94":1,"96":2,"100":1,"106":1,"107":1,"129":1,"135":1,"137":5,"139":2,"140":2,"142":2,"155":1,"156":1,"163":1,"166":1,"167":1,"168":4,"169":1,"170":1,"171":1,"173":1,"179":1,"180":1,"184":1,"189":1,"197":1,"207":1,"208":1,"210":1,"215":1,"220":1,"221":1,"225":1,"229":1,"237":1,"244":1,"246":1,"248":1,"251":1,"256":1,"260":1}}],["dev=gpu",{"2":{"215":1}}],["developer",{"2":{"106":1}}],["developed",{"2":{"100":1,"203":1}}],["deviate",{"2":{"256":1}}],["deviation",{"2":{"24":3,"69":1}}],["devicememory",{"2":{"5":3,"77":18,"131":1,"153":2,"168":2,"172":1,"220":18}}],["deviceiterator",{"2":{"5":3,"92":2,"107":2,"207":1}}],["device",{"0":{"160":1},"2":{"2":25,"3":32,"4":11,"5":3,"20":1,"37":1,"72":5,"73":4,"77":2,"78":1,"90":1,"91":3,"95":2,"106":5,"107":5,"123":4,"131":8,"132":2,"140":3,"153":1,"160":5,"180":1,"187":2,"189":1,"205":2,"207":1,"210":1,"215":1,"217":2,"220":1,"221":1,"228":1,"229":1,"235":2,"236":1,"237":1,"239":2,"246":1}}],["devices",{"2":{"0":1,"37":4,"140":1}}],["dev",{"2":{"2":1,"3":1,"4":2,"5":4,"20":2,"72":4,"73":3,"77":5,"78":4,"187":3,"215":2,"217":3,"228":4,"235":3,"236":2}}],["deepcopy",{"2":{"52":1}}],["deep",{"2":{"2":1,"19":1,"21":1,"24":6,"74":1,"100":1,"155":1,"158":1,"168":1,"169":1}}],["axes",{"2":{"225":3}}],["axislegend",{"2":{"206":1,"209":1,"232":1,"236":1,"255":1,"256":1,"259":1}}],["axis",{"2":{"206":1,"209":1,"220":44,"232":1,"236":1,"245":1,"248":1,"250":1,"255":1,"256":1,"259":2}}],["ax",{"2":{"206":4,"209":6,"232":4,"236":5,"245":4,"248":3,"255":4,"256":6,"259":11}}],["a100",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["ai",{"2":{"177":1}}],["aims",{"2":{"24":1}}],["a∈rd×d",{"2":{"149":1}}],["a=layer",{"2":{"140":1}}],["a×b×x",{"2":{"140":1}}],["aware",{"0":{"163":1},"2":{"123":2,"124":1,"163":4}}],["ah",{"2":{"117":1}}],["ahmadi",{"2":{"53":1}}],["ahmad",{"2":{"53":1}}],["author",{"2":{"74":2}}],["autojacvec=reversediffvjp",{"2":{"217":1}}],["autojacvec=zygotevjp",{"2":{"215":1,"217":1}}],["auto",{"2":{"142":1,"155":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["autoreactant",{"2":{"52":1}}],["autoreversediff",{"2":{"49":1,"52":2}}],["autoenzyme",{"2":{"49":1,"52":1,"107":2}}],["autofinitediff",{"2":{"49":1}}],["autoforwarddiff",{"2":{"27":1,"28":1,"49":1,"146":1,"151":1,"177":2}}],["autotracker",{"2":{"49":1,"52":1}}],["autozygote",{"2":{"27":1,"28":1,"49":1,"52":1,"77":2,"78":1,"150":1,"178":1,"179":1,"187":1,"195":2,"208":1,"217":1,"228":1,"236":2,"244":1,"258":1}}],["autodiff",{"2":{"17":2,"19":2,"98":1,"137":1,"144":1,"145":2,"175":2}}],["autoselection",{"2":{"2":1}}],["automatically",{"2":{"5":1,"7":2,"8":1,"17":3,"19":2,"66":2,"123":1,"142":1,"143":1,"144":1,"159":1,"183":1,"184":2,"185":1,"192":1,"213":1,"249":1}}],["automatic",{"0":{"26":1,"101":1,"131":1,"144":1,"162":1,"165":1,"175":1},"1":{"27":1,"28":1,"29":1,"102":1,"103":1,"104":1,"105":1,"145":1,"146":1,"147":1,"148":1,"149":1,"150":1,"151":1,"152":1,"176":1,"177":1,"178":1},"2":{"2":1,"3":1,"8":1,"29":1,"106":1,"131":2,"132":1,"137":1,"144":2,"153":1,"162":4,"164":1,"170":1,"177":1}}],["affected",{"2":{"149":1}}],["affects",{"2":{"63":1}}],["affine",{"2":{"69":1}}],["affine=false",{"2":{"69":5,"96":1}}],["affine=true",{"2":{"62":2,"69":20,"96":1,"116":1,"117":2}}],["afterwards",{"2":{"69":1}}],["after",{"2":{"17":2,"45":1,"59":1,"62":2,"64":1,"65":3,"69":4,"100":1,"115":1,"120":1,"123":1,"160":1,"179":14,"184":1}}],["abyss",{"2":{"170":1}}],["ability",{"2":{"133":1}}],["ab7d",{"2":{"112":3,"145":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["able",{"2":{"87":1,"181":1}}],["above",{"2":{"54":1,"62":2,"76":1,"148":1,"169":1,"172":1,"250":1}}],["about",{"2":{"45":1,"57":1,"100":1,"135":1,"171":1,"174":1,"184":1,"243":1,"250":1}}],["abstol",{"2":{"220":13}}],["abstol=1",{"2":{"215":1}}],["abstraction",{"2":{"90":1}}],["abstractions",{"2":{"20":1}}],["abstractexplicitcontainerlayer",{"2":{"87":2,"88":1}}],["abstractexplicitlayer",{"2":{"87":1}}],["abstractrecurrentcell",{"2":{"66":2}}],["abstractrng=utils",{"2":{"24":8,"25":24}}],["abstractrng",{"2":{"7":2,"8":2,"9":1,"10":1,"17":4,"24":3,"135":2,"140":2,"225":1,"232":1}}],["abstracttimeseriesdatabatchordering=batchlastindex",{"2":{"66":2}}],["abstractadtype",{"2":{"27":2,"28":1,"52":1}}],["abstractarrays",{"2":{"49":1}}],["abstractarray",{"2":{"13":2,"16":2,"19":1,"20":1,"24":16,"25":60,"28":1,"54":1,"55":1,"64":3,"67":6,"68":6,"69":1,"113":1,"140":2,"144":3,"148":1,"184":1,"185":1,"208":1,"209":1,"242":3}}],["abstractmatrix",{"2":{"18":2,"68":1,"135":1,"136":1,"214":1}}],["abstractvector",{"2":{"16":1,"18":1,"68":1,"249":1,"254":2}}],["abstractluxdistributedbacked",{"2":{"40":1}}],["abstractluxdistributedbackend",{"2":{"37":3,"38":2,"39":7,"41":1}}],["abstractluxwrapperlayer",{"2":{"7":4,"44":1,"87":1,"88":2,"111":1,"136":1,"214":1,"218":1}}],["abstractluxcontainerlayer",{"2":{"7":5,"87":1,"136":3,"184":2,"241":1}}],["abstractluxlayers",{"2":{"62":1}}],["abstractluxlayer",{"2":{"7":5,"8":2,"34":3,"43":4,"44":3,"45":1,"58":1,"62":4,"69":1,"87":1,"109":2,"112":3,"113":2,"135":2,"140":1,"214":3,"218":2,"225":2}}],["abstract",{"0":{"7":1},"2":{"3":2,"6":1,"7":3,"67":4,"208":1}}],["abstractdevice",{"2":{"2":1,"3":9,"4":4,"5":1}}],["abstractgpudevice",{"2":{"1":1,"132":1}}],["absolute",{"2":{"53":1,"155":1}}],["absolutely",{"2":{"52":1}}],["abs2",{"2":{"49":1,"53":1,"59":3,"145":2,"146":2,"147":2,"148":1,"242":3,"249":1}}],["abs",{"2":{"24":1,"68":1,"245":2,"254":1}}],["amazing",{"2":{"203":1}}],["am",{"2":{"147":1,"256":1}}],["ambiguous",{"2":{"55":1,"87":1}}],["amp",{"0":{"27":1,"154":1,"170":1},"1":{"155":1,"156":1,"157":1,"158":1,"159":1,"160":1,"171":1,"172":1,"173":1,"174":1,"175":1,"176":1,"177":1,"178":1,"179":1,"180":1},"2":{"27":2,"28":1,"52":1,"63":4,"66":1,"67":1,"68":1,"69":1,"98":2}}],["amd",{"2":{"3":1,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["amdgpudevice",{"2":{"4":2,"123":1,"132":2,"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["amdgpu",{"2":{"2":2,"3":1,"22":2,"37":1,"72":2,"77":1,"78":1,"80":2,"104":1,"124":1,"130":2,"180":2,"189":2,"197":2,"210":2,"221":2,"229":2,"237":2,"246":2,"251":2,"260":2}}],["average",{"2":{"250":3}}],["averages",{"2":{"39":2,"40":1}}],["avik",{"2":{"74":2}}],["avg",{"2":{"39":2}}],["avoids",{"2":{"218":1}}],["avoiding",{"2":{"146":1}}],["avoid",{"2":{"17":1,"66":1,"100":1,"123":1,"218":1,"220":1,"249":2}}],["available",{"2":{"13":2,"16":1,"18":1,"35":1,"41":1,"76":1,"79":1,"101":1,"106":1,"131":1,"132":1,"143":1,"159":1,"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["aka",{"2":{"16":1,"18":1}}],["academic",{"2":{"74":1}}],["achieved",{"2":{"63":1,"146":1}}],["achieve",{"2":{"24":1,"66":1}}],["acc",{"2":{"187":2,"195":8,"196":2,"217":4,"228":13}}],["accumulated",{"2":{"208":1}}],["accumulates",{"2":{"69":2}}],["accuracy",{"0":{"186":1},"2":{"13":1,"186":2,"187":102,"194":2,"195":4,"196":40,"216":2,"217":94,"219":18,"227":2,"228":212}}],["account",{"2":{"59":1}}],["according",{"2":{"24":1}}],["accordingly",{"2":{"19":2,"69":2}}],["accidental",{"2":{"57":1}}],["accelerate",{"2":{"167":1}}],["accelerating",{"2":{"19":1}}],["acceptance",{"2":{"249":1}}],["accept",{"2":{"137":1,"249":1}}],["accepts",{"2":{"70":1}}],["access",{"2":{"59":1,"62":2,"72":1,"73":1,"135":1,"136":1,"137":1}}],["accessors",{"2":{"46":1}}],["accessing",{"2":{"32":1,"34":1,"44":1}}],["acts",{"2":{"249":1}}],["action",{"2":{"60":1}}],["activates",{"2":{"166":1}}],["activated",{"2":{"66":4}}],["activate",{"2":{"49":1}}],["activation=tanh",{"2":{"66":1}}],["activation=identity",{"2":{"63":2,"67":4,"69":4}}],["activations",{"2":{"13":1,"16":1,"249":1}}],["activation",{"0":{"13":1,"15":1},"2":{"13":7,"15":12,"16":4,"18":3,"19":5,"59":1,"63":6,"66":3,"67":10,"69":12,"94":3,"96":3,"158":7,"166":1,"249":1,"256":3}}],["activity",{"2":{"49":2}}],["actively",{"2":{"85":1,"106":1}}],["active",{"2":{"32":2}}],["act=relu",{"2":{"59":2,"78":1}}],["actually",{"2":{"87":1,"133":1}}],["actual",{"2":{"45":1,"176":2,"236":1}}],["act",{"2":{"13":2,"19":2,"59":8,"78":5,"241":4}}],["across",{"2":{"0":1,"19":2,"39":2,"40":1,"41":1,"99":2,"120":3,"171":2,"250":1}}],["adjust",{"2":{"244":1}}],["adjoint",{"2":{"208":2}}],["adam",{"2":{"59":1,"77":15,"78":1,"107":1,"187":1,"195":1,"208":4,"217":1,"228":1,"234":3,"236":1,"244":1}}],["adapted",{"2":{"252":2}}],["adaptive=false",{"2":{"255":1,"256":1,"257":1,"259":1}}],["adaptivemeanpool",{"2":{"65":1}}],["adaptivemaxpool",{"2":{"65":1}}],["adaptive",{"2":{"65":3}}],["adaptivelppool",{"2":{"65":1,"97":1}}],["adaptor",{"2":{"34":4,"91":1,"193":1}}],["adapt",{"2":{"3":4,"32":6,"34":6}}],["ads",{"2":{"55":1}}],["adtype=autotracker",{"2":{"249":1}}],["adtypes",{"2":{"52":1,"144":1,"182":1,"191":1,"223":1,"231":1,"236":1,"239":1}}],["adtype",{"2":{"49":1,"258":1}}],["advances",{"2":{"17":1}}],["advancedvi",{"2":{"247":1}}],["advanced",{"0":{"201":1},"2":{"7":2,"106":1}}],["ad",{"0":{"28":1,"29":1},"2":{"16":1,"18":1,"27":6,"28":1,"29":1,"45":2,"52":3,"53":1,"55":2,"58":2,"77":2,"99":1,"100":3,"101":4,"102":2,"105":2,"144":4,"147":2,"154":1,"162":1,"170":1,"173":1,"175":2,"176":5,"177":2,"217":1,"236":1,"238":1,"240":2,"242":1,"245":1}}],["adding",{"2":{"124":1}}],["additional",{"0":{"79":1},"2":{"48":1,"49":1,"54":1,"79":1,"80":1,"139":1,"172":1,"177":1,"179":1}}],["additionally",{"2":{"7":1,"52":1,"53":1,"54":2,"101":1,"133":1,"140":1,"149":1,"153":1,"160":1,"188":1,"207":1,"242":1}}],["addition",{"2":{"15":1,"55":1}}],["address",{"2":{"98":1}}],["add",{"2":{"47":1,"55":3,"71":2,"72":5,"73":1,"76":2,"77":1,"79":1,"97":1,"104":1,"145":1,"161":3,"171":1,"172":1,"203":1}}],["added",{"2":{"15":1,"19":4,"53":2,"63":2,"65":3,"68":1,"69":4,"82":1,"97":1}}],["adds",{"2":{"14":1,"45":1,"171":1}}],["at=train",{"2":{"192":1,"213":1}}],["at=0",{"2":{"183":1}}],["atleast",{"2":{"19":1}}],["attempts",{"2":{"14":1,"18":1,"32":1}}],["at",{"2":{"4":2,"17":1,"39":2,"43":1,"53":1,"59":1,"63":2,"65":3,"66":1,"70":1,"77":1,"106":1,"116":4,"117":10,"124":1,"125":1,"130":1,"137":1,"166":2,"177":1,"184":1,"203":2,"244":4,"250":2,"254":2,"255":1}}],["april",{"2":{"74":1}}],["append",{"2":{"248":2}}],["appendix",{"0":{"180":1,"189":1,"197":1,"210":1,"221":1,"229":1,"237":1,"246":1,"251":1,"260":1}}],["appreciate",{"2":{"59":1}}],["approach",{"0":{"131":1},"2":{"106":1,"208":1}}],["approximation",{"2":{"68":1}}],["approximately",{"2":{"24":1}}],["approx",{"2":{"49":1}}],["appropriate",{"2":{"8":1,"24":1}}],["applications",{"2":{"100":1,"173":1}}],["applicable",{"2":{"2":1,"4":1}}],["applies",{"2":{"15":1,"19":1,"62":1,"69":6,"70":2,"120":1}}],["applied",{"2":{"8":1,"24":3,"53":2,"55":1,"62":1,"64":4,"69":4,"140":2}}],["applychain",{"2":{"220":3}}],["applying",{"2":{"17":2,"24":1,"62":1}}],["apply",{"0":{"13":1},"2":{"8":7,"52":5,"55":1,"62":1,"63":2,"64":2,"65":3,"77":3,"109":1,"111":1,"112":4,"113":3,"135":1,"136":1,"153":1,"166":1,"236":1,"249":1}}],["appleaccelerate",{"2":{"18":1}}],["apple",{"2":{"3":1,"80":1}}],["api",{"0":{"23":1,"52":1,"107":1,"185":1},"1":{"24":1,"25":1},"2":{"3":1,"13":2,"14":1,"15":2,"16":1,"17":2,"18":1,"19":4,"32":1,"39":3,"52":2,"53":2,"62":9,"68":2,"77":3,"82":1,"84":2,"89":1,"106":2,"107":1,"125":1,"139":1,"179":1,"180":1,"181":1,"185":1,"189":1,"190":1,"204":1,"210":1,"214":1,"218":1,"220":1,"221":1,"229":1,"235":1,"237":1,"246":1,"256":2}}],["astroinformatics",{"2":{"252":1}}],["ask",{"2":{"171":3}}],["asymmetric",{"2":{"63":2,"65":3}}],["assert",{"2":{"174":1,"249":1,"254":5}}],["associated",{"2":{"98":1}}],["assume",{"2":{"106":1,"254":1}}],["assumed",{"2":{"49":1}}],["assumes",{"2":{"11":1,"62":1,"96":1}}],["assigned",{"2":{"59":1}}],["assign",{"2":{"37":1}}],["as",{"2":{"3":2,"4":2,"5":3,"7":4,"13":2,"15":2,"17":2,"19":3,"24":5,"27":2,"32":1,"34":1,"45":1,"46":1,"48":2,"49":2,"50":4,"52":2,"53":8,"54":2,"55":1,"57":2,"59":10,"62":10,"63":1,"65":3,"66":3,"68":3,"69":1,"70":1,"71":1,"74":1,"78":2,"82":1,"94":2,"95":2,"100":2,"103":1,"106":6,"107":1,"110":1,"117":1,"120":1,"121":1,"123":2,"136":3,"137":2,"145":1,"146":1,"147":2,"148":1,"152":1,"153":2,"155":2,"165":1,"169":2,"170":1,"171":1,"173":1,"174":2,"176":1,"185":1,"190":2,"196":1,"204":1,"208":1,"214":1,"220":1,"225":1,"247":1,"249":6,"250":1,"256":2}}],["agent",{"2":{"116":1,"220":3,"244":4}}],["aggressive",{"2":{"90":1}}],["aggregation",{"2":{"53":1}}],["aggregated",{"2":{"53":1}}],["agg=sum",{"2":{"53":2}}],["agg=mean",{"2":{"53":1}}],["agg",{"2":{"53":26}}],["agnostic",{"2":{"3":3,"39":3,"160":1}}],["against",{"2":{"104":1,"149":1}}],["again",{"2":{"3":1,"120":1,"146":1}}],["algebras",{"2":{"177":1}}],["algebraic",{"2":{"146":1}}],["algorithm",{"2":{"2":1,"149":1,"247":1}}],["aligned",{"2":{"70":1}}],["align",{"2":{"70":2,"96":2,"97":1}}],["alias",{"2":{"54":1}}],["aliased",{"2":{"52":3}}],["aliases",{"2":{"15":1}}],["alternate",{"0":{"218":1}}],["alternatives",{"2":{"158":1}}],["alternatively",{"2":{"49":1,"66":2,"70":1,"76":1,"120":1,"155":1}}],["alternative",{"2":{"3":1}}],["alts",{"2":{"68":2}}],["already",{"2":{"60":1,"77":1,"98":1,"174":1,"235":1}}],["almost",{"2":{"32":1}}],["always",{"2":{"27":2,"32":1,"42":1,"43":1,"53":1,"55":1,"96":1,"138":1}}],["al",{"2":{"17":2,"24":2,"53":2,"252":1}}],["alpha=0",{"2":{"232":1,"236":2,"255":2,"256":4,"259":7}}],["alphadropout",{"2":{"64":4}}],["alpha",{"2":{"17":4,"249":2}}],["along",{"2":{"15":1,"19":1,"24":1,"53":1,"54":1,"62":1,"63":1,"64":2,"66":2,"69":1,"100":1,"111":1}}],["also",{"2":{"3":1,"7":1,"15":2,"24":2,"37":2,"43":1,"54":1,"55":2,"57":1,"59":1,"62":2,"64":5,"68":1,"69":7,"70":2,"76":1,"82":1,"90":1,"100":2,"135":2,"168":1,"171":1,"185":1,"187":1,"214":1,"217":2,"249":1,"250":1}}],["allocate",{"2":{"153":1}}],["allow",{"2":{"46":1,"49":1,"55":1,"90":1,"94":1,"155":1}}],["allowscalar",{"2":{"156":1,"212":1,"223":1,"239":1}}],["allows",{"2":{"6":1,"7":1,"8":1,"27":2,"39":2,"55":1,"59":1,"62":1,"66":1,"68":1,"81":1,"97":1,"136":1,"137":1,"217":1}}],["allreduce",{"2":{"39":3,"40":1,"122":1}}],["all",{"2":{"3":1,"6":1,"7":1,"8":1,"10":3,"24":1,"34":2,"39":5,"42":1,"43":3,"44":2,"48":2,"49":1,"52":2,"55":1,"56":1,"58":1,"62":3,"63":2,"65":6,"67":1,"68":2,"70":2,"77":1,"79":2,"90":1,"92":1,"99":3,"100":3,"106":1,"120":3,"123":1,"128":2,"135":1,"160":1,"166":1,"167":1,"169":1,"171":3,"172":1,"173":1,"207":1,"241":1,"248":1,"249":2}}],["artifact",{"2":{"180":1,"189":1,"210":1,"221":1,"229":1,"237":1,"246":1}}],["artificially",{"2":{"117":1}}],["artificial",{"2":{"24":2,"248":2}}],["arranged",{"2":{"171":1,"248":1}}],["arrayandtime",{"2":{"113":8,"114":1}}],["array",{"2":{"8":1,"13":5,"17":4,"19":5,"20":1,"22":3,"24":9,"28":2,"34":7,"53":1,"55":2,"59":1,"63":5,"65":6,"66":2,"67":4,"68":6,"69":16,"106":1,"107":2,"113":3,"146":1,"147":1,"169":1,"171":8,"172":1,"173":6,"194":1,"206":1,"208":1,"214":3,"215":1,"220":9,"248":4,"249":1,"255":1,"256":1,"257":1,"259":1}}],["arrays",{"0":{"156":1,"171":1,"172":1},"1":{"172":1},"2":{"3":1,"11":1,"16":2,"18":3,"20":7,"55":1,"56":1,"62":1,"67":1,"68":1,"70":5,"106":1,"135":1,"140":2,"153":2,"171":4,"173":1}}],["architecture",{"2":{"135":2,"140":1}}],["architectures",{"2":{"100":1}}],["architectural",{"2":{"135":1}}],["arbitrary",{"2":{"70":1,"217":1}}],["args",{"2":{"48":2,"49":4,"55":2,"57":6,"95":1}}],["argumenterror",{"2":{"244":1}}],["argument",{"2":{"8":2,"25":1,"59":1,"62":4,"66":1,"68":1,"87":1,"96":1,"97":1,"127":1}}],["arguments",{"2":{"2":2,"4":2,"8":1,"13":2,"15":1,"16":1,"17":2,"18":1,"19":4,"24":3,"27":2,"28":1,"32":2,"34":2,"40":1,"43":1,"45":3,"46":1,"48":3,"49":5,"52":3,"58":1,"59":2,"62":11,"63":4,"64":5,"65":9,"66":7,"67":8,"68":7,"69":9,"70":2,"84":1,"135":1,"169":1,"220":3}}],["arxiv",{"2":{"19":4,"24":1,"68":1,"69":2}}],["around",{"2":{"14":1,"27":2,"32":1,"63":2,"65":3,"100":1,"136":1,"148":1}}],["aren",{"2":{"202":1}}],["are",{"2":{"2":1,"3":2,"6":1,"7":2,"8":1,"11":1,"13":1,"18":1,"19":3,"20":1,"24":4,"28":2,"34":2,"35":1,"37":2,"42":2,"43":6,"45":4,"46":1,"49":2,"52":6,"53":6,"54":2,"55":1,"57":2,"59":2,"60":1,"67":3,"68":2,"69":5,"70":2,"76":1,"78":1,"79":1,"87":1,"95":1,"97":1,"99":2,"100":2,"103":3,"104":5,"105":2,"106":2,"107":1,"112":1,"115":1,"117":1,"120":1,"123":1,"124":1,"126":1,"130":1,"135":3,"136":2,"137":2,"138":1,"144":2,"145":2,"146":1,"149":1,"152":3,"153":1,"155":2,"158":1,"159":2,"160":1,"163":2,"164":1,"166":1,"167":4,"171":1,"173":1,"175":1,"184":2,"202":2,"203":2,"208":1,"214":1,"225":1,"249":1,"250":1,"255":1,"256":1}}],["animation",{"2":{"250":1}}],["analytical",{"2":{"243":4,"245":1}}],["anticlockwise",{"2":{"181":1,"183":4}}],["anonymous",{"2":{"94":1}}],["another",{"2":{"67":1,"136":1}}],["answers",{"2":{"81":1}}],["anything",{"2":{"135":1}}],["anywhere",{"2":{"87":1}}],["any",{"2":{"8":2,"11":1,"16":1,"18":1,"34":1,"42":1,"45":1,"49":1,"52":2,"53":3,"55":2,"59":2,"62":1,"66":3,"68":1,"90":2,"99":1,"105":1,"106":3,"107":1,"112":7,"116":1,"118":1,"123":1,"133":2,"135":2,"137":1,"140":1,"144":1,"153":1,"167":1,"172":1,"173":2,"174":1,"175":1,"188":1,"203":1,"217":1,"220":15,"226":1,"235":1}}],["an",{"2":{"2":5,"3":2,"4":1,"5":1,"8":3,"13":1,"17":4,"19":3,"24":19,"25":36,"27":2,"36":2,"39":1,"40":1,"45":1,"50":3,"52":2,"53":2,"55":1,"57":1,"59":1,"62":10,"63":3,"64":3,"65":3,"66":2,"67":8,"68":2,"69":7,"77":1,"87":1,"94":1,"96":2,"97":1,"100":1,"101":1,"103":1,"105":1,"106":1,"107":1,"110":1,"115":1,"117":1,"118":2,"124":1,"131":1,"144":1,"145":2,"146":1,"148":1,"155":1,"159":1,"161":2,"171":3,"174":1,"190":1,"202":1,"203":3,"208":1,"248":1,"249":1,"250":1}}],["andrea",{"2":{"19":1,"69":1}}],["and",{"0":{"30":1,"160":1,"186":1,"215":1,"222":1,"226":1},"1":{"31":1,"32":1,"33":1,"34":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1},"2":{"1":1,"2":10,"3":3,"4":7,"5":4,"7":7,"8":7,"13":1,"14":1,"15":1,"16":4,"17":2,"18":3,"19":20,"24":13,"28":1,"32":5,"34":2,"37":4,"39":2,"41":2,"42":2,"43":4,"44":7,"45":5,"46":3,"47":2,"48":2,"50":2,"52":8,"53":19,"54":2,"55":6,"56":1,"57":4,"58":1,"59":5,"60":1,"62":10,"63":9,"65":3,"66":31,"67":8,"68":3,"69":22,"70":8,"76":1,"77":4,"78":5,"81":2,"87":5,"92":2,"94":5,"95":3,"96":3,"97":4,"98":3,"99":2,"100":6,"103":4,"104":2,"106":6,"107":2,"114":1,"116":3,"117":6,"118":3,"120":6,"121":2,"123":1,"124":1,"126":1,"130":1,"131":3,"132":1,"133":2,"135":9,"136":8,"137":3,"138":1,"140":10,"142":1,"144":2,"145":5,"147":1,"148":1,"149":2,"150":2,"152":1,"153":5,"155":3,"158":2,"159":1,"160":5,"164":2,"166":2,"167":4,"169":2,"171":7,"172":2,"173":1,"174":3,"175":3,"176":1,"178":1,"179":3,"181":2,"183":3,"184":10,"185":1,"188":3,"192":1,"193":1,"196":1,"202":1,"203":3,"208":3,"213":1,"214":2,"217":2,"220":1,"232":1,"235":2,"236":1,"238":1,"241":1,"243":2,"248":1,"249":12,"250":3,"254":3,"255":2,"256":6}}],["a",{"0":{"33":1,"126":1,"129":1,"181":1,"184":1,"222":1,"225":1,"230":1,"238":1,"252":1,"256":1},"1":{"34":1,"182":1,"183":1,"184":1,"185":1,"186":1,"187":1,"188":1,"189":1,"223":1,"224":1,"225":1,"226":1,"227":1,"228":1,"229":1,"231":1,"232":1,"233":1,"234":1,"235":1,"236":1,"237":1,"239":1,"240":1,"241":1,"242":1,"243":1,"244":1,"245":1,"246":1,"253":1,"254":1,"255":1,"256":1,"257":1,"258":1,"259":1,"260":1},"2":{"0":1,"1":2,"2":5,"3":5,"4":5,"5":3,"6":1,"7":10,"8":12,"10":1,"11":4,"14":1,"15":3,"16":2,"17":4,"18":2,"19":2,"21":1,"24":20,"25":13,"27":2,"28":4,"32":9,"34":9,"37":2,"39":3,"40":1,"42":1,"43":4,"44":1,"45":3,"46":4,"47":2,"48":1,"49":4,"50":5,"52":5,"53":4,"54":5,"55":5,"57":3,"58":4,"59":21,"62":27,"63":21,"65":18,"66":43,"67":8,"68":10,"69":20,"70":6,"72":1,"73":1,"76":1,"77":2,"78":3,"81":1,"82":2,"87":9,"88":2,"89":2,"92":3,"98":2,"99":1,"100":4,"106":4,"107":3,"109":2,"110":1,"113":3,"114":1,"115":1,"116":3,"117":5,"119":2,"120":1,"123":1,"124":1,"125":1,"126":1,"128":2,"130":1,"131":7,"133":3,"135":8,"136":8,"137":2,"140":17,"142":1,"143":1,"144":6,"145":7,"146":2,"147":3,"148":1,"149":10,"152":1,"153":4,"155":5,"160":1,"163":1,"164":1,"168":2,"169":2,"170":3,"171":12,"172":1,"174":3,"175":12,"176":1,"177":2,"179":4,"181":1,"183":2,"184":3,"185":1,"190":2,"196":1,"202":1,"204":3,"207":1,"208":6,"214":3,"217":2,"218":1,"219":1,"226":1,"230":2,"233":1,"236":2,"238":2,"241":3,"242":1,"243":1,"247":1,"248":3,"249":7,"250":5,"254":9,"256":3,"257":1,"258":1}}]],"serializationVersion":2}';export{e as default}; diff --git a/previews/PR1000/assets/chunks/VPLocalSearchBox.DFDiEjIp.js b/previews/PR1000/assets/chunks/VPLocalSearchBox.D7LKkx_r.js similarity index 99% rename from previews/PR1000/assets/chunks/VPLocalSearchBox.DFDiEjIp.js rename to previews/PR1000/assets/chunks/VPLocalSearchBox.D7LKkx_r.js index 4b34fb8a07..c5a41b8b8a 100644 --- a/previews/PR1000/assets/chunks/VPLocalSearchBox.DFDiEjIp.js +++ b/previews/PR1000/assets/chunks/VPLocalSearchBox.D7LKkx_r.js @@ -1,4 +1,4 @@ -var Ft=Object.defineProperty;var Ot=(a,e,t)=>e in a?Ft(a,e,{enumerable:!0,configurable:!0,writable:!0,value:t}):a[e]=t;var Me=(a,e,t)=>Ot(a,typeof e!="symbol"?e+"":e,t);import{V as Rt,p as ie,h as me,aj as et,ak as Ct,al as Mt,q as $e,am as At,d as Lt,D as xe,an as tt,ao as Dt,ap as zt,s as Pt,aq as jt,v as Ae,P as he,O as Se,ar as Vt,as as $t,W as Bt,R as Wt,$ as Kt,o as H,b as Jt,j as S,a0 as Ut,k as L,at as qt,au as Gt,av as Ht,c as Z,n as st,e as _e,C as nt,F as it,a as fe,t as pe,aw as Qt,ax as rt,ay as Yt,a9 as Zt,af as Xt,az as es,_ as ts}from"./framework.DjZDIZsN.js";import{u as ss,d as ns}from"./theme.DRSgyssM.js";const is={root:()=>Rt(()=>import("./@localSearchIndexroot.CXVX33RZ.js"),[])};/*! +var Ft=Object.defineProperty;var Ot=(a,e,t)=>e in a?Ft(a,e,{enumerable:!0,configurable:!0,writable:!0,value:t}):a[e]=t;var Me=(a,e,t)=>Ot(a,typeof e!="symbol"?e+"":e,t);import{V as Rt,p as ie,h as me,aj as et,ak as Ct,al as Mt,q as $e,am as At,d as Lt,D as xe,an as tt,ao as Dt,ap as zt,s as Pt,aq as jt,v as Ae,P as he,O as Se,ar as Vt,as as $t,W as Bt,R as Wt,$ as Kt,o as H,b as Jt,j as S,a0 as Ut,k as L,at as qt,au as Gt,av as Ht,c as Z,n as st,e as _e,C as nt,F as it,a as fe,t as pe,aw as Qt,ax as rt,ay as Yt,a9 as Zt,af as Xt,az as es,_ as ts}from"./framework.DjZDIZsN.js";import{u as ss,d as ns}from"./theme.BD8xVYwz.js";const is={root:()=>Rt(()=>import("./@localSearchIndexroot.1OabY8x9.js"),[])};/*! * tabbable 6.2.0 * @license MIT, https://github.com/focus-trap/tabbable/blob/master/LICENSE */var mt=["input:not([inert])","select:not([inert])","textarea:not([inert])","a[href]:not([inert])","button:not([inert])","[tabindex]:not(slot):not([inert])","audio[controls]:not([inert])","video[controls]:not([inert])",'[contenteditable]:not([contenteditable="false"]):not([inert])',"details>summary:first-of-type:not([inert])","details:not([inert])"],Ne=mt.join(","),gt=typeof Element>"u",ae=gt?function(){}:Element.prototype.matches||Element.prototype.msMatchesSelector||Element.prototype.webkitMatchesSelector,Fe=!gt&&Element.prototype.getRootNode?function(a){var e;return a==null||(e=a.getRootNode)===null||e===void 0?void 0:e.call(a)}:function(a){return a==null?void 0:a.ownerDocument},Oe=function a(e,t){var s;t===void 0&&(t=!0);var n=e==null||(s=e.getAttribute)===null||s===void 0?void 0:s.call(e,"inert"),r=n===""||n==="true",i=r||t&&e&&a(e.parentNode);return i},rs=function(e){var t,s=e==null||(t=e.getAttribute)===null||t===void 0?void 0:t.call(e,"contenteditable");return s===""||s==="true"},bt=function(e,t,s){if(Oe(e))return[];var n=Array.prototype.slice.apply(e.querySelectorAll(Ne));return t&&ae.call(e,Ne)&&n.unshift(e),n=n.filter(s),n},yt=function a(e,t,s){for(var n=[],r=Array.from(e);r.length;){var i=r.shift();if(!Oe(i,!1))if(i.tagName==="SLOT"){var o=i.assignedElements(),l=o.length?o:i.children,c=a(l,!0,s);s.flatten?n.push.apply(n,c):n.push({scopeParent:i,candidates:c})}else{var h=ae.call(i,Ne);h&&s.filter(i)&&(t||!e.includes(i))&&n.push(i);var m=i.shadowRoot||typeof s.getShadowRoot=="function"&&s.getShadowRoot(i),f=!Oe(m,!1)&&(!s.shadowRootFilter||s.shadowRootFilter(i));if(m&&f){var b=a(m===!0?i.children:m.children,!0,s);s.flatten?n.push.apply(n,b):n.push({scopeParent:i,candidates:b})}else r.unshift.apply(r,i.children)}}return n},wt=function(e){return!isNaN(parseInt(e.getAttribute("tabindex"),10))},re=function(e){if(!e)throw new Error("No node provided");return e.tabIndex<0&&(/^(AUDIO|VIDEO|DETAILS)$/.test(e.tagName)||rs(e))&&!wt(e)?0:e.tabIndex},as=function(e,t){var s=re(e);return s<0&&t&&!wt(e)?0:s},os=function(e,t){return e.tabIndex===t.tabIndex?e.documentOrder-t.documentOrder:e.tabIndex-t.tabIndex},xt=function(e){return e.tagName==="INPUT"},ls=function(e){return xt(e)&&e.type==="hidden"},cs=function(e){var t=e.tagName==="DETAILS"&&Array.prototype.slice.apply(e.children).some(function(s){return s.tagName==="SUMMARY"});return t},us=function(e,t){for(var s=0;ssummary:first-of-type"),i=r?e.parentElement:e;if(ae.call(i,"details:not([open]) *"))return!0;if(!s||s==="full"||s==="legacy-full"){if(typeof n=="function"){for(var o=e;e;){var l=e.parentElement,c=Fe(e);if(l&&!l.shadowRoot&&n(l)===!0)return at(e);e.assignedSlot?e=e.assignedSlot:!l&&c!==e.ownerDocument?e=c.host:e=l}e=o}if(ps(e))return!e.getClientRects().length;if(s!=="legacy-full")return!0}else if(s==="non-zero-area")return at(e);return!1},ms=function(e){if(/^(INPUT|BUTTON|SELECT|TEXTAREA)$/.test(e.tagName))for(var t=e.parentElement;t;){if(t.tagName==="FIELDSET"&&t.disabled){for(var s=0;s=0)},bs=function a(e){var t=[],s=[];return e.forEach(function(n,r){var i=!!n.scopeParent,o=i?n.scopeParent:n,l=as(o,i),c=i?a(n.candidates):o;l===0?i?t.push.apply(t,c):t.push(o):s.push({documentOrder:r,tabIndex:l,item:n,isScope:i,content:c})}),s.sort(os).reduce(function(n,r){return r.isScope?n.push.apply(n,r.content):n.push(r.content),n},[]).concat(t)},ys=function(e,t){t=t||{};var s;return t.getShadowRoot?s=yt([e],t.includeContainer,{filter:Be.bind(null,t),flatten:!1,getShadowRoot:t.getShadowRoot,shadowRootFilter:gs}):s=bt(e,t.includeContainer,Be.bind(null,t)),bs(s)},ws=function(e,t){t=t||{};var s;return t.getShadowRoot?s=yt([e],t.includeContainer,{filter:Re.bind(null,t),flatten:!0,getShadowRoot:t.getShadowRoot}):s=bt(e,t.includeContainer,Re.bind(null,t)),s},oe=function(e,t){if(t=t||{},!e)throw new Error("No node provided");return ae.call(e,Ne)===!1?!1:Be(t,e)},xs=mt.concat("iframe").join(","),Le=function(e,t){if(t=t||{},!e)throw new Error("No node provided");return ae.call(e,xs)===!1?!1:Re(t,e)};/*! diff --git a/previews/PR1000/assets/chunks/theme.DRSgyssM.js b/previews/PR1000/assets/chunks/theme.BD8xVYwz.js similarity index 99% rename from previews/PR1000/assets/chunks/theme.DRSgyssM.js rename to previews/PR1000/assets/chunks/theme.BD8xVYwz.js index 88d12a9682..1c2a652d2b 100644 --- a/previews/PR1000/assets/chunks/theme.DRSgyssM.js +++ b/previews/PR1000/assets/chunks/theme.BD8xVYwz.js @@ -1,2 +1,2 @@ -const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=["assets/chunks/VPLocalSearchBox.DFDiEjIp.js","assets/chunks/framework.DjZDIZsN.js"])))=>i.map(i=>d[i]); -import{d as b,o as a,c as l,r as u,a as F,t as w,n as I,b as $,w as f,e as m,T as pe,_ as k,u as Le,i as Ke,f as qe,g as ve,h as y,j as p,k as i,l as G,m as ie,p as T,q as D,s as Z,v as R,x as fe,y as me,z as We,A as Je,B as K,F as M,C as B,D as Te,E as x,G as g,H as O,I as we,J as ee,K as j,L as W,M as Ye,N as Ne,O as le,P as Ie,Q as Me,R as te,S as Qe,U as Xe,V as Ze,W as Ce,X as he,Y as xe,Z as et,$ as tt,a0 as nt,a1 as Ae,a2 as st,a3 as ot,a4 as at,a5 as ye}from"./framework.DjZDIZsN.js";const rt=b({__name:"VPBadge",props:{text:{},type:{default:"tip"}},setup(o){return(e,t)=>(a(),l("span",{class:I(["VPBadge",e.type])},[u(e.$slots,"default",{},()=>[F(w(e.text),1)])],2))}}),it={key:0,class:"VPBackdrop"},lt=b({__name:"VPBackdrop",props:{show:{type:Boolean}},setup(o){return(e,t)=>(a(),$(pe,{name:"fade"},{default:f(()=>[e.show?(a(),l("div",it)):m("",!0)]),_:1}))}}),ct=k(lt,[["__scopeId","data-v-b06cdb19"]]),L=Le;function ut(o,e){let t,s=!1;return()=>{t&&clearTimeout(t),s?t=setTimeout(o,e):(o(),(s=!0)&&setTimeout(()=>s=!1,e))}}function ce(o){return/^\//.test(o)?o:`/${o}`}function _e(o){const{pathname:e,search:t,hash:s,protocol:n}=new URL(o,"http://a.com");if(Ke(o)||o.startsWith("#")||!n.startsWith("http")||!qe(e))return o;const{site:r}=L(),c=e.endsWith("/")||e.endsWith(".html")?o:o.replace(/(?:(^\.+)\/)?.*$/,`$1${e.replace(/(\.md)?$/,r.value.cleanUrls?"":".html")}${t}${s}`);return ve(c)}function Y({correspondingLink:o=!1}={}){const{site:e,localeIndex:t,page:s,theme:n,hash:r}=L(),c=y(()=>{var d,h;return{label:(d=e.value.locales[t.value])==null?void 0:d.label,link:((h=e.value.locales[t.value])==null?void 0:h.link)||(t.value==="root"?"/":`/${t.value}/`)}});return{localeLinks:y(()=>Object.entries(e.value.locales).flatMap(([d,h])=>c.value.label===h.label?[]:{text:h.label,link:dt(h.link||(d==="root"?"/":`/${d}/`),n.value.i18nRouting!==!1&&o,s.value.relativePath.slice(c.value.link.length-1),!e.value.cleanUrls)+r.value})),currentLang:c}}function dt(o,e,t,s){return e?o.replace(/\/$/,"")+ce(t.replace(/(^|\/)index\.md$/,"$1").replace(/\.md$/,s?".html":"")):o}const pt={class:"NotFound"},vt={class:"code"},ft={class:"title"},mt={class:"quote"},ht={class:"action"},_t=["href","aria-label"],bt=b({__name:"NotFound",setup(o){const{theme:e}=L(),{currentLang:t}=Y();return(s,n)=>{var r,c,v,d,h;return a(),l("div",pt,[p("p",vt,w(((r=i(e).notFound)==null?void 0:r.code)??"404"),1),p("h1",ft,w(((c=i(e).notFound)==null?void 0:c.title)??"PAGE NOT FOUND"),1),n[0]||(n[0]=p("div",{class:"divider"},null,-1)),p("blockquote",mt,w(((v=i(e).notFound)==null?void 0:v.quote)??"But if you don't change your direction, and if you keep looking, you may end up where you are heading."),1),p("div",ht,[p("a",{class:"link",href:i(ve)(i(t).link),"aria-label":((d=i(e).notFound)==null?void 0:d.linkLabel)??"go to home"},w(((h=i(e).notFound)==null?void 0:h.linkText)??"Take me home"),9,_t)])])}}}),kt=k(bt,[["__scopeId","data-v-951cab6c"]]);function Be(o,e){if(Array.isArray(o))return Q(o);if(o==null)return[];e=ce(e);const t=Object.keys(o).sort((n,r)=>r.split("/").length-n.split("/").length).find(n=>e.startsWith(ce(n))),s=t?o[t]:[];return Array.isArray(s)?Q(s):Q(s.items,s.base)}function gt(o){const e=[];let t=0;for(const s in o){const n=o[s];if(n.items){t=e.push(n);continue}e[t]||e.push({items:[]}),e[t].items.push(n)}return e}function $t(o){const e=[];function t(s){for(const n of s)n.text&&n.link&&e.push({text:n.text,link:n.link,docFooterText:n.docFooterText}),n.items&&t(n.items)}return t(o),e}function ue(o,e){return Array.isArray(e)?e.some(t=>ue(o,t)):G(o,e.link)?!0:e.items?ue(o,e.items):!1}function Q(o,e){return[...o].map(t=>{const s={...t},n=s.base||e;return n&&s.link&&(s.link=n+s.link),s.items&&(s.items=Q(s.items,n)),s})}function U(){const{frontmatter:o,page:e,theme:t}=L(),s=ie("(min-width: 960px)"),n=T(!1),r=y(()=>{const A=t.value.sidebar,N=e.value.relativePath;return A?Be(A,N):[]}),c=T(r.value);D(r,(A,N)=>{JSON.stringify(A)!==JSON.stringify(N)&&(c.value=r.value)});const v=y(()=>o.value.sidebar!==!1&&c.value.length>0&&o.value.layout!=="home"),d=y(()=>h?o.value.aside==null?t.value.aside==="left":o.value.aside==="left":!1),h=y(()=>o.value.layout==="home"?!1:o.value.aside!=null?!!o.value.aside:t.value.aside!==!1),P=y(()=>v.value&&s.value),_=y(()=>v.value?gt(c.value):[]);function V(){n.value=!0}function S(){n.value=!1}function C(){n.value?S():V()}return{isOpen:n,sidebar:c,sidebarGroups:_,hasSidebar:v,hasAside:h,leftAside:d,isSidebarEnabled:P,open:V,close:S,toggle:C}}function yt(o,e){let t;Z(()=>{t=o.value?document.activeElement:void 0}),R(()=>{window.addEventListener("keyup",s)}),fe(()=>{window.removeEventListener("keyup",s)});function s(n){n.key==="Escape"&&o.value&&(e(),t==null||t.focus())}}function Pt(o){const{page:e,hash:t}=L(),s=T(!1),n=y(()=>o.value.collapsed!=null),r=y(()=>!!o.value.link),c=T(!1),v=()=>{c.value=G(e.value.relativePath,o.value.link)};D([e,o,t],v),R(v);const d=y(()=>c.value?!0:o.value.items?ue(e.value.relativePath,o.value.items):!1),h=y(()=>!!(o.value.items&&o.value.items.length));Z(()=>{s.value=!!(n.value&&o.value.collapsed)}),me(()=>{(c.value||d.value)&&(s.value=!1)});function P(){n.value&&(s.value=!s.value)}return{collapsed:s,collapsible:n,isLink:r,isActiveLink:c,hasActiveLink:d,hasChildren:h,toggle:P}}function Vt(){const{hasSidebar:o}=U(),e=ie("(min-width: 960px)"),t=ie("(min-width: 1280px)");return{isAsideEnabled:y(()=>!t.value&&!e.value?!1:o.value?t.value:e.value)}}const de=[];function Ee(o){return typeof o.outline=="object"&&!Array.isArray(o.outline)&&o.outline.label||o.outlineTitle||"On this page"}function be(o){const e=[...document.querySelectorAll(".VPDoc :where(h1,h2,h3,h4,h5,h6)")].filter(t=>t.id&&t.hasChildNodes()).map(t=>{const s=Number(t.tagName[1]);return{element:t,title:St(t),link:"#"+t.id,level:s}});return Lt(e,o)}function St(o){let e="";for(const t of o.childNodes)if(t.nodeType===1){if(t.classList.contains("VPBadge")||t.classList.contains("header-anchor")||t.classList.contains("ignore-header"))continue;e+=t.textContent}else t.nodeType===3&&(e+=t.textContent);return e.trim()}function Lt(o,e){if(e===!1)return[];const t=(typeof e=="object"&&!Array.isArray(e)?e.level:e)||2,[s,n]=typeof t=="number"?[t,t]:t==="deep"?[2,6]:t;return Nt(o,s,n)}function Tt(o,e){const{isAsideEnabled:t}=Vt(),s=ut(r,100);let n=null;R(()=>{requestAnimationFrame(r),window.addEventListener("scroll",s)}),We(()=>{c(location.hash)}),fe(()=>{window.removeEventListener("scroll",s)});function r(){if(!t.value)return;const v=window.scrollY,d=window.innerHeight,h=document.body.offsetHeight,P=Math.abs(v+d-h)<1,_=de.map(({element:S,link:C})=>({link:C,top:wt(S)})).filter(({top:S})=>!Number.isNaN(S)).sort((S,C)=>S.top-C.top);if(!_.length){c(null);return}if(v<1){c(null);return}if(P){c(_[_.length-1].link);return}let V=null;for(const{link:S,top:C}of _){if(C>v+Je()+4)break;V=S}c(V)}function c(v){n&&n.classList.remove("active"),v==null?n=null:n=o.value.querySelector(`a[href="${decodeURIComponent(v)}"]`);const d=n;d?(d.classList.add("active"),e.value.style.top=d.offsetTop+39+"px",e.value.style.opacity="1"):(e.value.style.top="33px",e.value.style.opacity="0")}}function wt(o){let e=0;for(;o!==document.body;){if(o===null)return NaN;e+=o.offsetTop,o=o.offsetParent}return e}function Nt(o,e,t){de.length=0;const s=[],n=[];return o.forEach(r=>{const c={...r,children:[]};let v=n[n.length-1];for(;v&&v.level>=c.level;)n.pop(),v=n[n.length-1];if(c.element.classList.contains("ignore-header")||v&&"shouldIgnore"in v){n.push({level:c.level,shouldIgnore:!0});return}c.level>t||c.level{const n=K("VPDocOutlineItem",!0);return a(),l("ul",{class:I(["VPDocOutlineItem",t.root?"root":"nested"])},[(a(!0),l(M,null,B(t.headers,({children:r,link:c,title:v})=>(a(),l("li",null,[p("a",{class:"outline-link",href:c,onClick:e,title:v},w(v),9,It),r!=null&&r.length?(a(),$(n,{key:0,headers:r},null,8,["headers"])):m("",!0)]))),256))],2)}}}),He=k(Mt,[["__scopeId","data-v-3f927ebe"]]),Ct={class:"content"},At={"aria-level":"2",class:"outline-title",id:"doc-outline-aria-label",role:"heading"},Bt=b({__name:"VPDocAsideOutline",setup(o){const{frontmatter:e,theme:t}=L(),s=Te([]);x(()=>{s.value=be(e.value.outline??t.value.outline)});const n=T(),r=T();return Tt(n,r),(c,v)=>(a(),l("nav",{"aria-labelledby":"doc-outline-aria-label",class:I(["VPDocAsideOutline",{"has-outline":s.value.length>0}]),ref_key:"container",ref:n},[p("div",Ct,[p("div",{class:"outline-marker",ref_key:"marker",ref:r},null,512),p("div",At,w(i(Ee)(i(t))),1),g(He,{headers:s.value,root:!0},null,8,["headers"])])],2))}}),Et=k(Bt,[["__scopeId","data-v-b38bf2ff"]]),Ht={class:"VPDocAsideCarbonAds"},Ot=b({__name:"VPDocAsideCarbonAds",props:{carbonAds:{}},setup(o){const e=()=>null;return(t,s)=>(a(),l("div",Ht,[g(i(e),{"carbon-ads":t.carbonAds},null,8,["carbon-ads"])]))}}),Dt={class:"VPDocAside"},Ft=b({__name:"VPDocAside",setup(o){const{theme:e}=L();return(t,s)=>(a(),l("div",Dt,[u(t.$slots,"aside-top",{},void 0,!0),u(t.$slots,"aside-outline-before",{},void 0,!0),g(Et),u(t.$slots,"aside-outline-after",{},void 0,!0),s[0]||(s[0]=p("div",{class:"spacer"},null,-1)),u(t.$slots,"aside-ads-before",{},void 0,!0),i(e).carbonAds?(a(),$(Ot,{key:0,"carbon-ads":i(e).carbonAds},null,8,["carbon-ads"])):m("",!0),u(t.$slots,"aside-ads-after",{},void 0,!0),u(t.$slots,"aside-bottom",{},void 0,!0)]))}}),Rt=k(Ft,[["__scopeId","data-v-6d7b3c46"]]);function Ut(){const{theme:o,page:e}=L();return y(()=>{const{text:t="Edit this page",pattern:s=""}=o.value.editLink||{};let n;return typeof s=="function"?n=s(e.value):n=s.replace(/:path/g,e.value.filePath),{url:n,text:t}})}function zt(){const{page:o,theme:e,frontmatter:t}=L();return y(()=>{var h,P,_,V,S,C,A,N;const s=Be(e.value.sidebar,o.value.relativePath),n=$t(s),r=jt(n,E=>E.link.replace(/[?#].*$/,"")),c=r.findIndex(E=>G(o.value.relativePath,E.link)),v=((h=e.value.docFooter)==null?void 0:h.prev)===!1&&!t.value.prev||t.value.prev===!1,d=((P=e.value.docFooter)==null?void 0:P.next)===!1&&!t.value.next||t.value.next===!1;return{prev:v?void 0:{text:(typeof t.value.prev=="string"?t.value.prev:typeof t.value.prev=="object"?t.value.prev.text:void 0)??((_=r[c-1])==null?void 0:_.docFooterText)??((V=r[c-1])==null?void 0:V.text),link:(typeof t.value.prev=="object"?t.value.prev.link:void 0)??((S=r[c-1])==null?void 0:S.link)},next:d?void 0:{text:(typeof t.value.next=="string"?t.value.next:typeof t.value.next=="object"?t.value.next.text:void 0)??((C=r[c+1])==null?void 0:C.docFooterText)??((A=r[c+1])==null?void 0:A.text),link:(typeof t.value.next=="object"?t.value.next.link:void 0)??((N=r[c+1])==null?void 0:N.link)}}})}function jt(o,e){const t=new Set;return o.filter(s=>{const n=e(s);return t.has(n)?!1:t.add(n)})}const H=b({__name:"VPLink",props:{tag:{},href:{},noIcon:{type:Boolean},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.tag??(e.href?"a":"span")),s=y(()=>e.href&&we.test(e.href)||e.target==="_blank");return(n,r)=>(a(),$(O(t.value),{class:I(["VPLink",{link:n.href,"vp-external-link-icon":s.value,"no-icon":n.noIcon}]),href:n.href?i(_e)(n.href):void 0,target:n.target??(s.value?"_blank":void 0),rel:n.rel??(s.value?"noreferrer":void 0)},{default:f(()=>[u(n.$slots,"default")]),_:3},8,["class","href","target","rel"]))}}),Gt={class:"VPLastUpdated"},Kt=["datetime"],qt=b({__name:"VPDocFooterLastUpdated",setup(o){const{theme:e,page:t,lang:s}=L(),n=y(()=>new Date(t.value.lastUpdated)),r=y(()=>n.value.toISOString()),c=T("");return R(()=>{Z(()=>{var v,d,h;c.value=new Intl.DateTimeFormat((d=(v=e.value.lastUpdated)==null?void 0:v.formatOptions)!=null&&d.forceLocale?s.value:void 0,((h=e.value.lastUpdated)==null?void 0:h.formatOptions)??{dateStyle:"short",timeStyle:"short"}).format(n.value)})}),(v,d)=>{var h;return a(),l("p",Gt,[F(w(((h=i(e).lastUpdated)==null?void 0:h.text)||i(e).lastUpdatedText||"Last updated")+": ",1),p("time",{datetime:r.value},w(c.value),9,Kt)])}}}),Wt=k(qt,[["__scopeId","data-v-475f71b8"]]),Jt={key:0,class:"VPDocFooter"},Yt={key:0,class:"edit-info"},Qt={key:0,class:"edit-link"},Xt={key:1,class:"last-updated"},Zt={key:1,class:"prev-next","aria-labelledby":"doc-footer-aria-label"},xt={class:"pager"},en=["innerHTML"],tn=["innerHTML"],nn={class:"pager"},sn=["innerHTML"],on=["innerHTML"],an=b({__name:"VPDocFooter",setup(o){const{theme:e,page:t,frontmatter:s}=L(),n=Ut(),r=zt(),c=y(()=>e.value.editLink&&s.value.editLink!==!1),v=y(()=>t.value.lastUpdated),d=y(()=>c.value||v.value||r.value.prev||r.value.next);return(h,P)=>{var _,V,S,C;return d.value?(a(),l("footer",Jt,[u(h.$slots,"doc-footer-before",{},void 0,!0),c.value||v.value?(a(),l("div",Yt,[c.value?(a(),l("div",Qt,[g(H,{class:"edit-link-button",href:i(n).url,"no-icon":!0},{default:f(()=>[P[0]||(P[0]=p("span",{class:"vpi-square-pen edit-link-icon"},null,-1)),F(" "+w(i(n).text),1)]),_:1},8,["href"])])):m("",!0),v.value?(a(),l("div",Xt,[g(Wt)])):m("",!0)])):m("",!0),(_=i(r).prev)!=null&&_.link||(V=i(r).next)!=null&&V.link?(a(),l("nav",Zt,[P[1]||(P[1]=p("span",{class:"visually-hidden",id:"doc-footer-aria-label"},"Pager",-1)),p("div",xt,[(S=i(r).prev)!=null&&S.link?(a(),$(H,{key:0,class:"pager-link prev",href:i(r).prev.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.prev)||"Previous page"},null,8,en),p("span",{class:"title",innerHTML:i(r).prev.text},null,8,tn)]}),_:1},8,["href"])):m("",!0)]),p("div",nn,[(C=i(r).next)!=null&&C.link?(a(),$(H,{key:0,class:"pager-link next",href:i(r).next.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.next)||"Next page"},null,8,sn),p("span",{class:"title",innerHTML:i(r).next.text},null,8,on)]}),_:1},8,["href"])):m("",!0)])])):m("",!0)])):m("",!0)}}}),rn=k(an,[["__scopeId","data-v-4f9813fa"]]),ln={class:"container"},cn={class:"aside-container"},un={class:"aside-content"},dn={class:"content"},pn={class:"content-container"},vn={class:"main"},fn=b({__name:"VPDoc",setup(o){const{theme:e}=L(),t=ee(),{hasSidebar:s,hasAside:n,leftAside:r}=U(),c=y(()=>t.path.replace(/[./]+/g,"_").replace(/_html$/,""));return(v,d)=>{const h=K("Content");return a(),l("div",{class:I(["VPDoc",{"has-sidebar":i(s),"has-aside":i(n)}])},[u(v.$slots,"doc-top",{},void 0,!0),p("div",ln,[i(n)?(a(),l("div",{key:0,class:I(["aside",{"left-aside":i(r)}])},[d[0]||(d[0]=p("div",{class:"aside-curtain"},null,-1)),p("div",cn,[p("div",un,[g(Rt,null,{"aside-top":f(()=>[u(v.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(v.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(v.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(v.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(v.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(v.$slots,"aside-ads-after",{},void 0,!0)]),_:3})])])],2)):m("",!0),p("div",dn,[p("div",pn,[u(v.$slots,"doc-before",{},void 0,!0),p("main",vn,[g(h,{class:I(["vp-doc",[c.value,i(e).externalLinkIcon&&"external-link-icon-enabled"]])},null,8,["class"])]),g(rn,null,{"doc-footer-before":f(()=>[u(v.$slots,"doc-footer-before",{},void 0,!0)]),_:3}),u(v.$slots,"doc-after",{},void 0,!0)])])]),u(v.$slots,"doc-bottom",{},void 0,!0)],2)}}}),mn=k(fn,[["__scopeId","data-v-83890dd9"]]),hn=b({__name:"VPButton",props:{tag:{},size:{default:"medium"},theme:{default:"brand"},text:{},href:{},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.href&&we.test(e.href)),s=y(()=>e.tag||(e.href?"a":"button"));return(n,r)=>(a(),$(O(s.value),{class:I(["VPButton",[n.size,n.theme]]),href:n.href?i(_e)(n.href):void 0,target:e.target??(t.value?"_blank":void 0),rel:e.rel??(t.value?"noreferrer":void 0)},{default:f(()=>[F(w(n.text),1)]),_:1},8,["class","href","target","rel"]))}}),_n=k(hn,[["__scopeId","data-v-906d7fb4"]]),bn=["src","alt"],kn=b({inheritAttrs:!1,__name:"VPImage",props:{image:{},alt:{}},setup(o){return(e,t)=>{const s=K("VPImage",!0);return e.image?(a(),l(M,{key:0},[typeof e.image=="string"||"src"in e.image?(a(),l("img",j({key:0,class:"VPImage"},typeof e.image=="string"?e.$attrs:{...e.image,...e.$attrs},{src:i(ve)(typeof e.image=="string"?e.image:e.image.src),alt:e.alt??(typeof e.image=="string"?"":e.image.alt||"")}),null,16,bn)):(a(),l(M,{key:1},[g(s,j({class:"dark",image:e.image.dark,alt:e.image.alt},e.$attrs),null,16,["image","alt"]),g(s,j({class:"light",image:e.image.light,alt:e.image.alt},e.$attrs),null,16,["image","alt"])],64))],64)):m("",!0)}}}),X=k(kn,[["__scopeId","data-v-35a7d0b8"]]),gn={class:"container"},$n={class:"main"},yn={key:0,class:"name"},Pn=["innerHTML"],Vn=["innerHTML"],Sn=["innerHTML"],Ln={key:0,class:"actions"},Tn={key:0,class:"image"},wn={class:"image-container"},Nn=b({__name:"VPHero",props:{name:{},text:{},tagline:{},image:{},actions:{}},setup(o){const e=W("hero-image-slot-exists");return(t,s)=>(a(),l("div",{class:I(["VPHero",{"has-image":t.image||i(e)}])},[p("div",gn,[p("div",$n,[u(t.$slots,"home-hero-info-before",{},void 0,!0),u(t.$slots,"home-hero-info",{},()=>[t.name?(a(),l("h1",yn,[p("span",{innerHTML:t.name,class:"clip"},null,8,Pn)])):m("",!0),t.text?(a(),l("p",{key:1,innerHTML:t.text,class:"text"},null,8,Vn)):m("",!0),t.tagline?(a(),l("p",{key:2,innerHTML:t.tagline,class:"tagline"},null,8,Sn)):m("",!0)],!0),u(t.$slots,"home-hero-info-after",{},void 0,!0),t.actions?(a(),l("div",Ln,[(a(!0),l(M,null,B(t.actions,n=>(a(),l("div",{key:n.link,class:"action"},[g(_n,{tag:"a",size:"medium",theme:n.theme,text:n.text,href:n.link,target:n.target,rel:n.rel},null,8,["theme","text","href","target","rel"])]))),128))])):m("",!0),u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),t.image||i(e)?(a(),l("div",Tn,[p("div",wn,[s[0]||(s[0]=p("div",{class:"image-bg"},null,-1)),u(t.$slots,"home-hero-image",{},()=>[t.image?(a(),$(X,{key:0,class:"image-src",image:t.image},null,8,["image"])):m("",!0)],!0)])])):m("",!0)])],2))}}),In=k(Nn,[["__scopeId","data-v-955009fc"]]),Mn=b({__name:"VPHomeHero",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).hero?(a(),$(In,{key:0,class:"VPHomeHero",name:i(e).hero.name,text:i(e).hero.text,tagline:i(e).hero.tagline,image:i(e).hero.image,actions:i(e).hero.actions},{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before")]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info")]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after")]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after")]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image")]),_:3},8,["name","text","tagline","image","actions"])):m("",!0)}}),Cn={class:"box"},An={key:0,class:"icon"},Bn=["innerHTML"],En=["innerHTML"],Hn=["innerHTML"],On={key:4,class:"link-text"},Dn={class:"link-text-value"},Fn=b({__name:"VPFeature",props:{icon:{},title:{},details:{},link:{},linkText:{},rel:{},target:{}},setup(o){return(e,t)=>(a(),$(H,{class:"VPFeature",href:e.link,rel:e.rel,target:e.target,"no-icon":!0,tag:e.link?"a":"div"},{default:f(()=>[p("article",Cn,[typeof e.icon=="object"&&e.icon.wrap?(a(),l("div",An,[g(X,{image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])])):typeof e.icon=="object"?(a(),$(X,{key:1,image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])):e.icon?(a(),l("div",{key:2,class:"icon",innerHTML:e.icon},null,8,Bn)):m("",!0),p("h2",{class:"title",innerHTML:e.title},null,8,En),e.details?(a(),l("p",{key:3,class:"details",innerHTML:e.details},null,8,Hn)):m("",!0),e.linkText?(a(),l("div",On,[p("p",Dn,[F(w(e.linkText)+" ",1),t[0]||(t[0]=p("span",{class:"vpi-arrow-right link-text-icon"},null,-1))])])):m("",!0)])]),_:1},8,["href","rel","target","tag"]))}}),Rn=k(Fn,[["__scopeId","data-v-f5e9645b"]]),Un={key:0,class:"VPFeatures"},zn={class:"container"},jn={class:"items"},Gn=b({__name:"VPFeatures",props:{features:{}},setup(o){const e=o,t=y(()=>{const s=e.features.length;if(s){if(s===2)return"grid-2";if(s===3)return"grid-3";if(s%3===0)return"grid-6";if(s>3)return"grid-4"}else return});return(s,n)=>s.features?(a(),l("div",Un,[p("div",zn,[p("div",jn,[(a(!0),l(M,null,B(s.features,r=>(a(),l("div",{key:r.title,class:I(["item",[t.value]])},[g(Rn,{icon:r.icon,title:r.title,details:r.details,link:r.link,"link-text":r.linkText,rel:r.rel,target:r.target},null,8,["icon","title","details","link","link-text","rel","target"])],2))),128))])])])):m("",!0)}}),Kn=k(Gn,[["__scopeId","data-v-d0a190d7"]]),qn=b({__name:"VPHomeFeatures",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).features?(a(),$(Kn,{key:0,class:"VPHomeFeatures",features:i(e).features},null,8,["features"])):m("",!0)}}),Wn=b({__name:"VPHomeContent",setup(o){const{width:e}=Ye({initialWidth:0,includeScrollbar:!1});return(t,s)=>(a(),l("div",{class:"vp-doc container",style:Ne(i(e)?{"--vp-offset":`calc(50% - ${i(e)/2}px)`}:{})},[u(t.$slots,"default",{},void 0,!0)],4))}}),Jn=k(Wn,[["__scopeId","data-v-7a48a447"]]),Yn={class:"VPHome"},Qn=b({__name:"VPHome",setup(o){const{frontmatter:e}=L();return(t,s)=>{const n=K("Content");return a(),l("div",Yn,[u(t.$slots,"home-hero-before",{},void 0,!0),g(Mn,null,{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image",{},void 0,!0)]),_:3}),u(t.$slots,"home-hero-after",{},void 0,!0),u(t.$slots,"home-features-before",{},void 0,!0),g(qn),u(t.$slots,"home-features-after",{},void 0,!0),i(e).markdownStyles!==!1?(a(),$(Jn,{key:0},{default:f(()=>[g(n)]),_:1})):(a(),$(n,{key:1}))])}}}),Xn=k(Qn,[["__scopeId","data-v-cbb6ec48"]]),Zn={},xn={class:"VPPage"};function es(o,e){const t=K("Content");return a(),l("div",xn,[u(o.$slots,"page-top"),g(t),u(o.$slots,"page-bottom")])}const ts=k(Zn,[["render",es]]),ns=b({__name:"VPContent",setup(o){const{page:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>(a(),l("div",{class:I(["VPContent",{"has-sidebar":i(s),"is-home":i(t).layout==="home"}]),id:"VPContent"},[i(e).isNotFound?u(n.$slots,"not-found",{key:0},()=>[g(kt)],!0):i(t).layout==="page"?(a(),$(ts,{key:1},{"page-top":f(()=>[u(n.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(n.$slots,"page-bottom",{},void 0,!0)]),_:3})):i(t).layout==="home"?(a(),$(Xn,{key:2},{"home-hero-before":f(()=>[u(n.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(n.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(n.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(n.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(n.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(n.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(n.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(n.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(n.$slots,"home-features-after",{},void 0,!0)]),_:3})):i(t).layout&&i(t).layout!=="doc"?(a(),$(O(i(t).layout),{key:3})):(a(),$(mn,{key:4},{"doc-top":f(()=>[u(n.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(n.$slots,"doc-bottom",{},void 0,!0)]),"doc-footer-before":f(()=>[u(n.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(n.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(n.$slots,"doc-after",{},void 0,!0)]),"aside-top":f(()=>[u(n.$slots,"aside-top",{},void 0,!0)]),"aside-outline-before":f(()=>[u(n.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(n.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(n.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(n.$slots,"aside-ads-after",{},void 0,!0)]),"aside-bottom":f(()=>[u(n.$slots,"aside-bottom",{},void 0,!0)]),_:3}))],2))}}),ss=k(ns,[["__scopeId","data-v-91765379"]]),os={class:"container"},as=["innerHTML"],rs=["innerHTML"],is=b({__name:"VPFooter",setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>i(e).footer&&i(t).footer!==!1?(a(),l("footer",{key:0,class:I(["VPFooter",{"has-sidebar":i(s)}])},[p("div",os,[i(e).footer.message?(a(),l("p",{key:0,class:"message",innerHTML:i(e).footer.message},null,8,as)):m("",!0),i(e).footer.copyright?(a(),l("p",{key:1,class:"copyright",innerHTML:i(e).footer.copyright},null,8,rs)):m("",!0)])],2)):m("",!0)}}),ls=k(is,[["__scopeId","data-v-c970a860"]]);function cs(){const{theme:o,frontmatter:e}=L(),t=Te([]),s=y(()=>t.value.length>0);return x(()=>{t.value=be(e.value.outline??o.value.outline)}),{headers:t,hasLocalNav:s}}const us={class:"menu-text"},ds={class:"header"},ps={class:"outline"},vs=b({__name:"VPLocalNavOutlineDropdown",props:{headers:{},navHeight:{}},setup(o){const e=o,{theme:t}=L(),s=T(!1),n=T(0),r=T(),c=T();function v(_){var V;(V=r.value)!=null&&V.contains(_.target)||(s.value=!1)}D(s,_=>{if(_){document.addEventListener("click",v);return}document.removeEventListener("click",v)}),le("Escape",()=>{s.value=!1}),x(()=>{s.value=!1});function d(){s.value=!s.value,n.value=window.innerHeight+Math.min(window.scrollY-e.navHeight,0)}function h(_){_.target.classList.contains("outline-link")&&(c.value&&(c.value.style.transition="none"),Ie(()=>{s.value=!1}))}function P(){s.value=!1,window.scrollTo({top:0,left:0,behavior:"smooth"})}return(_,V)=>(a(),l("div",{class:"VPLocalNavOutlineDropdown",style:Ne({"--vp-vh":n.value+"px"}),ref_key:"main",ref:r},[_.headers.length>0?(a(),l("button",{key:0,onClick:d,class:I({open:s.value})},[p("span",us,w(i(Ee)(i(t))),1),V[0]||(V[0]=p("span",{class:"vpi-chevron-right icon"},null,-1))],2)):(a(),l("button",{key:1,onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)),g(pe,{name:"flyout"},{default:f(()=>[s.value?(a(),l("div",{key:0,ref_key:"items",ref:c,class:"items",onClick:h},[p("div",ds,[p("a",{class:"top-link",href:"#",onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)]),p("div",ps,[g(He,{headers:_.headers},null,8,["headers"])])],512)):m("",!0)]),_:1})],4))}}),fs=k(vs,[["__scopeId","data-v-bc9dc845"]]),ms={class:"container"},hs=["aria-expanded"],_s={class:"menu-text"},bs=b({__name:"VPLocalNav",props:{open:{type:Boolean}},emits:["open-menu"],setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U(),{headers:n}=cs(),{y:r}=Me(),c=T(0);R(()=>{c.value=parseInt(getComputedStyle(document.documentElement).getPropertyValue("--vp-nav-height"))}),x(()=>{n.value=be(t.value.outline??e.value.outline)});const v=y(()=>n.value.length===0),d=y(()=>v.value&&!s.value),h=y(()=>({VPLocalNav:!0,"has-sidebar":s.value,empty:v.value,fixed:d.value}));return(P,_)=>i(t).layout!=="home"&&(!d.value||i(r)>=c.value)?(a(),l("div",{key:0,class:I(h.value)},[p("div",ms,[i(s)?(a(),l("button",{key:0,class:"menu","aria-expanded":P.open,"aria-controls":"VPSidebarNav",onClick:_[0]||(_[0]=V=>P.$emit("open-menu"))},[_[1]||(_[1]=p("span",{class:"vpi-align-left menu-icon"},null,-1)),p("span",_s,w(i(e).sidebarMenuLabel||"Menu"),1)],8,hs)):m("",!0),g(fs,{headers:i(n),navHeight:c.value},null,8,["headers","navHeight"])])],2)):m("",!0)}}),ks=k(bs,[["__scopeId","data-v-070ab83d"]]);function gs(){const o=T(!1);function e(){o.value=!0,window.addEventListener("resize",n)}function t(){o.value=!1,window.removeEventListener("resize",n)}function s(){o.value?t():e()}function n(){window.outerWidth>=768&&t()}const r=ee();return D(()=>r.path,t),{isScreenOpen:o,openScreen:e,closeScreen:t,toggleScreen:s}}const $s={},ys={class:"VPSwitch",type:"button",role:"switch"},Ps={class:"check"},Vs={key:0,class:"icon"};function Ss(o,e){return a(),l("button",ys,[p("span",Ps,[o.$slots.default?(a(),l("span",Vs,[u(o.$slots,"default",{},void 0,!0)])):m("",!0)])])}const Ls=k($s,[["render",Ss],["__scopeId","data-v-4a1c76db"]]),Ts=b({__name:"VPSwitchAppearance",setup(o){const{isDark:e,theme:t}=L(),s=W("toggle-appearance",()=>{e.value=!e.value}),n=T("");return me(()=>{n.value=e.value?t.value.lightModeSwitchTitle||"Switch to light theme":t.value.darkModeSwitchTitle||"Switch to dark theme"}),(r,c)=>(a(),$(Ls,{title:n.value,class:"VPSwitchAppearance","aria-checked":i(e),onClick:i(s)},{default:f(()=>c[0]||(c[0]=[p("span",{class:"vpi-sun sun"},null,-1),p("span",{class:"vpi-moon moon"},null,-1)])),_:1},8,["title","aria-checked","onClick"]))}}),ke=k(Ts,[["__scopeId","data-v-e40a8bb6"]]),ws={key:0,class:"VPNavBarAppearance"},Ns=b({__name:"VPNavBarAppearance",setup(o){const{site:e}=L();return(t,s)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",ws,[g(ke)])):m("",!0)}}),Is=k(Ns,[["__scopeId","data-v-af096f4a"]]),ge=T();let Oe=!1,re=0;function Ms(o){const e=T(!1);if(te){!Oe&&Cs(),re++;const t=D(ge,s=>{var n,r,c;s===o.el.value||(n=o.el.value)!=null&&n.contains(s)?(e.value=!0,(r=o.onFocus)==null||r.call(o)):(e.value=!1,(c=o.onBlur)==null||c.call(o))});fe(()=>{t(),re--,re||As()})}return Qe(e)}function Cs(){document.addEventListener("focusin",De),Oe=!0,ge.value=document.activeElement}function As(){document.removeEventListener("focusin",De)}function De(){ge.value=document.activeElement}const Bs={class:"VPMenuLink"},Es=["innerHTML"],Hs=b({__name:"VPMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),l("div",Bs,[g(H,{class:I({active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Es)]),_:1},8,["class","href","target","rel","no-icon"])]))}}),ne=k(Hs,[["__scopeId","data-v-acbfed09"]]),Os={class:"VPMenuGroup"},Ds={key:0,class:"title"},Fs=b({__name:"VPMenuGroup",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",Os,[e.text?(a(),l("p",Ds,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),l(M,null,["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):m("",!0)],64))),256))]))}}),Rs=k(Fs,[["__scopeId","data-v-48c802d0"]]),Us={class:"VPMenu"},zs={key:0,class:"items"},js=b({__name:"VPMenu",props:{items:{}},setup(o){return(e,t)=>(a(),l("div",Us,[e.items?(a(),l("div",zs,[(a(!0),l(M,null,B(e.items,s=>(a(),l(M,{key:JSON.stringify(s)},["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):"component"in s?(a(),$(O(s.component),j({key:1,ref_for:!0},s.props),null,16)):(a(),$(Rs,{key:2,text:s.text,items:s.items},null,8,["text","items"]))],64))),128))])):m("",!0),u(e.$slots,"default",{},void 0,!0)]))}}),Gs=k(js,[["__scopeId","data-v-7dd3104a"]]),Ks=["aria-expanded","aria-label"],qs={key:0,class:"text"},Ws=["innerHTML"],Js={key:1,class:"vpi-more-horizontal icon"},Ys={class:"menu"},Qs=b({__name:"VPFlyout",props:{icon:{},button:{},label:{},items:{}},setup(o){const e=T(!1),t=T();Ms({el:t,onBlur:s});function s(){e.value=!1}return(n,r)=>(a(),l("div",{class:"VPFlyout",ref_key:"el",ref:t,onMouseenter:r[1]||(r[1]=c=>e.value=!0),onMouseleave:r[2]||(r[2]=c=>e.value=!1)},[p("button",{type:"button",class:"button","aria-haspopup":"true","aria-expanded":e.value,"aria-label":n.label,onClick:r[0]||(r[0]=c=>e.value=!e.value)},[n.button||n.icon?(a(),l("span",qs,[n.icon?(a(),l("span",{key:0,class:I([n.icon,"option-icon"])},null,2)):m("",!0),n.button?(a(),l("span",{key:1,innerHTML:n.button},null,8,Ws)):m("",!0),r[3]||(r[3]=p("span",{class:"vpi-chevron-down text-icon"},null,-1))])):(a(),l("span",Js))],8,Ks),p("div",Ys,[g(Gs,{items:n.items},{default:f(()=>[u(n.$slots,"default",{},void 0,!0)]),_:3},8,["items"])])],544))}}),$e=k(Qs,[["__scopeId","data-v-04f5c5e9"]]),Xs=["href","aria-label","innerHTML"],Zs=b({__name:"VPSocialLink",props:{icon:{},link:{},ariaLabel:{}},setup(o){const e=o,t=y(()=>typeof e.icon=="object"?e.icon.svg:``);return(s,n)=>(a(),l("a",{class:"VPSocialLink no-icon",href:s.link,"aria-label":s.ariaLabel??(typeof s.icon=="string"?s.icon:""),target:"_blank",rel:"noopener",innerHTML:t.value},null,8,Xs))}}),xs=k(Zs,[["__scopeId","data-v-717b8b75"]]),eo={class:"VPSocialLinks"},to=b({__name:"VPSocialLinks",props:{links:{}},setup(o){return(e,t)=>(a(),l("div",eo,[(a(!0),l(M,null,B(e.links,({link:s,icon:n,ariaLabel:r})=>(a(),$(xs,{key:s,icon:n,link:s,ariaLabel:r},null,8,["icon","link","ariaLabel"]))),128))]))}}),se=k(to,[["__scopeId","data-v-ee7a9424"]]),no={key:0,class:"group translations"},so={class:"trans-title"},oo={key:1,class:"group"},ao={class:"item appearance"},ro={class:"label"},io={class:"appearance-action"},lo={key:2,class:"group"},co={class:"item social-links"},uo=b({__name:"VPNavBarExtra",setup(o){const{site:e,theme:t}=L(),{localeLinks:s,currentLang:n}=Y({correspondingLink:!0}),r=y(()=>s.value.length&&n.value.label||e.value.appearance||t.value.socialLinks);return(c,v)=>r.value?(a(),$($e,{key:0,class:"VPNavBarExtra",label:"extra navigation"},{default:f(()=>[i(s).length&&i(n).label?(a(),l("div",no,[p("p",so,w(i(n).label),1),(a(!0),l(M,null,B(i(s),d=>(a(),$(ne,{key:d.link,item:d},null,8,["item"]))),128))])):m("",!0),i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",oo,[p("div",ao,[p("p",ro,w(i(t).darkModeSwitchLabel||"Appearance"),1),p("div",io,[g(ke)])])])):m("",!0),i(t).socialLinks?(a(),l("div",lo,[p("div",co,[g(se,{class:"social-links-list",links:i(t).socialLinks},null,8,["links"])])])):m("",!0)]),_:1})):m("",!0)}}),po=k(uo,[["__scopeId","data-v-925effce"]]),vo=["aria-expanded"],fo=b({__name:"VPNavBarHamburger",props:{active:{type:Boolean}},emits:["click"],setup(o){return(e,t)=>(a(),l("button",{type:"button",class:I(["VPNavBarHamburger",{active:e.active}]),"aria-label":"mobile navigation","aria-expanded":e.active,"aria-controls":"VPNavScreen",onClick:t[0]||(t[0]=s=>e.$emit("click"))},t[1]||(t[1]=[p("span",{class:"container"},[p("span",{class:"top"}),p("span",{class:"middle"}),p("span",{class:"bottom"})],-1)]),10,vo))}}),mo=k(fo,[["__scopeId","data-v-5dea55bf"]]),ho=["innerHTML"],_o=b({__name:"VPNavBarMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),$(H,{class:I({VPNavBarMenuLink:!0,active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,tabindex:"0"},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ho)]),_:1},8,["class","href","target","rel","no-icon"]))}}),bo=k(_o,[["__scopeId","data-v-956ec74c"]]),Fe=b({__name:"VPNavBarMenuGroup",props:{item:{}},setup(o){const e=o,{page:t}=L(),s=r=>"component"in r?!1:"link"in r?G(t.value.relativePath,r.link,!!e.item.activeMatch):r.items.some(s),n=y(()=>s(e.item));return(r,c)=>(a(),$($e,{class:I({VPNavBarMenuGroup:!0,active:i(G)(i(t).relativePath,r.item.activeMatch,!!r.item.activeMatch)||n.value}),button:r.item.text,items:r.item.items},null,8,["class","button","items"]))}}),ko={key:0,"aria-labelledby":"main-nav-aria-label",class:"VPNavBarMenu"},go=b({__name:"VPNavBarMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",ko,[s[0]||(s[0]=p("span",{id:"main-nav-aria-label",class:"visually-hidden"}," Main Navigation ",-1)),(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(bo,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props),null,16)):(a(),$(Fe,{key:2,item:n},null,8,["item"]))],64))),128))])):m("",!0)}}),$o=k(go,[["__scopeId","data-v-e6d46098"]]);function yo(o){const{localeIndex:e,theme:t}=L();function s(n){var C,A,N;const r=n.split("."),c=(C=t.value.search)==null?void 0:C.options,v=c&&typeof c=="object",d=v&&((N=(A=c.locales)==null?void 0:A[e.value])==null?void 0:N.translations)||null,h=v&&c.translations||null;let P=d,_=h,V=o;const S=r.pop();for(const E of r){let z=null;const q=V==null?void 0:V[E];q&&(z=V=q);const oe=_==null?void 0:_[E];oe&&(z=_=oe);const ae=P==null?void 0:P[E];ae&&(z=P=ae),q||(V=z),oe||(_=z),ae||(P=z)}return(P==null?void 0:P[S])??(_==null?void 0:_[S])??(V==null?void 0:V[S])??""}return s}const Po=["aria-label"],Vo={class:"DocSearch-Button-Container"},So={class:"DocSearch-Button-Placeholder"},Pe=b({__name:"VPNavBarSearchButton",setup(o){const t=yo({button:{buttonText:"Search",buttonAriaLabel:"Search"}});return(s,n)=>(a(),l("button",{type:"button",class:"DocSearch DocSearch-Button","aria-label":i(t)("button.buttonAriaLabel")},[p("span",Vo,[n[0]||(n[0]=p("span",{class:"vp-icon DocSearch-Search-Icon"},null,-1)),p("span",So,w(i(t)("button.buttonText")),1)]),n[1]||(n[1]=p("span",{class:"DocSearch-Button-Keys"},[p("kbd",{class:"DocSearch-Button-Key"}),p("kbd",{class:"DocSearch-Button-Key"},"K")],-1))],8,Po))}}),Lo={class:"VPNavBarSearch"},To={id:"local-search"},wo={key:1,id:"docsearch"},No=b({__name:"VPNavBarSearch",setup(o){const e=Xe(()=>Ze(()=>import("./VPLocalSearchBox.DFDiEjIp.js"),__vite__mapDeps([0,1]))),t=()=>null,{theme:s}=L(),n=T(!1),r=T(!1);R(()=>{});function c(){n.value||(n.value=!0,setTimeout(v,16))}function v(){const _=new Event("keydown");_.key="k",_.metaKey=!0,window.dispatchEvent(_),setTimeout(()=>{document.querySelector(".DocSearch-Modal")||v()},16)}function d(_){const V=_.target,S=V.tagName;return V.isContentEditable||S==="INPUT"||S==="SELECT"||S==="TEXTAREA"}const h=T(!1);le("k",_=>{(_.ctrlKey||_.metaKey)&&(_.preventDefault(),h.value=!0)}),le("/",_=>{d(_)||(_.preventDefault(),h.value=!0)});const P="local";return(_,V)=>{var S;return a(),l("div",Lo,[i(P)==="local"?(a(),l(M,{key:0},[h.value?(a(),$(i(e),{key:0,onClose:V[0]||(V[0]=C=>h.value=!1)})):m("",!0),p("div",To,[g(Pe,{onClick:V[1]||(V[1]=C=>h.value=!0)})])],64)):i(P)==="algolia"?(a(),l(M,{key:1},[n.value?(a(),$(i(t),{key:0,algolia:((S=i(s).search)==null?void 0:S.options)??i(s).algolia,onVnodeBeforeMount:V[2]||(V[2]=C=>r.value=!0)},null,8,["algolia"])):m("",!0),r.value?m("",!0):(a(),l("div",wo,[g(Pe,{onClick:c})]))],64)):m("",!0)])}}}),Io=b({__name:"VPNavBarSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavBarSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),Mo=k(Io,[["__scopeId","data-v-164c457f"]]),Co=["href","rel","target"],Ao=["innerHTML"],Bo={key:2},Eo=b({__name:"VPNavBarTitle",setup(o){const{site:e,theme:t}=L(),{hasSidebar:s}=U(),{currentLang:n}=Y(),r=y(()=>{var d;return typeof t.value.logoLink=="string"?t.value.logoLink:(d=t.value.logoLink)==null?void 0:d.link}),c=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.rel}),v=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.target});return(d,h)=>(a(),l("div",{class:I(["VPNavBarTitle",{"has-sidebar":i(s)}])},[p("a",{class:"title",href:r.value??i(_e)(i(n).link),rel:c.value,target:v.value},[u(d.$slots,"nav-bar-title-before",{},void 0,!0),i(t).logo?(a(),$(X,{key:0,class:"logo",image:i(t).logo},null,8,["image"])):m("",!0),i(t).siteTitle?(a(),l("span",{key:1,innerHTML:i(t).siteTitle},null,8,Ao)):i(t).siteTitle===void 0?(a(),l("span",Bo,w(i(e).title),1)):m("",!0),u(d.$slots,"nav-bar-title-after",{},void 0,!0)],8,Co)],2))}}),Ho=k(Eo,[["__scopeId","data-v-0f4f798b"]]),Oo={class:"items"},Do={class:"title"},Fo=b({__name:"VPNavBarTranslations",setup(o){const{theme:e}=L(),{localeLinks:t,currentLang:s}=Y({correspondingLink:!0});return(n,r)=>i(t).length&&i(s).label?(a(),$($e,{key:0,class:"VPNavBarTranslations",icon:"vpi-languages",label:i(e).langMenuLabel||"Change language"},{default:f(()=>[p("div",Oo,[p("p",Do,w(i(s).label),1),(a(!0),l(M,null,B(i(t),c=>(a(),$(ne,{key:c.link,item:c},null,8,["item"]))),128))])]),_:1},8,["label"])):m("",!0)}}),Ro=k(Fo,[["__scopeId","data-v-c80d9ad0"]]),Uo={class:"wrapper"},zo={class:"container"},jo={class:"title"},Go={class:"content"},Ko={class:"content-body"},qo=b({__name:"VPNavBar",props:{isScreenOpen:{type:Boolean}},emits:["toggle-screen"],setup(o){const e=o,{y:t}=Me(),{hasSidebar:s}=U(),{frontmatter:n}=L(),r=T({});return me(()=>{r.value={"has-sidebar":s.value,home:n.value.layout==="home",top:t.value===0,"screen-open":e.isScreenOpen}}),(c,v)=>(a(),l("div",{class:I(["VPNavBar",r.value])},[p("div",Uo,[p("div",zo,[p("div",jo,[g(Ho,null,{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),_:3})]),p("div",Go,[p("div",Ko,[u(c.$slots,"nav-bar-content-before",{},void 0,!0),g(No,{class:"search"}),g($o,{class:"menu"}),g(Ro,{class:"translations"}),g(Is,{class:"appearance"}),g(Mo,{class:"social-links"}),g(po,{class:"extra"}),u(c.$slots,"nav-bar-content-after",{},void 0,!0),g(mo,{class:"hamburger",active:c.isScreenOpen,onClick:v[0]||(v[0]=d=>c.$emit("toggle-screen"))},null,8,["active"])])])])]),v[1]||(v[1]=p("div",{class:"divider"},[p("div",{class:"divider-line"})],-1))],2))}}),Wo=k(qo,[["__scopeId","data-v-822684d1"]]),Jo={key:0,class:"VPNavScreenAppearance"},Yo={class:"text"},Qo=b({__name:"VPNavScreenAppearance",setup(o){const{site:e,theme:t}=L();return(s,n)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",Jo,[p("p",Yo,w(i(t).darkModeSwitchLabel||"Appearance"),1),g(ke)])):m("",!0)}}),Xo=k(Qo,[["__scopeId","data-v-ffb44008"]]),Zo=["innerHTML"],xo=b({__name:"VPNavScreenMenuLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Zo)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),ea=k(xo,[["__scopeId","data-v-735512b8"]]),ta=["innerHTML"],na=b({__name:"VPNavScreenMenuGroupLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuGroupLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ta)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),Re=k(na,[["__scopeId","data-v-372ae7c0"]]),sa={class:"VPNavScreenMenuGroupSection"},oa={key:0,class:"title"},aa=b({__name:"VPNavScreenMenuGroupSection",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",sa,[e.text?(a(),l("p",oa,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),$(Re,{key:s.text,item:s},null,8,["item"]))),128))]))}}),ra=k(aa,[["__scopeId","data-v-4b8941ac"]]),ia=["aria-controls","aria-expanded"],la=["innerHTML"],ca=["id"],ua={key:0,class:"item"},da={key:1,class:"item"},pa={key:2,class:"group"},va=b({__name:"VPNavScreenMenuGroup",props:{text:{},items:{}},setup(o){const e=o,t=T(!1),s=y(()=>`NavScreenGroup-${e.text.replace(" ","-").toLowerCase()}`);function n(){t.value=!t.value}return(r,c)=>(a(),l("div",{class:I(["VPNavScreenMenuGroup",{open:t.value}])},[p("button",{class:"button","aria-controls":s.value,"aria-expanded":t.value,onClick:n},[p("span",{class:"button-text",innerHTML:r.text},null,8,la),c[0]||(c[0]=p("span",{class:"vpi-plus button-icon"},null,-1))],8,ia),p("div",{id:s.value,class:"items"},[(a(!0),l(M,null,B(r.items,v=>(a(),l(M,{key:JSON.stringify(v)},["link"in v?(a(),l("div",ua,[g(Re,{item:v},null,8,["item"])])):"component"in v?(a(),l("div",da,[(a(),$(O(v.component),j({ref_for:!0},v.props,{"screen-menu":""}),null,16))])):(a(),l("div",pa,[g(ra,{text:v.text,items:v.items},null,8,["text","items"])]))],64))),128))],8,ca)],2))}}),Ue=k(va,[["__scopeId","data-v-875057a5"]]),fa={key:0,class:"VPNavScreenMenu"},ma=b({__name:"VPNavScreenMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",fa,[(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(ea,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props,{"screen-menu":""}),null,16)):(a(),$(Ue,{key:2,text:n.text||"",items:n.items},null,8,["text","items"]))],64))),128))])):m("",!0)}}),ha=b({__name:"VPNavScreenSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavScreenSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),_a={class:"list"},ba=b({__name:"VPNavScreenTranslations",setup(o){const{localeLinks:e,currentLang:t}=Y({correspondingLink:!0}),s=T(!1);function n(){s.value=!s.value}return(r,c)=>i(e).length&&i(t).label?(a(),l("div",{key:0,class:I(["VPNavScreenTranslations",{open:s.value}])},[p("button",{class:"title",onClick:n},[c[0]||(c[0]=p("span",{class:"vpi-languages icon lang"},null,-1)),F(" "+w(i(t).label)+" ",1),c[1]||(c[1]=p("span",{class:"vpi-chevron-down icon chevron"},null,-1))]),p("ul",_a,[(a(!0),l(M,null,B(i(e),v=>(a(),l("li",{key:v.link,class:"item"},[g(H,{class:"link",href:v.link},{default:f(()=>[F(w(v.text),1)]),_:2},1032,["href"])]))),128))])],2)):m("",!0)}}),ka=k(ba,[["__scopeId","data-v-362991c2"]]),ga={class:"container"},$a=b({__name:"VPNavScreen",props:{open:{type:Boolean}},setup(o){const e=T(null),t=Ce(te?document.body:null);return(s,n)=>(a(),$(pe,{name:"fade",onEnter:n[0]||(n[0]=r=>t.value=!0),onAfterLeave:n[1]||(n[1]=r=>t.value=!1)},{default:f(()=>[s.open?(a(),l("div",{key:0,class:"VPNavScreen",ref_key:"screen",ref:e,id:"VPNavScreen"},[p("div",ga,[u(s.$slots,"nav-screen-content-before",{},void 0,!0),g(ma,{class:"menu"}),g(ka,{class:"translations"}),g(Xo,{class:"appearance"}),g(ha,{class:"social-links"}),u(s.$slots,"nav-screen-content-after",{},void 0,!0)])],512)):m("",!0)]),_:3}))}}),ya=k($a,[["__scopeId","data-v-833aabba"]]),Pa={key:0,class:"VPNav"},Va=b({__name:"VPNav",setup(o){const{isScreenOpen:e,closeScreen:t,toggleScreen:s}=gs(),{frontmatter:n}=L(),r=y(()=>n.value.navbar!==!1);return he("close-screen",t),Z(()=>{te&&document.documentElement.classList.toggle("hide-nav",!r.value)}),(c,v)=>r.value?(a(),l("header",Pa,[g(Wo,{"is-screen-open":i(e),onToggleScreen:i(s)},{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(c.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(c.$slots,"nav-bar-content-after",{},void 0,!0)]),_:3},8,["is-screen-open","onToggleScreen"]),g(ya,{open:i(e)},{"nav-screen-content-before":f(()=>[u(c.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(c.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3},8,["open"])])):m("",!0)}}),Sa=k(Va,[["__scopeId","data-v-f1e365da"]]),La=["role","tabindex"],Ta={key:1,class:"items"},wa=b({__name:"VPSidebarItem",props:{item:{},depth:{}},setup(o){const e=o,{collapsed:t,collapsible:s,isLink:n,isActiveLink:r,hasActiveLink:c,hasChildren:v,toggle:d}=Pt(y(()=>e.item)),h=y(()=>v.value?"section":"div"),P=y(()=>n.value?"a":"div"),_=y(()=>v.value?e.depth+2===7?"p":`h${e.depth+2}`:"p"),V=y(()=>n.value?void 0:"button"),S=y(()=>[[`level-${e.depth}`],{collapsible:s.value},{collapsed:t.value},{"is-link":n.value},{"is-active":r.value},{"has-active":c.value}]);function C(N){"key"in N&&N.key!=="Enter"||!e.item.link&&d()}function A(){e.item.link&&d()}return(N,E)=>{const z=K("VPSidebarItem",!0);return a(),$(O(h.value),{class:I(["VPSidebarItem",S.value])},{default:f(()=>[N.item.text?(a(),l("div",j({key:0,class:"item",role:V.value},et(N.item.items?{click:C,keydown:C}:{},!0),{tabindex:N.item.items&&0}),[E[1]||(E[1]=p("div",{class:"indicator"},null,-1)),N.item.link?(a(),$(H,{key:0,tag:P.value,class:"link",href:N.item.link,rel:N.item.rel,target:N.item.target},{default:f(()=>[(a(),$(O(_.value),{class:"text",innerHTML:N.item.text},null,8,["innerHTML"]))]),_:1},8,["tag","href","rel","target"])):(a(),$(O(_.value),{key:1,class:"text",innerHTML:N.item.text},null,8,["innerHTML"])),N.item.collapsed!=null&&N.item.items&&N.item.items.length?(a(),l("div",{key:2,class:"caret",role:"button","aria-label":"toggle section",onClick:A,onKeydown:xe(A,["enter"]),tabindex:"0"},E[0]||(E[0]=[p("span",{class:"vpi-chevron-right caret-icon"},null,-1)]),32)):m("",!0)],16,La)):m("",!0),N.item.items&&N.item.items.length?(a(),l("div",Ta,[N.depth<5?(a(!0),l(M,{key:0},B(N.item.items,q=>(a(),$(z,{key:q.text,item:q,depth:N.depth+1},null,8,["item","depth"]))),128)):m("",!0)])):m("",!0)]),_:1},8,["class"])}}}),Na=k(wa,[["__scopeId","data-v-196b2e5f"]]),Ia=b({__name:"VPSidebarGroup",props:{items:{}},setup(o){const e=T(!0);let t=null;return R(()=>{t=setTimeout(()=>{t=null,e.value=!1},300)}),tt(()=>{t!=null&&(clearTimeout(t),t=null)}),(s,n)=>(a(!0),l(M,null,B(s.items,r=>(a(),l("div",{key:r.text,class:I(["group",{"no-transition":e.value}])},[g(Na,{item:r,depth:0},null,8,["item"])],2))),128))}}),Ma=k(Ia,[["__scopeId","data-v-9e426adc"]]),Ca={class:"nav",id:"VPSidebarNav","aria-labelledby":"sidebar-aria-label",tabindex:"-1"},Aa=b({__name:"VPSidebar",props:{open:{type:Boolean}},setup(o){const{sidebarGroups:e,hasSidebar:t}=U(),s=o,n=T(null),r=Ce(te?document.body:null);D([s,n],()=>{var v;s.open?(r.value=!0,(v=n.value)==null||v.focus()):r.value=!1},{immediate:!0,flush:"post"});const c=T(0);return D(e,()=>{c.value+=1},{deep:!0}),(v,d)=>i(t)?(a(),l("aside",{key:0,class:I(["VPSidebar",{open:v.open}]),ref_key:"navEl",ref:n,onClick:d[0]||(d[0]=nt(()=>{},["stop"]))},[d[2]||(d[2]=p("div",{class:"curtain"},null,-1)),p("nav",Ca,[d[1]||(d[1]=p("span",{class:"visually-hidden",id:"sidebar-aria-label"}," Sidebar Navigation ",-1)),u(v.$slots,"sidebar-nav-before",{},void 0,!0),(a(),$(Ma,{items:i(e),key:c.value},null,8,["items"])),u(v.$slots,"sidebar-nav-after",{},void 0,!0)])],2)):m("",!0)}}),Ba=k(Aa,[["__scopeId","data-v-18756405"]]),Ea=b({__name:"VPSkipLink",setup(o){const e=ee(),t=T();D(()=>e.path,()=>t.value.focus());function s({target:n}){const r=document.getElementById(decodeURIComponent(n.hash).slice(1));if(r){const c=()=>{r.removeAttribute("tabindex"),r.removeEventListener("blur",c)};r.setAttribute("tabindex","-1"),r.addEventListener("blur",c),r.focus(),window.scrollTo(0,0)}}return(n,r)=>(a(),l(M,null,[p("span",{ref_key:"backToTop",ref:t,tabindex:"-1"},null,512),p("a",{href:"#VPContent",class:"VPSkipLink visually-hidden",onClick:s}," Skip to content ")],64))}}),Ha=k(Ea,[["__scopeId","data-v-c3508ec8"]]),Oa=b({__name:"Layout",setup(o){const{isOpen:e,open:t,close:s}=U(),n=ee();D(()=>n.path,s),yt(e,s);const{frontmatter:r}=L(),c=Ae(),v=y(()=>!!c["home-hero-image"]);return he("hero-image-slot-exists",v),(d,h)=>{const P=K("Content");return i(r).layout!==!1?(a(),l("div",{key:0,class:I(["Layout",i(r).pageClass])},[u(d.$slots,"layout-top",{},void 0,!0),g(Ha),g(ct,{class:"backdrop",show:i(e),onClick:i(s)},null,8,["show","onClick"]),g(Sa,null,{"nav-bar-title-before":f(()=>[u(d.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(d.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(d.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(d.$slots,"nav-bar-content-after",{},void 0,!0)]),"nav-screen-content-before":f(()=>[u(d.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(d.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3}),g(ks,{open:i(e),onOpenMenu:i(t)},null,8,["open","onOpenMenu"]),g(Ba,{open:i(e)},{"sidebar-nav-before":f(()=>[u(d.$slots,"sidebar-nav-before",{},void 0,!0)]),"sidebar-nav-after":f(()=>[u(d.$slots,"sidebar-nav-after",{},void 0,!0)]),_:3},8,["open"]),g(ss,null,{"page-top":f(()=>[u(d.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(d.$slots,"page-bottom",{},void 0,!0)]),"not-found":f(()=>[u(d.$slots,"not-found",{},void 0,!0)]),"home-hero-before":f(()=>[u(d.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(d.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(d.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(d.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(d.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(d.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(d.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(d.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(d.$slots,"home-features-after",{},void 0,!0)]),"doc-footer-before":f(()=>[u(d.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(d.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(d.$slots,"doc-after",{},void 0,!0)]),"doc-top":f(()=>[u(d.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(d.$slots,"doc-bottom",{},void 0,!0)]),"aside-top":f(()=>[u(d.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(d.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(d.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(d.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(d.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(d.$slots,"aside-ads-after",{},void 0,!0)]),_:3}),g(ls),u(d.$slots,"layout-bottom",{},void 0,!0)],2)):(a(),$(P,{key:1}))}}}),Da=k(Oa,[["__scopeId","data-v-a9a9e638"]]),Fa={},Ra={class:"VPTeamPage"};function Ua(o,e){return a(),l("div",Ra,[u(o.$slots,"default")])}const Or=k(Fa,[["render",Ua],["__scopeId","data-v-c2f8e101"]]),za={},ja={class:"VPTeamPageTitle"},Ga={key:0,class:"title"},Ka={key:1,class:"lead"};function qa(o,e){return a(),l("div",ja,[o.$slots.title?(a(),l("h1",Ga,[u(o.$slots,"title",{},void 0,!0)])):m("",!0),o.$slots.lead?(a(),l("p",Ka,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0)])}const Dr=k(za,[["render",qa],["__scopeId","data-v-e277e15c"]]),Wa={},Ja={class:"VPTeamPageSection"},Ya={class:"title"},Qa={key:0,class:"title-text"},Xa={key:0,class:"lead"},Za={key:1,class:"members"};function xa(o,e){return a(),l("section",Ja,[p("div",Ya,[e[0]||(e[0]=p("div",{class:"title-line"},null,-1)),o.$slots.title?(a(),l("h2",Qa,[u(o.$slots,"title",{},void 0,!0)])):m("",!0)]),o.$slots.lead?(a(),l("p",Xa,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0),o.$slots.members?(a(),l("div",Za,[u(o.$slots,"members",{},void 0,!0)])):m("",!0)])}const Fr=k(Wa,[["render",xa],["__scopeId","data-v-d43bc49d"]]),er={class:"profile"},tr={class:"avatar"},nr=["src","alt"],sr={class:"data"},or={class:"name"},ar={key:0,class:"affiliation"},rr={key:0,class:"title"},ir={key:1,class:"at"},lr=["innerHTML"],cr={key:2,class:"links"},ur={key:0,class:"sp"},dr=b({__name:"VPTeamMembersItem",props:{size:{default:"medium"},member:{}},setup(o){return(e,t)=>(a(),l("article",{class:I(["VPTeamMembersItem",[e.size]])},[p("div",er,[p("figure",tr,[p("img",{class:"avatar-img",src:e.member.avatar,alt:e.member.name},null,8,nr)]),p("div",sr,[p("h1",or,w(e.member.name),1),e.member.title||e.member.org?(a(),l("p",ar,[e.member.title?(a(),l("span",rr,w(e.member.title),1)):m("",!0),e.member.title&&e.member.org?(a(),l("span",ir," @ ")):m("",!0),e.member.org?(a(),$(H,{key:2,class:I(["org",{link:e.member.orgLink}]),href:e.member.orgLink,"no-icon":""},{default:f(()=>[F(w(e.member.org),1)]),_:1},8,["class","href"])):m("",!0)])):m("",!0),e.member.desc?(a(),l("p",{key:1,class:"desc",innerHTML:e.member.desc},null,8,lr)):m("",!0),e.member.links?(a(),l("div",cr,[g(se,{links:e.member.links},null,8,["links"])])):m("",!0)])]),e.member.sponsor?(a(),l("div",ur,[g(H,{class:"sp-link",href:e.member.sponsor,"no-icon":""},{default:f(()=>[t[0]||(t[0]=p("span",{class:"vpi-heart sp-icon"},null,-1)),F(" "+w(e.member.actionText||"Sponsor"),1)]),_:1},8,["href"])])):m("",!0)],2))}}),pr=k(dr,[["__scopeId","data-v-f9987cb6"]]),vr={class:"container"},fr=b({__name:"VPTeamMembers",props:{size:{default:"medium"},members:{}},setup(o){const e=o,t=y(()=>[e.size,`count-${e.members.length}`]);return(s,n)=>(a(),l("div",{class:I(["VPTeamMembers",t.value])},[p("div",vr,[(a(!0),l(M,null,B(s.members,r=>(a(),l("div",{key:r.name,class:"item"},[g(pr,{size:s.size,member:r},null,8,["size","member"])]))),128))])],2))}}),Rr=k(fr,[["__scopeId","data-v-fba19bad"]]),Ve={Layout:Da,enhanceApp:({app:o})=>{o.component("Badge",rt)}},mr={};function hr(o,e){return e[0]||(e[0]=st('

    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    ',3))}const _r=k(mr,[["render",hr]]),br=b({__name:"VersionPicker",props:{screenMenu:{type:Boolean}},setup(o){const e=T([]),t=T("Versions"),s=T(!1);Le();const n=()=>typeof window<"u"&&(window.location.hostname==="localhost"||window.location.hostname==="127.0.0.1"),r=()=>{if(typeof window>"u")return"";const{origin:d,pathname:h}=window.location;if(d.includes("github.io")){const P=h.split("/").filter(Boolean),_=P.length>0?`/${P[0]}/`:"/";return`${d}${_}`}else return d},c=()=>new Promise(d=>{if(n()){d(!1);return}const h=setInterval(()=>{window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION&&(clearInterval(h),d(!0))},100);setTimeout(()=>{clearInterval(h),d(!1)},5e3)});return R(async()=>{if(!(typeof window>"u")){try{if(n()){const d=["dev"];e.value=d.map(h=>({text:h,link:"/"})),t.value="dev"}else{const d=await c(),h=y(()=>r());if(d&&window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION)e.value=window.DOC_VERSIONS.map(P=>({text:P,link:`${h.value}/${P}/`})),t.value=window.DOCUMENTER_CURRENT_VERSION;else{const P=["dev"];e.value=P.map(_=>({text:_,link:`${h.value}/${_}/`})),t.value="dev"}}}catch(d){console.warn("Error loading versions:",d);const h=["dev"],P=y(()=>r());e.value=h.map(_=>({text:_,link:`${P.value}/${_}/`})),t.value="dev"}s.value=!0}}),(d,h)=>s.value?(a(),l(M,{key:0},[!d.screenMenu&&e.value.length>0?(a(),$(Fe,{key:0,item:{text:t.value,items:e.value},class:"VPVersionPicker"},null,8,["item"])):d.screenMenu&&e.value.length>0?(a(),$(Ue,{key:1,text:t.value,items:e.value,class:"VPVersionPicker"},null,8,["text","items"])):m("",!0)],64)):m("",!0)}}),kr=k(br,[["__scopeId","data-v-d483b3a6"]]),gr=o=>{if(typeof document>"u")return{stabilizeScrollPosition:n=>async(...r)=>n(...r)};const e=document.documentElement;return{stabilizeScrollPosition:s=>async(...n)=>{const r=s(...n),c=o.value;if(!c)return r;const v=c.offsetTop-e.scrollTop;return await Ie(),e.scrollTop=c.offsetTop-v,r}}},ze="vitepress:tabSharedState",J=typeof localStorage<"u"?localStorage:null,je="vitepress:tabsSharedState",$r=()=>{const o=J==null?void 0:J.getItem(je);if(o)try{return JSON.parse(o)}catch{}return{}},yr=o=>{J&&J.setItem(je,JSON.stringify(o))},Pr=o=>{const e=ot({});D(()=>e.content,(t,s)=>{t&&s&&yr(t)},{deep:!0}),o.provide(ze,e)},Vr=(o,e)=>{const t=W(ze);if(!t)throw new Error("[vitepress-plugin-tabs] TabsSharedState should be injected");R(()=>{t.content||(t.content=$r())});const s=T(),n=y({get(){var d;const c=e.value,v=o.value;if(c){const h=(d=t.content)==null?void 0:d[c];if(h&&v.includes(h))return h}else{const h=s.value;if(h)return h}return v[0]},set(c){const v=e.value;v?t.content&&(t.content[v]=c):s.value=c}});return{selected:n,select:c=>{n.value=c}}};let Se=0;const Sr=()=>(Se++,""+Se);function Lr(){const o=Ae();return y(()=>{var s;const t=(s=o.default)==null?void 0:s.call(o);return t?t.filter(n=>typeof n.type=="object"&&"__name"in n.type&&n.type.__name==="PluginTabsTab"&&n.props).map(n=>{var r;return(r=n.props)==null?void 0:r.label}):[]})}const Ge="vitepress:tabSingleState",Tr=o=>{he(Ge,o)},wr=()=>{const o=W(Ge);if(!o)throw new Error("[vitepress-plugin-tabs] TabsSingleState should be injected");return o},Nr={class:"plugin-tabs"},Ir=["id","aria-selected","aria-controls","tabindex","onClick"],Mr=b({__name:"PluginTabs",props:{sharedStateKey:{}},setup(o){const e=o,t=Lr(),{selected:s,select:n}=Vr(t,at(e,"sharedStateKey")),r=T(),{stabilizeScrollPosition:c}=gr(r),v=c(n),d=T([]),h=_=>{var C;const V=t.value.indexOf(s.value);let S;_.key==="ArrowLeft"?S=V>=1?V-1:t.value.length-1:_.key==="ArrowRight"&&(S=V(a(),l("div",Nr,[p("div",{ref_key:"tablist",ref:r,class:"plugin-tabs--tab-list",role:"tablist",onKeydown:h},[(a(!0),l(M,null,B(i(t),S=>(a(),l("button",{id:`tab-${S}-${i(P)}`,ref_for:!0,ref_key:"buttonRefs",ref:d,key:S,role:"tab",class:"plugin-tabs--tab","aria-selected":S===i(s),"aria-controls":`panel-${S}-${i(P)}`,tabindex:S===i(s)?0:-1,onClick:()=>i(v)(S)},w(S),9,Ir))),128))],544),u(_.$slots,"default")]))}}),Cr=["id","aria-labelledby"],Ar=b({__name:"PluginTabsTab",props:{label:{}},setup(o){const{uid:e,selected:t}=wr();return(s,n)=>i(t)===s.label?(a(),l("div",{key:0,id:`panel-${s.label}-${i(e)}`,class:"plugin-tabs--content",role:"tabpanel",tabindex:"0","aria-labelledby":`tab-${s.label}-${i(e)}`},[u(s.$slots,"default",{},void 0,!0)],8,Cr)):m("",!0)}}),Br=k(Ar,[["__scopeId","data-v-9b0d03d2"]]),Er=o=>{Pr(o),o.component("PluginTabs",Mr),o.component("PluginTabsTab",Br)},Ur={extends:Ve,Layout(){return ye(Ve.Layout,null,{"aside-ads-before":()=>ye(_r)})},enhanceApp({app:o}){Er(o),o.component("VersionPicker",kr)}};export{Ur as R,Dr as V,Fr as a,Rr as b,Or as c,yo as d,L as u}; +const __vite__mapDeps=(i,m=__vite__mapDeps,d=(m.f||(m.f=["assets/chunks/VPLocalSearchBox.D7LKkx_r.js","assets/chunks/framework.DjZDIZsN.js"])))=>i.map(i=>d[i]); +import{d as b,o as a,c as l,r as u,a as F,t as w,n as I,b as $,w as f,e as m,T as pe,_ as k,u as Le,i as Ke,f as qe,g as ve,h as y,j as p,k as i,l as G,m as ie,p as T,q as D,s as Z,v as R,x as fe,y as me,z as We,A as Je,B as K,F as M,C as B,D as Te,E as x,G as g,H as O,I as we,J as ee,K as j,L as W,M as Ye,N as Ne,O as le,P as Ie,Q as Me,R as te,S as Qe,U as Xe,V as Ze,W as Ce,X as he,Y as xe,Z as et,$ as tt,a0 as nt,a1 as Ae,a2 as st,a3 as ot,a4 as at,a5 as ye}from"./framework.DjZDIZsN.js";const rt=b({__name:"VPBadge",props:{text:{},type:{default:"tip"}},setup(o){return(e,t)=>(a(),l("span",{class:I(["VPBadge",e.type])},[u(e.$slots,"default",{},()=>[F(w(e.text),1)])],2))}}),it={key:0,class:"VPBackdrop"},lt=b({__name:"VPBackdrop",props:{show:{type:Boolean}},setup(o){return(e,t)=>(a(),$(pe,{name:"fade"},{default:f(()=>[e.show?(a(),l("div",it)):m("",!0)]),_:1}))}}),ct=k(lt,[["__scopeId","data-v-b06cdb19"]]),L=Le;function ut(o,e){let t,s=!1;return()=>{t&&clearTimeout(t),s?t=setTimeout(o,e):(o(),(s=!0)&&setTimeout(()=>s=!1,e))}}function ce(o){return/^\//.test(o)?o:`/${o}`}function _e(o){const{pathname:e,search:t,hash:s,protocol:n}=new URL(o,"http://a.com");if(Ke(o)||o.startsWith("#")||!n.startsWith("http")||!qe(e))return o;const{site:r}=L(),c=e.endsWith("/")||e.endsWith(".html")?o:o.replace(/(?:(^\.+)\/)?.*$/,`$1${e.replace(/(\.md)?$/,r.value.cleanUrls?"":".html")}${t}${s}`);return ve(c)}function Y({correspondingLink:o=!1}={}){const{site:e,localeIndex:t,page:s,theme:n,hash:r}=L(),c=y(()=>{var d,h;return{label:(d=e.value.locales[t.value])==null?void 0:d.label,link:((h=e.value.locales[t.value])==null?void 0:h.link)||(t.value==="root"?"/":`/${t.value}/`)}});return{localeLinks:y(()=>Object.entries(e.value.locales).flatMap(([d,h])=>c.value.label===h.label?[]:{text:h.label,link:dt(h.link||(d==="root"?"/":`/${d}/`),n.value.i18nRouting!==!1&&o,s.value.relativePath.slice(c.value.link.length-1),!e.value.cleanUrls)+r.value})),currentLang:c}}function dt(o,e,t,s){return e?o.replace(/\/$/,"")+ce(t.replace(/(^|\/)index\.md$/,"$1").replace(/\.md$/,s?".html":"")):o}const pt={class:"NotFound"},vt={class:"code"},ft={class:"title"},mt={class:"quote"},ht={class:"action"},_t=["href","aria-label"],bt=b({__name:"NotFound",setup(o){const{theme:e}=L(),{currentLang:t}=Y();return(s,n)=>{var r,c,v,d,h;return a(),l("div",pt,[p("p",vt,w(((r=i(e).notFound)==null?void 0:r.code)??"404"),1),p("h1",ft,w(((c=i(e).notFound)==null?void 0:c.title)??"PAGE NOT FOUND"),1),n[0]||(n[0]=p("div",{class:"divider"},null,-1)),p("blockquote",mt,w(((v=i(e).notFound)==null?void 0:v.quote)??"But if you don't change your direction, and if you keep looking, you may end up where you are heading."),1),p("div",ht,[p("a",{class:"link",href:i(ve)(i(t).link),"aria-label":((d=i(e).notFound)==null?void 0:d.linkLabel)??"go to home"},w(((h=i(e).notFound)==null?void 0:h.linkText)??"Take me home"),9,_t)])])}}}),kt=k(bt,[["__scopeId","data-v-951cab6c"]]);function Be(o,e){if(Array.isArray(o))return Q(o);if(o==null)return[];e=ce(e);const t=Object.keys(o).sort((n,r)=>r.split("/").length-n.split("/").length).find(n=>e.startsWith(ce(n))),s=t?o[t]:[];return Array.isArray(s)?Q(s):Q(s.items,s.base)}function gt(o){const e=[];let t=0;for(const s in o){const n=o[s];if(n.items){t=e.push(n);continue}e[t]||e.push({items:[]}),e[t].items.push(n)}return e}function $t(o){const e=[];function t(s){for(const n of s)n.text&&n.link&&e.push({text:n.text,link:n.link,docFooterText:n.docFooterText}),n.items&&t(n.items)}return t(o),e}function ue(o,e){return Array.isArray(e)?e.some(t=>ue(o,t)):G(o,e.link)?!0:e.items?ue(o,e.items):!1}function Q(o,e){return[...o].map(t=>{const s={...t},n=s.base||e;return n&&s.link&&(s.link=n+s.link),s.items&&(s.items=Q(s.items,n)),s})}function U(){const{frontmatter:o,page:e,theme:t}=L(),s=ie("(min-width: 960px)"),n=T(!1),r=y(()=>{const A=t.value.sidebar,N=e.value.relativePath;return A?Be(A,N):[]}),c=T(r.value);D(r,(A,N)=>{JSON.stringify(A)!==JSON.stringify(N)&&(c.value=r.value)});const v=y(()=>o.value.sidebar!==!1&&c.value.length>0&&o.value.layout!=="home"),d=y(()=>h?o.value.aside==null?t.value.aside==="left":o.value.aside==="left":!1),h=y(()=>o.value.layout==="home"?!1:o.value.aside!=null?!!o.value.aside:t.value.aside!==!1),P=y(()=>v.value&&s.value),_=y(()=>v.value?gt(c.value):[]);function V(){n.value=!0}function S(){n.value=!1}function C(){n.value?S():V()}return{isOpen:n,sidebar:c,sidebarGroups:_,hasSidebar:v,hasAside:h,leftAside:d,isSidebarEnabled:P,open:V,close:S,toggle:C}}function yt(o,e){let t;Z(()=>{t=o.value?document.activeElement:void 0}),R(()=>{window.addEventListener("keyup",s)}),fe(()=>{window.removeEventListener("keyup",s)});function s(n){n.key==="Escape"&&o.value&&(e(),t==null||t.focus())}}function Pt(o){const{page:e,hash:t}=L(),s=T(!1),n=y(()=>o.value.collapsed!=null),r=y(()=>!!o.value.link),c=T(!1),v=()=>{c.value=G(e.value.relativePath,o.value.link)};D([e,o,t],v),R(v);const d=y(()=>c.value?!0:o.value.items?ue(e.value.relativePath,o.value.items):!1),h=y(()=>!!(o.value.items&&o.value.items.length));Z(()=>{s.value=!!(n.value&&o.value.collapsed)}),me(()=>{(c.value||d.value)&&(s.value=!1)});function P(){n.value&&(s.value=!s.value)}return{collapsed:s,collapsible:n,isLink:r,isActiveLink:c,hasActiveLink:d,hasChildren:h,toggle:P}}function Vt(){const{hasSidebar:o}=U(),e=ie("(min-width: 960px)"),t=ie("(min-width: 1280px)");return{isAsideEnabled:y(()=>!t.value&&!e.value?!1:o.value?t.value:e.value)}}const de=[];function Ee(o){return typeof o.outline=="object"&&!Array.isArray(o.outline)&&o.outline.label||o.outlineTitle||"On this page"}function be(o){const e=[...document.querySelectorAll(".VPDoc :where(h1,h2,h3,h4,h5,h6)")].filter(t=>t.id&&t.hasChildNodes()).map(t=>{const s=Number(t.tagName[1]);return{element:t,title:St(t),link:"#"+t.id,level:s}});return Lt(e,o)}function St(o){let e="";for(const t of o.childNodes)if(t.nodeType===1){if(t.classList.contains("VPBadge")||t.classList.contains("header-anchor")||t.classList.contains("ignore-header"))continue;e+=t.textContent}else t.nodeType===3&&(e+=t.textContent);return e.trim()}function Lt(o,e){if(e===!1)return[];const t=(typeof e=="object"&&!Array.isArray(e)?e.level:e)||2,[s,n]=typeof t=="number"?[t,t]:t==="deep"?[2,6]:t;return Nt(o,s,n)}function Tt(o,e){const{isAsideEnabled:t}=Vt(),s=ut(r,100);let n=null;R(()=>{requestAnimationFrame(r),window.addEventListener("scroll",s)}),We(()=>{c(location.hash)}),fe(()=>{window.removeEventListener("scroll",s)});function r(){if(!t.value)return;const v=window.scrollY,d=window.innerHeight,h=document.body.offsetHeight,P=Math.abs(v+d-h)<1,_=de.map(({element:S,link:C})=>({link:C,top:wt(S)})).filter(({top:S})=>!Number.isNaN(S)).sort((S,C)=>S.top-C.top);if(!_.length){c(null);return}if(v<1){c(null);return}if(P){c(_[_.length-1].link);return}let V=null;for(const{link:S,top:C}of _){if(C>v+Je()+4)break;V=S}c(V)}function c(v){n&&n.classList.remove("active"),v==null?n=null:n=o.value.querySelector(`a[href="${decodeURIComponent(v)}"]`);const d=n;d?(d.classList.add("active"),e.value.style.top=d.offsetTop+39+"px",e.value.style.opacity="1"):(e.value.style.top="33px",e.value.style.opacity="0")}}function wt(o){let e=0;for(;o!==document.body;){if(o===null)return NaN;e+=o.offsetTop,o=o.offsetParent}return e}function Nt(o,e,t){de.length=0;const s=[],n=[];return o.forEach(r=>{const c={...r,children:[]};let v=n[n.length-1];for(;v&&v.level>=c.level;)n.pop(),v=n[n.length-1];if(c.element.classList.contains("ignore-header")||v&&"shouldIgnore"in v){n.push({level:c.level,shouldIgnore:!0});return}c.level>t||c.level{const n=K("VPDocOutlineItem",!0);return a(),l("ul",{class:I(["VPDocOutlineItem",t.root?"root":"nested"])},[(a(!0),l(M,null,B(t.headers,({children:r,link:c,title:v})=>(a(),l("li",null,[p("a",{class:"outline-link",href:c,onClick:e,title:v},w(v),9,It),r!=null&&r.length?(a(),$(n,{key:0,headers:r},null,8,["headers"])):m("",!0)]))),256))],2)}}}),He=k(Mt,[["__scopeId","data-v-3f927ebe"]]),Ct={class:"content"},At={"aria-level":"2",class:"outline-title",id:"doc-outline-aria-label",role:"heading"},Bt=b({__name:"VPDocAsideOutline",setup(o){const{frontmatter:e,theme:t}=L(),s=Te([]);x(()=>{s.value=be(e.value.outline??t.value.outline)});const n=T(),r=T();return Tt(n,r),(c,v)=>(a(),l("nav",{"aria-labelledby":"doc-outline-aria-label",class:I(["VPDocAsideOutline",{"has-outline":s.value.length>0}]),ref_key:"container",ref:n},[p("div",Ct,[p("div",{class:"outline-marker",ref_key:"marker",ref:r},null,512),p("div",At,w(i(Ee)(i(t))),1),g(He,{headers:s.value,root:!0},null,8,["headers"])])],2))}}),Et=k(Bt,[["__scopeId","data-v-b38bf2ff"]]),Ht={class:"VPDocAsideCarbonAds"},Ot=b({__name:"VPDocAsideCarbonAds",props:{carbonAds:{}},setup(o){const e=()=>null;return(t,s)=>(a(),l("div",Ht,[g(i(e),{"carbon-ads":t.carbonAds},null,8,["carbon-ads"])]))}}),Dt={class:"VPDocAside"},Ft=b({__name:"VPDocAside",setup(o){const{theme:e}=L();return(t,s)=>(a(),l("div",Dt,[u(t.$slots,"aside-top",{},void 0,!0),u(t.$slots,"aside-outline-before",{},void 0,!0),g(Et),u(t.$slots,"aside-outline-after",{},void 0,!0),s[0]||(s[0]=p("div",{class:"spacer"},null,-1)),u(t.$slots,"aside-ads-before",{},void 0,!0),i(e).carbonAds?(a(),$(Ot,{key:0,"carbon-ads":i(e).carbonAds},null,8,["carbon-ads"])):m("",!0),u(t.$slots,"aside-ads-after",{},void 0,!0),u(t.$slots,"aside-bottom",{},void 0,!0)]))}}),Rt=k(Ft,[["__scopeId","data-v-6d7b3c46"]]);function Ut(){const{theme:o,page:e}=L();return y(()=>{const{text:t="Edit this page",pattern:s=""}=o.value.editLink||{};let n;return typeof s=="function"?n=s(e.value):n=s.replace(/:path/g,e.value.filePath),{url:n,text:t}})}function zt(){const{page:o,theme:e,frontmatter:t}=L();return y(()=>{var h,P,_,V,S,C,A,N;const s=Be(e.value.sidebar,o.value.relativePath),n=$t(s),r=jt(n,E=>E.link.replace(/[?#].*$/,"")),c=r.findIndex(E=>G(o.value.relativePath,E.link)),v=((h=e.value.docFooter)==null?void 0:h.prev)===!1&&!t.value.prev||t.value.prev===!1,d=((P=e.value.docFooter)==null?void 0:P.next)===!1&&!t.value.next||t.value.next===!1;return{prev:v?void 0:{text:(typeof t.value.prev=="string"?t.value.prev:typeof t.value.prev=="object"?t.value.prev.text:void 0)??((_=r[c-1])==null?void 0:_.docFooterText)??((V=r[c-1])==null?void 0:V.text),link:(typeof t.value.prev=="object"?t.value.prev.link:void 0)??((S=r[c-1])==null?void 0:S.link)},next:d?void 0:{text:(typeof t.value.next=="string"?t.value.next:typeof t.value.next=="object"?t.value.next.text:void 0)??((C=r[c+1])==null?void 0:C.docFooterText)??((A=r[c+1])==null?void 0:A.text),link:(typeof t.value.next=="object"?t.value.next.link:void 0)??((N=r[c+1])==null?void 0:N.link)}}})}function jt(o,e){const t=new Set;return o.filter(s=>{const n=e(s);return t.has(n)?!1:t.add(n)})}const H=b({__name:"VPLink",props:{tag:{},href:{},noIcon:{type:Boolean},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.tag??(e.href?"a":"span")),s=y(()=>e.href&&we.test(e.href)||e.target==="_blank");return(n,r)=>(a(),$(O(t.value),{class:I(["VPLink",{link:n.href,"vp-external-link-icon":s.value,"no-icon":n.noIcon}]),href:n.href?i(_e)(n.href):void 0,target:n.target??(s.value?"_blank":void 0),rel:n.rel??(s.value?"noreferrer":void 0)},{default:f(()=>[u(n.$slots,"default")]),_:3},8,["class","href","target","rel"]))}}),Gt={class:"VPLastUpdated"},Kt=["datetime"],qt=b({__name:"VPDocFooterLastUpdated",setup(o){const{theme:e,page:t,lang:s}=L(),n=y(()=>new Date(t.value.lastUpdated)),r=y(()=>n.value.toISOString()),c=T("");return R(()=>{Z(()=>{var v,d,h;c.value=new Intl.DateTimeFormat((d=(v=e.value.lastUpdated)==null?void 0:v.formatOptions)!=null&&d.forceLocale?s.value:void 0,((h=e.value.lastUpdated)==null?void 0:h.formatOptions)??{dateStyle:"short",timeStyle:"short"}).format(n.value)})}),(v,d)=>{var h;return a(),l("p",Gt,[F(w(((h=i(e).lastUpdated)==null?void 0:h.text)||i(e).lastUpdatedText||"Last updated")+": ",1),p("time",{datetime:r.value},w(c.value),9,Kt)])}}}),Wt=k(qt,[["__scopeId","data-v-475f71b8"]]),Jt={key:0,class:"VPDocFooter"},Yt={key:0,class:"edit-info"},Qt={key:0,class:"edit-link"},Xt={key:1,class:"last-updated"},Zt={key:1,class:"prev-next","aria-labelledby":"doc-footer-aria-label"},xt={class:"pager"},en=["innerHTML"],tn=["innerHTML"],nn={class:"pager"},sn=["innerHTML"],on=["innerHTML"],an=b({__name:"VPDocFooter",setup(o){const{theme:e,page:t,frontmatter:s}=L(),n=Ut(),r=zt(),c=y(()=>e.value.editLink&&s.value.editLink!==!1),v=y(()=>t.value.lastUpdated),d=y(()=>c.value||v.value||r.value.prev||r.value.next);return(h,P)=>{var _,V,S,C;return d.value?(a(),l("footer",Jt,[u(h.$slots,"doc-footer-before",{},void 0,!0),c.value||v.value?(a(),l("div",Yt,[c.value?(a(),l("div",Qt,[g(H,{class:"edit-link-button",href:i(n).url,"no-icon":!0},{default:f(()=>[P[0]||(P[0]=p("span",{class:"vpi-square-pen edit-link-icon"},null,-1)),F(" "+w(i(n).text),1)]),_:1},8,["href"])])):m("",!0),v.value?(a(),l("div",Xt,[g(Wt)])):m("",!0)])):m("",!0),(_=i(r).prev)!=null&&_.link||(V=i(r).next)!=null&&V.link?(a(),l("nav",Zt,[P[1]||(P[1]=p("span",{class:"visually-hidden",id:"doc-footer-aria-label"},"Pager",-1)),p("div",xt,[(S=i(r).prev)!=null&&S.link?(a(),$(H,{key:0,class:"pager-link prev",href:i(r).prev.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.prev)||"Previous page"},null,8,en),p("span",{class:"title",innerHTML:i(r).prev.text},null,8,tn)]}),_:1},8,["href"])):m("",!0)]),p("div",nn,[(C=i(r).next)!=null&&C.link?(a(),$(H,{key:0,class:"pager-link next",href:i(r).next.link},{default:f(()=>{var A;return[p("span",{class:"desc",innerHTML:((A=i(e).docFooter)==null?void 0:A.next)||"Next page"},null,8,sn),p("span",{class:"title",innerHTML:i(r).next.text},null,8,on)]}),_:1},8,["href"])):m("",!0)])])):m("",!0)])):m("",!0)}}}),rn=k(an,[["__scopeId","data-v-4f9813fa"]]),ln={class:"container"},cn={class:"aside-container"},un={class:"aside-content"},dn={class:"content"},pn={class:"content-container"},vn={class:"main"},fn=b({__name:"VPDoc",setup(o){const{theme:e}=L(),t=ee(),{hasSidebar:s,hasAside:n,leftAside:r}=U(),c=y(()=>t.path.replace(/[./]+/g,"_").replace(/_html$/,""));return(v,d)=>{const h=K("Content");return a(),l("div",{class:I(["VPDoc",{"has-sidebar":i(s),"has-aside":i(n)}])},[u(v.$slots,"doc-top",{},void 0,!0),p("div",ln,[i(n)?(a(),l("div",{key:0,class:I(["aside",{"left-aside":i(r)}])},[d[0]||(d[0]=p("div",{class:"aside-curtain"},null,-1)),p("div",cn,[p("div",un,[g(Rt,null,{"aside-top":f(()=>[u(v.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(v.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(v.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(v.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(v.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(v.$slots,"aside-ads-after",{},void 0,!0)]),_:3})])])],2)):m("",!0),p("div",dn,[p("div",pn,[u(v.$slots,"doc-before",{},void 0,!0),p("main",vn,[g(h,{class:I(["vp-doc",[c.value,i(e).externalLinkIcon&&"external-link-icon-enabled"]])},null,8,["class"])]),g(rn,null,{"doc-footer-before":f(()=>[u(v.$slots,"doc-footer-before",{},void 0,!0)]),_:3}),u(v.$slots,"doc-after",{},void 0,!0)])])]),u(v.$slots,"doc-bottom",{},void 0,!0)],2)}}}),mn=k(fn,[["__scopeId","data-v-83890dd9"]]),hn=b({__name:"VPButton",props:{tag:{},size:{default:"medium"},theme:{default:"brand"},text:{},href:{},target:{},rel:{}},setup(o){const e=o,t=y(()=>e.href&&we.test(e.href)),s=y(()=>e.tag||(e.href?"a":"button"));return(n,r)=>(a(),$(O(s.value),{class:I(["VPButton",[n.size,n.theme]]),href:n.href?i(_e)(n.href):void 0,target:e.target??(t.value?"_blank":void 0),rel:e.rel??(t.value?"noreferrer":void 0)},{default:f(()=>[F(w(n.text),1)]),_:1},8,["class","href","target","rel"]))}}),_n=k(hn,[["__scopeId","data-v-906d7fb4"]]),bn=["src","alt"],kn=b({inheritAttrs:!1,__name:"VPImage",props:{image:{},alt:{}},setup(o){return(e,t)=>{const s=K("VPImage",!0);return e.image?(a(),l(M,{key:0},[typeof e.image=="string"||"src"in e.image?(a(),l("img",j({key:0,class:"VPImage"},typeof e.image=="string"?e.$attrs:{...e.image,...e.$attrs},{src:i(ve)(typeof e.image=="string"?e.image:e.image.src),alt:e.alt??(typeof e.image=="string"?"":e.image.alt||"")}),null,16,bn)):(a(),l(M,{key:1},[g(s,j({class:"dark",image:e.image.dark,alt:e.image.alt},e.$attrs),null,16,["image","alt"]),g(s,j({class:"light",image:e.image.light,alt:e.image.alt},e.$attrs),null,16,["image","alt"])],64))],64)):m("",!0)}}}),X=k(kn,[["__scopeId","data-v-35a7d0b8"]]),gn={class:"container"},$n={class:"main"},yn={key:0,class:"name"},Pn=["innerHTML"],Vn=["innerHTML"],Sn=["innerHTML"],Ln={key:0,class:"actions"},Tn={key:0,class:"image"},wn={class:"image-container"},Nn=b({__name:"VPHero",props:{name:{},text:{},tagline:{},image:{},actions:{}},setup(o){const e=W("hero-image-slot-exists");return(t,s)=>(a(),l("div",{class:I(["VPHero",{"has-image":t.image||i(e)}])},[p("div",gn,[p("div",$n,[u(t.$slots,"home-hero-info-before",{},void 0,!0),u(t.$slots,"home-hero-info",{},()=>[t.name?(a(),l("h1",yn,[p("span",{innerHTML:t.name,class:"clip"},null,8,Pn)])):m("",!0),t.text?(a(),l("p",{key:1,innerHTML:t.text,class:"text"},null,8,Vn)):m("",!0),t.tagline?(a(),l("p",{key:2,innerHTML:t.tagline,class:"tagline"},null,8,Sn)):m("",!0)],!0),u(t.$slots,"home-hero-info-after",{},void 0,!0),t.actions?(a(),l("div",Ln,[(a(!0),l(M,null,B(t.actions,n=>(a(),l("div",{key:n.link,class:"action"},[g(_n,{tag:"a",size:"medium",theme:n.theme,text:n.text,href:n.link,target:n.target,rel:n.rel},null,8,["theme","text","href","target","rel"])]))),128))])):m("",!0),u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),t.image||i(e)?(a(),l("div",Tn,[p("div",wn,[s[0]||(s[0]=p("div",{class:"image-bg"},null,-1)),u(t.$slots,"home-hero-image",{},()=>[t.image?(a(),$(X,{key:0,class:"image-src",image:t.image},null,8,["image"])):m("",!0)],!0)])])):m("",!0)])],2))}}),In=k(Nn,[["__scopeId","data-v-955009fc"]]),Mn=b({__name:"VPHomeHero",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).hero?(a(),$(In,{key:0,class:"VPHomeHero",name:i(e).hero.name,text:i(e).hero.text,tagline:i(e).hero.tagline,image:i(e).hero.image,actions:i(e).hero.actions},{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before")]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info")]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after")]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after")]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image")]),_:3},8,["name","text","tagline","image","actions"])):m("",!0)}}),Cn={class:"box"},An={key:0,class:"icon"},Bn=["innerHTML"],En=["innerHTML"],Hn=["innerHTML"],On={key:4,class:"link-text"},Dn={class:"link-text-value"},Fn=b({__name:"VPFeature",props:{icon:{},title:{},details:{},link:{},linkText:{},rel:{},target:{}},setup(o){return(e,t)=>(a(),$(H,{class:"VPFeature",href:e.link,rel:e.rel,target:e.target,"no-icon":!0,tag:e.link?"a":"div"},{default:f(()=>[p("article",Cn,[typeof e.icon=="object"&&e.icon.wrap?(a(),l("div",An,[g(X,{image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])])):typeof e.icon=="object"?(a(),$(X,{key:1,image:e.icon,alt:e.icon.alt,height:e.icon.height||48,width:e.icon.width||48},null,8,["image","alt","height","width"])):e.icon?(a(),l("div",{key:2,class:"icon",innerHTML:e.icon},null,8,Bn)):m("",!0),p("h2",{class:"title",innerHTML:e.title},null,8,En),e.details?(a(),l("p",{key:3,class:"details",innerHTML:e.details},null,8,Hn)):m("",!0),e.linkText?(a(),l("div",On,[p("p",Dn,[F(w(e.linkText)+" ",1),t[0]||(t[0]=p("span",{class:"vpi-arrow-right link-text-icon"},null,-1))])])):m("",!0)])]),_:1},8,["href","rel","target","tag"]))}}),Rn=k(Fn,[["__scopeId","data-v-f5e9645b"]]),Un={key:0,class:"VPFeatures"},zn={class:"container"},jn={class:"items"},Gn=b({__name:"VPFeatures",props:{features:{}},setup(o){const e=o,t=y(()=>{const s=e.features.length;if(s){if(s===2)return"grid-2";if(s===3)return"grid-3";if(s%3===0)return"grid-6";if(s>3)return"grid-4"}else return});return(s,n)=>s.features?(a(),l("div",Un,[p("div",zn,[p("div",jn,[(a(!0),l(M,null,B(s.features,r=>(a(),l("div",{key:r.title,class:I(["item",[t.value]])},[g(Rn,{icon:r.icon,title:r.title,details:r.details,link:r.link,"link-text":r.linkText,rel:r.rel,target:r.target},null,8,["icon","title","details","link","link-text","rel","target"])],2))),128))])])])):m("",!0)}}),Kn=k(Gn,[["__scopeId","data-v-d0a190d7"]]),qn=b({__name:"VPHomeFeatures",setup(o){const{frontmatter:e}=L();return(t,s)=>i(e).features?(a(),$(Kn,{key:0,class:"VPHomeFeatures",features:i(e).features},null,8,["features"])):m("",!0)}}),Wn=b({__name:"VPHomeContent",setup(o){const{width:e}=Ye({initialWidth:0,includeScrollbar:!1});return(t,s)=>(a(),l("div",{class:"vp-doc container",style:Ne(i(e)?{"--vp-offset":`calc(50% - ${i(e)/2}px)`}:{})},[u(t.$slots,"default",{},void 0,!0)],4))}}),Jn=k(Wn,[["__scopeId","data-v-7a48a447"]]),Yn={class:"VPHome"},Qn=b({__name:"VPHome",setup(o){const{frontmatter:e}=L();return(t,s)=>{const n=K("Content");return a(),l("div",Yn,[u(t.$slots,"home-hero-before",{},void 0,!0),g(Mn,null,{"home-hero-info-before":f(()=>[u(t.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(t.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(t.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(t.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(t.$slots,"home-hero-image",{},void 0,!0)]),_:3}),u(t.$slots,"home-hero-after",{},void 0,!0),u(t.$slots,"home-features-before",{},void 0,!0),g(qn),u(t.$slots,"home-features-after",{},void 0,!0),i(e).markdownStyles!==!1?(a(),$(Jn,{key:0},{default:f(()=>[g(n)]),_:1})):(a(),$(n,{key:1}))])}}}),Xn=k(Qn,[["__scopeId","data-v-cbb6ec48"]]),Zn={},xn={class:"VPPage"};function es(o,e){const t=K("Content");return a(),l("div",xn,[u(o.$slots,"page-top"),g(t),u(o.$slots,"page-bottom")])}const ts=k(Zn,[["render",es]]),ns=b({__name:"VPContent",setup(o){const{page:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>(a(),l("div",{class:I(["VPContent",{"has-sidebar":i(s),"is-home":i(t).layout==="home"}]),id:"VPContent"},[i(e).isNotFound?u(n.$slots,"not-found",{key:0},()=>[g(kt)],!0):i(t).layout==="page"?(a(),$(ts,{key:1},{"page-top":f(()=>[u(n.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(n.$slots,"page-bottom",{},void 0,!0)]),_:3})):i(t).layout==="home"?(a(),$(Xn,{key:2},{"home-hero-before":f(()=>[u(n.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(n.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(n.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(n.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(n.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(n.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(n.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(n.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(n.$slots,"home-features-after",{},void 0,!0)]),_:3})):i(t).layout&&i(t).layout!=="doc"?(a(),$(O(i(t).layout),{key:3})):(a(),$(mn,{key:4},{"doc-top":f(()=>[u(n.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(n.$slots,"doc-bottom",{},void 0,!0)]),"doc-footer-before":f(()=>[u(n.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(n.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(n.$slots,"doc-after",{},void 0,!0)]),"aside-top":f(()=>[u(n.$slots,"aside-top",{},void 0,!0)]),"aside-outline-before":f(()=>[u(n.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(n.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(n.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(n.$slots,"aside-ads-after",{},void 0,!0)]),"aside-bottom":f(()=>[u(n.$slots,"aside-bottom",{},void 0,!0)]),_:3}))],2))}}),ss=k(ns,[["__scopeId","data-v-91765379"]]),os={class:"container"},as=["innerHTML"],rs=["innerHTML"],is=b({__name:"VPFooter",setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U();return(n,r)=>i(e).footer&&i(t).footer!==!1?(a(),l("footer",{key:0,class:I(["VPFooter",{"has-sidebar":i(s)}])},[p("div",os,[i(e).footer.message?(a(),l("p",{key:0,class:"message",innerHTML:i(e).footer.message},null,8,as)):m("",!0),i(e).footer.copyright?(a(),l("p",{key:1,class:"copyright",innerHTML:i(e).footer.copyright},null,8,rs)):m("",!0)])],2)):m("",!0)}}),ls=k(is,[["__scopeId","data-v-c970a860"]]);function cs(){const{theme:o,frontmatter:e}=L(),t=Te([]),s=y(()=>t.value.length>0);return x(()=>{t.value=be(e.value.outline??o.value.outline)}),{headers:t,hasLocalNav:s}}const us={class:"menu-text"},ds={class:"header"},ps={class:"outline"},vs=b({__name:"VPLocalNavOutlineDropdown",props:{headers:{},navHeight:{}},setup(o){const e=o,{theme:t}=L(),s=T(!1),n=T(0),r=T(),c=T();function v(_){var V;(V=r.value)!=null&&V.contains(_.target)||(s.value=!1)}D(s,_=>{if(_){document.addEventListener("click",v);return}document.removeEventListener("click",v)}),le("Escape",()=>{s.value=!1}),x(()=>{s.value=!1});function d(){s.value=!s.value,n.value=window.innerHeight+Math.min(window.scrollY-e.navHeight,0)}function h(_){_.target.classList.contains("outline-link")&&(c.value&&(c.value.style.transition="none"),Ie(()=>{s.value=!1}))}function P(){s.value=!1,window.scrollTo({top:0,left:0,behavior:"smooth"})}return(_,V)=>(a(),l("div",{class:"VPLocalNavOutlineDropdown",style:Ne({"--vp-vh":n.value+"px"}),ref_key:"main",ref:r},[_.headers.length>0?(a(),l("button",{key:0,onClick:d,class:I({open:s.value})},[p("span",us,w(i(Ee)(i(t))),1),V[0]||(V[0]=p("span",{class:"vpi-chevron-right icon"},null,-1))],2)):(a(),l("button",{key:1,onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)),g(pe,{name:"flyout"},{default:f(()=>[s.value?(a(),l("div",{key:0,ref_key:"items",ref:c,class:"items",onClick:h},[p("div",ds,[p("a",{class:"top-link",href:"#",onClick:P},w(i(t).returnToTopLabel||"Return to top"),1)]),p("div",ps,[g(He,{headers:_.headers},null,8,["headers"])])],512)):m("",!0)]),_:1})],4))}}),fs=k(vs,[["__scopeId","data-v-bc9dc845"]]),ms={class:"container"},hs=["aria-expanded"],_s={class:"menu-text"},bs=b({__name:"VPLocalNav",props:{open:{type:Boolean}},emits:["open-menu"],setup(o){const{theme:e,frontmatter:t}=L(),{hasSidebar:s}=U(),{headers:n}=cs(),{y:r}=Me(),c=T(0);R(()=>{c.value=parseInt(getComputedStyle(document.documentElement).getPropertyValue("--vp-nav-height"))}),x(()=>{n.value=be(t.value.outline??e.value.outline)});const v=y(()=>n.value.length===0),d=y(()=>v.value&&!s.value),h=y(()=>({VPLocalNav:!0,"has-sidebar":s.value,empty:v.value,fixed:d.value}));return(P,_)=>i(t).layout!=="home"&&(!d.value||i(r)>=c.value)?(a(),l("div",{key:0,class:I(h.value)},[p("div",ms,[i(s)?(a(),l("button",{key:0,class:"menu","aria-expanded":P.open,"aria-controls":"VPSidebarNav",onClick:_[0]||(_[0]=V=>P.$emit("open-menu"))},[_[1]||(_[1]=p("span",{class:"vpi-align-left menu-icon"},null,-1)),p("span",_s,w(i(e).sidebarMenuLabel||"Menu"),1)],8,hs)):m("",!0),g(fs,{headers:i(n),navHeight:c.value},null,8,["headers","navHeight"])])],2)):m("",!0)}}),ks=k(bs,[["__scopeId","data-v-070ab83d"]]);function gs(){const o=T(!1);function e(){o.value=!0,window.addEventListener("resize",n)}function t(){o.value=!1,window.removeEventListener("resize",n)}function s(){o.value?t():e()}function n(){window.outerWidth>=768&&t()}const r=ee();return D(()=>r.path,t),{isScreenOpen:o,openScreen:e,closeScreen:t,toggleScreen:s}}const $s={},ys={class:"VPSwitch",type:"button",role:"switch"},Ps={class:"check"},Vs={key:0,class:"icon"};function Ss(o,e){return a(),l("button",ys,[p("span",Ps,[o.$slots.default?(a(),l("span",Vs,[u(o.$slots,"default",{},void 0,!0)])):m("",!0)])])}const Ls=k($s,[["render",Ss],["__scopeId","data-v-4a1c76db"]]),Ts=b({__name:"VPSwitchAppearance",setup(o){const{isDark:e,theme:t}=L(),s=W("toggle-appearance",()=>{e.value=!e.value}),n=T("");return me(()=>{n.value=e.value?t.value.lightModeSwitchTitle||"Switch to light theme":t.value.darkModeSwitchTitle||"Switch to dark theme"}),(r,c)=>(a(),$(Ls,{title:n.value,class:"VPSwitchAppearance","aria-checked":i(e),onClick:i(s)},{default:f(()=>c[0]||(c[0]=[p("span",{class:"vpi-sun sun"},null,-1),p("span",{class:"vpi-moon moon"},null,-1)])),_:1},8,["title","aria-checked","onClick"]))}}),ke=k(Ts,[["__scopeId","data-v-e40a8bb6"]]),ws={key:0,class:"VPNavBarAppearance"},Ns=b({__name:"VPNavBarAppearance",setup(o){const{site:e}=L();return(t,s)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",ws,[g(ke)])):m("",!0)}}),Is=k(Ns,[["__scopeId","data-v-af096f4a"]]),ge=T();let Oe=!1,re=0;function Ms(o){const e=T(!1);if(te){!Oe&&Cs(),re++;const t=D(ge,s=>{var n,r,c;s===o.el.value||(n=o.el.value)!=null&&n.contains(s)?(e.value=!0,(r=o.onFocus)==null||r.call(o)):(e.value=!1,(c=o.onBlur)==null||c.call(o))});fe(()=>{t(),re--,re||As()})}return Qe(e)}function Cs(){document.addEventListener("focusin",De),Oe=!0,ge.value=document.activeElement}function As(){document.removeEventListener("focusin",De)}function De(){ge.value=document.activeElement}const Bs={class:"VPMenuLink"},Es=["innerHTML"],Hs=b({__name:"VPMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),l("div",Bs,[g(H,{class:I({active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Es)]),_:1},8,["class","href","target","rel","no-icon"])]))}}),ne=k(Hs,[["__scopeId","data-v-acbfed09"]]),Os={class:"VPMenuGroup"},Ds={key:0,class:"title"},Fs=b({__name:"VPMenuGroup",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",Os,[e.text?(a(),l("p",Ds,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),l(M,null,["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):m("",!0)],64))),256))]))}}),Rs=k(Fs,[["__scopeId","data-v-48c802d0"]]),Us={class:"VPMenu"},zs={key:0,class:"items"},js=b({__name:"VPMenu",props:{items:{}},setup(o){return(e,t)=>(a(),l("div",Us,[e.items?(a(),l("div",zs,[(a(!0),l(M,null,B(e.items,s=>(a(),l(M,{key:JSON.stringify(s)},["link"in s?(a(),$(ne,{key:0,item:s},null,8,["item"])):"component"in s?(a(),$(O(s.component),j({key:1,ref_for:!0},s.props),null,16)):(a(),$(Rs,{key:2,text:s.text,items:s.items},null,8,["text","items"]))],64))),128))])):m("",!0),u(e.$slots,"default",{},void 0,!0)]))}}),Gs=k(js,[["__scopeId","data-v-7dd3104a"]]),Ks=["aria-expanded","aria-label"],qs={key:0,class:"text"},Ws=["innerHTML"],Js={key:1,class:"vpi-more-horizontal icon"},Ys={class:"menu"},Qs=b({__name:"VPFlyout",props:{icon:{},button:{},label:{},items:{}},setup(o){const e=T(!1),t=T();Ms({el:t,onBlur:s});function s(){e.value=!1}return(n,r)=>(a(),l("div",{class:"VPFlyout",ref_key:"el",ref:t,onMouseenter:r[1]||(r[1]=c=>e.value=!0),onMouseleave:r[2]||(r[2]=c=>e.value=!1)},[p("button",{type:"button",class:"button","aria-haspopup":"true","aria-expanded":e.value,"aria-label":n.label,onClick:r[0]||(r[0]=c=>e.value=!e.value)},[n.button||n.icon?(a(),l("span",qs,[n.icon?(a(),l("span",{key:0,class:I([n.icon,"option-icon"])},null,2)):m("",!0),n.button?(a(),l("span",{key:1,innerHTML:n.button},null,8,Ws)):m("",!0),r[3]||(r[3]=p("span",{class:"vpi-chevron-down text-icon"},null,-1))])):(a(),l("span",Js))],8,Ks),p("div",Ys,[g(Gs,{items:n.items},{default:f(()=>[u(n.$slots,"default",{},void 0,!0)]),_:3},8,["items"])])],544))}}),$e=k(Qs,[["__scopeId","data-v-04f5c5e9"]]),Xs=["href","aria-label","innerHTML"],Zs=b({__name:"VPSocialLink",props:{icon:{},link:{},ariaLabel:{}},setup(o){const e=o,t=y(()=>typeof e.icon=="object"?e.icon.svg:``);return(s,n)=>(a(),l("a",{class:"VPSocialLink no-icon",href:s.link,"aria-label":s.ariaLabel??(typeof s.icon=="string"?s.icon:""),target:"_blank",rel:"noopener",innerHTML:t.value},null,8,Xs))}}),xs=k(Zs,[["__scopeId","data-v-717b8b75"]]),eo={class:"VPSocialLinks"},to=b({__name:"VPSocialLinks",props:{links:{}},setup(o){return(e,t)=>(a(),l("div",eo,[(a(!0),l(M,null,B(e.links,({link:s,icon:n,ariaLabel:r})=>(a(),$(xs,{key:s,icon:n,link:s,ariaLabel:r},null,8,["icon","link","ariaLabel"]))),128))]))}}),se=k(to,[["__scopeId","data-v-ee7a9424"]]),no={key:0,class:"group translations"},so={class:"trans-title"},oo={key:1,class:"group"},ao={class:"item appearance"},ro={class:"label"},io={class:"appearance-action"},lo={key:2,class:"group"},co={class:"item social-links"},uo=b({__name:"VPNavBarExtra",setup(o){const{site:e,theme:t}=L(),{localeLinks:s,currentLang:n}=Y({correspondingLink:!0}),r=y(()=>s.value.length&&n.value.label||e.value.appearance||t.value.socialLinks);return(c,v)=>r.value?(a(),$($e,{key:0,class:"VPNavBarExtra",label:"extra navigation"},{default:f(()=>[i(s).length&&i(n).label?(a(),l("div",no,[p("p",so,w(i(n).label),1),(a(!0),l(M,null,B(i(s),d=>(a(),$(ne,{key:d.link,item:d},null,8,["item"]))),128))])):m("",!0),i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",oo,[p("div",ao,[p("p",ro,w(i(t).darkModeSwitchLabel||"Appearance"),1),p("div",io,[g(ke)])])])):m("",!0),i(t).socialLinks?(a(),l("div",lo,[p("div",co,[g(se,{class:"social-links-list",links:i(t).socialLinks},null,8,["links"])])])):m("",!0)]),_:1})):m("",!0)}}),po=k(uo,[["__scopeId","data-v-925effce"]]),vo=["aria-expanded"],fo=b({__name:"VPNavBarHamburger",props:{active:{type:Boolean}},emits:["click"],setup(o){return(e,t)=>(a(),l("button",{type:"button",class:I(["VPNavBarHamburger",{active:e.active}]),"aria-label":"mobile navigation","aria-expanded":e.active,"aria-controls":"VPNavScreen",onClick:t[0]||(t[0]=s=>e.$emit("click"))},t[1]||(t[1]=[p("span",{class:"container"},[p("span",{class:"top"}),p("span",{class:"middle"}),p("span",{class:"bottom"})],-1)]),10,vo))}}),mo=k(fo,[["__scopeId","data-v-5dea55bf"]]),ho=["innerHTML"],_o=b({__name:"VPNavBarMenuLink",props:{item:{}},setup(o){const{page:e}=L();return(t,s)=>(a(),$(H,{class:I({VPNavBarMenuLink:!0,active:i(G)(i(e).relativePath,t.item.activeMatch||t.item.link,!!t.item.activeMatch)}),href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,tabindex:"0"},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ho)]),_:1},8,["class","href","target","rel","no-icon"]))}}),bo=k(_o,[["__scopeId","data-v-956ec74c"]]),Fe=b({__name:"VPNavBarMenuGroup",props:{item:{}},setup(o){const e=o,{page:t}=L(),s=r=>"component"in r?!1:"link"in r?G(t.value.relativePath,r.link,!!e.item.activeMatch):r.items.some(s),n=y(()=>s(e.item));return(r,c)=>(a(),$($e,{class:I({VPNavBarMenuGroup:!0,active:i(G)(i(t).relativePath,r.item.activeMatch,!!r.item.activeMatch)||n.value}),button:r.item.text,items:r.item.items},null,8,["class","button","items"]))}}),ko={key:0,"aria-labelledby":"main-nav-aria-label",class:"VPNavBarMenu"},go=b({__name:"VPNavBarMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",ko,[s[0]||(s[0]=p("span",{id:"main-nav-aria-label",class:"visually-hidden"}," Main Navigation ",-1)),(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(bo,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props),null,16)):(a(),$(Fe,{key:2,item:n},null,8,["item"]))],64))),128))])):m("",!0)}}),$o=k(go,[["__scopeId","data-v-e6d46098"]]);function yo(o){const{localeIndex:e,theme:t}=L();function s(n){var C,A,N;const r=n.split("."),c=(C=t.value.search)==null?void 0:C.options,v=c&&typeof c=="object",d=v&&((N=(A=c.locales)==null?void 0:A[e.value])==null?void 0:N.translations)||null,h=v&&c.translations||null;let P=d,_=h,V=o;const S=r.pop();for(const E of r){let z=null;const q=V==null?void 0:V[E];q&&(z=V=q);const oe=_==null?void 0:_[E];oe&&(z=_=oe);const ae=P==null?void 0:P[E];ae&&(z=P=ae),q||(V=z),oe||(_=z),ae||(P=z)}return(P==null?void 0:P[S])??(_==null?void 0:_[S])??(V==null?void 0:V[S])??""}return s}const Po=["aria-label"],Vo={class:"DocSearch-Button-Container"},So={class:"DocSearch-Button-Placeholder"},Pe=b({__name:"VPNavBarSearchButton",setup(o){const t=yo({button:{buttonText:"Search",buttonAriaLabel:"Search"}});return(s,n)=>(a(),l("button",{type:"button",class:"DocSearch DocSearch-Button","aria-label":i(t)("button.buttonAriaLabel")},[p("span",Vo,[n[0]||(n[0]=p("span",{class:"vp-icon DocSearch-Search-Icon"},null,-1)),p("span",So,w(i(t)("button.buttonText")),1)]),n[1]||(n[1]=p("span",{class:"DocSearch-Button-Keys"},[p("kbd",{class:"DocSearch-Button-Key"}),p("kbd",{class:"DocSearch-Button-Key"},"K")],-1))],8,Po))}}),Lo={class:"VPNavBarSearch"},To={id:"local-search"},wo={key:1,id:"docsearch"},No=b({__name:"VPNavBarSearch",setup(o){const e=Xe(()=>Ze(()=>import("./VPLocalSearchBox.D7LKkx_r.js"),__vite__mapDeps([0,1]))),t=()=>null,{theme:s}=L(),n=T(!1),r=T(!1);R(()=>{});function c(){n.value||(n.value=!0,setTimeout(v,16))}function v(){const _=new Event("keydown");_.key="k",_.metaKey=!0,window.dispatchEvent(_),setTimeout(()=>{document.querySelector(".DocSearch-Modal")||v()},16)}function d(_){const V=_.target,S=V.tagName;return V.isContentEditable||S==="INPUT"||S==="SELECT"||S==="TEXTAREA"}const h=T(!1);le("k",_=>{(_.ctrlKey||_.metaKey)&&(_.preventDefault(),h.value=!0)}),le("/",_=>{d(_)||(_.preventDefault(),h.value=!0)});const P="local";return(_,V)=>{var S;return a(),l("div",Lo,[i(P)==="local"?(a(),l(M,{key:0},[h.value?(a(),$(i(e),{key:0,onClose:V[0]||(V[0]=C=>h.value=!1)})):m("",!0),p("div",To,[g(Pe,{onClick:V[1]||(V[1]=C=>h.value=!0)})])],64)):i(P)==="algolia"?(a(),l(M,{key:1},[n.value?(a(),$(i(t),{key:0,algolia:((S=i(s).search)==null?void 0:S.options)??i(s).algolia,onVnodeBeforeMount:V[2]||(V[2]=C=>r.value=!0)},null,8,["algolia"])):m("",!0),r.value?m("",!0):(a(),l("div",wo,[g(Pe,{onClick:c})]))],64)):m("",!0)])}}}),Io=b({__name:"VPNavBarSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavBarSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),Mo=k(Io,[["__scopeId","data-v-164c457f"]]),Co=["href","rel","target"],Ao=["innerHTML"],Bo={key:2},Eo=b({__name:"VPNavBarTitle",setup(o){const{site:e,theme:t}=L(),{hasSidebar:s}=U(),{currentLang:n}=Y(),r=y(()=>{var d;return typeof t.value.logoLink=="string"?t.value.logoLink:(d=t.value.logoLink)==null?void 0:d.link}),c=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.rel}),v=y(()=>{var d;return typeof t.value.logoLink=="string"||(d=t.value.logoLink)==null?void 0:d.target});return(d,h)=>(a(),l("div",{class:I(["VPNavBarTitle",{"has-sidebar":i(s)}])},[p("a",{class:"title",href:r.value??i(_e)(i(n).link),rel:c.value,target:v.value},[u(d.$slots,"nav-bar-title-before",{},void 0,!0),i(t).logo?(a(),$(X,{key:0,class:"logo",image:i(t).logo},null,8,["image"])):m("",!0),i(t).siteTitle?(a(),l("span",{key:1,innerHTML:i(t).siteTitle},null,8,Ao)):i(t).siteTitle===void 0?(a(),l("span",Bo,w(i(e).title),1)):m("",!0),u(d.$slots,"nav-bar-title-after",{},void 0,!0)],8,Co)],2))}}),Ho=k(Eo,[["__scopeId","data-v-0f4f798b"]]),Oo={class:"items"},Do={class:"title"},Fo=b({__name:"VPNavBarTranslations",setup(o){const{theme:e}=L(),{localeLinks:t,currentLang:s}=Y({correspondingLink:!0});return(n,r)=>i(t).length&&i(s).label?(a(),$($e,{key:0,class:"VPNavBarTranslations",icon:"vpi-languages",label:i(e).langMenuLabel||"Change language"},{default:f(()=>[p("div",Oo,[p("p",Do,w(i(s).label),1),(a(!0),l(M,null,B(i(t),c=>(a(),$(ne,{key:c.link,item:c},null,8,["item"]))),128))])]),_:1},8,["label"])):m("",!0)}}),Ro=k(Fo,[["__scopeId","data-v-c80d9ad0"]]),Uo={class:"wrapper"},zo={class:"container"},jo={class:"title"},Go={class:"content"},Ko={class:"content-body"},qo=b({__name:"VPNavBar",props:{isScreenOpen:{type:Boolean}},emits:["toggle-screen"],setup(o){const e=o,{y:t}=Me(),{hasSidebar:s}=U(),{frontmatter:n}=L(),r=T({});return me(()=>{r.value={"has-sidebar":s.value,home:n.value.layout==="home",top:t.value===0,"screen-open":e.isScreenOpen}}),(c,v)=>(a(),l("div",{class:I(["VPNavBar",r.value])},[p("div",Uo,[p("div",zo,[p("div",jo,[g(Ho,null,{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),_:3})]),p("div",Go,[p("div",Ko,[u(c.$slots,"nav-bar-content-before",{},void 0,!0),g(No,{class:"search"}),g($o,{class:"menu"}),g(Ro,{class:"translations"}),g(Is,{class:"appearance"}),g(Mo,{class:"social-links"}),g(po,{class:"extra"}),u(c.$slots,"nav-bar-content-after",{},void 0,!0),g(mo,{class:"hamburger",active:c.isScreenOpen,onClick:v[0]||(v[0]=d=>c.$emit("toggle-screen"))},null,8,["active"])])])])]),v[1]||(v[1]=p("div",{class:"divider"},[p("div",{class:"divider-line"})],-1))],2))}}),Wo=k(qo,[["__scopeId","data-v-822684d1"]]),Jo={key:0,class:"VPNavScreenAppearance"},Yo={class:"text"},Qo=b({__name:"VPNavScreenAppearance",setup(o){const{site:e,theme:t}=L();return(s,n)=>i(e).appearance&&i(e).appearance!=="force-dark"&&i(e).appearance!=="force-auto"?(a(),l("div",Jo,[p("p",Yo,w(i(t).darkModeSwitchLabel||"Appearance"),1),g(ke)])):m("",!0)}}),Xo=k(Qo,[["__scopeId","data-v-ffb44008"]]),Zo=["innerHTML"],xo=b({__name:"VPNavScreenMenuLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,Zo)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),ea=k(xo,[["__scopeId","data-v-735512b8"]]),ta=["innerHTML"],na=b({__name:"VPNavScreenMenuGroupLink",props:{item:{}},setup(o){const e=W("close-screen");return(t,s)=>(a(),$(H,{class:"VPNavScreenMenuGroupLink",href:t.item.link,target:t.item.target,rel:t.item.rel,"no-icon":t.item.noIcon,onClick:i(e)},{default:f(()=>[p("span",{innerHTML:t.item.text},null,8,ta)]),_:1},8,["href","target","rel","no-icon","onClick"]))}}),Re=k(na,[["__scopeId","data-v-372ae7c0"]]),sa={class:"VPNavScreenMenuGroupSection"},oa={key:0,class:"title"},aa=b({__name:"VPNavScreenMenuGroupSection",props:{text:{},items:{}},setup(o){return(e,t)=>(a(),l("div",sa,[e.text?(a(),l("p",oa,w(e.text),1)):m("",!0),(a(!0),l(M,null,B(e.items,s=>(a(),$(Re,{key:s.text,item:s},null,8,["item"]))),128))]))}}),ra=k(aa,[["__scopeId","data-v-4b8941ac"]]),ia=["aria-controls","aria-expanded"],la=["innerHTML"],ca=["id"],ua={key:0,class:"item"},da={key:1,class:"item"},pa={key:2,class:"group"},va=b({__name:"VPNavScreenMenuGroup",props:{text:{},items:{}},setup(o){const e=o,t=T(!1),s=y(()=>`NavScreenGroup-${e.text.replace(" ","-").toLowerCase()}`);function n(){t.value=!t.value}return(r,c)=>(a(),l("div",{class:I(["VPNavScreenMenuGroup",{open:t.value}])},[p("button",{class:"button","aria-controls":s.value,"aria-expanded":t.value,onClick:n},[p("span",{class:"button-text",innerHTML:r.text},null,8,la),c[0]||(c[0]=p("span",{class:"vpi-plus button-icon"},null,-1))],8,ia),p("div",{id:s.value,class:"items"},[(a(!0),l(M,null,B(r.items,v=>(a(),l(M,{key:JSON.stringify(v)},["link"in v?(a(),l("div",ua,[g(Re,{item:v},null,8,["item"])])):"component"in v?(a(),l("div",da,[(a(),$(O(v.component),j({ref_for:!0},v.props,{"screen-menu":""}),null,16))])):(a(),l("div",pa,[g(ra,{text:v.text,items:v.items},null,8,["text","items"])]))],64))),128))],8,ca)],2))}}),Ue=k(va,[["__scopeId","data-v-875057a5"]]),fa={key:0,class:"VPNavScreenMenu"},ma=b({__name:"VPNavScreenMenu",setup(o){const{theme:e}=L();return(t,s)=>i(e).nav?(a(),l("nav",fa,[(a(!0),l(M,null,B(i(e).nav,n=>(a(),l(M,{key:JSON.stringify(n)},["link"in n?(a(),$(ea,{key:0,item:n},null,8,["item"])):"component"in n?(a(),$(O(n.component),j({key:1,ref_for:!0},n.props,{"screen-menu":""}),null,16)):(a(),$(Ue,{key:2,text:n.text||"",items:n.items},null,8,["text","items"]))],64))),128))])):m("",!0)}}),ha=b({__name:"VPNavScreenSocialLinks",setup(o){const{theme:e}=L();return(t,s)=>i(e).socialLinks?(a(),$(se,{key:0,class:"VPNavScreenSocialLinks",links:i(e).socialLinks},null,8,["links"])):m("",!0)}}),_a={class:"list"},ba=b({__name:"VPNavScreenTranslations",setup(o){const{localeLinks:e,currentLang:t}=Y({correspondingLink:!0}),s=T(!1);function n(){s.value=!s.value}return(r,c)=>i(e).length&&i(t).label?(a(),l("div",{key:0,class:I(["VPNavScreenTranslations",{open:s.value}])},[p("button",{class:"title",onClick:n},[c[0]||(c[0]=p("span",{class:"vpi-languages icon lang"},null,-1)),F(" "+w(i(t).label)+" ",1),c[1]||(c[1]=p("span",{class:"vpi-chevron-down icon chevron"},null,-1))]),p("ul",_a,[(a(!0),l(M,null,B(i(e),v=>(a(),l("li",{key:v.link,class:"item"},[g(H,{class:"link",href:v.link},{default:f(()=>[F(w(v.text),1)]),_:2},1032,["href"])]))),128))])],2)):m("",!0)}}),ka=k(ba,[["__scopeId","data-v-362991c2"]]),ga={class:"container"},$a=b({__name:"VPNavScreen",props:{open:{type:Boolean}},setup(o){const e=T(null),t=Ce(te?document.body:null);return(s,n)=>(a(),$(pe,{name:"fade",onEnter:n[0]||(n[0]=r=>t.value=!0),onAfterLeave:n[1]||(n[1]=r=>t.value=!1)},{default:f(()=>[s.open?(a(),l("div",{key:0,class:"VPNavScreen",ref_key:"screen",ref:e,id:"VPNavScreen"},[p("div",ga,[u(s.$slots,"nav-screen-content-before",{},void 0,!0),g(ma,{class:"menu"}),g(ka,{class:"translations"}),g(Xo,{class:"appearance"}),g(ha,{class:"social-links"}),u(s.$slots,"nav-screen-content-after",{},void 0,!0)])],512)):m("",!0)]),_:3}))}}),ya=k($a,[["__scopeId","data-v-833aabba"]]),Pa={key:0,class:"VPNav"},Va=b({__name:"VPNav",setup(o){const{isScreenOpen:e,closeScreen:t,toggleScreen:s}=gs(),{frontmatter:n}=L(),r=y(()=>n.value.navbar!==!1);return he("close-screen",t),Z(()=>{te&&document.documentElement.classList.toggle("hide-nav",!r.value)}),(c,v)=>r.value?(a(),l("header",Pa,[g(Wo,{"is-screen-open":i(e),onToggleScreen:i(s)},{"nav-bar-title-before":f(()=>[u(c.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(c.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(c.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(c.$slots,"nav-bar-content-after",{},void 0,!0)]),_:3},8,["is-screen-open","onToggleScreen"]),g(ya,{open:i(e)},{"nav-screen-content-before":f(()=>[u(c.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(c.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3},8,["open"])])):m("",!0)}}),Sa=k(Va,[["__scopeId","data-v-f1e365da"]]),La=["role","tabindex"],Ta={key:1,class:"items"},wa=b({__name:"VPSidebarItem",props:{item:{},depth:{}},setup(o){const e=o,{collapsed:t,collapsible:s,isLink:n,isActiveLink:r,hasActiveLink:c,hasChildren:v,toggle:d}=Pt(y(()=>e.item)),h=y(()=>v.value?"section":"div"),P=y(()=>n.value?"a":"div"),_=y(()=>v.value?e.depth+2===7?"p":`h${e.depth+2}`:"p"),V=y(()=>n.value?void 0:"button"),S=y(()=>[[`level-${e.depth}`],{collapsible:s.value},{collapsed:t.value},{"is-link":n.value},{"is-active":r.value},{"has-active":c.value}]);function C(N){"key"in N&&N.key!=="Enter"||!e.item.link&&d()}function A(){e.item.link&&d()}return(N,E)=>{const z=K("VPSidebarItem",!0);return a(),$(O(h.value),{class:I(["VPSidebarItem",S.value])},{default:f(()=>[N.item.text?(a(),l("div",j({key:0,class:"item",role:V.value},et(N.item.items?{click:C,keydown:C}:{},!0),{tabindex:N.item.items&&0}),[E[1]||(E[1]=p("div",{class:"indicator"},null,-1)),N.item.link?(a(),$(H,{key:0,tag:P.value,class:"link",href:N.item.link,rel:N.item.rel,target:N.item.target},{default:f(()=>[(a(),$(O(_.value),{class:"text",innerHTML:N.item.text},null,8,["innerHTML"]))]),_:1},8,["tag","href","rel","target"])):(a(),$(O(_.value),{key:1,class:"text",innerHTML:N.item.text},null,8,["innerHTML"])),N.item.collapsed!=null&&N.item.items&&N.item.items.length?(a(),l("div",{key:2,class:"caret",role:"button","aria-label":"toggle section",onClick:A,onKeydown:xe(A,["enter"]),tabindex:"0"},E[0]||(E[0]=[p("span",{class:"vpi-chevron-right caret-icon"},null,-1)]),32)):m("",!0)],16,La)):m("",!0),N.item.items&&N.item.items.length?(a(),l("div",Ta,[N.depth<5?(a(!0),l(M,{key:0},B(N.item.items,q=>(a(),$(z,{key:q.text,item:q,depth:N.depth+1},null,8,["item","depth"]))),128)):m("",!0)])):m("",!0)]),_:1},8,["class"])}}}),Na=k(wa,[["__scopeId","data-v-196b2e5f"]]),Ia=b({__name:"VPSidebarGroup",props:{items:{}},setup(o){const e=T(!0);let t=null;return R(()=>{t=setTimeout(()=>{t=null,e.value=!1},300)}),tt(()=>{t!=null&&(clearTimeout(t),t=null)}),(s,n)=>(a(!0),l(M,null,B(s.items,r=>(a(),l("div",{key:r.text,class:I(["group",{"no-transition":e.value}])},[g(Na,{item:r,depth:0},null,8,["item"])],2))),128))}}),Ma=k(Ia,[["__scopeId","data-v-9e426adc"]]),Ca={class:"nav",id:"VPSidebarNav","aria-labelledby":"sidebar-aria-label",tabindex:"-1"},Aa=b({__name:"VPSidebar",props:{open:{type:Boolean}},setup(o){const{sidebarGroups:e,hasSidebar:t}=U(),s=o,n=T(null),r=Ce(te?document.body:null);D([s,n],()=>{var v;s.open?(r.value=!0,(v=n.value)==null||v.focus()):r.value=!1},{immediate:!0,flush:"post"});const c=T(0);return D(e,()=>{c.value+=1},{deep:!0}),(v,d)=>i(t)?(a(),l("aside",{key:0,class:I(["VPSidebar",{open:v.open}]),ref_key:"navEl",ref:n,onClick:d[0]||(d[0]=nt(()=>{},["stop"]))},[d[2]||(d[2]=p("div",{class:"curtain"},null,-1)),p("nav",Ca,[d[1]||(d[1]=p("span",{class:"visually-hidden",id:"sidebar-aria-label"}," Sidebar Navigation ",-1)),u(v.$slots,"sidebar-nav-before",{},void 0,!0),(a(),$(Ma,{items:i(e),key:c.value},null,8,["items"])),u(v.$slots,"sidebar-nav-after",{},void 0,!0)])],2)):m("",!0)}}),Ba=k(Aa,[["__scopeId","data-v-18756405"]]),Ea=b({__name:"VPSkipLink",setup(o){const e=ee(),t=T();D(()=>e.path,()=>t.value.focus());function s({target:n}){const r=document.getElementById(decodeURIComponent(n.hash).slice(1));if(r){const c=()=>{r.removeAttribute("tabindex"),r.removeEventListener("blur",c)};r.setAttribute("tabindex","-1"),r.addEventListener("blur",c),r.focus(),window.scrollTo(0,0)}}return(n,r)=>(a(),l(M,null,[p("span",{ref_key:"backToTop",ref:t,tabindex:"-1"},null,512),p("a",{href:"#VPContent",class:"VPSkipLink visually-hidden",onClick:s}," Skip to content ")],64))}}),Ha=k(Ea,[["__scopeId","data-v-c3508ec8"]]),Oa=b({__name:"Layout",setup(o){const{isOpen:e,open:t,close:s}=U(),n=ee();D(()=>n.path,s),yt(e,s);const{frontmatter:r}=L(),c=Ae(),v=y(()=>!!c["home-hero-image"]);return he("hero-image-slot-exists",v),(d,h)=>{const P=K("Content");return i(r).layout!==!1?(a(),l("div",{key:0,class:I(["Layout",i(r).pageClass])},[u(d.$slots,"layout-top",{},void 0,!0),g(Ha),g(ct,{class:"backdrop",show:i(e),onClick:i(s)},null,8,["show","onClick"]),g(Sa,null,{"nav-bar-title-before":f(()=>[u(d.$slots,"nav-bar-title-before",{},void 0,!0)]),"nav-bar-title-after":f(()=>[u(d.$slots,"nav-bar-title-after",{},void 0,!0)]),"nav-bar-content-before":f(()=>[u(d.$slots,"nav-bar-content-before",{},void 0,!0)]),"nav-bar-content-after":f(()=>[u(d.$slots,"nav-bar-content-after",{},void 0,!0)]),"nav-screen-content-before":f(()=>[u(d.$slots,"nav-screen-content-before",{},void 0,!0)]),"nav-screen-content-after":f(()=>[u(d.$slots,"nav-screen-content-after",{},void 0,!0)]),_:3}),g(ks,{open:i(e),onOpenMenu:i(t)},null,8,["open","onOpenMenu"]),g(Ba,{open:i(e)},{"sidebar-nav-before":f(()=>[u(d.$slots,"sidebar-nav-before",{},void 0,!0)]),"sidebar-nav-after":f(()=>[u(d.$slots,"sidebar-nav-after",{},void 0,!0)]),_:3},8,["open"]),g(ss,null,{"page-top":f(()=>[u(d.$slots,"page-top",{},void 0,!0)]),"page-bottom":f(()=>[u(d.$slots,"page-bottom",{},void 0,!0)]),"not-found":f(()=>[u(d.$slots,"not-found",{},void 0,!0)]),"home-hero-before":f(()=>[u(d.$slots,"home-hero-before",{},void 0,!0)]),"home-hero-info-before":f(()=>[u(d.$slots,"home-hero-info-before",{},void 0,!0)]),"home-hero-info":f(()=>[u(d.$slots,"home-hero-info",{},void 0,!0)]),"home-hero-info-after":f(()=>[u(d.$slots,"home-hero-info-after",{},void 0,!0)]),"home-hero-actions-after":f(()=>[u(d.$slots,"home-hero-actions-after",{},void 0,!0)]),"home-hero-image":f(()=>[u(d.$slots,"home-hero-image",{},void 0,!0)]),"home-hero-after":f(()=>[u(d.$slots,"home-hero-after",{},void 0,!0)]),"home-features-before":f(()=>[u(d.$slots,"home-features-before",{},void 0,!0)]),"home-features-after":f(()=>[u(d.$slots,"home-features-after",{},void 0,!0)]),"doc-footer-before":f(()=>[u(d.$slots,"doc-footer-before",{},void 0,!0)]),"doc-before":f(()=>[u(d.$slots,"doc-before",{},void 0,!0)]),"doc-after":f(()=>[u(d.$slots,"doc-after",{},void 0,!0)]),"doc-top":f(()=>[u(d.$slots,"doc-top",{},void 0,!0)]),"doc-bottom":f(()=>[u(d.$slots,"doc-bottom",{},void 0,!0)]),"aside-top":f(()=>[u(d.$slots,"aside-top",{},void 0,!0)]),"aside-bottom":f(()=>[u(d.$slots,"aside-bottom",{},void 0,!0)]),"aside-outline-before":f(()=>[u(d.$slots,"aside-outline-before",{},void 0,!0)]),"aside-outline-after":f(()=>[u(d.$slots,"aside-outline-after",{},void 0,!0)]),"aside-ads-before":f(()=>[u(d.$slots,"aside-ads-before",{},void 0,!0)]),"aside-ads-after":f(()=>[u(d.$slots,"aside-ads-after",{},void 0,!0)]),_:3}),g(ls),u(d.$slots,"layout-bottom",{},void 0,!0)],2)):(a(),$(P,{key:1}))}}}),Da=k(Oa,[["__scopeId","data-v-a9a9e638"]]),Fa={},Ra={class:"VPTeamPage"};function Ua(o,e){return a(),l("div",Ra,[u(o.$slots,"default")])}const Or=k(Fa,[["render",Ua],["__scopeId","data-v-c2f8e101"]]),za={},ja={class:"VPTeamPageTitle"},Ga={key:0,class:"title"},Ka={key:1,class:"lead"};function qa(o,e){return a(),l("div",ja,[o.$slots.title?(a(),l("h1",Ga,[u(o.$slots,"title",{},void 0,!0)])):m("",!0),o.$slots.lead?(a(),l("p",Ka,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0)])}const Dr=k(za,[["render",qa],["__scopeId","data-v-e277e15c"]]),Wa={},Ja={class:"VPTeamPageSection"},Ya={class:"title"},Qa={key:0,class:"title-text"},Xa={key:0,class:"lead"},Za={key:1,class:"members"};function xa(o,e){return a(),l("section",Ja,[p("div",Ya,[e[0]||(e[0]=p("div",{class:"title-line"},null,-1)),o.$slots.title?(a(),l("h2",Qa,[u(o.$slots,"title",{},void 0,!0)])):m("",!0)]),o.$slots.lead?(a(),l("p",Xa,[u(o.$slots,"lead",{},void 0,!0)])):m("",!0),o.$slots.members?(a(),l("div",Za,[u(o.$slots,"members",{},void 0,!0)])):m("",!0)])}const Fr=k(Wa,[["render",xa],["__scopeId","data-v-d43bc49d"]]),er={class:"profile"},tr={class:"avatar"},nr=["src","alt"],sr={class:"data"},or={class:"name"},ar={key:0,class:"affiliation"},rr={key:0,class:"title"},ir={key:1,class:"at"},lr=["innerHTML"],cr={key:2,class:"links"},ur={key:0,class:"sp"},dr=b({__name:"VPTeamMembersItem",props:{size:{default:"medium"},member:{}},setup(o){return(e,t)=>(a(),l("article",{class:I(["VPTeamMembersItem",[e.size]])},[p("div",er,[p("figure",tr,[p("img",{class:"avatar-img",src:e.member.avatar,alt:e.member.name},null,8,nr)]),p("div",sr,[p("h1",or,w(e.member.name),1),e.member.title||e.member.org?(a(),l("p",ar,[e.member.title?(a(),l("span",rr,w(e.member.title),1)):m("",!0),e.member.title&&e.member.org?(a(),l("span",ir," @ ")):m("",!0),e.member.org?(a(),$(H,{key:2,class:I(["org",{link:e.member.orgLink}]),href:e.member.orgLink,"no-icon":""},{default:f(()=>[F(w(e.member.org),1)]),_:1},8,["class","href"])):m("",!0)])):m("",!0),e.member.desc?(a(),l("p",{key:1,class:"desc",innerHTML:e.member.desc},null,8,lr)):m("",!0),e.member.links?(a(),l("div",cr,[g(se,{links:e.member.links},null,8,["links"])])):m("",!0)])]),e.member.sponsor?(a(),l("div",ur,[g(H,{class:"sp-link",href:e.member.sponsor,"no-icon":""},{default:f(()=>[t[0]||(t[0]=p("span",{class:"vpi-heart sp-icon"},null,-1)),F(" "+w(e.member.actionText||"Sponsor"),1)]),_:1},8,["href"])])):m("",!0)],2))}}),pr=k(dr,[["__scopeId","data-v-f9987cb6"]]),vr={class:"container"},fr=b({__name:"VPTeamMembers",props:{size:{default:"medium"},members:{}},setup(o){const e=o,t=y(()=>[e.size,`count-${e.members.length}`]);return(s,n)=>(a(),l("div",{class:I(["VPTeamMembers",t.value])},[p("div",vr,[(a(!0),l(M,null,B(s.members,r=>(a(),l("div",{key:r.name,class:"item"},[g(pr,{size:s.size,member:r},null,8,["size","member"])]))),128))])],2))}}),Rr=k(fr,[["__scopeId","data-v-fba19bad"]]),Ve={Layout:Da,enhanceApp:({app:o})=>{o.component("Badge",rt)}},mr={};function hr(o,e){return e[0]||(e[0]=st('

    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    ',3))}const _r=k(mr,[["render",hr]]),br=b({__name:"VersionPicker",props:{screenMenu:{type:Boolean}},setup(o){const e=T([]),t=T("Versions"),s=T(!1);Le();const n=()=>typeof window<"u"&&(window.location.hostname==="localhost"||window.location.hostname==="127.0.0.1"),r=()=>{if(typeof window>"u")return"";const{origin:d,pathname:h}=window.location;if(d.includes("github.io")){const P=h.split("/").filter(Boolean),_=P.length>0?`/${P[0]}/`:"/";return`${d}${_}`}else return d},c=()=>new Promise(d=>{if(n()){d(!1);return}const h=setInterval(()=>{window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION&&(clearInterval(h),d(!0))},100);setTimeout(()=>{clearInterval(h),d(!1)},5e3)});return R(async()=>{if(!(typeof window>"u")){try{if(n()){const d=["dev"];e.value=d.map(h=>({text:h,link:"/"})),t.value="dev"}else{const d=await c(),h=y(()=>r());if(d&&window.DOC_VERSIONS&&window.DOCUMENTER_CURRENT_VERSION)e.value=window.DOC_VERSIONS.map(P=>({text:P,link:`${h.value}/${P}/`})),t.value=window.DOCUMENTER_CURRENT_VERSION;else{const P=["dev"];e.value=P.map(_=>({text:_,link:`${h.value}/${_}/`})),t.value="dev"}}}catch(d){console.warn("Error loading versions:",d);const h=["dev"],P=y(()=>r());e.value=h.map(_=>({text:_,link:`${P.value}/${_}/`})),t.value="dev"}s.value=!0}}),(d,h)=>s.value?(a(),l(M,{key:0},[!d.screenMenu&&e.value.length>0?(a(),$(Fe,{key:0,item:{text:t.value,items:e.value},class:"VPVersionPicker"},null,8,["item"])):d.screenMenu&&e.value.length>0?(a(),$(Ue,{key:1,text:t.value,items:e.value,class:"VPVersionPicker"},null,8,["text","items"])):m("",!0)],64)):m("",!0)}}),kr=k(br,[["__scopeId","data-v-d483b3a6"]]),gr=o=>{if(typeof document>"u")return{stabilizeScrollPosition:n=>async(...r)=>n(...r)};const e=document.documentElement;return{stabilizeScrollPosition:s=>async(...n)=>{const r=s(...n),c=o.value;if(!c)return r;const v=c.offsetTop-e.scrollTop;return await Ie(),e.scrollTop=c.offsetTop-v,r}}},ze="vitepress:tabSharedState",J=typeof localStorage<"u"?localStorage:null,je="vitepress:tabsSharedState",$r=()=>{const o=J==null?void 0:J.getItem(je);if(o)try{return JSON.parse(o)}catch{}return{}},yr=o=>{J&&J.setItem(je,JSON.stringify(o))},Pr=o=>{const e=ot({});D(()=>e.content,(t,s)=>{t&&s&&yr(t)},{deep:!0}),o.provide(ze,e)},Vr=(o,e)=>{const t=W(ze);if(!t)throw new Error("[vitepress-plugin-tabs] TabsSharedState should be injected");R(()=>{t.content||(t.content=$r())});const s=T(),n=y({get(){var d;const c=e.value,v=o.value;if(c){const h=(d=t.content)==null?void 0:d[c];if(h&&v.includes(h))return h}else{const h=s.value;if(h)return h}return v[0]},set(c){const v=e.value;v?t.content&&(t.content[v]=c):s.value=c}});return{selected:n,select:c=>{n.value=c}}};let Se=0;const Sr=()=>(Se++,""+Se);function Lr(){const o=Ae();return y(()=>{var s;const t=(s=o.default)==null?void 0:s.call(o);return t?t.filter(n=>typeof n.type=="object"&&"__name"in n.type&&n.type.__name==="PluginTabsTab"&&n.props).map(n=>{var r;return(r=n.props)==null?void 0:r.label}):[]})}const Ge="vitepress:tabSingleState",Tr=o=>{he(Ge,o)},wr=()=>{const o=W(Ge);if(!o)throw new Error("[vitepress-plugin-tabs] TabsSingleState should be injected");return o},Nr={class:"plugin-tabs"},Ir=["id","aria-selected","aria-controls","tabindex","onClick"],Mr=b({__name:"PluginTabs",props:{sharedStateKey:{}},setup(o){const e=o,t=Lr(),{selected:s,select:n}=Vr(t,at(e,"sharedStateKey")),r=T(),{stabilizeScrollPosition:c}=gr(r),v=c(n),d=T([]),h=_=>{var C;const V=t.value.indexOf(s.value);let S;_.key==="ArrowLeft"?S=V>=1?V-1:t.value.length-1:_.key==="ArrowRight"&&(S=V(a(),l("div",Nr,[p("div",{ref_key:"tablist",ref:r,class:"plugin-tabs--tab-list",role:"tablist",onKeydown:h},[(a(!0),l(M,null,B(i(t),S=>(a(),l("button",{id:`tab-${S}-${i(P)}`,ref_for:!0,ref_key:"buttonRefs",ref:d,key:S,role:"tab",class:"plugin-tabs--tab","aria-selected":S===i(s),"aria-controls":`panel-${S}-${i(P)}`,tabindex:S===i(s)?0:-1,onClick:()=>i(v)(S)},w(S),9,Ir))),128))],544),u(_.$slots,"default")]))}}),Cr=["id","aria-labelledby"],Ar=b({__name:"PluginTabsTab",props:{label:{}},setup(o){const{uid:e,selected:t}=wr();return(s,n)=>i(t)===s.label?(a(),l("div",{key:0,id:`panel-${s.label}-${i(e)}`,class:"plugin-tabs--content",role:"tabpanel",tabindex:"0","aria-labelledby":`tab-${s.label}-${i(e)}`},[u(s.$slots,"default",{},void 0,!0)],8,Cr)):m("",!0)}}),Br=k(Ar,[["__scopeId","data-v-9b0d03d2"]]),Er=o=>{Pr(o),o.component("PluginTabs",Mr),o.component("PluginTabsTab",Br)},Ur={extends:Ve,Layout(){return ye(Ve.Layout,null,{"aside-ads-before":()=>ye(_r)})},enhanceApp({app:o}){Er(o),o.component("VersionPicker",kr)}};export{Ur as R,Dr as V,Fr as a,Rr as b,Or as c,yo as d,L as u}; diff --git a/previews/PR1000/assets/ecosystem.md.C5I9rkF4.js b/previews/PR1000/assets/ecosystem.md.2XMefkkh.js similarity index 58% rename from previews/PR1000/assets/ecosystem.md.C5I9rkF4.js rename to previews/PR1000/assets/ecosystem.md.2XMefkkh.js index a144cab603..dca6b80b66 100644 --- a/previews/PR1000/assets/ecosystem.md.C5I9rkF4.js +++ b/previews/PR1000/assets/ecosystem.md.2XMefkkh.js @@ -1 +1 @@ -import{V as d,a as s,b as l,c as f}from"./chunks/theme.DRSgyssM.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/LuxTestUtils.jl"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; +import{V as d,a as s,b as l,c as f}from"./chunks/theme.BD8xVYwz.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/tree/main/lib/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/tree/main/lib/LuxTestUtils"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; diff --git a/previews/PR1000/assets/ecosystem.md.C5I9rkF4.lean.js b/previews/PR1000/assets/ecosystem.md.2XMefkkh.lean.js similarity index 58% rename from previews/PR1000/assets/ecosystem.md.C5I9rkF4.lean.js rename to previews/PR1000/assets/ecosystem.md.2XMefkkh.lean.js index a144cab603..dca6b80b66 100644 --- a/previews/PR1000/assets/ecosystem.md.C5I9rkF4.lean.js +++ b/previews/PR1000/assets/ecosystem.md.2XMefkkh.lean.js @@ -1 +1 @@ -import{V as d,a as s,b as l,c as f}from"./chunks/theme.DRSgyssM.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/LuxTestUtils.jl"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; +import{V as d,a as s,b as l,c as f}from"./chunks/theme.BD8xVYwz.js";import{c as k,G as t,w as a,k as e,o as L,a as n}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"page"},"headers":[],"relativePath":"ecosystem.md","filePath":"ecosystem.md","lastUpdated":null}'),j={name:"ecosystem.md"},J=Object.assign(j,{setup(v){const o=[{avatar:"https://github.com/SciML.png",name:"DiffEqFlux.jl",desc:"Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods",links:[{icon:"github",link:"https://github.com/SciML/DiffEqFlux.jl"}]},{avatar:"https://github.com/SciML.png",name:"SciMLSensitivity.jl",desc:"A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.",links:[{icon:"github",link:"https://github.com/SciML/SciMLSensitivity.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralPDE.jl",desc:"Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation",links:[{icon:"github",link:"https://github.com/SciML/NeuralPDE.jl"}]},{avatar:"https://github.com/SciML.png",name:"NeuralLyapunov.jl",desc:"A library for searching for neural Lyapunov functions in Julia",links:[{icon:"github",link:"https://github.com/SciML/NeuralLyapunov.jl"}]},{avatar:"https://github.com/SciML.png",name:"DeepEquilibriumNetworks.jl",desc:"Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence",links:[{icon:"github",link:"https://github.com/SciML/DeepEquilibriumNetworks.jl"}]},{avatar:"https://github.com/CosmologicalEmulators.png",name:"AbstractCosmologicalEmulators.jl",desc:"Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization",links:[{icon:"github",link:"https://github.com/CosmologicalEmulators/AbstractCosmologicalEmulators.jl"}]},{avatar:"https://github.com/impICNF.png",name:"ContinuousNormalizingFlows.jl",desc:"Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia",links:[{icon:"github",link:"https://github.com/impICNF/ContinuousNormalizingFlows.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"Sophon.jl",desc:"Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/Sophon.jl"}]},{avatar:"https://github.com/SciML.png",name:"DataDrivenDiffEq.jl",desc:"Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization",links:[{icon:"github",link:"https://github.com/SciML/DataDrivenDiffEq.jl"}]},{avatar:"https://github.com/YichengDWu.png",name:"NeuralGraphPDE.jl",desc:"Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks",links:[{icon:"github",link:"https://github.com/YichengDWu/NeuralGraphPDE.jl"}]},{avatar:"https://github.com/vavrines.png",name:"Solaris.jl",desc:"Lightweight module for fusing physical and neural models",links:[{icon:"github",link:"https://github.com/vavrines/Solaris.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"Boltz.jl",desc:" Accelerate your ML research using pre-built Deep Learning Models with Lux",links:[{icon:"github",link:"https://github.com/LuxDL/Boltz.jl"}]},{avatar:"https://github.com/JuliaGNI.png",name:"GeometricMachineLearning.jl",desc:"Structure Preserving Machine Learning Models in Julia",links:[{icon:"github",link:"https://github.com/JuliaGNI/GeometricMachineLearning.jl"}]},{avatar:"https://as1.ftcdn.net/jpg/01/09/84/42/220_F_109844212_NnLGUrn3RgMHQIuqSiLGlc9d419eK2dX.jpg",name:"Want to Add Your Package?",desc:'Open a PR in LuxDL/Lux.jl'}],r=[{avatar:"https://github.com/FluxML.png",name:"Zygote.jl",desc:"Lux.jl default choice for AD",links:[{icon:"github",link:"https://github.com/FluxML/Zygote.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Tracker.jl",desc:"Well tested and robust AD library (might fail on edge cases)",links:[{icon:"github",link:"https://github.com/FluxML/Tracker.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ForwardDiff.jl",desc:"For forward mode AD support",links:[{icon:"github",link:"https://github.com/JuliaDiff/ForwardDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"ReverseDiff.jl",desc:"Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)",links:[{icon:"github",link:"https://github.com/JuliaDiff/ReverseDiff.jl"}]},{avatar:"https://github.com/EnzymeAD.png",name:"Enzyme.jl",desc:"Experimental Support but will become the Future Default",links:[{icon:"github",link:"https://github.com/EnzymeAD/Enzyme.jl"}]}],u=[{avatar:"https://github.com/JuliaML.png",name:"MLUtils.jl",desc:"Utilities and abstractions for Machine Learning tasks",links:[{icon:"github",link:"https://github.com/JuliaML/MLUtils.jl"}]},{avatar:"https://github.com/JuliaML.png",name:"MLDatasets.jl",desc:"Utility package for accessing common Machine Learning datasets in Julia",links:[{icon:"github",link:"https://github.com/JuliaML/MLDatasets.jl"}]},{avatar:"https://github.com/JuliaImages.png",name:"Images.jl",desc:"An image library for Julia",links:[{icon:"github",link:"https://github.com/JuliaImages/Images.jl"}]},{avatar:"https://github.com/FluxML.png",name:"DataAugmentation.jl",desc:"Flexible data augmentation library for machine and deep learning",links:[{icon:"github",link:"https://github.com/FluxML/DataAugmentation.jl"}]}],m=[{avatar:"https://github.com/FluxML.png",name:"NNlib.jl",desc:"Neural Network primitives with multiple backends",links:[{icon:"github",link:"https://github.com/FluxML/NNlib.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxLib.jl",desc:"Backend for Lux.jl",links:[{icon:"github",link:"https://github.com/LuxDL/tree/main/lib/LuxLib.jl"}]}],c=[{avatar:"https://github.com/SciML.png",name:"Optimization.jl",desc:"Unified API for Optimization in Julia",links:[{icon:"github",link:"https://github.com/SciML/Optimization.jl"}]},{avatar:"https://github.com/FluxML.png",name:"Optimisers.jl",desc:"Optimisers.jl defines many standard optimisers and utilities for learning loops",links:[{icon:"github",link:"https://github.com/FluxML/Optimisers.jl"}]},{avatar:"https://github.com/FluxML.png",name:"ParameterSchedulers.jl",desc:"Common hyperparameter scheduling for ML",links:[{icon:"github",link:"https://github.com/FluxML/ParameterSchedulers.jl"}]}],g=[{avatar:"https://github.com/FluxML.png",name:"Functors.jl",desc:"Parameterise all the things",links:[{icon:"github",link:"https://github.com/FluxML/Functors.jl"}]},{avatar:"https://github.com/jonniedie.png",name:"ComponentArrays.jl",desc:"Arrays with arbitrarily nested named components",links:[{icon:"github",link:"https://github.com/jonniedie/ComponentArrays.jl"}]}],h=[{avatar:"https://github.com/JuliaLang.png",name:"Serialization.jl",desc:"Provides serialization of Julia objects",links:[{icon:"github",link:"https://github.com/JuliaLang/julia/tree/master/stdlib/Serialization"}]},{avatar:"https://github.com/JuliaIO.png",name:"JLD2.jl",desc:"HDF5-compatible file format in pure Julia",links:[{icon:"github",link:"https://github.com/JuliaIO/JLD2.jl"}]}],p=[{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDiff.jl",desc:"Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDiff.jl"}]},{avatar:"https://github.com/JuliaDiff.png",name:"FiniteDifferences.jl",desc:"High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)",links:[{icon:"github",link:"https://github.com/JuliaDiff/FiniteDifferences.jl"}]},{avatar:"https://github.com/aviatesk.png",name:"JET.jl",desc:"JET employs Julia's type inference system to detect potential bugs and type instabilities",links:[{icon:"github",link:"https://github.com/aviatesk/JET.jl"}]},{avatar:"https://github.com/LuxDL.png",name:"LuxTestUtils.jl",desc:"Collection of Functions useful for testing various packages in the Lux Ecosystem",links:[{icon:"github",link:"https://github.com/LuxDL/tree/main/lib/LuxTestUtils"}]}],b=[{avatar:"https://github.com/JuliaAI.png",name:"MLFlowClient.jl",desc:"Julia client for MLFlow",links:[{icon:"github",link:"https://github.com/JuliaAI/MLFlowClient.jl"}]},{avatar:"https://github.com/JuliaLogging.png",name:"TensorBoardLogger.jl",desc:"Easy peasy logging to TensorBoard with Julia",links:[{icon:"github",link:"https://github.com/JuliaLogging/TensorBoardLogger.jl"}]},{avatar:"https://github.com/avik-pal.png",name:"Wandb.jl",desc:"Unofficial Julia bindings for logging experiments to wandb.ai",links:[{icon:"github",link:"https://github.com/avik-pal/Wandb.jl"}]}];return(D,i)=>(L(),k("div",null,[t(e(f),null,{default:a(()=>[t(e(d),null,{title:a(()=>i[0]||(i[0]=[n("Ecosystem")])),_:1}),t(e(s),null,{title:a(()=>i[1]||(i[1]=[n("Frameworks Extending Lux.jl")])),members:a(()=>[t(e(l),{size:"small",members:o})]),_:1}),t(e(s),null,{title:a(()=>i[2]||(i[2]=[n("Automatic Differentiation")])),members:a(()=>[t(e(l),{size:"small",members:r})]),_:1}),t(e(s),null,{title:a(()=>i[3]||(i[3]=[n("Data Manipulation, Data Loading & Datasets")])),members:a(()=>[t(e(l),{size:"small",members:u})]),_:1}),t(e(s),null,{title:a(()=>i[4]||(i[4]=[n("Neural Network Primitives")])),members:a(()=>[t(e(l),{size:"small",members:m})]),_:1}),t(e(s),null,{title:a(()=>i[5]||(i[5]=[n("Optimization")])),members:a(()=>[t(e(l),{size:"small",members:c})]),_:1}),t(e(s),null,{title:a(()=>i[6]||(i[6]=[n("Parameter Manipulation")])),members:a(()=>[t(e(l),{size:"small",members:g})]),_:1}),t(e(s),null,{title:a(()=>i[7]||(i[7]=[n("Serialization")])),members:a(()=>[t(e(l),{size:"small",members:h})]),_:1}),t(e(s),null,{title:a(()=>i[8]||(i[8]=[n("Testing Utilities")])),members:a(()=>[t(e(l),{size:"small",members:p})]),_:1}),t(e(s),null,{title:a(()=>i[9]||(i[9]=[n("Training Visualization & Logging")])),members:a(()=>[t(e(l),{size:"small",members:b})]),_:1})]),_:1})]))}});export{x as __pageData,J as default}; diff --git a/previews/PR1000/assets/index.md.aWoZd1IO.js b/previews/PR1000/assets/index.md.BEFx1SYc.js similarity index 91% rename from previews/PR1000/assets/index.md.aWoZd1IO.js rename to previews/PR1000/assets/index.md.BEFx1SYc.js index 610ee2563a..cf0d828a25 100644 --- a/previews/PR1000/assets/index.md.aWoZd1IO.js +++ b/previews/PR1000/assets/index.md.BEFx1SYc.js @@ -1,12 +1,12 @@ -import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),e={name:"index.md"};function l(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`

    How to Install Lux.jl? ​

    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:

    julia
    julia> using Pkg
+import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const o=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),l={name:"index.md"};function e(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`
    How to Install Lux.jl? ​
    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:
    julia
    julia> using Pkg
 julia> Pkg.add("Lux")
    If you want to use the latest unreleased version of Lux.jl, you can run the following command: (in most cases the released version will be same as the version on github)
    julia
    julia> using Pkg
-julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
+julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
 Pkg.add("LuxCUDA")
 # or
 Pkg.add(["CUDA", "cuDNN"])
    julia
    using Pkg
 Pkg.add("AMDGPU")
    julia
    using Pkg
 Pkg.add("Metal")
    julia
    using Pkg
-Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
+Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
 
 const dev = gpu_device()
    julia
    using Lux, AMDGPU
 
@@ -15,7 +15,7 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const g
 const dev = gpu_device()
    julia
    using Lux, oneAPI
 
 const dev = gpu_device()
    Want XLA Support? ​
    Install the following package:
    julia
    using Pkg;
-Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
+Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
 Reactant.set_default_backend("cpu") # default
 
 const dev = xla_device()
    julia
    using Reactant, Lux
@@ -24,4 +24,4 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const g
 const dev = xla_device()
    julia
    using Reactant, Lux
 Reactant.set_default_backend("tpu")
 
-const dev = xla_device()
    `,15)]))}const E=i(e,[["render",l]]);export{g as __pageData,E as default};
+const dev = xla_device()
    `,15)]))}const E=i(l,[["render",e]]);export{o as __pageData,E as default}; diff --git a/previews/PR1000/assets/index.md.aWoZd1IO.lean.js b/previews/PR1000/assets/index.md.BEFx1SYc.lean.js similarity index 91% rename from previews/PR1000/assets/index.md.aWoZd1IO.lean.js rename to previews/PR1000/assets/index.md.BEFx1SYc.lean.js index 610ee2563a..cf0d828a25 100644 --- a/previews/PR1000/assets/index.md.aWoZd1IO.lean.js +++ b/previews/PR1000/assets/index.md.BEFx1SYc.lean.js @@ -1,12 +1,12 @@ -import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),e={name:"index.md"};function l(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`

    How to Install Lux.jl? ​

    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:

    julia
    julia> using Pkg
+import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const o=JSON.parse('{"title":"","description":"","frontmatter":{"layout":"home","hero":{"name":"LuxDL Docs","text":"Elegant & Performant Scientific Machine Learning in JuliaLang","tagline":"A Pure Julia Deep Learning Framework designed for Scientific Machine Learning","actions":[{"theme":"brand","text":"Tutorials","link":"/tutorials"},{"theme":"alt","text":"Ecosystem","link":"/ecosystem"},{"theme":"alt","text":"API Reference 📚","link":"/api/Lux/layers"},{"theme":"alt","text":"View on GitHub","link":"https://github.com/LuxDL/Lux.jl"}],"image":{"src":"/lux-logo.svg","alt":"Lux.jl"}},"features":[{"icon":"🚀","title":"Fast & Extendible","details":"Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.","link":"/introduction"},{"icon":"🧑‍🔬","title":"SciML ❤️ Lux","details":"Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.","link":"https://sciml.ai/"},{"icon":"🧩","title":"Uniquely Composable","details":"Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).","link":"/api/Lux/contrib#Training"},{"icon":"🧪","title":"Well Tested","details":"Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.","link":"/api/Testing_Functionality/LuxTestUtils"}]},"headers":[],"relativePath":"index.md","filePath":"index.md","lastUpdated":null}'),l={name:"index.md"};function e(p,s,h,k,d,r){return n(),a("div",null,s[0]||(s[0]=[t(`
    How to Install Lux.jl? ​
    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:
    julia
    julia> using Pkg
 julia> Pkg.add("Lux")
    If you want to use the latest unreleased version of Lux.jl, you can run the following command: (in most cases the released version will be same as the version on github)
    julia
    julia> using Pkg
-julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
+julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
 Pkg.add("LuxCUDA")
 # or
 Pkg.add(["CUDA", "cuDNN"])
    julia
    using Pkg
 Pkg.add("AMDGPU")
    julia
    using Pkg
 Pkg.add("Metal")
    julia
    using Pkg
-Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
+Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
 
 const dev = gpu_device()
    julia
    using Lux, AMDGPU
 
@@ -15,7 +15,7 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const g
 const dev = gpu_device()
    julia
    using Lux, oneAPI
 
 const dev = gpu_device()
    Want XLA Support? ​
    Install the following package:
    julia
    using Pkg;
-Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
+Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
 Reactant.set_default_backend("cpu") # default
 
 const dev = xla_device()
    julia
    using Reactant, Lux
@@ -24,4 +24,4 @@ import{_ as i,c as a,a2 as t,o as n}from"./chunks/framework.DjZDIZsN.js";const g
 const dev = xla_device()
    julia
    using Reactant, Lux
 Reactant.set_default_backend("tpu")
 
-const dev = xla_device()
    `,15)]))}const E=i(e,[["render",l]]);export{g as __pageData,E as default};
+const dev = xla_device()
    `,15)]))}const E=i(l,[["render",e]]);export{o as __pageData,E as default}; diff --git a/previews/PR1000/assets/manual_debugging.md.L4FxOj9s.js b/previews/PR1000/assets/manual_debugging.md.Cd8vyuD9.js similarity index 99% rename from previews/PR1000/assets/manual_debugging.md.L4FxOj9s.js rename to previews/PR1000/assets/manual_debugging.md.Cd8vyuD9.js index 9bcdce9bc3..748f9ec258 100644 --- a/previews/PR1000/assets/manual_debugging.md.L4FxOj9s.js +++ b/previews/PR1000/assets/manual_debugging.md.Cd8vyuD9.js @@ -40,7 +40,7 @@ import{_ as a,c as i,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g [ Info: Input Type: Matrix{Float32} | Input Structure: (3, 2). [ Info: Running Layer: Dense(1 => 1) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2)! ┌ Error: Layer Dense(1 => 1) failed!! This layer is present at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2). -└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-10/julialang/lux-dot-jl/src/contrib/debug.jl:102 +└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/src/contrib/debug.jl:102 DimensionMismatch("A has shape (1, 1) but B has shape (3, 2)")

    See now we know that model.layers.layer_2.layers.layer_2 is the problematic layer. Let us fix that layer and see what happens:

    julia
    model = Chain(Dense(1 => 16, relu),
     Chain(
         Dense(16 => 3),  
diff --git a/previews/PR1000/assets/manual_debugging.md.L4FxOj9s.lean.js b/previews/PR1000/assets/manual_debugging.md.Cd8vyuD9.lean.js
similarity index 99%
rename from previews/PR1000/assets/manual_debugging.md.L4FxOj9s.lean.js
rename to previews/PR1000/assets/manual_debugging.md.Cd8vyuD9.lean.js
index 9bcdce9bc3..748f9ec258 100644
--- a/previews/PR1000/assets/manual_debugging.md.L4FxOj9s.lean.js
+++ b/previews/PR1000/assets/manual_debugging.md.Cd8vyuD9.lean.js
@@ -40,7 +40,7 @@ import{_ as a,c as i,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g
 [ Info: Input Type: Matrix{Float32} | Input Structure: (3, 2).
 [ Info: Running Layer: Dense(1 => 1) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2)!
 ┌ Error: Layer Dense(1 => 1) failed!! This layer is present at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2).
-└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-10/julialang/lux-dot-jl/src/contrib/debug.jl:102
+└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/src/contrib/debug.jl:102
 DimensionMismatch("A has shape (1, 1) but B has shape (3, 2)")

    See now we know that model.layers.layer_2.layers.layer_2 is the problematic layer. Let us fix that layer and see what happens:

    julia
    model = Chain(Dense(1 => 16, relu),
     Chain(
         Dense(16 => 3),  
diff --git a/previews/PR1000/assets/manual_dispatch_custom_input.md.BycrtfJi.js b/previews/PR1000/assets/manual_dispatch_custom_input.md.BrY2DpLL.js
similarity index 97%
rename from previews/PR1000/assets/manual_dispatch_custom_input.md.BycrtfJi.js
rename to previews/PR1000/assets/manual_dispatch_custom_input.md.BrY2DpLL.js
index 8a2bf7a153..1910f338f3 100644
--- a/previews/PR1000/assets/manual_dispatch_custom_input.md.BycrtfJi.js
+++ b/previews/PR1000/assets/manual_dispatch_custom_input.md.BrY2DpLL.js
@@ -1,4 +1,4 @@
-import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Dispatching on Custom Input Types","description":"","frontmatter":{},"headers":[],"relativePath":"manual/dispatch_custom_input.md","filePath":"manual/dispatch_custom_input.md","lastUpdated":null}'),h={name:"manual/dispatch_custom_input.md"};function p(e,s,l,k,r,d){return t(),a("div",null,s[0]||(s[0]=[n(`
    Dispatching on Custom Input Types ​
    Which function should participate in dispatch? ​
    • Defining a dispatch on (::Layer)(x::MyInputType, ps, st::NamedTuple) is inconvenient, since it requires the user to define a new method for every layer type.
    • (::AbstractLuxLayer)(x::MyInputType, ps, st::NamedTuple) doesn't work.
    • Instead, we need to define the dispatch on Lux.apply(::AbstractLuxLayer, x::MyInputType, ps, st::NamedTuple).
    Concrete Example ​
    Consider Neural ODEs. In these models, often time we want to every iteration of the neural network to take the current time as input. Here, we won't go through implementing an entire Neural ODE model. Instead we will define a time dependent version of Chain.
    Time-Dependent Chain Implementation ​
    julia
    using Lux, Random
+import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Dispatching on Custom Input Types","description":"","frontmatter":{},"headers":[],"relativePath":"manual/dispatch_custom_input.md","filePath":"manual/dispatch_custom_input.md","lastUpdated":null}'),h={name:"manual/dispatch_custom_input.md"};function p(l,s,e,k,r,d){return t(),a("div",null,s[0]||(s[0]=[n(`
    Dispatching on Custom Input Types ​
    Which function should participate in dispatch? ​
    • Defining a dispatch on (::Layer)(x::MyInputType, ps, st::NamedTuple) is inconvenient, since it requires the user to define a new method for every layer type.
    • (::AbstractLuxLayer)(x::MyInputType, ps, st::NamedTuple) doesn't work.
    • Instead, we need to define the dispatch on Lux.apply(::AbstractLuxLayer, x::MyInputType, ps, st::NamedTuple).
    Concrete Example ​
    Consider Neural ODEs. In these models, often time we want to every iteration of the neural network to take the current time as input. Here, we won't go through implementing an entire Neural ODE model. Instead we will define a time dependent version of Chain.
    Time-Dependent Chain Implementation ​
    julia
    using Lux, Random
 
 struct TDChain{L <: NamedTuple} <: Lux.AbstractLuxWrapperLayer{:layers}
     layers::L
@@ -36,11 +36,11 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g
 
 Closest candidates are:
   apply(!Matched::AbstractLuxLayer, ::Any, ::Any, ::Any)
-   @ LuxCore ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/src/LuxCore.jl:154
+   @ LuxCore /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/src/LuxCore.jl:154
   apply(!Matched::AbstractLuxLayer, !Matched::Tracker.TrackedArray, ::Any, ::Any)
-   @ LuxCoreArrayInterfaceTrackerExt ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/ext/LuxCoreArrayInterfaceTrackerExt.jl:19
+   @ LuxCoreArrayInterfaceTrackerExt /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/ext/LuxCoreArrayInterfaceTrackerExt.jl:19
   apply(!Matched::AbstractLuxLayer, !Matched::ReverseDiff.TrackedArray, ::Any, ::Any)
-   @ LuxCoreArrayInterfaceReverseDiffExt ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:20
+   @ LuxCoreArrayInterfaceReverseDiffExt /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:20
   ...
    Writing the Correct Dispatch Rules ​
    • Create a Custom Layer storing the time.
    julia
    struct ArrayAndTime{A <: AbstractArray, T <: Real}
     array::A
     time::T
diff --git a/previews/PR1000/assets/manual_dispatch_custom_input.md.BycrtfJi.lean.js b/previews/PR1000/assets/manual_dispatch_custom_input.md.BrY2DpLL.lean.js
similarity index 97%
rename from previews/PR1000/assets/manual_dispatch_custom_input.md.BycrtfJi.lean.js
rename to previews/PR1000/assets/manual_dispatch_custom_input.md.BrY2DpLL.lean.js
index 8a2bf7a153..1910f338f3 100644
--- a/previews/PR1000/assets/manual_dispatch_custom_input.md.BycrtfJi.lean.js
+++ b/previews/PR1000/assets/manual_dispatch_custom_input.md.BrY2DpLL.lean.js
@@ -1,4 +1,4 @@
-import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Dispatching on Custom Input Types","description":"","frontmatter":{},"headers":[],"relativePath":"manual/dispatch_custom_input.md","filePath":"manual/dispatch_custom_input.md","lastUpdated":null}'),h={name:"manual/dispatch_custom_input.md"};function p(e,s,l,k,r,d){return t(),a("div",null,s[0]||(s[0]=[n(`
    Dispatching on Custom Input Types ​
    Which function should participate in dispatch? ​
    • Defining a dispatch on (::Layer)(x::MyInputType, ps, st::NamedTuple) is inconvenient, since it requires the user to define a new method for every layer type.
    • (::AbstractLuxLayer)(x::MyInputType, ps, st::NamedTuple) doesn't work.
    • Instead, we need to define the dispatch on Lux.apply(::AbstractLuxLayer, x::MyInputType, ps, st::NamedTuple).
    Concrete Example ​
    Consider Neural ODEs. In these models, often time we want to every iteration of the neural network to take the current time as input. Here, we won't go through implementing an entire Neural ODE model. Instead we will define a time dependent version of Chain.
    Time-Dependent Chain Implementation ​
    julia
    using Lux, Random
+import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Dispatching on Custom Input Types","description":"","frontmatter":{},"headers":[],"relativePath":"manual/dispatch_custom_input.md","filePath":"manual/dispatch_custom_input.md","lastUpdated":null}'),h={name:"manual/dispatch_custom_input.md"};function p(l,s,e,k,r,d){return t(),a("div",null,s[0]||(s[0]=[n(`
    Dispatching on Custom Input Types ​
    Which function should participate in dispatch? ​
    • Defining a dispatch on (::Layer)(x::MyInputType, ps, st::NamedTuple) is inconvenient, since it requires the user to define a new method for every layer type.
    • (::AbstractLuxLayer)(x::MyInputType, ps, st::NamedTuple) doesn't work.
    • Instead, we need to define the dispatch on Lux.apply(::AbstractLuxLayer, x::MyInputType, ps, st::NamedTuple).
    Concrete Example ​
    Consider Neural ODEs. In these models, often time we want to every iteration of the neural network to take the current time as input. Here, we won't go through implementing an entire Neural ODE model. Instead we will define a time dependent version of Chain.
    Time-Dependent Chain Implementation ​
    julia
    using Lux, Random
 
 struct TDChain{L <: NamedTuple} <: Lux.AbstractLuxWrapperLayer{:layers}
     layers::L
@@ -36,11 +36,11 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g
 
 Closest candidates are:
   apply(!Matched::AbstractLuxLayer, ::Any, ::Any, ::Any)
-   @ LuxCore ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/src/LuxCore.jl:154
+   @ LuxCore /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/src/LuxCore.jl:154
   apply(!Matched::AbstractLuxLayer, !Matched::Tracker.TrackedArray, ::Any, ::Any)
-   @ LuxCoreArrayInterfaceTrackerExt ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/ext/LuxCoreArrayInterfaceTrackerExt.jl:19
+   @ LuxCoreArrayInterfaceTrackerExt /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/ext/LuxCoreArrayInterfaceTrackerExt.jl:19
   apply(!Matched::AbstractLuxLayer, !Matched::ReverseDiff.TrackedArray, ::Any, ::Any)
-   @ LuxCoreArrayInterfaceReverseDiffExt ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:20
+   @ LuxCoreArrayInterfaceReverseDiffExt /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:20
   ...
    Writing the Correct Dispatch Rules ​
    • Create a Custom Layer storing the time.
    julia
    struct ArrayAndTime{A <: AbstractArray, T <: Real}
     array::A
     time::T
diff --git a/previews/PR1000/assets/manual_freezing_model_parameters.md.DQ5B11bP.js b/previews/PR1000/assets/manual_freezing_model_parameters.md.Cvglf3j-.js
similarity index 97%
rename from previews/PR1000/assets/manual_freezing_model_parameters.md.DQ5B11bP.js
rename to previews/PR1000/assets/manual_freezing_model_parameters.md.Cvglf3j-.js
index 5aff4803a4..491551db57 100644
--- a/previews/PR1000/assets/manual_freezing_model_parameters.md.DQ5B11bP.js
+++ b/previews/PR1000/assets/manual_freezing_model_parameters.md.Cvglf3j-.js
@@ -1,4 +1,4 @@
-import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Freezing Model Parameters","description":"","frontmatter":{},"headers":[],"relativePath":"manual/freezing_model_parameters.md","filePath":"manual/freezing_model_parameters.md","lastUpdated":null}'),h={name:"manual/freezing_model_parameters.md"};function l(t,s,p,k,r,d){return e(),a("div",null,s[0]||(s[0]=[n(`
    Freezing Model Parameters ​
    Warning
    API for freezing parameters should be considered experimental at this point.
    In this manual entry, we will go over how to freeze certain parameters in a model.
    Freezing Layers of a Particular Kind ​
    To freeze a particular kind of layer, let's say Dense in the following example. We can use Lux.Experimental.layer_map and freeze layers if they are of type Dense.
    julia
    using Lux, Random, Functors
+import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Freezing Model Parameters","description":"","frontmatter":{},"headers":[],"relativePath":"manual/freezing_model_parameters.md","filePath":"manual/freezing_model_parameters.md","lastUpdated":null}'),h={name:"manual/freezing_model_parameters.md"};function l(t,s,p,k,r,E){return e(),a("div",null,s[0]||(s[0]=[n(`
    Freezing Model Parameters ​
    Warning
    API for freezing parameters should be considered experimental at this point.
    In this manual entry, we will go over how to freeze certain parameters in a model.
    Freezing Layers of a Particular Kind ​
    To freeze a particular kind of layer, let's say Dense in the following example. We can use Lux.Experimental.layer_map and freeze layers if they are of type Dense.
    julia
    using Lux, Random, Functors
 
 rng = Xoshiro(0)
 
@@ -17,7 +17,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g
 
 model_frozen, ps_frozen, st_frozen = Lux.Experimental.layer_map(freeze_dense, model, ps, st)
 
-model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))
    Freezing by Layer Name ​
    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.
    julia
    
+model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))
    Freezing by Layer Name ​
    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.
    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
@@ -26,7 +26,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g
 function freeze_dense(d::Dense, ps, st, _)
     return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
 end
-freeze_dense(l, ps, st, _) = (l, ps, st)
    Freezing Part of the Parameters ​
    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.
    julia
    
+freeze_dense(l, ps, st, _) = (l, ps, st)
    Freezing Part of the Parameters ​
    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.
    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight,))
diff --git a/previews/PR1000/assets/manual_freezing_model_parameters.md.DQ5B11bP.lean.js b/previews/PR1000/assets/manual_freezing_model_parameters.md.Cvglf3j-.lean.js
similarity index 97%
rename from previews/PR1000/assets/manual_freezing_model_parameters.md.DQ5B11bP.lean.js
rename to previews/PR1000/assets/manual_freezing_model_parameters.md.Cvglf3j-.lean.js
index 5aff4803a4..491551db57 100644
--- a/previews/PR1000/assets/manual_freezing_model_parameters.md.DQ5B11bP.lean.js
+++ b/previews/PR1000/assets/manual_freezing_model_parameters.md.Cvglf3j-.lean.js
@@ -1,4 +1,4 @@
-import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Freezing Model Parameters","description":"","frontmatter":{},"headers":[],"relativePath":"manual/freezing_model_parameters.md","filePath":"manual/freezing_model_parameters.md","lastUpdated":null}'),h={name:"manual/freezing_model_parameters.md"};function l(t,s,p,k,r,d){return e(),a("div",null,s[0]||(s[0]=[n(`
    Freezing Model Parameters ​
    Warning
    API for freezing parameters should be considered experimental at this point.
    In this manual entry, we will go over how to freeze certain parameters in a model.
    Freezing Layers of a Particular Kind ​
    To freeze a particular kind of layer, let's say Dense in the following example. We can use Lux.Experimental.layer_map and freeze layers if they are of type Dense.
    julia
    using Lux, Random, Functors
+import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g=JSON.parse('{"title":"Freezing Model Parameters","description":"","frontmatter":{},"headers":[],"relativePath":"manual/freezing_model_parameters.md","filePath":"manual/freezing_model_parameters.md","lastUpdated":null}'),h={name:"manual/freezing_model_parameters.md"};function l(t,s,p,k,r,E){return e(),a("div",null,s[0]||(s[0]=[n(`
    Freezing Model Parameters ​
    Warning
    API for freezing parameters should be considered experimental at this point.
    In this manual entry, we will go over how to freeze certain parameters in a model.
    Freezing Layers of a Particular Kind ​
    To freeze a particular kind of layer, let's say Dense in the following example. We can use Lux.Experimental.layer_map and freeze layers if they are of type Dense.
    julia
    using Lux, Random, Functors
 
 rng = Xoshiro(0)
 
@@ -17,7 +17,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g
 
 model_frozen, ps_frozen, st_frozen = Lux.Experimental.layer_map(freeze_dense, model, ps, st)
 
-model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))
    Freezing by Layer Name ​
    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.
    julia
    
+model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))
    Freezing by Layer Name ​
    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.
    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
@@ -26,7 +26,7 @@ import{_ as i,c as a,a2 as n,o as e}from"./chunks/framework.DjZDIZsN.js";const g
 function freeze_dense(d::Dense, ps, st, _)
     return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
 end
-freeze_dense(l, ps, st, _) = (l, ps, st)
    Freezing Part of the Parameters ​
    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.
    julia
    
+freeze_dense(l, ps, st, _) = (l, ps, st)
    Freezing Part of the Parameters ​
    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.
    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight,))
diff --git a/previews/PR1000/assets/manual_migrate_from_flux.md.B2ZBcJwH.js b/previews/PR1000/assets/manual_migrate_from_flux.md.CdtVRtI7.js
similarity index 98%
rename from previews/PR1000/assets/manual_migrate_from_flux.md.B2ZBcJwH.js
rename to previews/PR1000/assets/manual_migrate_from_flux.md.CdtVRtI7.js
index f9f7d28c96..fe51032c16 100644
--- a/previews/PR1000/assets/manual_migrate_from_flux.md.B2ZBcJwH.js
+++ b/previews/PR1000/assets/manual_migrate_from_flux.md.CdtVRtI7.js
@@ -1,4 +1,4 @@
-import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
+import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
 
 model = Chain(Dense(2 => 4), BatchNorm(4, relu), Dense(4 => 2))
 rng = Random.default_rng()
@@ -18,7 +18,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZ
 
 model(x)
 
-gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
+gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
 
 struct LuxLinear <: Lux.AbstractLuxLayer
     init_A
@@ -56,7 +56,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZ
 # Needed so that both \`A\` and \`B\` can be transferred between devices
 Flux.@functor FluxLinear
 
-(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
+(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
 model = LuxLinear(randn(rng, 2, 4), randn(rng, 4, 2))
 x = randn(rng, 2, 1)
 
diff --git a/previews/PR1000/assets/manual_migrate_from_flux.md.B2ZBcJwH.lean.js b/previews/PR1000/assets/manual_migrate_from_flux.md.CdtVRtI7.lean.js
similarity index 98%
rename from previews/PR1000/assets/manual_migrate_from_flux.md.B2ZBcJwH.lean.js
rename to previews/PR1000/assets/manual_migrate_from_flux.md.CdtVRtI7.lean.js
index f9f7d28c96..fe51032c16 100644
--- a/previews/PR1000/assets/manual_migrate_from_flux.md.B2ZBcJwH.lean.js
+++ b/previews/PR1000/assets/manual_migrate_from_flux.md.CdtVRtI7.lean.js
@@ -1,4 +1,4 @@
-import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
+import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZsN.js";const x=JSON.parse('{"title":"Migrating from Flux to Lux","description":"","frontmatter":{},"headers":[],"relativePath":"manual/migrate_from_flux.md","filePath":"manual/migrate_from_flux.md","lastUpdated":null}'),h={name:"manual/migrate_from_flux.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.24ex",height:"1.645ex",role:"img",focusable:"false",viewBox:"0 -716 4525.9 727","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.697ex",height:"1.62ex",role:"img",focusable:"false",viewBox:"0 -716 750 716","aria-hidden":"true"},E={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"1.717ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 759 683","aria-hidden":"true"};function o(y,i,c,u,F,m){return t(),a("div",null,[i[10]||(i[10]=l(`
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
 
 model = Chain(Dense(2 => 4), BatchNorm(4, relu), Dense(4 => 2))
 rng = Random.default_rng()
@@ -18,7 +18,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZ
 
 model(x)
 
-gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
+gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    `,6)),s("p",null,[i[6]||(i[6]=n("Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes ")),s("mjx-container",p,[(t(),a("svg",k,i[0]||(i[0]=[l('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"×"),s("mi",null,"B"),s("mo",null,"×"),s("mi",null,"x")])],-1))]),i[7]||(i[7]=n(" where ")),s("mjx-container",r,[(t(),a("svg",d,i[2]||(i[2]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D434",d:"M208 74Q208 50 254 46Q272 46 272 35Q272 34 270 22Q267 8 264 4T251 0Q249 0 239 0T205 1T141 2Q70 2 50 0H42Q35 7 35 11Q37 38 48 46H62Q132 49 164 96Q170 102 345 401T523 704Q530 716 547 716H555H572Q578 707 578 706L606 383Q634 60 636 57Q641 46 701 46Q726 46 726 36Q726 34 723 22Q720 7 718 4T704 0Q701 0 690 0T651 1T578 2Q484 2 455 0H443Q437 6 437 9T439 27Q443 40 445 43L449 46H469Q523 49 533 63L521 213H283L249 155Q208 86 208 74ZM516 260Q516 271 504 416T490 562L463 519Q447 492 400 412L310 260L413 259Q516 259 516 260Z",style:{"stroke-width":"3"}})])])],-1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A")])],-1))]),i[8]||(i[8]=n(" is not trainable and ")),s("mjx-container",E,[(t(),a("svg",g,i[4]||(i[4]=[s("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[s("g",{"data-mml-node":"math"},[s("g",{"data-mml-node":"mi"},[s("path",{"data-c":"1D435",d:"M231 637Q204 637 199 638T194 649Q194 676 205 682Q206 683 335 683Q594 683 608 681Q671 671 713 636T756 544Q756 480 698 429T565 360L555 357Q619 348 660 311T702 219Q702 146 630 78T453 1Q446 0 242 0Q42 0 39 2Q35 5 35 10Q35 17 37 24Q42 43 47 45Q51 46 62 46H68Q95 46 128 49Q142 52 147 61Q150 65 219 339T288 628Q288 635 231 637ZM649 544Q649 574 634 600T585 634Q578 636 493 637Q473 637 451 637T416 636H403Q388 635 384 626Q382 622 352 506Q352 503 351 500L320 374H401Q482 374 494 376Q554 386 601 434T649 544ZM595 229Q595 273 572 302T512 336Q506 337 429 337Q311 337 310 336Q310 334 293 263T258 122L240 52Q240 48 252 48T333 46Q422 46 429 47Q491 54 543 105T595 229Z",style:{"stroke-width":"3"}})])])],-1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"B")])],-1))]),i[9]||(i[9]=n(" is trainable."))]),i[11]||(i[11]=l(`
    julia
    using Lux, Random, NNlib, Zygote
 
 struct LuxLinear <: Lux.AbstractLuxLayer
     init_A
@@ -56,7 +56,7 @@ import{_ as e,c as a,a2 as l,j as s,a as n,o as t}from"./chunks/framework.DjZDIZ
 # Needed so that both \`A\` and \`B\` can be transferred between devices
 Flux.@functor FluxLinear
 
-(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
+(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
 model = LuxLinear(randn(rng, 2, 4), randn(rng, 4, 2))
 x = randn(rng, 2, 1)
 
diff --git a/previews/PR1000/assets/manual_nested_autodiff.md.CDY_fLt0.js b/previews/PR1000/assets/manual_nested_autodiff.md.BHR1mjww.js
similarity index 99%
rename from previews/PR1000/assets/manual_nested_autodiff.md.CDY_fLt0.js
rename to previews/PR1000/assets/manual_nested_autodiff.md.BHR1mjww.js
index a5a6d73d65..635645f623 100644
--- a/previews/PR1000/assets/manual_nested_autodiff.md.CDY_fLt0.js
+++ b/previews/PR1000/assets/manual_nested_autodiff.md.BHR1mjww.js
@@ -1,4 +1,4 @@
-import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js";const j=JSON.parse('{"title":"Nested Automatic Differentiation","description":"","frontmatter":{},"headers":[],"relativePath":"manual/nested_autodiff.md","filePath":"manual/nested_autodiff.md","lastUpdated":null}'),h={name:"manual/nested_autodiff.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.178ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4498.7 886","aria-hidden":"true"},d={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"7.009ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 3098 886","aria-hidden":"true"},o={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.439ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5056 1199","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"33.692ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 14891.7 2978.9","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"9.913ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4381.7 886","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},m={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.167ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 9355.6 2978.9","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.661ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.843ex",height:"2.565ex",role:"img",focusable:"false",viewBox:"0 -841.7 1698.8 1133.9","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.269ex",height:"1.902ex",role:"img",focusable:"false",viewBox:"0 -683 1445 840.8","aria-hidden":"true"},b={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.837ex",height:"1.359ex",role:"img",focusable:"false",viewBox:"0 -443 812 600.8","aria-hidden":"true"};function v(w,i,H,D,B,L){return e(),n("div",null,[i[32]||(i[32]=t(`
    Nested Automatic Differentiation ​
    Note
    This is a relatively new feature in Lux, so there might be some rough edges. If you encounter any issues, please let us know by opening an issue on the GitHub repository.
    In this manual, we will explore how to use automatic differentiation (AD) inside your layers or loss functions and have Lux automatically switch the AD backend with a faster one when needed.
    Tip
    Don't wan't Lux to do this switching for you? You can disable it by setting the automatic_nested_ad_switching Preference to false.
    Remember that if you are using ForwardDiff inside a Zygote call, it will drop gradients (with a warning message), so it is not recommended to use this combination.
    Let's explore this using some questions that were posted on the Julia Discourse forum.
    julia
    using ADTypes, Lux, LinearAlgebra, Zygote, ForwardDiff, Random, StableRNGs
+import{_ as e,c as n,a2 as t,j as s,a,o as l}from"./chunks/framework.DjZDIZsN.js";const j=JSON.parse('{"title":"Nested Automatic Differentiation","description":"","frontmatter":{},"headers":[],"relativePath":"manual/nested_autodiff.md","filePath":"manual/nested_autodiff.md","lastUpdated":null}'),h={name:"manual/nested_autodiff.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.178ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4498.7 886","aria-hidden":"true"},d={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"7.009ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 3098 886","aria-hidden":"true"},o={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.439ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5056 1199","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"33.692ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 14891.7 2978.9","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"9.913ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4381.7 886","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},m={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.167ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 9355.6 2978.9","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.661ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.843ex",height:"2.565ex",role:"img",focusable:"false",viewBox:"0 -841.7 1698.8 1133.9","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.269ex",height:"1.902ex",role:"img",focusable:"false",viewBox:"0 -683 1445 840.8","aria-hidden":"true"},b={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.837ex",height:"1.359ex",role:"img",focusable:"false",viewBox:"0 -443 812 600.8","aria-hidden":"true"};function v(w,i,H,D,B,L){return l(),n("div",null,[i[32]||(i[32]=t(`
    Nested Automatic Differentiation ​
    Note
    This is a relatively new feature in Lux, so there might be some rough edges. If you encounter any issues, please let us know by opening an issue on the GitHub repository.
    In this manual, we will explore how to use automatic differentiation (AD) inside your layers or loss functions and have Lux automatically switch the AD backend with a faster one when needed.
    Tip
    Don't wan't Lux to do this switching for you? You can disable it by setting the automatic_nested_ad_switching Preference to false.
    Remember that if you are using ForwardDiff inside a Zygote call, it will drop gradients (with a warning message), so it is not recommended to use this combination.
    Let's explore this using some questions that were posted on the Julia Discourse forum.
    julia
    using ADTypes, Lux, LinearAlgebra, Zygote, ForwardDiff, Random, StableRNGs
 using ComponentArrays, FiniteDiff
    First let's set the stage using some minor changes that need to be made for this feature to work:
    • Switching only works if a StatefulLuxLayer is being used, with the following function calls:
      • For operations on the inputs:
        • (<some-function> ∘ <StatefulLuxLayer>)(x::AbstractArray)
        • (<StatefulLuxLayer> ∘ <some-function>)(x::AbstractArray)
        • (<StatefulLuxLayer>)(x::AbstractArray)
      • For operations on the parameters:
        • (<some-function> ∘ Base.Fix1(<StatefulLuxLayer>, x))(ps)
        • (Base.Fix1(<StatefulLuxLayer>, x) ∘ <some-function>)(ps)
        • (Base.Fix1(<StatefulLuxLayer>, x))(ps)
    • Currently we have custom routines implemented for:
    • Switching only happens for ChainRules compatible AD libraries.
    We plan to capture DifferentiationInterface, and Enzyme.autodiff calls in the future (PRs are welcome).
    Tip
    @compact uses StatefulLuxLayers internally, so you can directly use these features inside a layer generated by @compact.
    Loss Function containing Jacobian Computation ​
    This problem comes from @facusapienza on Discourse. In this case, we want to add a regularization term to the neural DE based on first-order derivatives. The neural DE part is not important here and we can demonstrate this easily with a standard neural network.
    julia
    function loss_function1(model, x, ps, st, y)
     # Make it a stateful layer
     smodel = StatefulLuxLayer{true}(model, ps, st)
@@ -22,9 +22,9 @@ import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js
 
 println("∞-norm(∂x - ∂x_fd): ", norm(∂x .- ∂x_fd, Inf))
 println("∞-norm(∂ps - ∂ps_fd): ", norm(ComponentArray(∂ps) .- ∂ps_fd, Inf))
    ┌ Warning: \`training\` is set to \`Val{true}()\` but is not being used within an autodiff call (gradient, jacobian, etc...). This will be slow. If you are using a \`Lux.jl\` model, set it to inference (test) mode using \`LuxCore.testmode\`. Reliance on this behavior is discouraged, and is not guaranteed by Semantic Versioning, and might be removed without a deprecation cycle. It is recommended to fix this issue in your code.
-└ @ LuxLib.Utils ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxLib/I9RHW/src/utils.jl:309
+└ @ LuxLib.Utils /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxLib/src/utils.jl:309
 ┌ Warning: \`training\` is set to \`Val{true}()\` but is not being used within an autodiff call (gradient, jacobian, etc...). This will be slow. If you are using a \`Lux.jl\` model, set it to inference (test) mode using \`LuxCore.testmode\`. Reliance on this behavior is discouraged, and is not guaranteed by Semantic Versioning, and might be removed without a deprecation cycle. It is recommended to fix this issue in your code.
-└ @ LuxLib.Utils ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxLib/I9RHW/src/utils.jl:309
+└ @ LuxLib.Utils /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxLib/src/utils.jl:309
 ∞-norm(∂x - ∂x_fd): 0.00046014786
 ∞-norm(∂ps - ∂ps_fd): 0.00068473816
    That's pretty good, of course you will have some error from the finite differences calculation.
    Using Batched Jacobian for Multiple Inputs ​
    Notice that in this example the Jacobian J consists on the full matrix of derivatives of smodel with respect the different inputs in x. In many cases, we are interested in computing the Jacobian with respect to each input individually, avoiding the unnecessary calculation of zero entries of the Jacobian. This can be achieved with batched_jacobian to parse the calculation of the Jacobian per each single input. Using the same example from the previous section:
    julia
    model = Chain(Dense(2 => 4, tanh), Dense(4 => 2))
 ps, st = Lux.setup(StableRNG(0), model)
@@ -80,7 +80,7 @@ import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js
 
 println("∞-norm(∂x - ∂x_fd): ", norm(∂x .- ∂x_fd, Inf))
 println("∞-norm(∂ps - ∂ps_fd): ", norm(ComponentArray(∂ps) .- ∂ps_fd, Inf))
    ∞-norm(∂x - ∂x_fd): 4.172325e-6
-∞-norm(∂ps - ∂ps_fd): 4.5776367e-5
    Hutchinson Trace Estimation ​
    `,51)),s("p",null,[i[6]||(i[6]=a("Hutchinson Trace Estimation often shows up in machine learning literature to provide a fast estimate of the trace of a Jacobian Matrix. This is based off of ")),i[7]||(i[7]=s("a",{href:"https://www.nowozin.net/sebastian/blog/thoughts-on-trace-estimation-in-deep-learning.html",target:"_blank",rel:"noreferrer"},"Hutchinson 1990",-1)),i[8]||(i[8]=a(" which computes the estimated trace of a matrix ")),s("mjx-container",p,[(e(),n("svg",k,i[0]||(i[0]=[t('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[9]||(i[9]=a(" using random vectors ")),s("mjx-container",d,[(e(),n("svg",r,i[2]||(i[2]=[t('',1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"v"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D")])])])],-1))]),i[10]||(i[10]=a(" s.t. ")),s("mjx-container",o,[(e(),n("svg",g,i[4]||(i[4]=[t('',1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("mi",null,"v"),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mi",null,"I")])],-1))]),i[11]||(i[11]=a("."))]),s("mjx-container",Q,[(e(),n("svg",E,i[12]||(i[12]=[t('',1)]))),i[13]||(i[13]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"A"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mi",null,"A"),s("mi",null,"v"),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"A"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),s("p",null,[i[16]||(i[16]=a("We can use this to compute the trace of a Jacobian Matrix ")),s("mjx-container",T,[(e(),n("svg",c,i[14]||(i[14]=[t('',1)]))),i[15]||(i[15]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[17]||(i[17]=a(" using the following algorithm:"))]),s("mjx-container",y,[(e(),n("svg",m,i[18]||(i[18]=[t('',1)]))),i[19]||(i[19]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"J"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[33]||(i[33]=s("p",null,"Note that we can compute this using two methods:",-1)),s("ol",null,[s("li",null,[s("p",null,[i[22]||(i[22]=a("Compute ")),s("mjx-container",u,[(e(),n("svg",F,i[20]||(i[20]=[t('',1)]))),i[21]||(i[21]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J")])],-1))]),i[23]||(i[23]=a(" using a Vector-Jacobian product and then do a matrix-vector product to get the trace."))])]),s("li",null,[s("p",null,[i[26]||(i[26]=a("Compute ")),s("mjx-container",C,[(e(),n("svg",f,i[24]||(i[24]=[t('',1)]))),i[25]||(i[25]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[27]||(i[27]=a(" using a Jacobian-Vector product and then do a matrix-vector product to get the trace."))])])]),s("p",null,[i[30]||(i[30]=a("For simplicity, we will use a single sample of ")),s("mjx-container",b,[(e(),n("svg",x,i[28]||(i[28]=[t('',1)]))),i[29]||(i[29]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[31]||(i[31]=a(" to compute the trace. Additionally, we will fix the sample to ensure that our tests against the finite difference implementation are not affected by the randomness in the sample."))]),i[34]||(i[34]=t(`
    Computing using the Vector-Jacobian Product ​
    julia
    function hutchinson_trace_vjp(model, x, ps, st, v)
+∞-norm(∂ps - ∂ps_fd): 4.5776367e-5
    Hutchinson Trace Estimation ​
    `,51)),s("p",null,[i[6]||(i[6]=a("Hutchinson Trace Estimation often shows up in machine learning literature to provide a fast estimate of the trace of a Jacobian Matrix. This is based off of ")),i[7]||(i[7]=s("a",{href:"https://www.nowozin.net/sebastian/blog/thoughts-on-trace-estimation-in-deep-learning.html",target:"_blank",rel:"noreferrer"},"Hutchinson 1990",-1)),i[8]||(i[8]=a(" which computes the estimated trace of a matrix ")),s("mjx-container",p,[(l(),n("svg",k,i[0]||(i[0]=[t('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[9]||(i[9]=a(" using random vectors ")),s("mjx-container",d,[(l(),n("svg",r,i[2]||(i[2]=[t('',1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"v"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D")])])])],-1))]),i[10]||(i[10]=a(" s.t. ")),s("mjx-container",o,[(l(),n("svg",g,i[4]||(i[4]=[t('',1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("mi",null,"v"),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mi",null,"I")])],-1))]),i[11]||(i[11]=a("."))]),s("mjx-container",Q,[(l(),n("svg",E,i[12]||(i[12]=[t('',1)]))),i[13]||(i[13]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"A"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mi",null,"A"),s("mi",null,"v"),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"A"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),s("p",null,[i[16]||(i[16]=a("We can use this to compute the trace of a Jacobian Matrix ")),s("mjx-container",T,[(l(),n("svg",c,i[14]||(i[14]=[t('',1)]))),i[15]||(i[15]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[17]||(i[17]=a(" using the following algorithm:"))]),s("mjx-container",y,[(l(),n("svg",m,i[18]||(i[18]=[t('',1)]))),i[19]||(i[19]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"J"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[33]||(i[33]=s("p",null,"Note that we can compute this using two methods:",-1)),s("ol",null,[s("li",null,[s("p",null,[i[22]||(i[22]=a("Compute ")),s("mjx-container",u,[(l(),n("svg",F,i[20]||(i[20]=[t('',1)]))),i[21]||(i[21]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J")])],-1))]),i[23]||(i[23]=a(" using a Vector-Jacobian product and then do a matrix-vector product to get the trace."))])]),s("li",null,[s("p",null,[i[26]||(i[26]=a("Compute ")),s("mjx-container",C,[(l(),n("svg",f,i[24]||(i[24]=[t('',1)]))),i[25]||(i[25]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[27]||(i[27]=a(" using a Jacobian-Vector product and then do a matrix-vector product to get the trace."))])])]),s("p",null,[i[30]||(i[30]=a("For simplicity, we will use a single sample of ")),s("mjx-container",b,[(l(),n("svg",x,i[28]||(i[28]=[t('',1)]))),i[29]||(i[29]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[31]||(i[31]=a(" to compute the trace. Additionally, we will fix the sample to ensure that our tests against the finite difference implementation are not affected by the randomness in the sample."))]),i[34]||(i[34]=t(`
    Computing using the Vector-Jacobian Product ​
    julia
    function hutchinson_trace_vjp(model, x, ps, st, v)
     smodel = StatefulLuxLayer{true}(model, ps, st)
     vjp = vector_jacobian_product(smodel, AutoZygote(), x, v)
     return sum(batched_matmul(reshape(vjp, 1, :, size(vjp, ndims(vjp))),
@@ -116,4 +116,4 @@ import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js
     norm(ComponentArray(∂ps_full_jacobian) .- ComponentArray(∂ps_vjp), Inf))
    ∞-norm(∂x using vjp): 0.0
 ∞-norm(∂ps using vjp): 0.0
 ∞-norm(∂x using full jacobian): 9.536743e-7
-∞-norm(∂ps using full jacobian): 1.4305115e-6
    `,20))])}const _=l(h,[["render",v]]);export{j as __pageData,_ as default};
+∞-norm(∂ps using full jacobian): 1.4305115e-6
    `,20))])}const _=e(h,[["render",v]]);export{j as __pageData,_ as default};
diff --git a/previews/PR1000/assets/manual_nested_autodiff.md.CDY_fLt0.lean.js b/previews/PR1000/assets/manual_nested_autodiff.md.BHR1mjww.lean.js
similarity index 99%
rename from previews/PR1000/assets/manual_nested_autodiff.md.CDY_fLt0.lean.js
rename to previews/PR1000/assets/manual_nested_autodiff.md.BHR1mjww.lean.js
index a5a6d73d65..635645f623 100644
--- a/previews/PR1000/assets/manual_nested_autodiff.md.CDY_fLt0.lean.js
+++ b/previews/PR1000/assets/manual_nested_autodiff.md.BHR1mjww.lean.js
@@ -1,4 +1,4 @@
-import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js";const j=JSON.parse('{"title":"Nested Automatic Differentiation","description":"","frontmatter":{},"headers":[],"relativePath":"manual/nested_autodiff.md","filePath":"manual/nested_autodiff.md","lastUpdated":null}'),h={name:"manual/nested_autodiff.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.178ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4498.7 886","aria-hidden":"true"},d={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"7.009ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 3098 886","aria-hidden":"true"},o={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.439ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5056 1199","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"33.692ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 14891.7 2978.9","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"9.913ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4381.7 886","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},m={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.167ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 9355.6 2978.9","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.661ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.843ex",height:"2.565ex",role:"img",focusable:"false",viewBox:"0 -841.7 1698.8 1133.9","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.269ex",height:"1.902ex",role:"img",focusable:"false",viewBox:"0 -683 1445 840.8","aria-hidden":"true"},b={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.837ex",height:"1.359ex",role:"img",focusable:"false",viewBox:"0 -443 812 600.8","aria-hidden":"true"};function v(w,i,H,D,B,L){return e(),n("div",null,[i[32]||(i[32]=t(`
    Nested Automatic Differentiation ​
    Note
    This is a relatively new feature in Lux, so there might be some rough edges. If you encounter any issues, please let us know by opening an issue on the GitHub repository.
    In this manual, we will explore how to use automatic differentiation (AD) inside your layers or loss functions and have Lux automatically switch the AD backend with a faster one when needed.
    Tip
    Don't wan't Lux to do this switching for you? You can disable it by setting the automatic_nested_ad_switching Preference to false.
    Remember that if you are using ForwardDiff inside a Zygote call, it will drop gradients (with a warning message), so it is not recommended to use this combination.
    Let's explore this using some questions that were posted on the Julia Discourse forum.
    julia
    using ADTypes, Lux, LinearAlgebra, Zygote, ForwardDiff, Random, StableRNGs
+import{_ as e,c as n,a2 as t,j as s,a,o as l}from"./chunks/framework.DjZDIZsN.js";const j=JSON.parse('{"title":"Nested Automatic Differentiation","description":"","frontmatter":{},"headers":[],"relativePath":"manual/nested_autodiff.md","filePath":"manual/nested_autodiff.md","lastUpdated":null}'),h={name:"manual/nested_autodiff.md"},p={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},k={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.178ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4498.7 886","aria-hidden":"true"},d={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"7.009ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 3098 886","aria-hidden":"true"},o={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.791ex"},xmlns:"http://www.w3.org/2000/svg",width:"11.439ex",height:"2.713ex",role:"img",focusable:"false",viewBox:"0 -849.5 5056 1199","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"33.692ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 14891.7 2978.9","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.09ex"},xmlns:"http://www.w3.org/2000/svg",width:"9.913ex",height:"2.004ex",role:"img",focusable:"false",viewBox:"0 -846 4381.7 886","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},m={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-2.819ex"},xmlns:"http://www.w3.org/2000/svg",width:"21.167ex",height:"6.74ex",role:"img",focusable:"false",viewBox:"0 -1733 9355.6 2978.9","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.661ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.843ex",height:"2.565ex",role:"img",focusable:"false",viewBox:"0 -841.7 1698.8 1133.9","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},f={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"3.269ex",height:"1.902ex",role:"img",focusable:"false",viewBox:"0 -683 1445 840.8","aria-hidden":"true"},b={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},x={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.357ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.837ex",height:"1.359ex",role:"img",focusable:"false",viewBox:"0 -443 812 600.8","aria-hidden":"true"};function v(w,i,H,D,B,L){return l(),n("div",null,[i[32]||(i[32]=t(`
    Nested Automatic Differentiation ​
    Note
    This is a relatively new feature in Lux, so there might be some rough edges. If you encounter any issues, please let us know by opening an issue on the GitHub repository.
    In this manual, we will explore how to use automatic differentiation (AD) inside your layers or loss functions and have Lux automatically switch the AD backend with a faster one when needed.
    Tip
    Don't wan't Lux to do this switching for you? You can disable it by setting the automatic_nested_ad_switching Preference to false.
    Remember that if you are using ForwardDiff inside a Zygote call, it will drop gradients (with a warning message), so it is not recommended to use this combination.
    Let's explore this using some questions that were posted on the Julia Discourse forum.
    julia
    using ADTypes, Lux, LinearAlgebra, Zygote, ForwardDiff, Random, StableRNGs
 using ComponentArrays, FiniteDiff
    First let's set the stage using some minor changes that need to be made for this feature to work:
    • Switching only works if a StatefulLuxLayer is being used, with the following function calls:
      • For operations on the inputs:
        • (<some-function> ∘ <StatefulLuxLayer>)(x::AbstractArray)
        • (<StatefulLuxLayer> ∘ <some-function>)(x::AbstractArray)
        • (<StatefulLuxLayer>)(x::AbstractArray)
      • For operations on the parameters:
        • (<some-function> ∘ Base.Fix1(<StatefulLuxLayer>, x))(ps)
        • (Base.Fix1(<StatefulLuxLayer>, x) ∘ <some-function>)(ps)
        • (Base.Fix1(<StatefulLuxLayer>, x))(ps)
    • Currently we have custom routines implemented for:
    • Switching only happens for ChainRules compatible AD libraries.
    We plan to capture DifferentiationInterface, and Enzyme.autodiff calls in the future (PRs are welcome).
    Tip
    @compact uses StatefulLuxLayers internally, so you can directly use these features inside a layer generated by @compact.
    Loss Function containing Jacobian Computation ​
    This problem comes from @facusapienza on Discourse. In this case, we want to add a regularization term to the neural DE based on first-order derivatives. The neural DE part is not important here and we can demonstrate this easily with a standard neural network.
    julia
    function loss_function1(model, x, ps, st, y)
     # Make it a stateful layer
     smodel = StatefulLuxLayer{true}(model, ps, st)
@@ -22,9 +22,9 @@ import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js
 
 println("∞-norm(∂x - ∂x_fd): ", norm(∂x .- ∂x_fd, Inf))
 println("∞-norm(∂ps - ∂ps_fd): ", norm(ComponentArray(∂ps) .- ∂ps_fd, Inf))
    ┌ Warning: \`training\` is set to \`Val{true}()\` but is not being used within an autodiff call (gradient, jacobian, etc...). This will be slow. If you are using a \`Lux.jl\` model, set it to inference (test) mode using \`LuxCore.testmode\`. Reliance on this behavior is discouraged, and is not guaranteed by Semantic Versioning, and might be removed without a deprecation cycle. It is recommended to fix this issue in your code.
-└ @ LuxLib.Utils ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxLib/I9RHW/src/utils.jl:309
+└ @ LuxLib.Utils /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxLib/src/utils.jl:309
 ┌ Warning: \`training\` is set to \`Val{true}()\` but is not being used within an autodiff call (gradient, jacobian, etc...). This will be slow. If you are using a \`Lux.jl\` model, set it to inference (test) mode using \`LuxCore.testmode\`. Reliance on this behavior is discouraged, and is not guaranteed by Semantic Versioning, and might be removed without a deprecation cycle. It is recommended to fix this issue in your code.
-└ @ LuxLib.Utils ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxLib/I9RHW/src/utils.jl:309
+└ @ LuxLib.Utils /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxLib/src/utils.jl:309
 ∞-norm(∂x - ∂x_fd): 0.00046014786
 ∞-norm(∂ps - ∂ps_fd): 0.00068473816
    That's pretty good, of course you will have some error from the finite differences calculation.
    Using Batched Jacobian for Multiple Inputs ​
    Notice that in this example the Jacobian J consists on the full matrix of derivatives of smodel with respect the different inputs in x. In many cases, we are interested in computing the Jacobian with respect to each input individually, avoiding the unnecessary calculation of zero entries of the Jacobian. This can be achieved with batched_jacobian to parse the calculation of the Jacobian per each single input. Using the same example from the previous section:
    julia
    model = Chain(Dense(2 => 4, tanh), Dense(4 => 2))
 ps, st = Lux.setup(StableRNG(0), model)
@@ -80,7 +80,7 @@ import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js
 
 println("∞-norm(∂x - ∂x_fd): ", norm(∂x .- ∂x_fd, Inf))
 println("∞-norm(∂ps - ∂ps_fd): ", norm(ComponentArray(∂ps) .- ∂ps_fd, Inf))
    ∞-norm(∂x - ∂x_fd): 4.172325e-6
-∞-norm(∂ps - ∂ps_fd): 4.5776367e-5
    Hutchinson Trace Estimation ​
    `,51)),s("p",null,[i[6]||(i[6]=a("Hutchinson Trace Estimation often shows up in machine learning literature to provide a fast estimate of the trace of a Jacobian Matrix. This is based off of ")),i[7]||(i[7]=s("a",{href:"https://www.nowozin.net/sebastian/blog/thoughts-on-trace-estimation-in-deep-learning.html",target:"_blank",rel:"noreferrer"},"Hutchinson 1990",-1)),i[8]||(i[8]=a(" which computes the estimated trace of a matrix ")),s("mjx-container",p,[(e(),n("svg",k,i[0]||(i[0]=[t('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[9]||(i[9]=a(" using random vectors ")),s("mjx-container",d,[(e(),n("svg",r,i[2]||(i[2]=[t('',1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"v"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D")])])])],-1))]),i[10]||(i[10]=a(" s.t. ")),s("mjx-container",o,[(e(),n("svg",g,i[4]||(i[4]=[t('',1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("mi",null,"v"),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mi",null,"I")])],-1))]),i[11]||(i[11]=a("."))]),s("mjx-container",Q,[(e(),n("svg",E,i[12]||(i[12]=[t('',1)]))),i[13]||(i[13]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"A"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mi",null,"A"),s("mi",null,"v"),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"A"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),s("p",null,[i[16]||(i[16]=a("We can use this to compute the trace of a Jacobian Matrix ")),s("mjx-container",T,[(e(),n("svg",c,i[14]||(i[14]=[t('',1)]))),i[15]||(i[15]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[17]||(i[17]=a(" using the following algorithm:"))]),s("mjx-container",y,[(e(),n("svg",m,i[18]||(i[18]=[t('',1)]))),i[19]||(i[19]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"J"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[33]||(i[33]=s("p",null,"Note that we can compute this using two methods:",-1)),s("ol",null,[s("li",null,[s("p",null,[i[22]||(i[22]=a("Compute ")),s("mjx-container",u,[(e(),n("svg",F,i[20]||(i[20]=[t('',1)]))),i[21]||(i[21]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J")])],-1))]),i[23]||(i[23]=a(" using a Vector-Jacobian product and then do a matrix-vector product to get the trace."))])]),s("li",null,[s("p",null,[i[26]||(i[26]=a("Compute ")),s("mjx-container",C,[(e(),n("svg",f,i[24]||(i[24]=[t('',1)]))),i[25]||(i[25]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[27]||(i[27]=a(" using a Jacobian-Vector product and then do a matrix-vector product to get the trace."))])])]),s("p",null,[i[30]||(i[30]=a("For simplicity, we will use a single sample of ")),s("mjx-container",b,[(e(),n("svg",x,i[28]||(i[28]=[t('',1)]))),i[29]||(i[29]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[31]||(i[31]=a(" to compute the trace. Additionally, we will fix the sample to ensure that our tests against the finite difference implementation are not affected by the randomness in the sample."))]),i[34]||(i[34]=t(`
    Computing using the Vector-Jacobian Product ​
    julia
    function hutchinson_trace_vjp(model, x, ps, st, v)
+∞-norm(∂ps - ∂ps_fd): 4.5776367e-5
    Hutchinson Trace Estimation ​
    `,51)),s("p",null,[i[6]||(i[6]=a("Hutchinson Trace Estimation often shows up in machine learning literature to provide a fast estimate of the trace of a Jacobian Matrix. This is based off of ")),i[7]||(i[7]=s("a",{href:"https://www.nowozin.net/sebastian/blog/thoughts-on-trace-estimation-in-deep-learning.html",target:"_blank",rel:"noreferrer"},"Hutchinson 1990",-1)),i[8]||(i[8]=a(" which computes the estimated trace of a matrix ")),s("mjx-container",p,[(l(),n("svg",k,i[0]||(i[0]=[t('',1)]))),i[1]||(i[1]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"A"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[9]||(i[9]=a(" using random vectors ")),s("mjx-container",d,[(l(),n("svg",r,i[2]||(i[2]=[t('',1)]))),i[3]||(i[3]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"v"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D")])])])],-1))]),i[10]||(i[10]=a(" s.t. ")),s("mjx-container",o,[(l(),n("svg",g,i[4]||(i[4]=[t('',1)]))),i[5]||(i[5]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("mi",null,"v"),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mi",null,"I")])],-1))]),i[11]||(i[11]=a("."))]),s("mjx-container",Q,[(l(),n("svg",E,i[12]||(i[12]=[t('',1)]))),i[13]||(i[13]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"A"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"E")]),s("mrow",{"data-mjx-texclass":"INNER"},[s("mo",{"data-mjx-texclass":"OPEN"},"["),s("msup",null,[s("mi",null,"v"),s("mi",null,"T")]),s("mi",null,"A"),s("mi",null,"v"),s("mo",{"data-mjx-texclass":"CLOSE"},"]")]),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"A"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),s("p",null,[i[16]||(i[16]=a("We can use this to compute the trace of a Jacobian Matrix ")),s("mjx-container",T,[(l(),n("svg",c,i[14]||(i[14]=[t('',1)]))),i[15]||(i[15]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("mo",null,"∈"),s("msup",null,[s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",{mathvariant:"double-struck"},"R")]),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"D"),s("mo",null,"×"),s("mi",null,"D")])])])],-1))]),i[17]||(i[17]=a(" using the following algorithm:"))]),s("mjx-container",y,[(l(),n("svg",m,i[18]||(i[18]=[t('',1)]))),i[19]||(i[19]=s("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[s("mtext",null,"Tr"),s("mo",{stretchy:"false"},"("),s("mi",null,"J"),s("mo",{stretchy:"false"},")"),s("mo",null,"="),s("mfrac",null,[s("mn",null,"1"),s("mi",null,"V")]),s("munderover",null,[s("mo",{"data-mjx-texclass":"OP"},"∑"),s("mrow",{"data-mjx-texclass":"ORD"},[s("mi",null,"i"),s("mo",null,"="),s("mn",null,"1")]),s("mi",null,"V")]),s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[33]||(i[33]=s("p",null,"Note that we can compute this using two methods:",-1)),s("ol",null,[s("li",null,[s("p",null,[i[22]||(i[22]=a("Compute ")),s("mjx-container",u,[(l(),n("svg",F,i[20]||(i[20]=[t('',1)]))),i[21]||(i[21]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msubsup",null,[s("mi",null,"v"),s("mi",null,"i"),s("mi",null,"T")]),s("mi",null,"J")])],-1))]),i[23]||(i[23]=a(" using a Vector-Jacobian product and then do a matrix-vector product to get the trace."))])]),s("li",null,[s("p",null,[i[26]||(i[26]=a("Compute ")),s("mjx-container",C,[(l(),n("svg",f,i[24]||(i[24]=[t('',1)]))),i[25]||(i[25]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("mi",null,"J"),s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[27]||(i[27]=a(" using a Jacobian-Vector product and then do a matrix-vector product to get the trace."))])])]),s("p",null,[i[30]||(i[30]=a("For simplicity, we will use a single sample of ")),s("mjx-container",b,[(l(),n("svg",x,i[28]||(i[28]=[t('',1)]))),i[29]||(i[29]=s("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[s("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[s("msub",null,[s("mi",null,"v"),s("mi",null,"i")])])],-1))]),i[31]||(i[31]=a(" to compute the trace. Additionally, we will fix the sample to ensure that our tests against the finite difference implementation are not affected by the randomness in the sample."))]),i[34]||(i[34]=t(`
    Computing using the Vector-Jacobian Product ​
    julia
    function hutchinson_trace_vjp(model, x, ps, st, v)
     smodel = StatefulLuxLayer{true}(model, ps, st)
     vjp = vector_jacobian_product(smodel, AutoZygote(), x, v)
     return sum(batched_matmul(reshape(vjp, 1, :, size(vjp, ndims(vjp))),
@@ -116,4 +116,4 @@ import{_ as l,c as n,a2 as t,j as s,a,o as e}from"./chunks/framework.DjZDIZsN.js
     norm(ComponentArray(∂ps_full_jacobian) .- ComponentArray(∂ps_vjp), Inf))
    ∞-norm(∂x using vjp): 0.0
 ∞-norm(∂ps using vjp): 0.0
 ∞-norm(∂x using full jacobian): 9.536743e-7
-∞-norm(∂ps using full jacobian): 1.4305115e-6
    `,20))])}const _=l(h,[["render",v]]);export{j as __pageData,_ as default};
+∞-norm(∂ps using full jacobian): 1.4305115e-6
    `,20))])}const _=e(h,[["render",v]]);export{j as __pageData,_ as default};
diff --git a/previews/PR1000/assets/manual_weight_initializers.md.DnHa7Oz9.js b/previews/PR1000/assets/manual_weight_initializers.md.pgcyJMH_.js
similarity index 89%
rename from previews/PR1000/assets/manual_weight_initializers.md.DnHa7Oz9.js
rename to previews/PR1000/assets/manual_weight_initializers.md.pgcyJMH_.js
index 02fef41eca..3fa492b605 100644
--- a/previews/PR1000/assets/manual_weight_initializers.md.DnHa7Oz9.js
+++ b/previews/PR1000/assets/manual_weight_initializers.md.pgcyJMH_.js
@@ -18,10 +18,10 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const c
   0.486214   0.321506  -0.306641  0.145296   0.206476
    To generate weights directly on GPU, pass in a CUDA.RNG. For a complete list of supported RNG types, see Supported RNG Types.
    julia
    using LuxCUDA
 
 weights = kaiming_normal(CUDA.default_rng(), 2, 5)
    2×5 CuArray{Float32, 2, CUDA.DeviceMemory}:
-  0.108558   0.944269  -0.0671347   0.509523   0.00845341
- -0.279856  -0.507971  -0.178438   -0.112973  -1.44735
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
- 0.542428-1.17063im   -0.586636+0.809742im  …  -0.599194-0.446424im
- 0.407158+0.421916im   -1.24735+1.2547im        -0.71315+0.916298im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
+ -1.24304    1.00968   -0.756944   -0.509345  -0.343497
+ -0.335454  -0.655999  -0.0236516   0.697709   0.9543
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
+  1.51066-0.410399im    0.148313+0.795943im  …  -0.266121+0.532213im
+ 0.548101+0.283859im  -0.0162294-0.608428im      0.964663+0.16569im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
 weights = weights_init(rng, dims...)
 
 # Or
diff --git a/previews/PR1000/assets/manual_weight_initializers.md.DnHa7Oz9.lean.js b/previews/PR1000/assets/manual_weight_initializers.md.pgcyJMH_.lean.js
similarity index 89%
rename from previews/PR1000/assets/manual_weight_initializers.md.DnHa7Oz9.lean.js
rename to previews/PR1000/assets/manual_weight_initializers.md.pgcyJMH_.lean.js
index 02fef41eca..3fa492b605 100644
--- a/previews/PR1000/assets/manual_weight_initializers.md.DnHa7Oz9.lean.js
+++ b/previews/PR1000/assets/manual_weight_initializers.md.pgcyJMH_.lean.js
@@ -18,10 +18,10 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const c
   0.486214   0.321506  -0.306641  0.145296   0.206476
    To generate weights directly on GPU, pass in a CUDA.RNG. For a complete list of supported RNG types, see Supported RNG Types.
    julia
    using LuxCUDA
 
 weights = kaiming_normal(CUDA.default_rng(), 2, 5)
    2×5 CuArray{Float32, 2, CUDA.DeviceMemory}:
-  0.108558   0.944269  -0.0671347   0.509523   0.00845341
- -0.279856  -0.507971  -0.178438   -0.112973  -1.44735
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
- 0.542428-1.17063im   -0.586636+0.809742im  …  -0.599194-0.446424im
- 0.407158+0.421916im   -1.24735+1.2547im        -0.71315+0.916298im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
+ -1.24304    1.00968   -0.756944   -0.509345  -0.343497
+ -0.335454  -0.655999  -0.0236516   0.697709   0.9543
    You can also generate Complex Numbers:
    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
+  1.51066-0.410399im    0.148313+0.795943im  …  -0.266121+0.532213im
+ 0.548101+0.283859im  -0.0162294-0.608428im      0.964663+0.16569im
    Quick examples ​
    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.
    julia
    weights = init(rng, dims...)
    The rng is optional, if not specified a default one will be used.
    julia
    weights = init(dims...)
    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.
    julia
    weights_init = init(rng; kwargs...)
 weights = weights_init(rng, dims...)
 
 # Or
diff --git a/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.Bfi8AeMz.js b/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.Bfi8AeMz.js
new file mode 100644
index 0000000000..b398d472eb
--- /dev/null
+++ b/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.Bfi8AeMz.js
@@ -0,0 +1,286 @@
+import{_ as e,c as i,a2 as n,j as A,a as t,o as a}from"./chunks/framework.DjZDIZsN.js";const G=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),h={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},d={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},m={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},R={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function X(O,s,P,Z,N,w){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
+      OptimizationOptimJL, Printf, Random, SciMLSensitivity
+using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=t("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=t(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=t(", ")),A("mjx-container",d,[(a(),i("svg",r,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=t(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
+    M = m₁ + m₂
+    r₁ = m₂ / M .* path
+    r₂ = -m₁ / M .* path
+    return r₁, r₂
+end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=t("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",g,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
+    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
+
+    if size(soln, 1) == 2
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+
+        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
+        p, M, e = model_params
+    else
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+        p = soln[3, :]
+        e = soln[4, :]
+    end
+
+    r = p ./ (1 .+ e .* cos.(χ))
+    x = r .* cos.(ϕ)
+    y = r .* sin.(ϕ)
+
+    orbit = vcat(x', y')
+    return orbit
+end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
+    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
+    b = (v[3:end] .- v[1:(end - 2)]) / 2
+    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
+    return [a; b; c] / dt
+end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
+    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
+    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
+    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
+    return [a; b; c] / (dt^2)
+end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
+    x = orbit[1, :]
+    y = orbit[2, :]
+
+    Ixx = x .^ 2
+    Iyy = y .^ 2
+    Ixy = x .* y
+    trace = Ixx .+ Iyy
+
+    if component[1] == 1 && component[2] == 1
+        tmp = Ixx .- trace ./ 3
+    elseif component[1] == 2 && component[2] == 2
+        tmp = Iyy .- trace ./ 3
+    else
+        tmp = Ixy
+    end
+
+    return mass .* tmp
+end
+
+function h_22_quadrupole_components(dt, orbit, component, mass=1)
+    mtensor = orbit2tensor(orbit, component, mass)
+    mtensor_ddot = d2_dt2(mtensor, dt)
+    return 2 * mtensor_ddot
+end
+
+function h_22_quadrupole(dt, orbit, mass=1)
+    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
+    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
+    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
+    return h11, h12, h22
+end
+
+function h_22_strain_one_body(dt::T, orbit) where {T}
+    h11, h12, h22 = h_22_quadrupole(dt, orbit)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
+    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
+    h11 = h11_1 + h11_2
+    h12 = h12_1 + h12_2
+    h22 = h22_1 + h22_2
+    return h11, h12, h22
+end
+
+function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
+    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
+
+    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
+
+    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
+    @assert mass_ratio≤1 "mass_ratio must be <= 1"
+    @assert mass_ratio≥0 "mass_ratio must be non-negative"
+
+    orbit = soln2orbit(soln, model_params)
+    if mass_ratio > 0
+        m₂ = inv(T(1) + mass_ratio)
+        m₁ = mass_ratio * m₂
+
+        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
+        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
+    else
+        waveform = h_22_strain_one_body(dt, orbit)
+    end
+    return waveform
+end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",C,[(a(),i("svg",o,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=t("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=t(", ")),A("mjx-container",F,[(a(),i("svg",u,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=t(", and ")),A("mjx-container",c,[(a(),i("svg",B,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=t(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
+    χ, ϕ = u
+
+    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
+    denom = sqrt((p - 2)^2 - 4 * e^2)
+
+    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
+    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
+
+    return [χ̇, ϕ̇]
+end
+
+mass_ratio = 0.0         # test particle
+u0 = Float64[π, 0.0]     # initial conditions
+datasize = 250
+tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
+tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
+dt_data = tsteps[2] - tsteps[1]
+dt = 100.0
+const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
+soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
+waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
+
+    axislegend(ax, [[l, s]], ["Waveform Data"])
+
+    fig
+end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
+    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
+ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[-5.067828f-5; 0.00020338836; 1.4273281f-5; -4.4280547f-5; -4.2506847f-5; 7.748175f-5; 6.288575f-6; 3.7066755f-5; 0.00014335215; -0.00011426929; 0.0001108622; 8.886729f-5; -0.00023507916; -0.00026994065; -2.1895053f-6; -4.1372066f-5; 0.00014377595; -1.2547869f-5; 0.00015402514; -6.948349f-5; 9.75148f-7; -2.2759441f-5; 5.7592948f-5; -9.247271f-6; -7.653275f-5; -5.4306223f-5; -5.6007382f-5; 4.515377f-5; -9.931654f-5; 0.00011081037; -2.5950818f-5; -2.3206787f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-6.4881f-5 5.1595423f-5 -0.00010973665 0.00011888758 -7.869763f-5 2.6731343f-5 -4.3378386f-5 -0.0001431499 2.6663824f-5 -0.00017339565 -2.6334403f-5 9.1598034f-5 -0.00015508423 -4.310734f-5 1.6837905f-5 1.7922774f-5 -7.000959f-5 -4.674141f-5 3.0365556f-5 -2.1681205f-6 2.4398596f-5 1.1775482f-5 9.9300596f-5 0.0001105504 9.565299f-6 -4.7076144f-5 9.590853f-5 -4.885141f-5 0.000120821234 -0.00022250146 -9.05917f-5 -3.2679833f-5; 0.00012063581 -6.609236f-5 -8.302315f-5 -1.954174f-5 -6.6735214f-5 -1.8397034f-6 -2.99155f-5 0.00013611194 -0.00010268538 7.048608f-5 -1.6720745f-5 7.6480974f-5 8.878988f-5 3.9338465f-5 -2.3043342f-5 1.4160479f-5 -3.0309327f-5 -4.2313906f-5 -5.78722f-5 -1.2361765f-5 0.00011966063 -8.0163416f-5 3.7680657f-5 -3.4606725f-5 -9.543387f-5 6.8428395f-5 1.155613f-5 -0.000115886956 -6.523307f-5 5.2380037f-5 0.00024232482 7.973879f-6; 8.224085f-5 -1.21905605f-5 2.7365168f-5 4.015555f-5 -4.8491387f-5 5.793325f-5 7.1500195f-5 7.0172675f-5 -8.642905f-5 1.9702306f-5 4.9804526f-5 9.729075f-5 9.6269556f-5 -9.517981f-5 -0.000108945256 -6.5701315f-5 -0.00014163546 0.00020532864 6.4517335f-5 -0.00010703346 2.3381364f-5 4.1316358f-5 -9.1240385f-5 2.8189028f-5 3.4416404f-5 8.883044f-5 -9.182712f-5 0.00014107842 5.8567588f-5 8.330891f-5 8.732947f-5 2.7254235f-5; 2.0640122f-5 -6.5564316f-5 -9.267124f-5 1.2782187f-5 -2.1939577f-5 4.8729362f-5 -0.0002059266 0.00019216529 -0.00010824198 -0.000106641404 -0.00011950945 4.0736777f-5 3.8151837f-5 1.5811183f-5 -5.0110833f-5 -1.12530515f-5 -9.231862f-5 0.00014269506 -9.049237f-5 0.0001928238 -8.382982f-5 -3.1066495f-6 1.396399f-5 3.5595588f-6 -0.0001678254 -5.0562965f-5 5.644873f-5 0.0001424172 0.00012270945 1.297231f-5 6.82423f-5 1.0279729f-5; -1.3927525f-5 -2.5807853f-5 -3.353609f-5 -0.0001509003 -5.633148f-5 2.3138953f-5 -9.295884f-5 -1.6964543f-5 8.622773f-6 -2.287433f-5 -9.8104494f-5 0.00019548782 -1.4208715f-5 4.2834552f-5 -0.00014059727 0.00012722421 7.491649f-5 -0.00013340326 -7.4468226f-5 6.701388f-7 7.389381f-6 7.241207f-6 0.0001258489 2.1600781f-5 -6.4029395f-5 9.2974806f-5 -0.00013893936 -2.0481652f-6 9.337237f-5 1.4617574f-6 -4.278323f-5 -4.6443496f-5; -9.862447f-6 -8.921982f-5 9.193483f-5 4.696958f-5 -7.964992f-6 -0.00015717202 1.1457437f-5 4.2228417f-5 1.0273722f-5 -0.0001314355 -6.983743f-5 3.2823362f-5 -2.2050655f-5 4.167267f-5 8.6878565f-5 0.00012557801 9.7389f-5 5.7838948f-5 0.00013859413 -5.4861524f-5 9.106602f-5 0.00010086939 0.00014309268 -1.2072048f-5 -5.2197997f-6 4.112339f-5 0.00015406324 -3.9871407f-5 5.1895193f-5 8.516548f-5 6.1933075f-5 -9.757812f-5; 0.000100224905 3.8019178f-5 -0.00011406402 0.00020252966 0.00012403002 -7.962413f-6 -7.4583564f-5 6.3843974f-5 0.00011404545 0.000169954 8.429819f-5 4.996831f-5 0.00011434548 0.00016386562 -6.057491f-5 -9.412545f-5 -0.00015873568 -6.482431f-5 3.091892f-5 -2.159468f-5 0.00012109969 -7.771308f-5 -1.8478474f-6 -9.604292f-6 -0.00010614015 -0.00013608659 -3.699296f-5 -9.4771065f-5 -0.000106533116 -1.4093011f-5 -0.00010621838 -1.599139f-5; -6.859555f-5 5.9557868f-5 0.00013538415 -5.856615f-5 -1.4068297f-6 -9.1875416f-5 4.9636124f-5 -2.6894352f-6 0.00010794412 3.837797f-5 0.00023890898 -0.00011218314 3.951066f-5 7.6334916f-5 -6.472669f-5 2.6779146f-5 1.4812273f-5 -8.1917766f-5 1.3783435f-5 -0.00013009476 0.00010798479 9.0076355f-5 -5.1770206f-5 -3.248121f-5 1.4516136f-5 7.808087f-5 -4.9713708f-5 -4.637024f-5 0.00012679571 7.44449f-5 -0.00014616112 -9.48301f-5; -5.843995f-5 -7.837018f-5 -0.00015722003 0.00013298752 0.0002058001 0.0001131393 -6.0574115f-5 6.8372574f-6 8.129709f-5 5.0710118f-5 0.00025877086 -0.00015811148 -2.5584899f-5 -0.00010511612 -0.00012036738 9.864298f-5 -4.5218094f-6 -0.00011798466 2.9166674f-6 0.00014341663 -5.8281497f-5 4.9651182f-5 -6.645968f-5 -0.0001173513 2.0992076f-5 -1.2117198f-5 -0.00013775482 -5.2643958f-5 -8.244777f-6 -6.679681f-5 -0.000112732036 2.2170127f-5; 1.8069002f-5 -8.4170046f-5 4.027615f-5 -9.122345f-5 1.6444257f-5 8.069179f-6 -4.988504f-5 0.00018525976 7.757772f-5 -0.000103358674 -5.7851914f-5 3.672725f-5 0.00014628147 0.00011684927 0.00014759402 0.00010566242 0.0001684325 7.3146985f-5 -0.00029925458 -9.248588f-5 -0.00010037117 2.386982f-5 -8.735626f-5 -0.00014760795 8.600574f-5 -0.00010550478 -1.5243796f-5 -2.6065078f-5 0.00014419388 9.088046f-5 4.867762f-5 4.917452f-5; -6.598519f-5 -6.0307575f-5 -7.217373f-5 -7.9992154f-5 -0.00020351214 8.730133f-5 3.311596f-5 -7.431629f-5 -6.286525f-6 -4.801173f-5 0.0001289165 4.1723186f-5 -8.892634f-5 0.00024919503 4.605369f-5 8.194185f-5 -4.1749663f-5 -4.7821406f-5 -7.79361f-5 -4.722093f-5 -5.6371475f-5 -2.9041228f-5 -4.2099673f-5 -4.402712f-5 -2.151333f-5 0.00010635854 -0.00018420393 7.879363f-5 6.56917f-5 -0.00014872105 -0.00025311403 0.00016434335; -3.2588643f-5 -6.82938f-5 -8.62755f-5 -2.488788f-5 -0.00010869599 -1.875883f-5 -5.000146f-5 0.00010093272 -0.00017555879 8.3838204f-5 -0.00019008624 -2.0509233f-5 9.822452f-5 -0.00014449273 -4.487349f-5 -0.0001129706 5.2817784f-5 0.00010259473 1.2052635f-5 0.0001414271 1.6518059f-5 0.000114337774 -9.06657f-5 3.4899254f-5 2.4568328f-5 3.0560605f-5 0.000109304965 -4.3656746f-5 5.3483916f-5 5.271978f-5 -6.0059614f-5 6.0993232f-5; 4.6849014f-5 0.00015922435 3.7304446f-5 0.00010522433 -1.7149052f-5 9.699794f-5 8.679175f-5 -5.980967f-5 0.000102687576 -0.00011774394 -9.3231676f-5 -7.073589f-5 9.4761046f-5 -2.0640855f-5 -0.00023659732 -0.00015139587 -7.254155f-5 0.00018710685 -7.661277f-5 -6.958926f-5 -0.00023375152 0.000113982176 6.118777f-5 -0.00014834842 -2.048975f-5 2.8104418f-5 2.5483473f-6 6.996796f-5 -9.3646326f-5 -0.00014223685 -0.00017213685 -0.00018663933; -0.00011808671 4.8037993f-5 1.6431522f-5 -0.00015378026 7.1910195f-5 -0.0001872632 -5.120849f-5 -0.00010370612 0.00029613348 3.4528523f-5 -7.646572f-5 5.0855837f-5 9.754051f-5 6.584373f-5 -4.716242f-5 -2.1120639f-5 -0.00011250277 8.575611f-5 -1.6247295f-5 -0.00027942262 1.3625263f-6 8.212421f-5 6.171591f-5 6.960599f-5 -1.3251189f-5 0.00010215443 0.0001165171 -2.5964511f-5 4.8122438f-5 4.640835f-5 0.00014826102 -7.550899f-5; -6.171692f-5 -2.2494147f-5 -0.00011052692 0.000108468135 -6.2695945f-5 -9.421945f-5 0.00014297807 3.1365198f-5 8.234304f-5 7.663826f-6 -0.000109010085 -8.211707f-6 -4.7824436f-5 -0.0001506865 -7.138279f-5 0.00012205401 0.00010132722 -2.9440233f-5 8.614374f-5 -3.23056f-5 5.3609398f-5 9.664525f-5 -1.596922f-5 -8.9840825f-5 5.949551f-5 1.0887881f-5 4.2916592f-5 -0.00010351466 0.00012603638 5.6517703f-5 6.545845f-5 0.00019466775; -6.0357146f-5 -3.048198f-5 1.9146099f-5 4.148812f-5 -2.4972449f-5 -9.497651f-5 -5.7457888f-5 0.00010517364 6.8643836f-5 -5.117873f-6 9.27461f-5 1.2731097f-5 2.0752132f-5 -1.23914115f-5 3.9276125f-7 1.6265903f-5 8.1468446f-5 9.052403f-5 0.00018212575 7.8478355f-5 1.3240985f-5 -6.920781f-5 1.9615041f-5 -0.00020374315 3.897066f-5 -0.0001280138 -6.9336143f-6 0.000103044265 0.000104789826 4.8267525f-6 6.81914f-5 -0.00012826166; -2.809343f-5 -3.0831015f-5 2.2828628f-5 8.898851f-5 1.2071551f-5 -3.803355f-5 -1.0437871f-5 5.422938f-5 4.867486f-5 -4.518063f-5 7.922069f-5 -0.00010169225 0.0001434924 -7.677246f-5 0.000111034875 -5.2173058f-5 1.2276159f-6 -2.8245557f-5 9.942706f-5 1.1442849f-5 1.463341f-6 -5.2849533f-5 -4.7457823f-5 -5.7791447f-5 -2.5199992f-5 2.5344176f-5 9.2255f-5 5.7867932f-5 -4.42645f-5 -0.00022660938 -0.00029660418 -9.220186f-5; 6.735818f-5 -4.176811f-5 -1.2232209f-5 -0.0001059559 5.754733f-5 -6.4020445f-5 -0.00014015102 0.00011234368 3.3669865f-5 -0.00012898138 -0.0001243019 7.9176105f-5 0.00021939976 -1.7735352f-6 1.3753903f-5 3.0978037f-5 1.261377f-5 3.2221094f-5 1.1791008f-5 0.00012344854 2.0633179f-5 -0.00019241248 3.0955096f-5 -4.4071912f-5 0.00013447642 0.00015082232 2.4748131f-5 7.190226f-5 5.500368f-5 -2.3531034f-6 0.000106698426 -0.00015557921; 0.00015004253 7.1710674f-5 7.18664f-5 -4.449272f-5 0.00010724826 -3.0034582f-5 -6.1318984f-5 -7.845766f-6 7.125385f-5 -1.1303766f-5 4.3301923f-5 1.4665254f-5 7.303844f-5 0.00012035262 3.3127606f-5 -1.6960375f-5 9.650369f-5 -0.000110265166 1.5607555f-5 -7.302818f-5 2.9097244f-5 -0.00014719421 1.1049523f-5 -2.7170749f-5 6.9879985f-5 -0.00017216924 -5.5424134f-5 0.00015734922 -2.3573297f-5 -9.351214f-6 -4.482439f-5 0.00015532046; 9.200195f-5 -7.894702f-5 6.909601f-5 6.397365f-5 -7.337188f-5 1.810738f-5 -1.3611387f-5 -0.00011573212 6.84041f-5 0.00016948365 -3.591823f-5 -0.00010666039 5.9029382f-5 -1.1825928f-5 5.6072877f-5 4.840527f-5 0.00019966012 -7.507182f-5 3.281313f-5 -0.00026297724 0.00013116715 -6.360724f-5 1.34186175f-5 -5.0054907f-5 -6.6365785f-5 -5.7354497f-5 -0.00027880043 5.8056135f-5 4.771379f-6 0.00023913936 -0.00027277894 0.00013627937; -3.8475675f-5 0.00013499877 -0.00011172844 -1.5792652f-5 1.7460216f-5 8.990932f-5 -0.00013876938 0.00010837003 2.5286476f-5 4.0916253f-5 -8.68343f-5 2.7274678f-5 -0.0002117695 -4.198623f-5 2.5651116f-5 -4.3864326f-5 0.00010112666 8.5957945f-5 -0.00011121263 4.618323f-5 -2.0187681f-5 0.00010101195 -6.718644f-5 0.0001685784 -6.6461784f-5 -8.18679f-5 5.8931157f-5 6.0350863f-5 0.000110743706 2.1969163f-5 8.4774925f-5 2.8408233f-5; 0.00015168458 -0.00015583177 9.526762f-7 3.8325594f-5 3.7966631f-6 -2.4170578f-5 -5.1147792f-5 -1.8475313f-5 2.11474f-6 2.0240308f-5 -0.00019447393 7.808402f-5 -1.9867588f-5 2.6745403f-5 2.3344088f-5 -7.562572f-5 3.0028186f-5 0.00020181146 0.0001235826 -4.559302f-5 0.00010591059 0.00026848647 -3.44617f-5 2.3216366f-5 8.163823f-5 -1.3018025f-5 0.00011558951 -6.941131f-5 -0.00019635934 -8.809247f-5 0.00015325508 -0.00014338388; 1.7577355f-5 -0.00011430754 -2.3156954f-5 -1.96567f-5 -0.000108622946 8.5745494f-5 -6.518311f-6 -4.53642f-5 -5.5961747f-5 0.00012402056 5.957273f-5 0.0001436606 -2.3027369f-5 -2.1112419f-5 1.8754206f-5 6.52711f-5 -6.917144f-5 0.00015894721 8.440349f-5 0.00015213025 4.88502f-7 -7.422043f-5 4.4104203f-5 -5.2935447f-5 3.839207f-5 -0.00026345407 -0.00027999925 -0.00010818485 0.000100323494 0.00013953423 -0.00015231865 9.962925f-5; 3.5571054f-5 -7.7061304f-5 -0.00021089091 0.000117353586 -8.080046f-6 0.00020298584 -7.7734745f-5 0.000111111476 0.00021550554 -0.00011431153 -5.765442f-5 -9.9738056f-5 -0.00012480488 0.00012828007 -7.215347f-5 -9.8592405f-5 -5.0616338f-5 6.370253f-5 -1.0569401f-5 -8.1941704f-5 -9.611313f-5 4.9776292f-5 7.01193f-5 -3.1117284f-5 -0.00011037468 -0.00011528408 -7.007478f-5 -0.0001132747 2.4702704f-5 -0.00013523488 0.000120989585 -0.0002573811; 0.00026173593 1.283452f-5 -1.7065893f-5 -0.000104993436 -1.4819496f-5 0.00016174733 5.1968495f-6 0.00012912843 1.9695099f-5 0.00014575227 6.3173524f-5 -5.2360924f-6 -5.6768888f-5 -0.00014571872 -0.0001785546 5.9773887f-5 -0.00018953496 6.7306304f-5 0.0001568174 7.134748f-5 6.8328976f-5 -3.3585914f-6 -5.0178074f-5 2.9536714f-5 1.9834733f-5 4.025718f-5 -0.00021762686 9.823387f-5 2.9596104f-5 -5.052526f-5 0.00011400497 0.0001491102; 3.034911f-5 0.00011196634 6.7014895f-5 -0.00013202667 -4.6789377f-5 8.572312f-7 9.169984f-5 -2.3223301f-5 -0.00010554397 8.7856f-5 5.0881303f-5 -9.156455f-5 -6.564096f-5 3.4504257f-5 -0.00016190331 -6.2145766f-5 1.9165122f-5 -7.877776f-5 -4.080841f-5 2.136334f-5 -7.858364f-5 -8.119439f-6 0.00013435273 4.4416214f-5 -0.00012370772 0.000101974394 -4.802835f-6 -0.00010022334 1.2407417f-5 -0.00015726443 -0.00010571935 -3.07934f-5; 8.026358f-5 -2.2175438f-5 -0.00012479624 -4.0228583f-6 -0.0002115195 2.9970723f-5 -4.1155407f-5 0.00014840579 5.4885195f-6 0.00029171017 0.00013645006 -0.00017074464 0.00018228586 -0.00011459783 3.7666694f-5 -1.6101341f-5 0.0001842996 -0.0001169029 -1.4438652f-5 -6.3499065f-5 -9.643319f-5 -0.00017683755 5.7190664f-5 -3.922391f-5 -3.1976626f-5 -2.109656f-5 -3.90291f-6 -0.00013517434 0.00016720014 -0.00013073278 8.5466956f-5 -1.526848f-5; -4.2852924f-5 -0.00015807984 -8.3653595f-6 2.5182915f-6 -0.00013412097 -9.557661f-5 8.1717204f-5 -5.246506f-5 -1.7424688f-6 9.216007f-5 -7.319666f-5 -0.00027948042 0.00015722163 3.1111696f-5 -0.00020530884 -5.4546465f-5 -6.29937f-6 -0.0001428365 3.9120237f-5 3.721716f-5 -6.6093584f-5 -1.8950981f-6 -9.6505835f-5 2.1517824f-5 5.098992f-5 3.5149867f-5 0.0001302061 0.00012702291 -7.683339f-5 0.00011238969 2.6854445f-6 4.7272497f-5; -4.8100424f-5 4.516353f-5 0.000111997026 5.204853f-6 -6.4593376f-5 0.00010572687 3.921466f-6 6.905398f-6 -0.00015121218 1.7533444f-5 -3.0009418f-5 -3.6821777f-5 -0.0001382609 -0.00013444631 -0.000118799144 -0.00011516807 -2.658559f-5 -1.0310642f-5 -0.00013526081 8.310802f-6 1.39192125f-5 4.0677514f-5 5.7376667f-5 -4.978068f-5 0.00015903164 3.915949f-8 -1.5798567f-5 5.3141244f-5 -0.00014180606 -0.0001386008 0.00013365413 0.00013656895; 0.00017168031 -2.2275335f-5 -9.818207f-5 -6.38929f-5 -9.4072755f-5 -3.0191156f-6 8.873516f-5 6.795865f-5 -0.000121237004 7.997962f-5 0.00013511072 0.00013809897 -2.5103287f-5 0.00012684942 -0.0001443395 -9.857521f-5 0.00017308586 4.2674477f-5 -1.6791339f-5 -8.397967f-5 1.0641746f-5 -5.0464263f-5 -0.0001410602 -2.9612102f-5 1.5865919f-5 6.131211f-5 -1.00230445f-5 -2.2032244f-5 1.0033375f-5 -0.00012724352 -1.5285226f-5 0.00014202898; -1.4121472f-5 -5.0044982f-5 7.915148f-5 0.00010705557 -4.692474f-5 4.8653448f-5 -6.4731226f-5 -7.822046f-5 -1.0700496f-5 2.0036923f-6 0.00012267969 3.603697f-7 1.9522387f-5 -0.00010675869 -0.00013979523 -4.262446f-5 -6.636593f-5 -2.5301544f-5 0.00017978139 8.460204f-5 -9.998411f-5 -0.0001354939 -9.499201f-6 -4.361895f-5 -0.00013717679 4.524789f-5 0.0001629717 7.41474f-5 -0.0001341515 0.00012045276 -8.077927f-5 1.8191115f-6; 8.464025f-5 -0.0001911386 -0.00013323688 -5.179797f-5 3.0535775f-5 -3.6069952f-5 5.7020614f-5 -6.241533f-5 -3.7568614f-5 -9.843058f-5 0.00024355052 -0.00013379584 -6.849716f-5 1.1521302f-6 -1.6933034f-5 6.922868f-5 0.00015571411 -8.287445f-5 4.693961f-5 0.00018321241 7.1558397f-6 -0.00023000097 -2.3251116f-6 9.687749f-5 8.636353f-5 8.079613f-5 6.684031f-5 4.8694303f-5 -7.006521f-6 -5.833168f-7 -4.4962962f-5 -6.118613f-6], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-0.00010431886 7.263726f-5 0.0001153885 2.3408782f-6 -0.00014803729 0.00017549854 -1.4270489f-5 9.409741f-5 -2.1240825f-5 0.00027308802 -2.6860275f-6 1.595533f-5 -8.200385f-5 0.00014178251 -1.6477046f-5 -9.237305f-5 -0.00011124783 1.087667f-5 0.00022355978 2.2964196f-5 -9.138082f-5 -3.8097627f-5 -7.44995f-5 -0.00022302984 -9.659961f-5 -0.00034645712 9.354396f-5 6.146441f-5 -9.705188f-5 -4.282779f-5 8.976634f-5 -4.020095f-5; 6.706114f-5 0.0002107104 1.2080838f-5 6.554694f-5 6.5647335f-5 3.832331f-5 -0.00016438964 -2.266784f-5 -7.713031f-5 -0.00018027052 6.2050116f-5 -5.3583874f-5 0.00016199157 -2.413137f-5 1.9695848f-5 0.000119893564 2.6232174f-5 -2.5299496f-5 7.291109f-5 -0.0002123351 0.0001043167 -3.4432705f-5 -9.066305f-5 -0.00017630673 -5.5120025f-5 0.00013201217 -0.00011529041 0.00013865373 5.0689f-5 8.1196495f-6 -0.00010693036 9.6504766f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
+
+const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
+    Chain(
+        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
+        layer_2 = Dense(1 => 32, cos),  # 64 parameters
+        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
+        layer_4 = Dense(32 => 2),       # 66 parameters
+    ),
+)         # Total: 1_186 parameters,
+          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",m,[(a(),i("svg",V,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",T,[(a(),i("svg",U,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=t("where, ")),A("mjx-container",q,[(a(),i("svg",b,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=t(", ")),A("mjx-container",D,[(a(),i("svg",z,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=t(", and ")),A("mjx-container",R,[(a(),i("svg",K,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=t(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
+    χ, ϕ = u
+    p, M, e = ode_model_params
+
+    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
+    # it, however, in general, we should use \`st\` to store the state of the neural network.
+    y = 1 .+ nn_model([first(u)], nn_params)
+
+    numer = (1 + e * cos(χ))^2
+    denom = M * (p^(3 / 2))
+
+    χ̇ = (numer / denom) * y[1]
+    ϕ̇ = (numer / denom) * y[2]
+
+    return [χ̇, ϕ̇]
+end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
+waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    axislegend(ax, [[l1, s1], [l2, s2]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
+
+    fig
+end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
+
+function loss(θ)
+    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
+    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
+    return mseloss(pred_waveform, waveform)
+end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0007098617733391764
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
+
+function callback(θ, l)
+    push!(losses, l)
+    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
+    return false
+end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
+optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
+optprob = Optimization.OptimizationProblem(optf, params)
+res = Optimization.solve(
+    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
+    callback, maxiters=1000)
    retcode: Success
+u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [-5.067828169553457e-5; 0.0002033883647527833; 1.4273280612531504e-5; -4.428054671737328e-5; -4.250684651193247e-5; 7.748175266906924e-5; 6.288575150393261e-6; 3.7066754884979796e-5; 0.00014335215382743272; -0.00011426929268051497; 0.00011086220183643978; 8.886728755885818e-5; -0.0002350791619390836; -0.00026994064683101955; -2.1895052668672963e-6; -4.137206633454982e-5; 0.00014377594925456472; -1.254786911885069e-5; 0.0001540251396362168; -6.948348891458086e-5; 9.75148054747745e-7; -2.275944098073367e-5; 5.759294799648488e-5; -9.247271009369957e-6; -7.65327495172892e-5; -5.430622331914719e-5; -5.6007382227096824e-5; 4.515377077039578e-5; -9.931653767119681e-5; 0.00011081036791418321; -2.595081787143777e-5; -2.3206786863723692e-5;;], bias = [-5.310483371894584e-17, -3.1011765244248956e-17, 1.2287450168839055e-17, 3.141598635647141e-17, -4.114087861200215e-17, -1.087215296099596e-16, -5.37164552124023e-19, 4.2209422355381365e-17, 1.3438114389553043e-18, -3.747220297384205e-17, -1.5831630774748768e-17, 1.2461438073596785e-16, -1.4656832046366923e-16, 8.680182063003482e-17, -2.656384109725839e-18, -4.3500024256058437e-17, 7.393138650556651e-17, -1.2508691195987064e-17, 1.6063779055314245e-16, 2.5819551063395286e-17, 1.742596203254984e-18, -1.739224497562247e-18, -3.550440848984027e-17, -1.8956723722780767e-18, -7.511083048774223e-17, 1.7363671075209743e-18, -1.260939208188153e-17, 1.8950031747943242e-17, -6.621050923994785e-17, 2.2468062521676514e-16, -5.786119776188296e-17, -1.7438878875030686e-17]), layer_3 = (weight = [-6.488304392744379e-5 5.1593381391560795e-5 -0.00010973868961057376 0.00011888554104759725 -7.869967270719461e-5 2.6729300549302098e-5 -4.338042821812204e-5 -0.00014315193879495047 2.6661781481641573e-5 -0.0001733976871434768 -2.6336444756196498e-5 9.159599244983697e-5 -0.00015508627006391803 -4.3109380607051135e-5 1.6835862658147272e-5 1.792073158528541e-5 -7.001163361902599e-5 -4.674345222495249e-5 3.036351404834028e-5 -2.1701625825915297e-6 2.439655395962534e-5 1.1773439615602058e-5 9.929855401085215e-5 0.00011054835591155347 9.563256531943786e-6 -4.707818629296995e-5 9.590648801920449e-5 -4.885345082923397e-5 0.0001208191922671903 -0.0002225035034313972 -9.059373882676194e-5 -3.268187469414162e-5; 0.00012063635157848382 -6.609181646762502e-5 -8.302260603794181e-5 -1.9541194621509316e-5 -6.673466915070392e-5 -1.8391583189296476e-6 -2.9914954854560117e-5 0.0001361124807708773 -0.00010268483375237967 7.048662454782473e-5 -1.6720200353106822e-5 7.6481518856795e-5 8.879042369972584e-5 3.933901033906554e-5 -2.3042796517059236e-5 1.4161023766989025e-5 -3.0308782431336086e-5 -4.231336125211143e-5 -5.7871654195867056e-5 -1.2361219621183299e-5 0.00011966117680732812 -8.016287093322083e-5 3.7681201586580233e-5 -3.460617945800818e-5 -9.54333233557773e-5 6.842894007878977e-5 1.1556674774525573e-5 -0.00011588641128308456 -6.523252587338435e-5 5.238058228589095e-5 0.00024232536751971285 7.974424345790755e-6; 8.224337639854286e-5 -1.218803148938489e-5 2.7367696618746988e-5 4.015808068084576e-5 -4.848885813975649e-5 5.793578080352111e-5 7.150272418380917e-5 7.017520388922748e-5 -8.642651968535577e-5 1.970483450145065e-5 4.980705549109859e-5 9.72932799938929e-5 9.627208479078838e-5 -9.517727752818883e-5 -0.00010894272722202147 -6.569878618596989e-5 -0.00014163293539446397 0.00020533117336322745 6.451986397017856e-5 -0.00010703093297911067 2.3383893365242713e-5 4.1318886958613766e-5 -9.12378557970847e-5 2.819155693737988e-5 3.441893264403294e-5 8.88329711031072e-5 -9.182458901909231e-5 0.000141080946066501 5.8570116978216505e-5 8.331143604459376e-5 8.733199925496392e-5 2.7256763730982073e-5; 2.0639607059931383e-5 -6.556483098836061e-5 -9.26717530500157e-5 1.2781671971535676e-5 -2.19400917966395e-5 4.872884706412406e-5 -0.00020592711193319547 0.00019216477262955044 -0.0001082424915192628 -0.00010664191908733837 -0.00011950996672101251 4.0736262194625994e-5 3.815132181910655e-5 1.5810668587742027e-5 -5.011134821404839e-5 -1.1253566340090383e-5 -9.231913831615613e-5 0.0001426945472804816 -9.049288750691079e-5 0.00019282329044938032 -8.383033677535268e-5 -3.1071643160232267e-6 1.3963474843987855e-5 3.5590439484819587e-6 -0.00016782591960225285 -5.056347985817206e-5 5.644821608723057e-5 0.00014241667845919624 0.00012270893235392902 1.297179547248012e-5 6.824178304225082e-5 1.0279214580006563e-5; -1.3928808630423847e-5 -2.580913592300915e-5 -3.3537372519475696e-5 -0.00015090158835866352 -5.633276218779253e-5 2.3137669768206132e-5 -9.296012563046942e-5 -1.6965826080393775e-5 8.621489496172741e-6 -2.2875613513145147e-5 -9.810577744266379e-5 0.0001954865410423597 -1.4209998016843574e-5 4.283326866226591e-5 -0.00014059855589059123 0.00012722292462947907 7.49152067781239e-5 -0.00013340454011105866 -7.446950899084912e-5 6.688554319983476e-7 7.388097810490849e-6 7.2399238431729244e-6 0.0001258476231212468 2.1599497767766602e-5 -6.403067804987242e-5 9.297352239608012e-5 -0.0001389406416367069 -2.049448585437825e-6 9.337108799607962e-5 1.4604740497695822e-6 -4.2784514380599464e-5 -4.6444779445863475e-5; -9.859231075038756e-6 -8.92166064731826e-5 9.193804707326044e-5 4.697279636463018e-5 -7.961775925591352e-6 -0.00015716880462417928 1.1460652596785891e-5 4.223163322609779e-5 1.0276937966449545e-5 -0.00013143228735129704 -6.983421208855968e-5 3.282657761888761e-5 -2.204743960464444e-5 4.16758846695159e-5 8.68817804003514e-5 0.00012558122379420223 9.739221404019534e-5 5.784216390754152e-5 0.00013859734444125957 -5.485830862011259e-5 9.106923225050862e-5 0.00010087260288134235 0.00014309589441114764 -1.206883208208888e-5 -5.216583947857849e-6 4.1126604439399654e-5 0.00015406645233457655 -3.986819156847103e-5 5.189840866665336e-5 8.516869255986725e-5 6.19362906873599e-5 -9.757490643817917e-5; 0.00010022519278392329 3.801946592103462e-5 -0.00011406373518001133 0.00020252994441945248 0.000124030307108128 -7.962125107938336e-6 -7.458327594393708e-5 6.384426238766085e-5 0.00011404573602886837 0.00016995428892370348 8.429847808839796e-5 4.9968596487828436e-5 0.00011434576741704361 0.0001638659131113016 -6.057462009083322e-5 -9.412515931943289e-5 -0.00015873539372295175 -6.482401944411764e-5 3.0919208131320155e-5 -2.1594392035681557e-5 0.0001210999806626343 -7.771278900507687e-5 -1.8475592587357495e-6 -9.604004060733888e-6 -0.000106139860816243 -0.00013608629809640216 -3.699267030059231e-5 -9.477077686641228e-5 -0.00010653282801102781 -1.4092722392875731e-5 -0.00010621809191251024 -1.5991102498432955e-5; -6.85945405234782e-5 5.955887999559289e-5 0.00013538516313245464 -5.856513863138032e-5 -1.4058173587873332e-6 -9.187440346751602e-5 4.963713681034698e-5 -2.688422825954933e-6 0.00010794513317930726 3.837898211029819e-5 0.0002389099909964718 -0.00011218212503765081 3.951167408783001e-5 7.633592858151902e-5 -6.472568102937236e-5 2.6780157979814266e-5 1.481328499037518e-5 -8.19167534314287e-5 1.378444730332931e-5 -0.00013009375007415393 0.0001079858057823773 9.007736761530801e-5 -5.176919413089905e-5 -3.248019757621968e-5 1.4517148058514547e-5 7.808188192255654e-5 -4.971269564295335e-5 -4.636922941018939e-5 0.000126796724628076 7.444591127976329e-5 -0.00014616011222416605 -9.482908981910036e-5; -5.844121978835604e-5 -7.837145376351397e-5 -0.00015722129746674127 0.00013298625281212424 0.00020579882702922013 0.00011313802979320028 -6.057538548585531e-5 6.835987097737537e-6 8.129581992879848e-5 5.070884735805013e-5 0.0002587695883460528 -0.00015811274779362782 -2.55861693391912e-5 -0.00010511739374003677 -0.00012036865218643604 9.864171015696166e-5 -4.523079677439581e-6 -0.00011798592886290081 2.915397136620633e-6 0.0001434153630737371 -5.82827676214142e-5 4.964991176283433e-5 -6.646095125571264e-5 -0.0001173525713045014 2.099080602537575e-5 -1.2118468201797582e-5 -0.0001377560881325826 -5.26452283326858e-5 -8.246047126437611e-6 -6.679808275176214e-5 -0.00011273330592185735 2.2168856322686e-5; 1.807031263771555e-5 -8.41687352815843e-5 4.027746114181075e-5 -9.122214318020622e-5 1.6445567655606466e-5 8.070489332307503e-6 -4.988372814958256e-5 0.0001852610738412444 7.757903300690267e-5 -0.0001033573636389419 -5.785060342285634e-5 3.672856188378627e-5 0.00014628277937180577 0.00011685057767197896 0.00014759533302159713 0.00010566372735658319 0.00016843380659584684 7.314829496200851e-5 -0.0002992532673827499 -9.248457275797153e-5 -0.00010036986271851036 2.3871129872274463e-5 -8.735494988039545e-5 -0.00014760663829045643 8.600705119780681e-5 -0.00010550347282085964 -1.5242485585335465e-5 -2.606376723386703e-5 0.00014419519071699367 9.088176817632428e-5 4.8678929363374415e-5 4.9175829974698705e-5; -6.598751317803042e-5 -6.030990155271338e-5 -7.217605709890307e-5 -7.999448100896822e-5 -0.00020351446985593347 8.729900034517088e-5 3.311363431997184e-5 -7.431861561350837e-5 -6.288851661567153e-6 -4.8014057053713016e-5 0.0001289141673375904 4.1720859346913115e-5 -8.892866574317183e-5 0.00024919270251556733 4.605136379971683e-5 8.19395226906025e-5 -4.1751989581145076e-5 -4.782373255445095e-5 -7.793842635442057e-5 -4.7223255048506476e-5 -5.63738012963482e-5 -2.9043554629668638e-5 -4.210199952218848e-5 -4.4029447987912596e-5 -2.1515656069153655e-5 0.00010635621035611067 -0.00018420626113517394 7.879130314243693e-5 6.568936985126093e-5 -0.00014872338035935024 -0.00025311636093398925 0.000164341023239769; -3.258934873928717e-5 -6.829450274425075e-5 -8.627620252854646e-5 -2.488858615878919e-5 -0.000108696694842814 -1.8759535924339176e-5 -5.000216686357272e-5 0.00010093201137306672 -0.00017555949296410188 8.38374982045109e-5 -0.00019008694642297169 -2.05099385134345e-5 9.822381263758857e-5 -0.00014449343860221752 -4.487419562837569e-5 -0.00011297130538923156 5.281707864879642e-5 0.0001025940219910863 1.2051929042638165e-5 0.0001414263898545934 1.6517353396471644e-5 0.00011433706833441294 -9.066640621357982e-5 3.489854872851174e-5 2.456762205793661e-5 3.055989884115363e-5 0.0001093042593488524 -4.3657451788470566e-5 5.348321075826649e-5 5.2719075137710264e-5 -6.0060319702182694e-5 6.099252652302867e-5; 4.684605982778897e-5 0.00015922139527554951 3.730149137131014e-5 0.00010522137699458063 -1.7152006424891324e-5 9.699498286513317e-5 8.678879339289912e-5 -5.9812624844068647e-5 0.00010268462164837113 -0.00011774689528815062 -9.323463064342426e-5 -7.073884671976903e-5 9.475809156010668e-5 -2.0643809424456035e-5 -0.0002366002740962891 -0.00015139882498611479 -7.254450656895337e-5 0.00018710389803904177 -7.661572499622105e-5 -6.959221400743762e-5 -0.00023375447245039922 0.00011397922162008577 6.118481496391263e-5 -0.00014835137745090697 -2.0492704155778817e-5 2.8101463154730067e-5 2.545392813903837e-6 6.996500405525111e-5 -9.364928019190132e-5 -0.00014223980588562476 -0.00017213980072834412 -0.00018664228310068032; -0.00011808607541993425 4.8038628844860654e-5 1.6432158188171453e-5 -0.0001537796242725119 7.191083113625497e-5 -0.0001872625625789189 -5.120785590647832e-5 -0.00010370548716550833 0.0002961341187959484 3.4529158253777705e-5 -7.646508766143015e-5 5.085647262362489e-5 9.7541148531797e-5 6.58443632322203e-5 -4.7161783188722274e-5 -2.1120003114129957e-5 -0.00011250213447297528 8.57567456454027e-5 -1.6246659472613986e-5 -0.0002794219808517045 1.363162025290119e-6 8.212484226291877e-5 6.171654513068318e-5 6.960662267133479e-5 -1.3250552913316052e-5 0.0001021550660856051 0.000116517733656427 -2.596387548933425e-5 4.812307360893733e-5 4.640898532429348e-5 0.00014826165988538294 -7.550835744282355e-5; -6.17154759238133e-5 -2.249270200262119e-5 -0.00011052547360301221 0.00010846958012556371 -6.269449967654701e-5 -9.421800804554029e-5 0.0001429795125732551 3.136664291381576e-5 8.234448306032229e-5 7.665271012258093e-6 -0.00010900864016727386 -8.210261868921642e-6 -4.782299164798104e-5 -0.0001506850488949556 -7.13813455076535e-5 0.0001220554519095803 0.00010132866428469075 -2.9438787707441644e-5 8.61451813834961e-5 -3.230415622234766e-5 5.361084241307146e-5 9.664669469933737e-5 -1.5967776009313548e-5 -8.983938040906761e-5 5.9496953879748424e-5 1.088932604765186e-5 4.291803689560512e-5 -0.0001035132185206405 0.00012603782361842033 5.6519148172840214e-5 6.545989582085756e-5 0.00019466919603295402; -6.035587200219031e-5 -3.048070636485676e-5 1.9147372703658136e-5 4.1489395634045756e-5 -2.497117481740846e-5 -9.497523485239959e-5 -5.7456613895707984e-5 0.00010517491403246128 6.864511026744694e-5 -5.116598977995619e-6 9.274737470061827e-5 1.2732370968505513e-5 2.075340574175034e-5 -1.2390137336297228e-5 3.9403538861096767e-7 1.626717760612538e-5 8.14697204341642e-5 9.05253045212093e-5 0.0001821270206428582 7.847962927283407e-5 1.3242259164174353e-5 -6.920653601898014e-5 1.9616315447848574e-5 -0.00020374188086040435 3.897193248943069e-5 -0.00012801252075493572 -6.932340121209963e-6 0.0001030455395530517 0.00010479109999235196 4.828026639213363e-6 6.819267667109008e-5 -0.00012826038352701946; -2.8094946705911407e-5 -3.083253303418634e-5 2.28271104925644e-5 8.898699471965577e-5 1.207003371301013e-5 -3.803506896010253e-5 -1.0439388993198004e-5 5.42278607236792e-5 4.867334374931775e-5 -4.518214753706244e-5 7.921917441772954e-5 -0.00010169376454077221 0.00014349088761731043 -7.677398078600014e-5 0.00011103335729340141 -5.2174575634941974e-5 1.2260983704556828e-6 -2.8247074246699347e-5 9.942554189830378e-5 1.1441331857990563e-5 1.461823480987718e-6 -5.285105051816366e-5 -4.745934037136632e-5 -5.7792964603229816e-5 -2.5201509365519844e-5 2.534265824133726e-5 9.225348495884073e-5 5.786641433142276e-5 -4.426601716194696e-5 -0.00022661089732072592 -0.0002966056968806676 -9.220337650267332e-5; 6.735962678692488e-5 -4.1766666167990785e-5 -1.2230763882255954e-5 -0.00010595445483443115 5.75487764716636e-5 -6.401900018784194e-5 -0.0001401495784240279 0.00011234512648559416 3.367131002493477e-5 -0.00012897993574836596 -0.00012430046181646445 7.917755001005653e-5 0.00021940120941039688 -1.7720901543366787e-6 1.3755347840630702e-5 3.097948219408506e-5 1.2615215286920207e-5 3.2222539534702144e-5 1.179245272839102e-5 0.0001234499840346639 2.063462388366766e-5 -0.00019241103573688025 3.0956541099726786e-5 -4.40704671792656e-5 0.00013447786195217666 0.00015082376639372836 2.4749576273166298e-5 7.190370242358394e-5 5.500512351761415e-5 -2.351658263368668e-6 0.00010669987104381709 -0.00015557776633170152; 0.00015004433877961992 7.171248505085587e-5 7.186821237169626e-5 -4.4490909958627636e-5 0.00010725007375199672 -3.0032770909973542e-5 -6.131717312186528e-5 -7.843955090580263e-6 7.125566405206153e-5 -1.1301954900639649e-5 4.3303734100979687e-5 1.4667064688319504e-5 7.304025272798725e-5 0.0001203544299370227 3.312941696108749e-5 -1.6958564474267366e-5 9.650549979609595e-5 -0.00011026335489973004 1.5609365773470854e-5 -7.302637171037346e-5 2.9099054854809883e-5 -0.0001471923977349845 1.1051333940625235e-5 -2.716893762212287e-5 6.98817959074871e-5 -0.0001721674278358903 -5.542232323009273e-5 0.00015735103003876595 -2.357148589894786e-5 -9.349403471653878e-6 -4.482258084849063e-5 0.00015532227477720406; 9.200155002588563e-5 -7.894742336531445e-5 6.909560581959909e-5 6.397325162368042e-5 -7.337227905111243e-5 1.81069772469319e-5 -1.3611788720594416e-5 -0.00011573251865513483 6.840369863028096e-5 0.00016948325176315674 -3.591863239591659e-5 -0.00010666078921815155 5.9028980128354384e-5 -1.1826329658209398e-5 5.607247478735757e-5 4.8404866926718095e-5 0.00019965971844328957 -7.507221923617522e-5 3.281272622814832e-5 -0.00026297763722425135 0.00013116674929449456 -6.360763834265864e-5 1.3418215555135889e-5 -5.0055308723243565e-5 -6.63661867048385e-5 -5.73548986477575e-5 -0.00027880083687079906 5.80557334859591e-5 4.770976854049545e-6 0.00023913896259444115 -0.00027277934166079255 0.00013627896808145697; -3.847454513714503e-5 0.00013499990222799242 -0.00011172731115191943 -1.5791521720241938e-5 1.746134534943557e-5 8.991045092290287e-5 -0.00013876824755396457 0.00010837116337672065 2.5287605465845612e-5 4.091738287416977e-5 -8.683316787181636e-5 2.7275808177650394e-5 -0.0002117683668545441 -4.1985100098565496e-5 2.5652245538656208e-5 -4.386319574376392e-5 0.00010112778750072644 8.595907479349758e-5 -0.00011121149668878578 4.6184361279588816e-5 -2.018655118416289e-5 0.00010101308202894625 -6.718531017366678e-5 0.00016857952345919098 -6.646065389912549e-5 -8.186676655139642e-5 5.893228649103207e-5 6.0351993168584786e-5 0.00011074483540898065 2.197029266272924e-5 8.477605501713446e-5 2.8409362356176015e-5; 0.00015168579624614677 -0.00015583055870494473 9.538919617630628e-7 3.832680953664729e-5 3.7978788815051777e-6 -2.4169362640261138e-5 -5.114657620551897e-5 -1.847409772281022e-5 2.1159557864773197e-6 2.0241523762792873e-5 -0.00019447271848448938 7.808523581943265e-5 -1.986637222118892e-5 2.674661913358162e-5 2.334530358084579e-5 -7.56245056062654e-5 3.0029402060874113e-5 0.00020181267676989644 0.0001235838214971321 -4.559180457263342e-5 0.00010591180829678763 0.00026848768970112405 -3.4460484365718736e-5 2.3217581420886376e-5 8.163944652651308e-5 -1.3016809370038249e-5 0.00011559072367263004 -6.941009565607624e-5 -0.0001963581228291365 -8.809125507369087e-5 0.0001532562971452538 -0.00014338266124368565; 1.757703004370722e-5 -0.0001143078671567958 -2.315727859711487e-5 -1.965702457271244e-5 -0.00010862327061220267 8.574516952178291e-5 -6.518635550652207e-6 -4.5364523507647356e-5 -5.5962072188025316e-5 0.00012402023541183133 5.957240402024154e-5 0.00014366027145554176 -2.3027693792000462e-5 -2.1112743602468684e-5 1.875388162700245e-5 6.527077759058019e-5 -6.91717624114733e-5 0.00015894688377995764 8.440316279559432e-5 0.00015212992267590782 4.881772291117934e-7 -7.422075312677048e-5 4.410387820355415e-5 -5.293577222362373e-5 3.839174579736624e-5 -0.00026345439717731614 -0.0002799995755460038 -0.00010818517792829506 0.000100323169058197 0.0001395339012695561 -0.00015231897585652043 9.96289262864811e-5; 3.5567809000901426e-5 -7.706454964950166e-5 -0.00021089415570622416 0.00011735034019785079 -8.083291894757975e-6 0.00020298259875836352 -7.773799045844129e-5 0.00011110823067702369 0.00021550229536935868 -0.00011431477508174869 -5.7657664391610264e-5 -9.974130119097646e-5 -0.00012480812477309142 0.0001282768203701277 -7.215671251196544e-5 -9.859565073292652e-5 -5.0619583297677294e-5 6.369928412005192e-5 -1.0572646196095261e-5 -8.19449491509571e-5 -9.611637545185293e-5 4.977304667682311e-5 7.011605300341022e-5 -3.112052984669926e-5 -0.00011037792903970437 -0.0001152873232322168 -7.007802235329321e-5 -0.0001132779438457816 2.469945827307626e-5 -0.00013523812270157737 0.0001209863399444701 -0.00025738433799142086; 0.00026173845471244757 1.2837046954176047e-5 -1.7063366256061072e-5 -0.00010499090931947774 -1.4816968912365366e-5 0.00016174985866423866 5.1993764759212034e-6 0.00012913095487643582 1.9697625605151052e-5 0.00014575480089883838 6.317605074623253e-5 -5.233565486875458e-6 -5.676636065581032e-5 -0.000145716190315948 -0.000178552077711665 5.977641412069611e-5 -0.00018953242907405934 6.730883105088676e-5 0.00015681993171932752 7.13500070491646e-5 6.833150327334844e-5 -3.3560644238060075e-6 -5.0175547228302636e-5 2.9539240488211657e-5 1.9837260326726308e-5 4.025970555854449e-5 -0.00021762433389771325 9.82363948355292e-5 2.9598630492229406e-5 -5.052273409778598e-5 0.00011400749862556023 0.00014911272809737047; 3.034694161183524e-5 0.00011196417396575523 6.701272574869138e-5 -0.0001320288436821541 -4.679154592881643e-5 8.550622234496614e-7 9.169767069547579e-5 -2.3225470459100275e-5 -0.00010554614057612856 8.78538332196153e-5 5.0879133750331555e-5 -9.156671952072757e-5 -6.564313010158575e-5 3.4502088210235905e-5 -0.00016190548270092207 -6.214793468675314e-5 1.916295256675445e-5 -7.877992978169433e-5 -4.0810577802704876e-5 2.1361170338108406e-5 -7.858580723254803e-5 -8.121608362079694e-6 0.0001343505565907006 4.441404523245204e-5 -0.0001237098920958899 0.00010197222506544903 -4.80500405533108e-6 -0.00010022550925927883 1.2405247771001035e-5 -0.00015726659406058402 -0.00010572152025847508 -3.0795567754784686e-5; 8.026301135713982e-5 -2.2176003524236732e-5 -0.0001247968009306606 -4.023423782662333e-6 -0.00021152006023158516 2.9970157145265214e-5 -4.1155971969154164e-5 0.00014840522114126738 5.487954028918456e-6 0.00029170960103101106 0.0001364494985568802 -0.00017074520085998894 0.0001822852957661807 -0.00011459839673306628 3.7666128507673295e-5 -1.6101906778665383e-5 0.00018429903345122147 -0.00011690346376099852 -1.4439217677594005e-5 -6.349963017101542e-5 -9.64337575464998e-5 -0.00017683811686993795 5.719009847552347e-5 -3.922447443302018e-5 -3.1977191367793196e-5 -2.1097126348202013e-5 -3.903475528666147e-6 -0.00013517490672939649 0.00016719957263148342 -0.00013073334205956205 8.546639061704991e-5 -1.5269046100464052e-5; -4.285495149137722e-5 -0.00015808186933328908 -8.367387156383673e-6 2.5162637734446107e-6 -0.00013412299947280758 -9.557863583954096e-5 8.171517631804133e-5 -5.246708812823869e-5 -1.7444965328947305e-6 9.215804541550998e-5 -7.319868516109516e-5 -0.00027948244448684073 0.00015721960019390505 3.110966875056439e-5 -0.0002053108669065728 -5.454849221909746e-5 -6.301397636464194e-6 -0.00014283853440478876 3.911820979507831e-5 3.7215132149372895e-5 -6.609561157579041e-5 -1.8971257737095413e-6 -9.650786293836533e-5 2.1515796586256786e-5 5.0987890542689164e-5 3.5147839778305374e-5 0.00013020407143581424 0.00012702088363535395 -7.683541977943642e-5 0.00011238765958065533 2.683416848824828e-6 4.7270469192383486e-5; -4.81021370076556e-5 4.5161818924709434e-5 0.00011199531312668745 5.2031405532107285e-6 -6.459508855258904e-5 0.00010572515478926155 3.919753489032578e-6 6.903685552249666e-6 -0.00015121389420599422 1.7531731863723377e-5 -3.0011130512125973e-5 -3.682348957298774e-5 -0.0001382626168931522 -0.00013444802149092396 -0.00011880085636293478 -0.00011516978243961795 -2.658730314649844e-5 -1.0312354753107738e-5 -0.00013526252129011354 8.309089061376478e-6 1.3917499963287668e-5 4.067580179181019e-5 5.7374953975530736e-5 -4.978239208939439e-5 0.00015902992621632663 3.7446948785966035e-8 -1.5800279434907896e-5 5.3139531736822475e-5 -0.00014180776903302748 -0.00013860250597716513 0.00013365241924403315 0.00013656724062385112; 0.0001716806770309658 -2.2274967104751507e-5 -9.818170232392436e-5 -6.3892529706206e-5 -9.407238698258605e-5 -3.0187476022524926e-6 8.873553148078214e-5 6.795901996156597e-5 -0.00012123663631427866 7.99799860578801e-5 0.0001351110883297239 0.00013809933867378477 -2.510291900762605e-5 0.00012684979143159005 -0.00014433913323955196 -9.857484414169174e-5 0.00017308623197095634 4.2674845335571465e-5 -1.679097053056454e-5 -8.397930227147059e-5 1.064211361154162e-5 -5.046389500421066e-5 -0.00014105983003900941 -2.9611733534690534e-5 1.5866286648817994e-5 6.131247633675491e-5 -1.0022676488003671e-5 -2.2031875541867535e-5 1.003374260108281e-5 -0.0001272431503275569 -1.5284858218362233e-5 0.00014202934531568027; -1.4122403897882985e-5 -5.004591393831892e-5 7.915055148453373e-5 0.00010705464201430764 -4.692567044719153e-5 4.8652516581687163e-5 -6.473215789617122e-5 -7.822139041115574e-5 -1.070142779814479e-5 2.0027607225820363e-6 0.00012267875690861888 3.5943811747557506e-7 1.9521455853597164e-5 -0.00010675962430272944 -0.00013979616078165446 -4.262539032231118e-5 -6.636686184525886e-5 -2.5302475749296935e-5 0.0001797804577128186 8.46011095765146e-5 -9.998503837584602e-5 -0.0001354948329188775 -9.500132635249613e-6 -4.3619882760933495e-5 -0.00013717771825929085 4.5246957326643204e-5 0.00016297076918101878 7.414646797963396e-5 -0.00013415243329097305 0.00012045182638951863 -8.078019918491195e-5 1.8181799097218416e-6; 8.464045504599007e-5 -0.00019113839834670673 -0.00013323667689899772 -5.1797763714314024e-5 3.05359831175501e-5 -3.606974450729679e-5 5.702082171712219e-5 -6.241512482727741e-5 -3.756840623890042e-5 -9.84303728698095e-5 0.00024355072355621558 -0.00013379563051483503 -6.849695227782469e-5 1.1523379756114767e-6 -1.693282638069095e-5 6.92288854255075e-5 0.00015571431549989628 -8.287424452344741e-5 4.693981873362311e-5 0.00018321261859757007 7.156047561982724e-6 -0.00023000075770749568 -2.324903786468119e-6 9.687769898238739e-5 8.636373650307874e-5 8.07963410837309e-5 6.684051595886362e-5 4.869451049973391e-5 -7.006313350468768e-6 -5.831089962910616e-7 -4.4962754597401675e-5 -6.1184050506944375e-6], bias = [-2.042037274184826e-9, 5.450538238276147e-10, 2.5289934987238694e-9, -5.148148177753555e-10, -1.2833426980289933e-9, 3.2157841518188328e-9, 2.881843239953348e-10, 1.0123517173553542e-9, -1.2702900780703617e-9, 1.3104570071210386e-9, -2.326512247765953e-9, -7.056591985203844e-10, -2.954497048078123e-9, 6.357343856216914e-10, 1.4448305944960618e-9, 1.2741375330697258e-9, -1.5175691242038057e-9, 1.4450914722628342e-9, 1.8109587064243945e-9, -4.019722143932669e-10, 1.1298182630716548e-9, 1.2157589486701711e-9, -3.2476707575809744e-10, -3.245467265503101e-9, 2.5269332221808443e-9, -2.1689905677970475e-9, -5.654624220598772e-10, -2.0276934697509827e-9, -1.7125428650239895e-9, 3.680361289786686e-10, -9.315934878893783e-10, 2.0782255714890612e-10]), layer_4 = (weight = [-0.0007904217429736376 -0.0006134657177658567 -0.0005707143385862188 -0.0006837621003302308 -0.0008341402321897578 -0.0005106042139339528 -0.0007003734721873451 -0.0005920055483634651 -0.0007073437719725264 -0.0004130149265874651 -0.0006887888845526638 -0.0006701476440012 -0.0007681066177949802 -0.0005443204642705102 -0.0007025799817840762 -0.0007784759925414032 -0.0007973507560408622 -0.0006752262663322216 -0.0004625431346698359 -0.0006631387855498853 -0.0007774837715133924 -0.0007242005769272898 -0.0007606024799357885 -0.0009091325645501539 -0.0007827024388413978 -0.0010325599774881834 -0.0005925590173431629 -0.0006246384804114001 -0.0007831547945311147 -0.0007289307705100168 -0.0005963366240466802 -0.0007263039337167419; 0.00029680230859930646 0.00044045159335233126 0.00024182199209542147 0.00029528814104235834 0.0002953885231182095 0.00026806443487209774 6.535156190517071e-5 0.00020707335391228292 0.00015261088000378096 4.947066717177559e-5 0.00029179127530466097 0.00017615732457630542 0.0003917327014566102 0.00020560982886453667 0.0002494370336098561 0.00034963475290870455 0.0002559733568663764 0.00020444168978222695 0.0003026522687613105 1.740610020494017e-5 0.00033405789094706863 0.00019530848486119698 0.00013907814899625118 5.343438806550787e-5 0.00017462112606557147 0.0003617533259373307 0.00011445079170911656 0.0003683949033299282 0.00028043017727226373 0.00023786085032865976 0.0001228108338075376 0.00032624596727140253], bias = [-0.0006861029847291114, 0.00022974120193819142]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
+
+    lines!(ax, losses; linewidth=4, alpha=0.75)
+    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
+
+    fig
+end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
+waveform_nn_trained = first(compute_waveform(
+    dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
+    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
+        alpha=0.5, strokewidth=2, markersize=12)
+
+    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
+        position=:lb)
+
+    fig
+end
    Appendix ​
    julia
    using InteractiveUtils
+InteractiveUtils.versioninfo()
+
+if @isdefined(MLDataDevices)
+    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
+        println()
+        CUDA.versioninfo()
+    end
+
+    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
+        println()
+        AMDGPU.versioninfo()
+    end
+end
    Julia Version 1.10.6
+Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
+Build Info:
+  Official https://julialang.org/ release
+Platform Info:
+  OS: Linux (x86_64-linux-gnu)
+  CPU: 128 × AMD EPYC 7502 32-Core Processor
+  WORD_SIZE: 64
+  LIBM: libopenlibm
+  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
+Environment:
+  JULIA_CPU_THREADS = 16
+  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
+  JULIA_PKG_SERVER = 
+  JULIA_NUM_THREADS = 16
+  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
+  JULIA_PKG_PRECOMPILE_AUTO = 0
+  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const L=e(h,[["render",X]]);export{G as __pageData,L as default};
diff --git a/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.Bfi8AeMz.lean.js b/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.Bfi8AeMz.lean.js
new file mode 100644
index 0000000000..b398d472eb
--- /dev/null
+++ b/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.Bfi8AeMz.lean.js
@@ -0,0 +1,286 @@
+import{_ as e,c as i,a2 as n,j as A,a as t,o as a}from"./chunks/framework.DjZDIZsN.js";const G=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),h={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},d={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},r={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},g={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},C={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},F={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},u={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},c={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},m={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},V={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},D={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},z={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},R={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function X(O,s,P,Z,N,w){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
+      OptimizationOptimJL, Printf, Random, SciMLSensitivity
+using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=t("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=t(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=t(", ")),A("mjx-container",d,[(a(),i("svg",r,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=t(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
+    M = m₁ + m₂
+    r₁ = m₂ / M .* path
+    r₂ = -m₁ / M .* path
+    return r₁, r₂
+end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=t("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",g,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
+    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
+
+    if size(soln, 1) == 2
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+
+        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
+        p, M, e = model_params
+    else
+        χ = soln[1, :]
+        ϕ = soln[2, :]
+        p = soln[3, :]
+        e = soln[4, :]
+    end
+
+    r = p ./ (1 .+ e .* cos.(χ))
+    x = r .* cos.(ϕ)
+    y = r .* sin.(ϕ)
+
+    orbit = vcat(x', y')
+    return orbit
+end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
+    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
+    b = (v[3:end] .- v[1:(end - 2)]) / 2
+    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
+    return [a; b; c] / dt
+end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
+    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
+    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
+    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
+    return [a; b; c] / (dt^2)
+end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
+    x = orbit[1, :]
+    y = orbit[2, :]
+
+    Ixx = x .^ 2
+    Iyy = y .^ 2
+    Ixy = x .* y
+    trace = Ixx .+ Iyy
+
+    if component[1] == 1 && component[2] == 1
+        tmp = Ixx .- trace ./ 3
+    elseif component[1] == 2 && component[2] == 2
+        tmp = Iyy .- trace ./ 3
+    else
+        tmp = Ixy
+    end
+
+    return mass .* tmp
+end
+
+function h_22_quadrupole_components(dt, orbit, component, mass=1)
+    mtensor = orbit2tensor(orbit, component, mass)
+    mtensor_ddot = d2_dt2(mtensor, dt)
+    return 2 * mtensor_ddot
+end
+
+function h_22_quadrupole(dt, orbit, mass=1)
+    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
+    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
+    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
+    return h11, h12, h22
+end
+
+function h_22_strain_one_body(dt::T, orbit) where {T}
+    h11, h12, h22 = h_22_quadrupole(dt, orbit)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
+    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
+    h11 = h11_1 + h11_2
+    h12 = h12_1 + h12_2
+    h22 = h22_1 + h22_2
+    return h11, h12, h22
+end
+
+function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
+    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
+
+    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
+
+    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
+
+    h₊ = h11 - h22
+    hₓ = T(2) * h12
+
+    scaling_const = √(T(π) / 5)
+    return scaling_const * h₊, -scaling_const * hₓ
+end
+
+function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
+    @assert mass_ratio≤1 "mass_ratio must be <= 1"
+    @assert mass_ratio≥0 "mass_ratio must be non-negative"
+
+    orbit = soln2orbit(soln, model_params)
+    if mass_ratio > 0
+        m₂ = inv(T(1) + mass_ratio)
+        m₁ = mass_ratio * m₂
+
+        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
+        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
+    else
+        waveform = h_22_strain_one_body(dt, orbit)
+    end
+    return waveform
+end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",C,[(a(),i("svg",o,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=t("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=t(", ")),A("mjx-container",F,[(a(),i("svg",u,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=t(", and ")),A("mjx-container",c,[(a(),i("svg",B,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=t(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
+    χ, ϕ = u
+
+    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
+    denom = sqrt((p - 2)^2 - 4 * e^2)
+
+    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
+    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
+
+    return [χ̇, ϕ̇]
+end
+
+mass_ratio = 0.0         # test particle
+u0 = Float64[π, 0.0]     # initial conditions
+datasize = 250
+tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
+tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
+dt_data = tsteps[2] - tsteps[1]
+dt = 100.0
+const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
+soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
+waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
+
+    axislegend(ax, [[l, s]], ["Waveform Data"])
+
+    fig
+end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
+    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
+    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
+ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[-5.067828f-5; 0.00020338836; 1.4273281f-5; -4.4280547f-5; -4.2506847f-5; 7.748175f-5; 6.288575f-6; 3.7066755f-5; 0.00014335215; -0.00011426929; 0.0001108622; 8.886729f-5; -0.00023507916; -0.00026994065; -2.1895053f-6; -4.1372066f-5; 0.00014377595; -1.2547869f-5; 0.00015402514; -6.948349f-5; 9.75148f-7; -2.2759441f-5; 5.7592948f-5; -9.247271f-6; -7.653275f-5; -5.4306223f-5; -5.6007382f-5; 4.515377f-5; -9.931654f-5; 0.00011081037; -2.5950818f-5; -2.3206787f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-6.4881f-5 5.1595423f-5 -0.00010973665 0.00011888758 -7.869763f-5 2.6731343f-5 -4.3378386f-5 -0.0001431499 2.6663824f-5 -0.00017339565 -2.6334403f-5 9.1598034f-5 -0.00015508423 -4.310734f-5 1.6837905f-5 1.7922774f-5 -7.000959f-5 -4.674141f-5 3.0365556f-5 -2.1681205f-6 2.4398596f-5 1.1775482f-5 9.9300596f-5 0.0001105504 9.565299f-6 -4.7076144f-5 9.590853f-5 -4.885141f-5 0.000120821234 -0.00022250146 -9.05917f-5 -3.2679833f-5; 0.00012063581 -6.609236f-5 -8.302315f-5 -1.954174f-5 -6.6735214f-5 -1.8397034f-6 -2.99155f-5 0.00013611194 -0.00010268538 7.048608f-5 -1.6720745f-5 7.6480974f-5 8.878988f-5 3.9338465f-5 -2.3043342f-5 1.4160479f-5 -3.0309327f-5 -4.2313906f-5 -5.78722f-5 -1.2361765f-5 0.00011966063 -8.0163416f-5 3.7680657f-5 -3.4606725f-5 -9.543387f-5 6.8428395f-5 1.155613f-5 -0.000115886956 -6.523307f-5 5.2380037f-5 0.00024232482 7.973879f-6; 8.224085f-5 -1.21905605f-5 2.7365168f-5 4.015555f-5 -4.8491387f-5 5.793325f-5 7.1500195f-5 7.0172675f-5 -8.642905f-5 1.9702306f-5 4.9804526f-5 9.729075f-5 9.6269556f-5 -9.517981f-5 -0.000108945256 -6.5701315f-5 -0.00014163546 0.00020532864 6.4517335f-5 -0.00010703346 2.3381364f-5 4.1316358f-5 -9.1240385f-5 2.8189028f-5 3.4416404f-5 8.883044f-5 -9.182712f-5 0.00014107842 5.8567588f-5 8.330891f-5 8.732947f-5 2.7254235f-5; 2.0640122f-5 -6.5564316f-5 -9.267124f-5 1.2782187f-5 -2.1939577f-5 4.8729362f-5 -0.0002059266 0.00019216529 -0.00010824198 -0.000106641404 -0.00011950945 4.0736777f-5 3.8151837f-5 1.5811183f-5 -5.0110833f-5 -1.12530515f-5 -9.231862f-5 0.00014269506 -9.049237f-5 0.0001928238 -8.382982f-5 -3.1066495f-6 1.396399f-5 3.5595588f-6 -0.0001678254 -5.0562965f-5 5.644873f-5 0.0001424172 0.00012270945 1.297231f-5 6.82423f-5 1.0279729f-5; -1.3927525f-5 -2.5807853f-5 -3.353609f-5 -0.0001509003 -5.633148f-5 2.3138953f-5 -9.295884f-5 -1.6964543f-5 8.622773f-6 -2.287433f-5 -9.8104494f-5 0.00019548782 -1.4208715f-5 4.2834552f-5 -0.00014059727 0.00012722421 7.491649f-5 -0.00013340326 -7.4468226f-5 6.701388f-7 7.389381f-6 7.241207f-6 0.0001258489 2.1600781f-5 -6.4029395f-5 9.2974806f-5 -0.00013893936 -2.0481652f-6 9.337237f-5 1.4617574f-6 -4.278323f-5 -4.6443496f-5; -9.862447f-6 -8.921982f-5 9.193483f-5 4.696958f-5 -7.964992f-6 -0.00015717202 1.1457437f-5 4.2228417f-5 1.0273722f-5 -0.0001314355 -6.983743f-5 3.2823362f-5 -2.2050655f-5 4.167267f-5 8.6878565f-5 0.00012557801 9.7389f-5 5.7838948f-5 0.00013859413 -5.4861524f-5 9.106602f-5 0.00010086939 0.00014309268 -1.2072048f-5 -5.2197997f-6 4.112339f-5 0.00015406324 -3.9871407f-5 5.1895193f-5 8.516548f-5 6.1933075f-5 -9.757812f-5; 0.000100224905 3.8019178f-5 -0.00011406402 0.00020252966 0.00012403002 -7.962413f-6 -7.4583564f-5 6.3843974f-5 0.00011404545 0.000169954 8.429819f-5 4.996831f-5 0.00011434548 0.00016386562 -6.057491f-5 -9.412545f-5 -0.00015873568 -6.482431f-5 3.091892f-5 -2.159468f-5 0.00012109969 -7.771308f-5 -1.8478474f-6 -9.604292f-6 -0.00010614015 -0.00013608659 -3.699296f-5 -9.4771065f-5 -0.000106533116 -1.4093011f-5 -0.00010621838 -1.599139f-5; -6.859555f-5 5.9557868f-5 0.00013538415 -5.856615f-5 -1.4068297f-6 -9.1875416f-5 4.9636124f-5 -2.6894352f-6 0.00010794412 3.837797f-5 0.00023890898 -0.00011218314 3.951066f-5 7.6334916f-5 -6.472669f-5 2.6779146f-5 1.4812273f-5 -8.1917766f-5 1.3783435f-5 -0.00013009476 0.00010798479 9.0076355f-5 -5.1770206f-5 -3.248121f-5 1.4516136f-5 7.808087f-5 -4.9713708f-5 -4.637024f-5 0.00012679571 7.44449f-5 -0.00014616112 -9.48301f-5; -5.843995f-5 -7.837018f-5 -0.00015722003 0.00013298752 0.0002058001 0.0001131393 -6.0574115f-5 6.8372574f-6 8.129709f-5 5.0710118f-5 0.00025877086 -0.00015811148 -2.5584899f-5 -0.00010511612 -0.00012036738 9.864298f-5 -4.5218094f-6 -0.00011798466 2.9166674f-6 0.00014341663 -5.8281497f-5 4.9651182f-5 -6.645968f-5 -0.0001173513 2.0992076f-5 -1.2117198f-5 -0.00013775482 -5.2643958f-5 -8.244777f-6 -6.679681f-5 -0.000112732036 2.2170127f-5; 1.8069002f-5 -8.4170046f-5 4.027615f-5 -9.122345f-5 1.6444257f-5 8.069179f-6 -4.988504f-5 0.00018525976 7.757772f-5 -0.000103358674 -5.7851914f-5 3.672725f-5 0.00014628147 0.00011684927 0.00014759402 0.00010566242 0.0001684325 7.3146985f-5 -0.00029925458 -9.248588f-5 -0.00010037117 2.386982f-5 -8.735626f-5 -0.00014760795 8.600574f-5 -0.00010550478 -1.5243796f-5 -2.6065078f-5 0.00014419388 9.088046f-5 4.867762f-5 4.917452f-5; -6.598519f-5 -6.0307575f-5 -7.217373f-5 -7.9992154f-5 -0.00020351214 8.730133f-5 3.311596f-5 -7.431629f-5 -6.286525f-6 -4.801173f-5 0.0001289165 4.1723186f-5 -8.892634f-5 0.00024919503 4.605369f-5 8.194185f-5 -4.1749663f-5 -4.7821406f-5 -7.79361f-5 -4.722093f-5 -5.6371475f-5 -2.9041228f-5 -4.2099673f-5 -4.402712f-5 -2.151333f-5 0.00010635854 -0.00018420393 7.879363f-5 6.56917f-5 -0.00014872105 -0.00025311403 0.00016434335; -3.2588643f-5 -6.82938f-5 -8.62755f-5 -2.488788f-5 -0.00010869599 -1.875883f-5 -5.000146f-5 0.00010093272 -0.00017555879 8.3838204f-5 -0.00019008624 -2.0509233f-5 9.822452f-5 -0.00014449273 -4.487349f-5 -0.0001129706 5.2817784f-5 0.00010259473 1.2052635f-5 0.0001414271 1.6518059f-5 0.000114337774 -9.06657f-5 3.4899254f-5 2.4568328f-5 3.0560605f-5 0.000109304965 -4.3656746f-5 5.3483916f-5 5.271978f-5 -6.0059614f-5 6.0993232f-5; 4.6849014f-5 0.00015922435 3.7304446f-5 0.00010522433 -1.7149052f-5 9.699794f-5 8.679175f-5 -5.980967f-5 0.000102687576 -0.00011774394 -9.3231676f-5 -7.073589f-5 9.4761046f-5 -2.0640855f-5 -0.00023659732 -0.00015139587 -7.254155f-5 0.00018710685 -7.661277f-5 -6.958926f-5 -0.00023375152 0.000113982176 6.118777f-5 -0.00014834842 -2.048975f-5 2.8104418f-5 2.5483473f-6 6.996796f-5 -9.3646326f-5 -0.00014223685 -0.00017213685 -0.00018663933; -0.00011808671 4.8037993f-5 1.6431522f-5 -0.00015378026 7.1910195f-5 -0.0001872632 -5.120849f-5 -0.00010370612 0.00029613348 3.4528523f-5 -7.646572f-5 5.0855837f-5 9.754051f-5 6.584373f-5 -4.716242f-5 -2.1120639f-5 -0.00011250277 8.575611f-5 -1.6247295f-5 -0.00027942262 1.3625263f-6 8.212421f-5 6.171591f-5 6.960599f-5 -1.3251189f-5 0.00010215443 0.0001165171 -2.5964511f-5 4.8122438f-5 4.640835f-5 0.00014826102 -7.550899f-5; -6.171692f-5 -2.2494147f-5 -0.00011052692 0.000108468135 -6.2695945f-5 -9.421945f-5 0.00014297807 3.1365198f-5 8.234304f-5 7.663826f-6 -0.000109010085 -8.211707f-6 -4.7824436f-5 -0.0001506865 -7.138279f-5 0.00012205401 0.00010132722 -2.9440233f-5 8.614374f-5 -3.23056f-5 5.3609398f-5 9.664525f-5 -1.596922f-5 -8.9840825f-5 5.949551f-5 1.0887881f-5 4.2916592f-5 -0.00010351466 0.00012603638 5.6517703f-5 6.545845f-5 0.00019466775; -6.0357146f-5 -3.048198f-5 1.9146099f-5 4.148812f-5 -2.4972449f-5 -9.497651f-5 -5.7457888f-5 0.00010517364 6.8643836f-5 -5.117873f-6 9.27461f-5 1.2731097f-5 2.0752132f-5 -1.23914115f-5 3.9276125f-7 1.6265903f-5 8.1468446f-5 9.052403f-5 0.00018212575 7.8478355f-5 1.3240985f-5 -6.920781f-5 1.9615041f-5 -0.00020374315 3.897066f-5 -0.0001280138 -6.9336143f-6 0.000103044265 0.000104789826 4.8267525f-6 6.81914f-5 -0.00012826166; -2.809343f-5 -3.0831015f-5 2.2828628f-5 8.898851f-5 1.2071551f-5 -3.803355f-5 -1.0437871f-5 5.422938f-5 4.867486f-5 -4.518063f-5 7.922069f-5 -0.00010169225 0.0001434924 -7.677246f-5 0.000111034875 -5.2173058f-5 1.2276159f-6 -2.8245557f-5 9.942706f-5 1.1442849f-5 1.463341f-6 -5.2849533f-5 -4.7457823f-5 -5.7791447f-5 -2.5199992f-5 2.5344176f-5 9.2255f-5 5.7867932f-5 -4.42645f-5 -0.00022660938 -0.00029660418 -9.220186f-5; 6.735818f-5 -4.176811f-5 -1.2232209f-5 -0.0001059559 5.754733f-5 -6.4020445f-5 -0.00014015102 0.00011234368 3.3669865f-5 -0.00012898138 -0.0001243019 7.9176105f-5 0.00021939976 -1.7735352f-6 1.3753903f-5 3.0978037f-5 1.261377f-5 3.2221094f-5 1.1791008f-5 0.00012344854 2.0633179f-5 -0.00019241248 3.0955096f-5 -4.4071912f-5 0.00013447642 0.00015082232 2.4748131f-5 7.190226f-5 5.500368f-5 -2.3531034f-6 0.000106698426 -0.00015557921; 0.00015004253 7.1710674f-5 7.18664f-5 -4.449272f-5 0.00010724826 -3.0034582f-5 -6.1318984f-5 -7.845766f-6 7.125385f-5 -1.1303766f-5 4.3301923f-5 1.4665254f-5 7.303844f-5 0.00012035262 3.3127606f-5 -1.6960375f-5 9.650369f-5 -0.000110265166 1.5607555f-5 -7.302818f-5 2.9097244f-5 -0.00014719421 1.1049523f-5 -2.7170749f-5 6.9879985f-5 -0.00017216924 -5.5424134f-5 0.00015734922 -2.3573297f-5 -9.351214f-6 -4.482439f-5 0.00015532046; 9.200195f-5 -7.894702f-5 6.909601f-5 6.397365f-5 -7.337188f-5 1.810738f-5 -1.3611387f-5 -0.00011573212 6.84041f-5 0.00016948365 -3.591823f-5 -0.00010666039 5.9029382f-5 -1.1825928f-5 5.6072877f-5 4.840527f-5 0.00019966012 -7.507182f-5 3.281313f-5 -0.00026297724 0.00013116715 -6.360724f-5 1.34186175f-5 -5.0054907f-5 -6.6365785f-5 -5.7354497f-5 -0.00027880043 5.8056135f-5 4.771379f-6 0.00023913936 -0.00027277894 0.00013627937; -3.8475675f-5 0.00013499877 -0.00011172844 -1.5792652f-5 1.7460216f-5 8.990932f-5 -0.00013876938 0.00010837003 2.5286476f-5 4.0916253f-5 -8.68343f-5 2.7274678f-5 -0.0002117695 -4.198623f-5 2.5651116f-5 -4.3864326f-5 0.00010112666 8.5957945f-5 -0.00011121263 4.618323f-5 -2.0187681f-5 0.00010101195 -6.718644f-5 0.0001685784 -6.6461784f-5 -8.18679f-5 5.8931157f-5 6.0350863f-5 0.000110743706 2.1969163f-5 8.4774925f-5 2.8408233f-5; 0.00015168458 -0.00015583177 9.526762f-7 3.8325594f-5 3.7966631f-6 -2.4170578f-5 -5.1147792f-5 -1.8475313f-5 2.11474f-6 2.0240308f-5 -0.00019447393 7.808402f-5 -1.9867588f-5 2.6745403f-5 2.3344088f-5 -7.562572f-5 3.0028186f-5 0.00020181146 0.0001235826 -4.559302f-5 0.00010591059 0.00026848647 -3.44617f-5 2.3216366f-5 8.163823f-5 -1.3018025f-5 0.00011558951 -6.941131f-5 -0.00019635934 -8.809247f-5 0.00015325508 -0.00014338388; 1.7577355f-5 -0.00011430754 -2.3156954f-5 -1.96567f-5 -0.000108622946 8.5745494f-5 -6.518311f-6 -4.53642f-5 -5.5961747f-5 0.00012402056 5.957273f-5 0.0001436606 -2.3027369f-5 -2.1112419f-5 1.8754206f-5 6.52711f-5 -6.917144f-5 0.00015894721 8.440349f-5 0.00015213025 4.88502f-7 -7.422043f-5 4.4104203f-5 -5.2935447f-5 3.839207f-5 -0.00026345407 -0.00027999925 -0.00010818485 0.000100323494 0.00013953423 -0.00015231865 9.962925f-5; 3.5571054f-5 -7.7061304f-5 -0.00021089091 0.000117353586 -8.080046f-6 0.00020298584 -7.7734745f-5 0.000111111476 0.00021550554 -0.00011431153 -5.765442f-5 -9.9738056f-5 -0.00012480488 0.00012828007 -7.215347f-5 -9.8592405f-5 -5.0616338f-5 6.370253f-5 -1.0569401f-5 -8.1941704f-5 -9.611313f-5 4.9776292f-5 7.01193f-5 -3.1117284f-5 -0.00011037468 -0.00011528408 -7.007478f-5 -0.0001132747 2.4702704f-5 -0.00013523488 0.000120989585 -0.0002573811; 0.00026173593 1.283452f-5 -1.7065893f-5 -0.000104993436 -1.4819496f-5 0.00016174733 5.1968495f-6 0.00012912843 1.9695099f-5 0.00014575227 6.3173524f-5 -5.2360924f-6 -5.6768888f-5 -0.00014571872 -0.0001785546 5.9773887f-5 -0.00018953496 6.7306304f-5 0.0001568174 7.134748f-5 6.8328976f-5 -3.3585914f-6 -5.0178074f-5 2.9536714f-5 1.9834733f-5 4.025718f-5 -0.00021762686 9.823387f-5 2.9596104f-5 -5.052526f-5 0.00011400497 0.0001491102; 3.034911f-5 0.00011196634 6.7014895f-5 -0.00013202667 -4.6789377f-5 8.572312f-7 9.169984f-5 -2.3223301f-5 -0.00010554397 8.7856f-5 5.0881303f-5 -9.156455f-5 -6.564096f-5 3.4504257f-5 -0.00016190331 -6.2145766f-5 1.9165122f-5 -7.877776f-5 -4.080841f-5 2.136334f-5 -7.858364f-5 -8.119439f-6 0.00013435273 4.4416214f-5 -0.00012370772 0.000101974394 -4.802835f-6 -0.00010022334 1.2407417f-5 -0.00015726443 -0.00010571935 -3.07934f-5; 8.026358f-5 -2.2175438f-5 -0.00012479624 -4.0228583f-6 -0.0002115195 2.9970723f-5 -4.1155407f-5 0.00014840579 5.4885195f-6 0.00029171017 0.00013645006 -0.00017074464 0.00018228586 -0.00011459783 3.7666694f-5 -1.6101341f-5 0.0001842996 -0.0001169029 -1.4438652f-5 -6.3499065f-5 -9.643319f-5 -0.00017683755 5.7190664f-5 -3.922391f-5 -3.1976626f-5 -2.109656f-5 -3.90291f-6 -0.00013517434 0.00016720014 -0.00013073278 8.5466956f-5 -1.526848f-5; -4.2852924f-5 -0.00015807984 -8.3653595f-6 2.5182915f-6 -0.00013412097 -9.557661f-5 8.1717204f-5 -5.246506f-5 -1.7424688f-6 9.216007f-5 -7.319666f-5 -0.00027948042 0.00015722163 3.1111696f-5 -0.00020530884 -5.4546465f-5 -6.29937f-6 -0.0001428365 3.9120237f-5 3.721716f-5 -6.6093584f-5 -1.8950981f-6 -9.6505835f-5 2.1517824f-5 5.098992f-5 3.5149867f-5 0.0001302061 0.00012702291 -7.683339f-5 0.00011238969 2.6854445f-6 4.7272497f-5; -4.8100424f-5 4.516353f-5 0.000111997026 5.204853f-6 -6.4593376f-5 0.00010572687 3.921466f-6 6.905398f-6 -0.00015121218 1.7533444f-5 -3.0009418f-5 -3.6821777f-5 -0.0001382609 -0.00013444631 -0.000118799144 -0.00011516807 -2.658559f-5 -1.0310642f-5 -0.00013526081 8.310802f-6 1.39192125f-5 4.0677514f-5 5.7376667f-5 -4.978068f-5 0.00015903164 3.915949f-8 -1.5798567f-5 5.3141244f-5 -0.00014180606 -0.0001386008 0.00013365413 0.00013656895; 0.00017168031 -2.2275335f-5 -9.818207f-5 -6.38929f-5 -9.4072755f-5 -3.0191156f-6 8.873516f-5 6.795865f-5 -0.000121237004 7.997962f-5 0.00013511072 0.00013809897 -2.5103287f-5 0.00012684942 -0.0001443395 -9.857521f-5 0.00017308586 4.2674477f-5 -1.6791339f-5 -8.397967f-5 1.0641746f-5 -5.0464263f-5 -0.0001410602 -2.9612102f-5 1.5865919f-5 6.131211f-5 -1.00230445f-5 -2.2032244f-5 1.0033375f-5 -0.00012724352 -1.5285226f-5 0.00014202898; -1.4121472f-5 -5.0044982f-5 7.915148f-5 0.00010705557 -4.692474f-5 4.8653448f-5 -6.4731226f-5 -7.822046f-5 -1.0700496f-5 2.0036923f-6 0.00012267969 3.603697f-7 1.9522387f-5 -0.00010675869 -0.00013979523 -4.262446f-5 -6.636593f-5 -2.5301544f-5 0.00017978139 8.460204f-5 -9.998411f-5 -0.0001354939 -9.499201f-6 -4.361895f-5 -0.00013717679 4.524789f-5 0.0001629717 7.41474f-5 -0.0001341515 0.00012045276 -8.077927f-5 1.8191115f-6; 8.464025f-5 -0.0001911386 -0.00013323688 -5.179797f-5 3.0535775f-5 -3.6069952f-5 5.7020614f-5 -6.241533f-5 -3.7568614f-5 -9.843058f-5 0.00024355052 -0.00013379584 -6.849716f-5 1.1521302f-6 -1.6933034f-5 6.922868f-5 0.00015571411 -8.287445f-5 4.693961f-5 0.00018321241 7.1558397f-6 -0.00023000097 -2.3251116f-6 9.687749f-5 8.636353f-5 8.079613f-5 6.684031f-5 4.8694303f-5 -7.006521f-6 -5.833168f-7 -4.4962962f-5 -6.118613f-6], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-0.00010431886 7.263726f-5 0.0001153885 2.3408782f-6 -0.00014803729 0.00017549854 -1.4270489f-5 9.409741f-5 -2.1240825f-5 0.00027308802 -2.6860275f-6 1.595533f-5 -8.200385f-5 0.00014178251 -1.6477046f-5 -9.237305f-5 -0.00011124783 1.087667f-5 0.00022355978 2.2964196f-5 -9.138082f-5 -3.8097627f-5 -7.44995f-5 -0.00022302984 -9.659961f-5 -0.00034645712 9.354396f-5 6.146441f-5 -9.705188f-5 -4.282779f-5 8.976634f-5 -4.020095f-5; 6.706114f-5 0.0002107104 1.2080838f-5 6.554694f-5 6.5647335f-5 3.832331f-5 -0.00016438964 -2.266784f-5 -7.713031f-5 -0.00018027052 6.2050116f-5 -5.3583874f-5 0.00016199157 -2.413137f-5 1.9695848f-5 0.000119893564 2.6232174f-5 -2.5299496f-5 7.291109f-5 -0.0002123351 0.0001043167 -3.4432705f-5 -9.066305f-5 -0.00017630673 -5.5120025f-5 0.00013201217 -0.00011529041 0.00013865373 5.0689f-5 8.1196495f-6 -0.00010693036 9.6504766f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
+
+const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
+    Chain(
+        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
+        layer_2 = Dense(1 => 32, cos),  # 64 parameters
+        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
+        layer_4 = Dense(32 => 2),       # 66 parameters
+    ),
+)         # Total: 1_186 parameters,
+          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",m,[(a(),i("svg",V,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",T,[(a(),i("svg",U,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=t("where, ")),A("mjx-container",q,[(a(),i("svg",b,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=t(", ")),A("mjx-container",D,[(a(),i("svg",z,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=t(", and ")),A("mjx-container",R,[(a(),i("svg",K,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=t(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
+    χ, ϕ = u
+    p, M, e = ode_model_params
+
+    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
+    # it, however, in general, we should use \`st\` to store the state of the neural network.
+    y = 1 .+ nn_model([first(u)], nn_params)
+
+    numer = (1 + e * cos(χ))^2
+    denom = M * (p^(3 / 2))
+
+    χ̇ = (numer / denom) * y[1]
+    ϕ̇ = (numer / denom) * y[2]
+
+    return [χ̇, ϕ̇]
+end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
+waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
+
+    axislegend(ax, [[l1, s1], [l2, s2]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
+
+    fig
+end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
+
+function loss(θ)
+    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
+    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
+    return mseloss(pred_waveform, waveform)
+end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0007098617733391764
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
+
+function callback(θ, l)
+    push!(losses, l)
+    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
+    return false
+end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
+optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
+optprob = Optimization.OptimizationProblem(optf, params)
+res = Optimization.solve(
+    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
+    callback, maxiters=1000)
    retcode: Success
+u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [-5.067828169553457e-5; 0.0002033883647527833; 1.4273280612531504e-5; -4.428054671737328e-5; -4.250684651193247e-5; 7.748175266906924e-5; 6.288575150393261e-6; 3.7066754884979796e-5; 0.00014335215382743272; -0.00011426929268051497; 0.00011086220183643978; 8.886728755885818e-5; -0.0002350791619390836; -0.00026994064683101955; -2.1895052668672963e-6; -4.137206633454982e-5; 0.00014377594925456472; -1.254786911885069e-5; 0.0001540251396362168; -6.948348891458086e-5; 9.75148054747745e-7; -2.275944098073367e-5; 5.759294799648488e-5; -9.247271009369957e-6; -7.65327495172892e-5; -5.430622331914719e-5; -5.6007382227096824e-5; 4.515377077039578e-5; -9.931653767119681e-5; 0.00011081036791418321; -2.595081787143777e-5; -2.3206786863723692e-5;;], bias = [-5.310483371894584e-17, -3.1011765244248956e-17, 1.2287450168839055e-17, 3.141598635647141e-17, -4.114087861200215e-17, -1.087215296099596e-16, -5.37164552124023e-19, 4.2209422355381365e-17, 1.3438114389553043e-18, -3.747220297384205e-17, -1.5831630774748768e-17, 1.2461438073596785e-16, -1.4656832046366923e-16, 8.680182063003482e-17, -2.656384109725839e-18, -4.3500024256058437e-17, 7.393138650556651e-17, -1.2508691195987064e-17, 1.6063779055314245e-16, 2.5819551063395286e-17, 1.742596203254984e-18, -1.739224497562247e-18, -3.550440848984027e-17, -1.8956723722780767e-18, -7.511083048774223e-17, 1.7363671075209743e-18, -1.260939208188153e-17, 1.8950031747943242e-17, -6.621050923994785e-17, 2.2468062521676514e-16, -5.786119776188296e-17, -1.7438878875030686e-17]), layer_3 = (weight = [-6.488304392744379e-5 5.1593381391560795e-5 -0.00010973868961057376 0.00011888554104759725 -7.869967270719461e-5 2.6729300549302098e-5 -4.338042821812204e-5 -0.00014315193879495047 2.6661781481641573e-5 -0.0001733976871434768 -2.6336444756196498e-5 9.159599244983697e-5 -0.00015508627006391803 -4.3109380607051135e-5 1.6835862658147272e-5 1.792073158528541e-5 -7.001163361902599e-5 -4.674345222495249e-5 3.036351404834028e-5 -2.1701625825915297e-6 2.439655395962534e-5 1.1773439615602058e-5 9.929855401085215e-5 0.00011054835591155347 9.563256531943786e-6 -4.707818629296995e-5 9.590648801920449e-5 -4.885345082923397e-5 0.0001208191922671903 -0.0002225035034313972 -9.059373882676194e-5 -3.268187469414162e-5; 0.00012063635157848382 -6.609181646762502e-5 -8.302260603794181e-5 -1.9541194621509316e-5 -6.673466915070392e-5 -1.8391583189296476e-6 -2.9914954854560117e-5 0.0001361124807708773 -0.00010268483375237967 7.048662454782473e-5 -1.6720200353106822e-5 7.6481518856795e-5 8.879042369972584e-5 3.933901033906554e-5 -2.3042796517059236e-5 1.4161023766989025e-5 -3.0308782431336086e-5 -4.231336125211143e-5 -5.7871654195867056e-5 -1.2361219621183299e-5 0.00011966117680732812 -8.016287093322083e-5 3.7681201586580233e-5 -3.460617945800818e-5 -9.54333233557773e-5 6.842894007878977e-5 1.1556674774525573e-5 -0.00011588641128308456 -6.523252587338435e-5 5.238058228589095e-5 0.00024232536751971285 7.974424345790755e-6; 8.224337639854286e-5 -1.218803148938489e-5 2.7367696618746988e-5 4.015808068084576e-5 -4.848885813975649e-5 5.793578080352111e-5 7.150272418380917e-5 7.017520388922748e-5 -8.642651968535577e-5 1.970483450145065e-5 4.980705549109859e-5 9.72932799938929e-5 9.627208479078838e-5 -9.517727752818883e-5 -0.00010894272722202147 -6.569878618596989e-5 -0.00014163293539446397 0.00020533117336322745 6.451986397017856e-5 -0.00010703093297911067 2.3383893365242713e-5 4.1318886958613766e-5 -9.12378557970847e-5 2.819155693737988e-5 3.441893264403294e-5 8.88329711031072e-5 -9.182458901909231e-5 0.000141080946066501 5.8570116978216505e-5 8.331143604459376e-5 8.733199925496392e-5 2.7256763730982073e-5; 2.0639607059931383e-5 -6.556483098836061e-5 -9.26717530500157e-5 1.2781671971535676e-5 -2.19400917966395e-5 4.872884706412406e-5 -0.00020592711193319547 0.00019216477262955044 -0.0001082424915192628 -0.00010664191908733837 -0.00011950996672101251 4.0736262194625994e-5 3.815132181910655e-5 1.5810668587742027e-5 -5.011134821404839e-5 -1.1253566340090383e-5 -9.231913831615613e-5 0.0001426945472804816 -9.049288750691079e-5 0.00019282329044938032 -8.383033677535268e-5 -3.1071643160232267e-6 1.3963474843987855e-5 3.5590439484819587e-6 -0.00016782591960225285 -5.056347985817206e-5 5.644821608723057e-5 0.00014241667845919624 0.00012270893235392902 1.297179547248012e-5 6.824178304225082e-5 1.0279214580006563e-5; -1.3928808630423847e-5 -2.580913592300915e-5 -3.3537372519475696e-5 -0.00015090158835866352 -5.633276218779253e-5 2.3137669768206132e-5 -9.296012563046942e-5 -1.6965826080393775e-5 8.621489496172741e-6 -2.2875613513145147e-5 -9.810577744266379e-5 0.0001954865410423597 -1.4209998016843574e-5 4.283326866226591e-5 -0.00014059855589059123 0.00012722292462947907 7.49152067781239e-5 -0.00013340454011105866 -7.446950899084912e-5 6.688554319983476e-7 7.388097810490849e-6 7.2399238431729244e-6 0.0001258476231212468 2.1599497767766602e-5 -6.403067804987242e-5 9.297352239608012e-5 -0.0001389406416367069 -2.049448585437825e-6 9.337108799607962e-5 1.4604740497695822e-6 -4.2784514380599464e-5 -4.6444779445863475e-5; -9.859231075038756e-6 -8.92166064731826e-5 9.193804707326044e-5 4.697279636463018e-5 -7.961775925591352e-6 -0.00015716880462417928 1.1460652596785891e-5 4.223163322609779e-5 1.0276937966449545e-5 -0.00013143228735129704 -6.983421208855968e-5 3.282657761888761e-5 -2.204743960464444e-5 4.16758846695159e-5 8.68817804003514e-5 0.00012558122379420223 9.739221404019534e-5 5.784216390754152e-5 0.00013859734444125957 -5.485830862011259e-5 9.106923225050862e-5 0.00010087260288134235 0.00014309589441114764 -1.206883208208888e-5 -5.216583947857849e-6 4.1126604439399654e-5 0.00015406645233457655 -3.986819156847103e-5 5.189840866665336e-5 8.516869255986725e-5 6.19362906873599e-5 -9.757490643817917e-5; 0.00010022519278392329 3.801946592103462e-5 -0.00011406373518001133 0.00020252994441945248 0.000124030307108128 -7.962125107938336e-6 -7.458327594393708e-5 6.384426238766085e-5 0.00011404573602886837 0.00016995428892370348 8.429847808839796e-5 4.9968596487828436e-5 0.00011434576741704361 0.0001638659131113016 -6.057462009083322e-5 -9.412515931943289e-5 -0.00015873539372295175 -6.482401944411764e-5 3.0919208131320155e-5 -2.1594392035681557e-5 0.0001210999806626343 -7.771278900507687e-5 -1.8475592587357495e-6 -9.604004060733888e-6 -0.000106139860816243 -0.00013608629809640216 -3.699267030059231e-5 -9.477077686641228e-5 -0.00010653282801102781 -1.4092722392875731e-5 -0.00010621809191251024 -1.5991102498432955e-5; -6.85945405234782e-5 5.955887999559289e-5 0.00013538516313245464 -5.856513863138032e-5 -1.4058173587873332e-6 -9.187440346751602e-5 4.963713681034698e-5 -2.688422825954933e-6 0.00010794513317930726 3.837898211029819e-5 0.0002389099909964718 -0.00011218212503765081 3.951167408783001e-5 7.633592858151902e-5 -6.472568102937236e-5 2.6780157979814266e-5 1.481328499037518e-5 -8.19167534314287e-5 1.378444730332931e-5 -0.00013009375007415393 0.0001079858057823773 9.007736761530801e-5 -5.176919413089905e-5 -3.248019757621968e-5 1.4517148058514547e-5 7.808188192255654e-5 -4.971269564295335e-5 -4.636922941018939e-5 0.000126796724628076 7.444591127976329e-5 -0.00014616011222416605 -9.482908981910036e-5; -5.844121978835604e-5 -7.837145376351397e-5 -0.00015722129746674127 0.00013298625281212424 0.00020579882702922013 0.00011313802979320028 -6.057538548585531e-5 6.835987097737537e-6 8.129581992879848e-5 5.070884735805013e-5 0.0002587695883460528 -0.00015811274779362782 -2.55861693391912e-5 -0.00010511739374003677 -0.00012036865218643604 9.864171015696166e-5 -4.523079677439581e-6 -0.00011798592886290081 2.915397136620633e-6 0.0001434153630737371 -5.82827676214142e-5 4.964991176283433e-5 -6.646095125571264e-5 -0.0001173525713045014 2.099080602537575e-5 -1.2118468201797582e-5 -0.0001377560881325826 -5.26452283326858e-5 -8.246047126437611e-6 -6.679808275176214e-5 -0.00011273330592185735 2.2168856322686e-5; 1.807031263771555e-5 -8.41687352815843e-5 4.027746114181075e-5 -9.122214318020622e-5 1.6445567655606466e-5 8.070489332307503e-6 -4.988372814958256e-5 0.0001852610738412444 7.757903300690267e-5 -0.0001033573636389419 -5.785060342285634e-5 3.672856188378627e-5 0.00014628277937180577 0.00011685057767197896 0.00014759533302159713 0.00010566372735658319 0.00016843380659584684 7.314829496200851e-5 -0.0002992532673827499 -9.248457275797153e-5 -0.00010036986271851036 2.3871129872274463e-5 -8.735494988039545e-5 -0.00014760663829045643 8.600705119780681e-5 -0.00010550347282085964 -1.5242485585335465e-5 -2.606376723386703e-5 0.00014419519071699367 9.088176817632428e-5 4.8678929363374415e-5 4.9175829974698705e-5; -6.598751317803042e-5 -6.030990155271338e-5 -7.217605709890307e-5 -7.999448100896822e-5 -0.00020351446985593347 8.729900034517088e-5 3.311363431997184e-5 -7.431861561350837e-5 -6.288851661567153e-6 -4.8014057053713016e-5 0.0001289141673375904 4.1720859346913115e-5 -8.892866574317183e-5 0.00024919270251556733 4.605136379971683e-5 8.19395226906025e-5 -4.1751989581145076e-5 -4.782373255445095e-5 -7.793842635442057e-5 -4.7223255048506476e-5 -5.63738012963482e-5 -2.9043554629668638e-5 -4.210199952218848e-5 -4.4029447987912596e-5 -2.1515656069153655e-5 0.00010635621035611067 -0.00018420626113517394 7.879130314243693e-5 6.568936985126093e-5 -0.00014872338035935024 -0.00025311636093398925 0.000164341023239769; -3.258934873928717e-5 -6.829450274425075e-5 -8.627620252854646e-5 -2.488858615878919e-5 -0.000108696694842814 -1.8759535924339176e-5 -5.000216686357272e-5 0.00010093201137306672 -0.00017555949296410188 8.38374982045109e-5 -0.00019008694642297169 -2.05099385134345e-5 9.822381263758857e-5 -0.00014449343860221752 -4.487419562837569e-5 -0.00011297130538923156 5.281707864879642e-5 0.0001025940219910863 1.2051929042638165e-5 0.0001414263898545934 1.6517353396471644e-5 0.00011433706833441294 -9.066640621357982e-5 3.489854872851174e-5 2.456762205793661e-5 3.055989884115363e-5 0.0001093042593488524 -4.3657451788470566e-5 5.348321075826649e-5 5.2719075137710264e-5 -6.0060319702182694e-5 6.099252652302867e-5; 4.684605982778897e-5 0.00015922139527554951 3.730149137131014e-5 0.00010522137699458063 -1.7152006424891324e-5 9.699498286513317e-5 8.678879339289912e-5 -5.9812624844068647e-5 0.00010268462164837113 -0.00011774689528815062 -9.323463064342426e-5 -7.073884671976903e-5 9.475809156010668e-5 -2.0643809424456035e-5 -0.0002366002740962891 -0.00015139882498611479 -7.254450656895337e-5 0.00018710389803904177 -7.661572499622105e-5 -6.959221400743762e-5 -0.00023375447245039922 0.00011397922162008577 6.118481496391263e-5 -0.00014835137745090697 -2.0492704155778817e-5 2.8101463154730067e-5 2.545392813903837e-6 6.996500405525111e-5 -9.364928019190132e-5 -0.00014223980588562476 -0.00017213980072834412 -0.00018664228310068032; -0.00011808607541993425 4.8038628844860654e-5 1.6432158188171453e-5 -0.0001537796242725119 7.191083113625497e-5 -0.0001872625625789189 -5.120785590647832e-5 -0.00010370548716550833 0.0002961341187959484 3.4529158253777705e-5 -7.646508766143015e-5 5.085647262362489e-5 9.7541148531797e-5 6.58443632322203e-5 -4.7161783188722274e-5 -2.1120003114129957e-5 -0.00011250213447297528 8.57567456454027e-5 -1.6246659472613986e-5 -0.0002794219808517045 1.363162025290119e-6 8.212484226291877e-5 6.171654513068318e-5 6.960662267133479e-5 -1.3250552913316052e-5 0.0001021550660856051 0.000116517733656427 -2.596387548933425e-5 4.812307360893733e-5 4.640898532429348e-5 0.00014826165988538294 -7.550835744282355e-5; -6.17154759238133e-5 -2.249270200262119e-5 -0.00011052547360301221 0.00010846958012556371 -6.269449967654701e-5 -9.421800804554029e-5 0.0001429795125732551 3.136664291381576e-5 8.234448306032229e-5 7.665271012258093e-6 -0.00010900864016727386 -8.210261868921642e-6 -4.782299164798104e-5 -0.0001506850488949556 -7.13813455076535e-5 0.0001220554519095803 0.00010132866428469075 -2.9438787707441644e-5 8.61451813834961e-5 -3.230415622234766e-5 5.361084241307146e-5 9.664669469933737e-5 -1.5967776009313548e-5 -8.983938040906761e-5 5.9496953879748424e-5 1.088932604765186e-5 4.291803689560512e-5 -0.0001035132185206405 0.00012603782361842033 5.6519148172840214e-5 6.545989582085756e-5 0.00019466919603295402; -6.035587200219031e-5 -3.048070636485676e-5 1.9147372703658136e-5 4.1489395634045756e-5 -2.497117481740846e-5 -9.497523485239959e-5 -5.7456613895707984e-5 0.00010517491403246128 6.864511026744694e-5 -5.116598977995619e-6 9.274737470061827e-5 1.2732370968505513e-5 2.075340574175034e-5 -1.2390137336297228e-5 3.9403538861096767e-7 1.626717760612538e-5 8.14697204341642e-5 9.05253045212093e-5 0.0001821270206428582 7.847962927283407e-5 1.3242259164174353e-5 -6.920653601898014e-5 1.9616315447848574e-5 -0.00020374188086040435 3.897193248943069e-5 -0.00012801252075493572 -6.932340121209963e-6 0.0001030455395530517 0.00010479109999235196 4.828026639213363e-6 6.819267667109008e-5 -0.00012826038352701946; -2.8094946705911407e-5 -3.083253303418634e-5 2.28271104925644e-5 8.898699471965577e-5 1.207003371301013e-5 -3.803506896010253e-5 -1.0439388993198004e-5 5.42278607236792e-5 4.867334374931775e-5 -4.518214753706244e-5 7.921917441772954e-5 -0.00010169376454077221 0.00014349088761731043 -7.677398078600014e-5 0.00011103335729340141 -5.2174575634941974e-5 1.2260983704556828e-6 -2.8247074246699347e-5 9.942554189830378e-5 1.1441331857990563e-5 1.461823480987718e-6 -5.285105051816366e-5 -4.745934037136632e-5 -5.7792964603229816e-5 -2.5201509365519844e-5 2.534265824133726e-5 9.225348495884073e-5 5.786641433142276e-5 -4.426601716194696e-5 -0.00022661089732072592 -0.0002966056968806676 -9.220337650267332e-5; 6.735962678692488e-5 -4.1766666167990785e-5 -1.2230763882255954e-5 -0.00010595445483443115 5.75487764716636e-5 -6.401900018784194e-5 -0.0001401495784240279 0.00011234512648559416 3.367131002493477e-5 -0.00012897993574836596 -0.00012430046181646445 7.917755001005653e-5 0.00021940120941039688 -1.7720901543366787e-6 1.3755347840630702e-5 3.097948219408506e-5 1.2615215286920207e-5 3.2222539534702144e-5 1.179245272839102e-5 0.0001234499840346639 2.063462388366766e-5 -0.00019241103573688025 3.0956541099726786e-5 -4.40704671792656e-5 0.00013447786195217666 0.00015082376639372836 2.4749576273166298e-5 7.190370242358394e-5 5.500512351761415e-5 -2.351658263368668e-6 0.00010669987104381709 -0.00015557776633170152; 0.00015004433877961992 7.171248505085587e-5 7.186821237169626e-5 -4.4490909958627636e-5 0.00010725007375199672 -3.0032770909973542e-5 -6.131717312186528e-5 -7.843955090580263e-6 7.125566405206153e-5 -1.1301954900639649e-5 4.3303734100979687e-5 1.4667064688319504e-5 7.304025272798725e-5 0.0001203544299370227 3.312941696108749e-5 -1.6958564474267366e-5 9.650549979609595e-5 -0.00011026335489973004 1.5609365773470854e-5 -7.302637171037346e-5 2.9099054854809883e-5 -0.0001471923977349845 1.1051333940625235e-5 -2.716893762212287e-5 6.98817959074871e-5 -0.0001721674278358903 -5.542232323009273e-5 0.00015735103003876595 -2.357148589894786e-5 -9.349403471653878e-6 -4.482258084849063e-5 0.00015532227477720406; 9.200155002588563e-5 -7.894742336531445e-5 6.909560581959909e-5 6.397325162368042e-5 -7.337227905111243e-5 1.81069772469319e-5 -1.3611788720594416e-5 -0.00011573251865513483 6.840369863028096e-5 0.00016948325176315674 -3.591863239591659e-5 -0.00010666078921815155 5.9028980128354384e-5 -1.1826329658209398e-5 5.607247478735757e-5 4.8404866926718095e-5 0.00019965971844328957 -7.507221923617522e-5 3.281272622814832e-5 -0.00026297763722425135 0.00013116674929449456 -6.360763834265864e-5 1.3418215555135889e-5 -5.0055308723243565e-5 -6.63661867048385e-5 -5.73548986477575e-5 -0.00027880083687079906 5.80557334859591e-5 4.770976854049545e-6 0.00023913896259444115 -0.00027277934166079255 0.00013627896808145697; -3.847454513714503e-5 0.00013499990222799242 -0.00011172731115191943 -1.5791521720241938e-5 1.746134534943557e-5 8.991045092290287e-5 -0.00013876824755396457 0.00010837116337672065 2.5287605465845612e-5 4.091738287416977e-5 -8.683316787181636e-5 2.7275808177650394e-5 -0.0002117683668545441 -4.1985100098565496e-5 2.5652245538656208e-5 -4.386319574376392e-5 0.00010112778750072644 8.595907479349758e-5 -0.00011121149668878578 4.6184361279588816e-5 -2.018655118416289e-5 0.00010101308202894625 -6.718531017366678e-5 0.00016857952345919098 -6.646065389912549e-5 -8.186676655139642e-5 5.893228649103207e-5 6.0351993168584786e-5 0.00011074483540898065 2.197029266272924e-5 8.477605501713446e-5 2.8409362356176015e-5; 0.00015168579624614677 -0.00015583055870494473 9.538919617630628e-7 3.832680953664729e-5 3.7978788815051777e-6 -2.4169362640261138e-5 -5.114657620551897e-5 -1.847409772281022e-5 2.1159557864773197e-6 2.0241523762792873e-5 -0.00019447271848448938 7.808523581943265e-5 -1.986637222118892e-5 2.674661913358162e-5 2.334530358084579e-5 -7.56245056062654e-5 3.0029402060874113e-5 0.00020181267676989644 0.0001235838214971321 -4.559180457263342e-5 0.00010591180829678763 0.00026848768970112405 -3.4460484365718736e-5 2.3217581420886376e-5 8.163944652651308e-5 -1.3016809370038249e-5 0.00011559072367263004 -6.941009565607624e-5 -0.0001963581228291365 -8.809125507369087e-5 0.0001532562971452538 -0.00014338266124368565; 1.757703004370722e-5 -0.0001143078671567958 -2.315727859711487e-5 -1.965702457271244e-5 -0.00010862327061220267 8.574516952178291e-5 -6.518635550652207e-6 -4.5364523507647356e-5 -5.5962072188025316e-5 0.00012402023541183133 5.957240402024154e-5 0.00014366027145554176 -2.3027693792000462e-5 -2.1112743602468684e-5 1.875388162700245e-5 6.527077759058019e-5 -6.91717624114733e-5 0.00015894688377995764 8.440316279559432e-5 0.00015212992267590782 4.881772291117934e-7 -7.422075312677048e-5 4.410387820355415e-5 -5.293577222362373e-5 3.839174579736624e-5 -0.00026345439717731614 -0.0002799995755460038 -0.00010818517792829506 0.000100323169058197 0.0001395339012695561 -0.00015231897585652043 9.96289262864811e-5; 3.5567809000901426e-5 -7.706454964950166e-5 -0.00021089415570622416 0.00011735034019785079 -8.083291894757975e-6 0.00020298259875836352 -7.773799045844129e-5 0.00011110823067702369 0.00021550229536935868 -0.00011431477508174869 -5.7657664391610264e-5 -9.974130119097646e-5 -0.00012480812477309142 0.0001282768203701277 -7.215671251196544e-5 -9.859565073292652e-5 -5.0619583297677294e-5 6.369928412005192e-5 -1.0572646196095261e-5 -8.19449491509571e-5 -9.611637545185293e-5 4.977304667682311e-5 7.011605300341022e-5 -3.112052984669926e-5 -0.00011037792903970437 -0.0001152873232322168 -7.007802235329321e-5 -0.0001132779438457816 2.469945827307626e-5 -0.00013523812270157737 0.0001209863399444701 -0.00025738433799142086; 0.00026173845471244757 1.2837046954176047e-5 -1.7063366256061072e-5 -0.00010499090931947774 -1.4816968912365366e-5 0.00016174985866423866 5.1993764759212034e-6 0.00012913095487643582 1.9697625605151052e-5 0.00014575480089883838 6.317605074623253e-5 -5.233565486875458e-6 -5.676636065581032e-5 -0.000145716190315948 -0.000178552077711665 5.977641412069611e-5 -0.00018953242907405934 6.730883105088676e-5 0.00015681993171932752 7.13500070491646e-5 6.833150327334844e-5 -3.3560644238060075e-6 -5.0175547228302636e-5 2.9539240488211657e-5 1.9837260326726308e-5 4.025970555854449e-5 -0.00021762433389771325 9.82363948355292e-5 2.9598630492229406e-5 -5.052273409778598e-5 0.00011400749862556023 0.00014911272809737047; 3.034694161183524e-5 0.00011196417396575523 6.701272574869138e-5 -0.0001320288436821541 -4.679154592881643e-5 8.550622234496614e-7 9.169767069547579e-5 -2.3225470459100275e-5 -0.00010554614057612856 8.78538332196153e-5 5.0879133750331555e-5 -9.156671952072757e-5 -6.564313010158575e-5 3.4502088210235905e-5 -0.00016190548270092207 -6.214793468675314e-5 1.916295256675445e-5 -7.877992978169433e-5 -4.0810577802704876e-5 2.1361170338108406e-5 -7.858580723254803e-5 -8.121608362079694e-6 0.0001343505565907006 4.441404523245204e-5 -0.0001237098920958899 0.00010197222506544903 -4.80500405533108e-6 -0.00010022550925927883 1.2405247771001035e-5 -0.00015726659406058402 -0.00010572152025847508 -3.0795567754784686e-5; 8.026301135713982e-5 -2.2176003524236732e-5 -0.0001247968009306606 -4.023423782662333e-6 -0.00021152006023158516 2.9970157145265214e-5 -4.1155971969154164e-5 0.00014840522114126738 5.487954028918456e-6 0.00029170960103101106 0.0001364494985568802 -0.00017074520085998894 0.0001822852957661807 -0.00011459839673306628 3.7666128507673295e-5 -1.6101906778665383e-5 0.00018429903345122147 -0.00011690346376099852 -1.4439217677594005e-5 -6.349963017101542e-5 -9.64337575464998e-5 -0.00017683811686993795 5.719009847552347e-5 -3.922447443302018e-5 -3.1977191367793196e-5 -2.1097126348202013e-5 -3.903475528666147e-6 -0.00013517490672939649 0.00016719957263148342 -0.00013073334205956205 8.546639061704991e-5 -1.5269046100464052e-5; -4.285495149137722e-5 -0.00015808186933328908 -8.367387156383673e-6 2.5162637734446107e-6 -0.00013412299947280758 -9.557863583954096e-5 8.171517631804133e-5 -5.246708812823869e-5 -1.7444965328947305e-6 9.215804541550998e-5 -7.319868516109516e-5 -0.00027948244448684073 0.00015721960019390505 3.110966875056439e-5 -0.0002053108669065728 -5.454849221909746e-5 -6.301397636464194e-6 -0.00014283853440478876 3.911820979507831e-5 3.7215132149372895e-5 -6.609561157579041e-5 -1.8971257737095413e-6 -9.650786293836533e-5 2.1515796586256786e-5 5.0987890542689164e-5 3.5147839778305374e-5 0.00013020407143581424 0.00012702088363535395 -7.683541977943642e-5 0.00011238765958065533 2.683416848824828e-6 4.7270469192383486e-5; -4.81021370076556e-5 4.5161818924709434e-5 0.00011199531312668745 5.2031405532107285e-6 -6.459508855258904e-5 0.00010572515478926155 3.919753489032578e-6 6.903685552249666e-6 -0.00015121389420599422 1.7531731863723377e-5 -3.0011130512125973e-5 -3.682348957298774e-5 -0.0001382626168931522 -0.00013444802149092396 -0.00011880085636293478 -0.00011516978243961795 -2.658730314649844e-5 -1.0312354753107738e-5 -0.00013526252129011354 8.309089061376478e-6 1.3917499963287668e-5 4.067580179181019e-5 5.7374953975530736e-5 -4.978239208939439e-5 0.00015902992621632663 3.7446948785966035e-8 -1.5800279434907896e-5 5.3139531736822475e-5 -0.00014180776903302748 -0.00013860250597716513 0.00013365241924403315 0.00013656724062385112; 0.0001716806770309658 -2.2274967104751507e-5 -9.818170232392436e-5 -6.3892529706206e-5 -9.407238698258605e-5 -3.0187476022524926e-6 8.873553148078214e-5 6.795901996156597e-5 -0.00012123663631427866 7.99799860578801e-5 0.0001351110883297239 0.00013809933867378477 -2.510291900762605e-5 0.00012684979143159005 -0.00014433913323955196 -9.857484414169174e-5 0.00017308623197095634 4.2674845335571465e-5 -1.679097053056454e-5 -8.397930227147059e-5 1.064211361154162e-5 -5.046389500421066e-5 -0.00014105983003900941 -2.9611733534690534e-5 1.5866286648817994e-5 6.131247633675491e-5 -1.0022676488003671e-5 -2.2031875541867535e-5 1.003374260108281e-5 -0.0001272431503275569 -1.5284858218362233e-5 0.00014202934531568027; -1.4122403897882985e-5 -5.004591393831892e-5 7.915055148453373e-5 0.00010705464201430764 -4.692567044719153e-5 4.8652516581687163e-5 -6.473215789617122e-5 -7.822139041115574e-5 -1.070142779814479e-5 2.0027607225820363e-6 0.00012267875690861888 3.5943811747557506e-7 1.9521455853597164e-5 -0.00010675962430272944 -0.00013979616078165446 -4.262539032231118e-5 -6.636686184525886e-5 -2.5302475749296935e-5 0.0001797804577128186 8.46011095765146e-5 -9.998503837584602e-5 -0.0001354948329188775 -9.500132635249613e-6 -4.3619882760933495e-5 -0.00013717771825929085 4.5246957326643204e-5 0.00016297076918101878 7.414646797963396e-5 -0.00013415243329097305 0.00012045182638951863 -8.078019918491195e-5 1.8181799097218416e-6; 8.464045504599007e-5 -0.00019113839834670673 -0.00013323667689899772 -5.1797763714314024e-5 3.05359831175501e-5 -3.606974450729679e-5 5.702082171712219e-5 -6.241512482727741e-5 -3.756840623890042e-5 -9.84303728698095e-5 0.00024355072355621558 -0.00013379563051483503 -6.849695227782469e-5 1.1523379756114767e-6 -1.693282638069095e-5 6.92288854255075e-5 0.00015571431549989628 -8.287424452344741e-5 4.693981873362311e-5 0.00018321261859757007 7.156047561982724e-6 -0.00023000075770749568 -2.324903786468119e-6 9.687769898238739e-5 8.636373650307874e-5 8.07963410837309e-5 6.684051595886362e-5 4.869451049973391e-5 -7.006313350468768e-6 -5.831089962910616e-7 -4.4962754597401675e-5 -6.1184050506944375e-6], bias = [-2.042037274184826e-9, 5.450538238276147e-10, 2.5289934987238694e-9, -5.148148177753555e-10, -1.2833426980289933e-9, 3.2157841518188328e-9, 2.881843239953348e-10, 1.0123517173553542e-9, -1.2702900780703617e-9, 1.3104570071210386e-9, -2.326512247765953e-9, -7.056591985203844e-10, -2.954497048078123e-9, 6.357343856216914e-10, 1.4448305944960618e-9, 1.2741375330697258e-9, -1.5175691242038057e-9, 1.4450914722628342e-9, 1.8109587064243945e-9, -4.019722143932669e-10, 1.1298182630716548e-9, 1.2157589486701711e-9, -3.2476707575809744e-10, -3.245467265503101e-9, 2.5269332221808443e-9, -2.1689905677970475e-9, -5.654624220598772e-10, -2.0276934697509827e-9, -1.7125428650239895e-9, 3.680361289786686e-10, -9.315934878893783e-10, 2.0782255714890612e-10]), layer_4 = (weight = [-0.0007904217429736376 -0.0006134657177658567 -0.0005707143385862188 -0.0006837621003302308 -0.0008341402321897578 -0.0005106042139339528 -0.0007003734721873451 -0.0005920055483634651 -0.0007073437719725264 -0.0004130149265874651 -0.0006887888845526638 -0.0006701476440012 -0.0007681066177949802 -0.0005443204642705102 -0.0007025799817840762 -0.0007784759925414032 -0.0007973507560408622 -0.0006752262663322216 -0.0004625431346698359 -0.0006631387855498853 -0.0007774837715133924 -0.0007242005769272898 -0.0007606024799357885 -0.0009091325645501539 -0.0007827024388413978 -0.0010325599774881834 -0.0005925590173431629 -0.0006246384804114001 -0.0007831547945311147 -0.0007289307705100168 -0.0005963366240466802 -0.0007263039337167419; 0.00029680230859930646 0.00044045159335233126 0.00024182199209542147 0.00029528814104235834 0.0002953885231182095 0.00026806443487209774 6.535156190517071e-5 0.00020707335391228292 0.00015261088000378096 4.947066717177559e-5 0.00029179127530466097 0.00017615732457630542 0.0003917327014566102 0.00020560982886453667 0.0002494370336098561 0.00034963475290870455 0.0002559733568663764 0.00020444168978222695 0.0003026522687613105 1.740610020494017e-5 0.00033405789094706863 0.00019530848486119698 0.00013907814899625118 5.343438806550787e-5 0.00017462112606557147 0.0003617533259373307 0.00011445079170911656 0.0003683949033299282 0.00028043017727226373 0.00023786085032865976 0.0001228108338075376 0.00032624596727140253], bias = [-0.0006861029847291114, 0.00022974120193819142]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
+
+    lines!(ax, losses; linewidth=4, alpha=0.75)
+    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
+
+    fig
+end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
+soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
+waveform_nn_trained = first(compute_waveform(
+    dt_data, soln_nn, mass_ratio, ode_model_params))
+
+begin
+    fig = Figure()
+    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
+
+    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
+    s1 = scatter!(
+        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
+    s2 = scatter!(
+        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
+
+    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
+    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
+        alpha=0.5, strokewidth=2, markersize=12)
+
+    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
+        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
+        position=:lb)
+
+    fig
+end
    Appendix ​
    julia
    using InteractiveUtils
+InteractiveUtils.versioninfo()
+
+if @isdefined(MLDataDevices)
+    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
+        println()
+        CUDA.versioninfo()
+    end
+
+    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
+        println()
+        AMDGPU.versioninfo()
+    end
+end
    Julia Version 1.10.6
+Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
+Build Info:
+  Official https://julialang.org/ release
+Platform Info:
+  OS: Linux (x86_64-linux-gnu)
+  CPU: 128 × AMD EPYC 7502 32-Core Processor
+  WORD_SIZE: 64
+  LIBM: libopenlibm
+  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
+Environment:
+  JULIA_CPU_THREADS = 16
+  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
+  JULIA_PKG_SERVER = 
+  JULIA_NUM_THREADS = 16
+  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
+  JULIA_PKG_PRECOMPILE_AUTO = 0
+  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const L=e(h,[["render",X]]);export{G as __pageData,L as default};
diff --git a/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.DNi4Y6LY.js b/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.DNi4Y6LY.js
deleted file mode 100644
index f299a14add..0000000000
--- a/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.DNi4Y6LY.js
+++ /dev/null
@@ -1,286 +0,0 @@
-import{_ as t,c as i,a2 as n,j as A,a as h,o as a}from"./chunks/framework.DjZDIZsN.js";const S=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),e={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},C={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},m={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},q={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},V={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},D={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function X(R,s,L,w,j,W){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
-      OptimizationOptimJL, Printf, Random, SciMLSensitivity
-using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=h("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=h(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=h(", ")),A("mjx-container",r,[(a(),i("svg",d,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=h(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
-    M = m₁ + m₂
-    r₁ = m₂ / M .* path
-    r₂ = -m₁ / M .* path
-    return r₁, r₂
-end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=h("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",o,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
-    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
-
-    if size(soln, 1) == 2
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-
-        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
-        p, M, e = model_params
-    else
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-        p = soln[3, :]
-        e = soln[4, :]
-    end
-
-    r = p ./ (1 .+ e .* cos.(χ))
-    x = r .* cos.(ϕ)
-    y = r .* sin.(ϕ)
-
-    orbit = vcat(x', y')
-    return orbit
-end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
-    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
-    b = (v[3:end] .- v[1:(end - 2)]) / 2
-    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
-    return [a; b; c] / dt
-end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
-    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
-    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
-    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
-    return [a; b; c] / (dt^2)
-end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
-    x = orbit[1, :]
-    y = orbit[2, :]
-
-    Ixx = x .^ 2
-    Iyy = y .^ 2
-    Ixy = x .* y
-    trace = Ixx .+ Iyy
-
-    if component[1] == 1 && component[2] == 1
-        tmp = Ixx .- trace ./ 3
-    elseif component[1] == 2 && component[2] == 2
-        tmp = Iyy .- trace ./ 3
-    else
-        tmp = Ixy
-    end
-
-    return mass .* tmp
-end
-
-function h_22_quadrupole_components(dt, orbit, component, mass=1)
-    mtensor = orbit2tensor(orbit, component, mass)
-    mtensor_ddot = d2_dt2(mtensor, dt)
-    return 2 * mtensor_ddot
-end
-
-function h_22_quadrupole(dt, orbit, mass=1)
-    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
-    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
-    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
-    return h11, h12, h22
-end
-
-function h_22_strain_one_body(dt::T, orbit) where {T}
-    h11, h12, h22 = h_22_quadrupole(dt, orbit)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
-    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
-    h11 = h11_1 + h11_2
-    h12 = h12_1 + h12_2
-    h22 = h22_1 + h22_2
-    return h11, h12, h22
-end
-
-function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
-    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
-
-    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
-
-    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
-    @assert mass_ratio≤1 "mass_ratio must be <= 1"
-    @assert mass_ratio≥0 "mass_ratio must be non-negative"
-
-    orbit = soln2orbit(soln, model_params)
-    if mass_ratio > 0
-        m₂ = inv(T(1) + mass_ratio)
-        m₁ = mass_ratio * m₂
-
-        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
-        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
-    else
-        waveform = h_22_strain_one_body(dt, orbit)
-    end
-    return waveform
-end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",g,[(a(),i("svg",C,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=h("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=h(", ")),A("mjx-container",u,[(a(),i("svg",c,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=h(", and ")),A("mjx-container",m,[(a(),i("svg",F,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=h(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
-    χ, ϕ = u
-
-    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
-    denom = sqrt((p - 2)^2 - 4 * e^2)
-
-    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
-    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
-
-    return [χ̇, ϕ̇]
-end
-
-mass_ratio = 0.0         # test particle
-u0 = Float64[π, 0.0]     # initial conditions
-datasize = 250
-tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
-tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
-dt_data = tsteps[2] - tsteps[1]
-dt = 100.0
-const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
-soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
-waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
-
-    axislegend(ax, [[l, s]], ["Waveform Data"])
-
-    fig
-end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
-    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
-ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[-8.733466f-6; -0.00010797254; -6.034535f-6; 7.889705f-5; -1.9253392f-5; -4.612653f-5; 7.144778f-5; -8.932017f-5; -4.7168105f-5; 9.798891f-5; -2.1554477f-5; 4.7397516f-5; 0.00016439424; -4.36064f-5; -6.190079f-5; 6.240796f-5; 0.000105714986; 4.7386144f-5; 2.5381063f-5; 4.9141305f-5; 0.00016239625; 0.00013147807; -0.00017642805; -0.00012611185; 3.0797285f-6; -3.7879247f-5; -1.162709f-6; -0.00016794122; 6.32018f-5; 0.00012728853; 9.064739f-5; -1.5429378f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-2.8177601f-5 -3.936497f-5 -7.657458f-5 0.00013857176 -6.853039f-7 -5.7781497f-5 -5.1510856f-6 3.9836163f-5 0.00012864341 -6.5123946f-5 5.465938f-5 -6.377014f-5 7.9869205f-5 -9.11579f-5 -5.0697596f-5 -1.4907303f-6 -3.207327f-5 -6.482217f-5 2.0175543f-5 9.1325936f-5 1.7174554f-5 6.39053f-5 -4.3893502f-5 -0.00011969867 -6.29855f-5 1.9702458f-5 -7.764152f-6 1.9585927f-6 7.7942015f-5 0.00010650995 -8.729355f-5 -3.0301662f-5; 5.0932027f-5 -3.3675303f-6 0.00010612095 -9.0758826f-5 -6.206741f-5 2.0948732f-5 6.1964456f-5 -8.0493766f-5 9.118536f-5 9.570129f-5 -0.00025590206 -8.4574596f-5 0.00011611364 0.00015786408 -0.0001325249 -1.472367f-5 2.512089f-6 5.8462672f-5 5.2352f-5 -4.69843f-5 -0.00024784482 -4.0580584f-5 -3.3547942f-6 7.7080156f-5 -5.3825785f-5 -0.00012691545 -0.00018019084 3.0379057f-5 -1.2276903f-5 -0.00022681341 0.00017366803 -3.039549f-6; 0.00010137846 -0.0001010016 -2.4273795f-6 -4.770885f-6 6.701655f-5 0.00017136613 -0.0001906262 -1.0672023f-6 0.00012427552 -3.5586767f-5 8.55602f-5 -8.816304f-7 -4.3871598f-5 0.0001563755 -2.259477f-5 4.3149896f-5 -0.00011615122 -1.5891324f-5 -7.964024f-5 1.08097465f-5 3.3177996f-5 0.0001521374 -0.00012456263 4.1607735f-5 -1.4163393f-5 3.9037423f-5 -4.4479628f-5 -3.5756973f-5 7.252298f-5 -0.00011933267 -7.04208f-5 -5.54682f-5; -8.33601f-5 0.00011739388 -1.6886734f-5 -9.9290606f-5 0.00018311598 0.00016209438 -0.00012484103 1.6862592f-5 -0.00012521839 -2.3408023f-5 9.5157055f-5 -6.6844086f-6 5.5988305f-5 0.00012589077 -3.7646063f-5 -0.00011733096 5.898576f-5 -3.2169828f-5 0.00016092892 -0.000102152466 -0.00013990636 -0.00021372396 0.00013839726 -2.180175f-5 -5.539738f-6 0.00017145288 -6.4497464f-5 0.00010669653 2.4381146f-5 -8.664719f-5 -0.00010137241 -0.00010135833; 6.767996f-6 7.979176f-5 0.00010768214 4.6958052f-5 8.351189f-5 5.7730363f-6 -7.494067f-5 2.7757571f-5 -5.5647397f-5 4.7348538f-5 6.738859f-5 -1.7598553f-5 6.2236526f-5 -0.00010106445 6.811721f-5 -1.7854622f-5 0.00014259979 -3.1129297f-5 2.758307f-5 -0.00021257819 -2.0857597f-5 -0.00015710465 0.000115748146 -9.06086f-5 0.00012642707 -6.715701f-5 -2.3087605f-5 0.00014910278 0.00017517559 0.00012419421 -2.7270444f-5 0.00010731906; 5.8766418f-5 -0.00015708971 2.6431195f-5 -0.000103879465 2.2029622f-5 4.658656f-5 -0.00014493923 7.2192146f-5 -2.4574167f-5 0.00013603218 0.00013269106 8.4467785f-5 9.409519f-5 -0.00016357354 1.9661726f-5 0.0001598959 -0.00014350242 -1.5808671f-5 1.605866f-5 -3.59838f-5 1.479661f-5 1.8547255f-5 4.254646f-5 -9.158475f-5 -0.00013229327 -1.3919468f-5 -1.956731f-5 -5.49026f-5 -3.0198875f-5 3.3093645f-5 0.00019555296 9.743678f-5; 6.583076f-5 -1.9019628f-5 -2.3915383f-5 -8.268265f-5 8.264759f-5 0.000113464965 0.00019279975 5.6251043f-5 -3.9762614f-5 -8.271291f-5 6.1018898f-5 -7.841602f-5 -1.2949409f-5 0.00013066275 -1.5204919f-5 -0.00012379164 4.110425f-5 -6.5780056f-5 -1.7211657f-7 -4.4427048f-5 -1.1617677f-6 -0.000109266395 -0.00018221501 -0.00013746502 0.0002607908 -1.48712015f-5 1.9145657f-5 -6.3632055f-5 -3.2119045f-5 -0.00013502994 -5.4510227f-5 -0.0001521188; 1.6407535f-5 -1.9113382f-5 0.00012204645 -7.556568f-5 -1.6685279f-5 -1.5900963f-5 6.0559876f-5 -1.5047506f-5 -8.193592f-5 -2.9995304f-5 -1.0388885f-5 -3.103509f-5 -0.00017250188 3.3705473f-5 -4.3459997f-5 -4.783416f-5 2.276422f-6 5.006527f-6 -0.00015602457 2.369959f-5 3.2234122f-5 -0.00012310114 -2.8645542f-5 5.168618f-5 0.00017521593 -3.8919454f-5 0.00016759252 4.4987402f-5 2.9359122f-5 -9.714238f-5 -8.786319f-6 9.830501f-5; 1.0688757f-6 -5.82454f-5 3.961646f-5 5.784082f-5 2.6230875f-5 -4.110797f-6 -1.2114489f-5 9.190244f-5 5.677286f-5 -0.000111494875 7.906678f-5 2.2018543f-5 -5.5081386f-5 -9.197007f-6 -6.6534965f-5 -6.8599526f-5 8.8153116f-5 2.2247887f-5 -3.938612f-5 -2.3031727f-5 -0.00015165182 0.00014176163 -0.00012151037 2.591397f-5 -0.00015541828 9.689811f-5 -0.000120833436 -5.9194394f-5 -2.0094647f-5 1.9722645f-5 1.8458933f-5 0.00016762815; 2.9945206f-5 -0.0002136077 4.2307387f-5 -0.00014859898 1.6029056f-5 0.00013049919 -0.000121818346 0.00011504457 -3.660613f-5 0.00013795843 1.5452692f-5 6.207062f-5 -4.9132796f-5 -5.6330246f-5 7.049317f-5 -0.0001624591 0.00020378857 5.0185274f-5 -6.563782f-6 2.5135363f-5 -0.00013978432 6.94403f-5 -5.7515543f-5 6.4258464f-5 -8.477129f-5 2.7108229f-6 -6.27297f-5 -0.00011133804 -0.000118937336 -2.491081f-5 -6.5520544f-5 -1.6221407f-5; 0.0001490329 -3.518714f-5 6.838341f-5 1.806367f-5 -1.2177103f-5 4.8077785f-5 4.8774464f-6 2.5176876f-5 -2.393502f-5 -0.00017052738 -8.472462f-5 2.6156655f-5 3.1476753f-5 -3.198646f-5 1.7295855f-5 -6.5615815f-5 -6.457243f-5 4.945713f-5 -6.379944f-5 0.0001725941 -5.566336f-5 -0.00013105926 -2.3267385f-5 0.00013213923 5.1767678f-5 4.8656395f-5 0.00011115292 2.2263952f-5 9.953958f-6 -5.85899f-5 -1.4107241f-5 0.00011483148; -4.1415296f-5 1.3916467f-5 -0.00011940488 -9.851123f-5 7.3574214f-5 4.740405f-5 -6.785301f-5 0.00019942185 0.00012728834 -0.00016363562 -0.00012561417 -1.6128222f-5 -0.00019293881 4.5368397f-6 4.713364f-5 -0.00013651798 -6.237191f-5 -0.00019911939 -0.00017180503 2.6719978f-5 -4.0074345f-5 -1.359718f-5 -8.151072f-5 5.2146228f-5 -1.9828212f-5 7.461231f-5 0.00013041316 -9.9392455f-5 8.682513f-5 0.00013921101 -5.4267803f-5 -0.00015905981; -4.4684915f-5 -5.462887f-5 0.00012135054 -4.510381f-5 -3.475299f-5 0.00010407074 2.2008955f-5 -0.00013626451 -2.2398599f-5 -2.954842f-5 0.00015306505 5.706277f-6 -3.3192628f-5 0.00020575941 6.463371f-5 -9.893465f-5 8.663553f-5 -0.00011787723 -0.00014052064 5.1350125f-5 -1.1286868f-6 0.00011688338 0.00013804293 4.82576f-5 -5.091659f-5 5.691207f-6 -1.1650249f-6 0.00010991485 7.0106557f-6 -5.4914188f-5 0.000103280305 -2.2072318f-5; 3.8976705f-5 0.00010581013 4.4845918f-5 0.00028989412 -2.1131393f-5 -0.00010667203 0.00022406431 6.0003676f-5 -5.046534f-5 0.00013748402 0.00016624035 0.00016925113 -7.0310954f-5 -8.645576f-5 0.000104551196 0.00012043912 -0.0001673082 4.826957f-5 -0.00021466143 0.00023090826 -0.00013504991 -2.8148033f-5 8.9335634f-5 2.1197551f-5 7.634965f-6 -0.00018328545 0.00011402708 4.8938728f-5 8.409785f-5 -9.03842f-5 -0.00015597348 -1.9681711f-5; -0.00017860696 0.00016289309 2.1554613f-5 7.65075f-5 -0.0001572884 -4.904941f-5 0.00013334364 8.8976296f-5 0.00011627484 3.0017205f-5 -4.347116f-6 -7.407668f-5 3.352131f-5 -9.5505995f-5 -6.2805775f-5 -3.8850536f-5 -5.6758072f-5 4.625631f-8 -2.8606568f-5 -8.249394f-5 -8.704262f-5 -1.5149345f-5 -9.393877f-6 -5.6657373f-5 3.1522475f-5 -1.299464f-5 -4.861005f-5 7.5044234f-5 -0.00013518587 -7.636546f-5 3.0268136f-6 3.7342343f-5; -1.5246345f-6 -0.00013971738 -4.301577f-5 -1.3734039f-5 0.00028718126 8.435068f-5 0.00015791062 4.3378295f-5 -4.043131f-5 0.00013740394 -0.00012764876 -5.627069f-5 3.8953523f-5 0.00019921537 -8.471506f-5 4.6630197f-5 0.00012246286 -1.7031091f-6 2.0175803f-5 0.00017496663 0.00012815923 4.6586156f-6 9.724115f-5 -0.00014763075 -0.00013615302 0.00021714495 7.7586665f-6 -9.8543445f-5 -3.5388155f-5 -0.0001154024 -3.734798f-5 -0.0002094385; -7.722984f-5 0.00012643216 -0.0001336082 2.5638423f-5 -8.438335f-5 -5.9111102f-5 -4.578554f-5 -7.396497f-6 -6.326183f-5 3.0497316f-5 -0.000102561884 -0.000101490295 0.00013105836 -0.00019522628 3.417967f-5 -0.00011023943 -1.3394814f-5 0.00010105535 -4.936928f-5 0.00019615894 -9.0929265f-5 0.00017270022 9.9116936f-5 1.238063f-5 3.634942f-5 -0.0001720305 -6.9169415f-5 -0.000103437924 1.6598398f-5 -0.00011501487 -0.00020326376 -0.00016199409; -1.0493395f-5 -1.843547f-5 3.4408877f-6 1.48939225f-5 0.00016012433 -1.5305499f-5 1.7829227f-5 7.7386605f-5 2.445304f-5 -4.7624897f-5 -1.5142531f-5 -9.343951f-6 0.00013970489 -0.0001309227 -0.0002114561 0.00018979193 3.2718573f-5 9.215379f-5 0.000120665085 -0.00010754408 6.665616f-6 -7.775345f-5 -6.5193824f-5 4.0987623f-5 3.918128f-5 3.6471756f-5 -4.6466415f-5 0.00012628127 5.7103025f-6 -5.55399f-5 -7.507195f-5 -2.8647954f-5; -6.847904f-5 -9.9914185f-5 -0.000105565225 1.3994042f-5 -0.00018054691 -4.7752743f-5 -2.5410349f-5 7.5910316f-6 0.000115343246 -0.00015416865 -2.0336203f-5 -6.3315434f-5 0.0001612588 -4.540509f-5 1.9162975f-5 6.950722f-5 7.672867f-5 -6.484223f-5 -1.9393967f-5 1.4184581f-5 0.00014736147 -8.958512f-5 0.00018367275 -2.6434365f-5 6.584939f-6 0.00010185486 8.6197484f-5 -3.4724155f-6 -3.9978455f-5 2.361536f-5 -2.6316737f-5 0.0002298531; -0.00015590824 3.233036f-5 -4.891627f-6 -0.000112197886 -3.579302f-5 -2.305788f-6 0.0002067958 -2.0661932f-5 2.812922f-6 -3.7179456f-5 -1.7453644f-5 6.0095757f-5 3.4720404f-5 -0.00017240063 -2.5926716f-5 -0.00013413644 0.0001581374 0.00014044714 -0.00013199625 6.1015515f-5 8.263777f-5 -0.00010763351 0.0001268077 -1.8969367f-5 4.3797056f-5 7.2108494f-5 5.4863922f-5 0.00014671938 -7.480865f-5 -5.1882493f-5 0.00016848213 -5.266863f-5; -0.00018211977 0.00022923049 -5.0249986f-5 1.5011439f-5 8.912166f-5 -9.50752f-5 3.0659136f-5 -0.00015601957 -4.408726f-5 8.5128704f-5 -0.00012687575 -4.7730693f-5 -9.324386f-5 1.5833908f-5 4.891327f-5 1.8481105f-5 0.00023573374 -4.9181544f-5 -0.0001148964 4.768313f-6 0.00026578025 -1.9075253f-5 -2.9713647f-5 -4.0804738f-5 9.9373574f-5 0.00020834891 1.2000563f-5 -0.00012790448 5.3762087f-5 4.0613824f-5 6.384502f-5 -0.00016239902; 6.825154f-5 5.2034096f-7 -3.3355213f-5 4.9806255f-5 2.6543269f-6 -8.8090885f-5 1.3807705f-5 -4.163206f-5 0.00011294925 -0.00015741195 0.00010468527 -1.8770994f-5 -1.7840232f-5 -9.5402196f-5 -6.371465f-5 -0.00017467847 -0.00023158824 0.00024108063 0.000104571554 6.284469f-5 -0.00015179072 -6.076992f-5 1.831456f-5 -5.2111507f-5 1.0137547f-5 -1.0156169f-5 -0.000111388974 -0.000139489 8.460674f-5 0.0001405082 7.1764507f-6 0.0002610488; -7.00232f-5 -5.1067367f-5 0.00014329309 -6.036692f-5 0.0002614541 -9.751939f-6 -0.00011116908 1.45649965f-5 5.4732864f-5 0.00010501139 -2.877626f-5 -0.00021153178 -1.9726102f-5 -0.00017698524 -7.0890484f-5 -0.00021033199 0.00012432237 3.8996743f-5 -0.00011144479 0.00011738168 0.00026127408 -6.956277f-5 -1.3386461f-5 -0.00017813583 3.5562014f-5 7.980341f-5 -0.000105268555 -7.7263f-5 4.97281f-5 2.7082115f-5 2.170017f-5 0.000103214596; 0.00020855725 0.00011113887 4.70926f-6 0.00013785521 1.884335f-5 9.6773845f-5 0.00011414899 -4.9486604f-5 -7.360017f-6 -7.687813f-5 -6.0845636f-5 -0.00012735029 -9.817477f-5 -0.0001946802 3.3830744f-5 -0.00012810805 -0.00012485104 -1.0283695f-5 -0.000101897385 0.00020154905 -0.00010027012 6.4827036f-5 -8.700263f-5 0.00020038383 5.8320256f-5 -1.971239f-5 -4.4582062f-5 8.605977f-5 3.7068062f-6 8.007526f-5 1.0883958f-5 -3.441287f-5; -7.422574f-5 -7.489358f-5 -0.0002489197 3.7079844f-5 9.6897085f-5 1.0863249f-5 -0.00024051977 3.988985f-5 -0.00030288193 3.255059f-5 -3.1009444f-5 -8.407521f-5 0.00018992538 0.00017652047 6.387699f-5 6.62641f-5 -5.659702f-5 -4.528007f-5 -2.4961199f-5 -6.458119f-5 -7.740639f-6 -8.641964f-5 5.9170667f-5 0.00019641886 7.1408605f-5 7.491677f-5 0.00015996007 -2.14441f-5 -0.00012596443 1.5148395f-5 -0.00020513525 7.203257f-5; -2.159809f-5 -2.1345286f-5 0.000114871524 -0.00021547108 4.5988298f-5 -0.00021723151 5.452951f-5 5.8848036f-6 -0.0001589268 -0.0002506956 0.00018307757 2.7498834f-5 0.00012473844 -3.9340946f-5 0.0001878797 -1.2492364f-6 -9.198892f-5 1.8248566f-5 0.000116399024 1.5051434f-5 0.00034781685 0.000156159 -1.0037205f-5 0.00010480752 -9.164158f-5 2.6788684f-6 0.00016894897 5.7068297f-5 0.00010959305 0.00014088652 -7.1304785f-5 5.224269f-5; 6.238324f-5 9.117311f-5 0.00020454697 0.000208475 0.00018094428 -3.228266f-5 -4.411761f-5 -0.00012658827 -0.00015168184 0.00010113172 0.000105583036 0.00012806694 -5.9978844f-5 -1.2934191f-5 -3.4510533f-5 -6.6202236f-5 -6.6260574f-5 0.0001058354 -0.00020347247 0.00019539056 -4.2171858f-5 0.00020606014 -2.4118353f-5 0.00012581136 7.688511f-5 -0.00010003266 8.23604f-5 -3.0226212f-5 4.6036577f-5 -5.898498f-5 -6.406751f-7 -0.0001438071; 0.00015314828 -5.8805148f-5 -5.2923174f-6 0.0001559641 7.959421f-5 0.00013301839 -4.3907394f-6 7.376904f-5 0.00012489593 4.809692f-5 0.000106285006 -0.00014742208 2.0827318f-5 1.4279862f-5 1.27891335f-5 -0.000102658276 2.7062184f-5 7.4395102f-6 -8.629122f-5 1.6947888f-5 -1.9900364f-5 -1.0614642f-5 -1.1935723f-5 -2.9863071f-5 -0.00010493953 -5.5913297f-5 -2.2774684f-5 -0.00014706138 -0.00017280485 1.1295227f-5 -7.577581f-5 0.00013971627; -7.24564f-5 -0.00019400741 -0.00020466995 -8.613493f-5 -1.8114002f-5 -0.00013081147 -4.010616f-5 0.00017401333 -9.880749f-5 -0.00010273883 -7.1364964f-5 4.2208332f-5 0.000110427754 -0.00013644555 5.977523f-5 -3.1731386f-6 5.8735186f-5 5.9781487f-5 0.00012411928 -7.0713344f-5 -1.4844751f-5 0.00021108685 -7.310388f-5 -0.00012904903 0.00016850662 -0.00017227576 -4.35704f-5 8.4781626f-5 -0.000118789314 9.910206f-6 -0.00017377545 0.000104917126; 3.421374f-5 0.0001784863 -1.1484191f-5 -0.000101836085 0.00024135718 -0.00017500338 -1.0642813f-5 -3.1498173f-5 4.254024f-5 -4.286035f-5 0.00015602878 -5.6244044f-5 0.00011752606 -0.00018053921 -0.000118252945 -7.63437f-5 6.47034f-6 0.0001642431 6.444494f-5 9.384959f-5 0.00015255544 -8.760447f-6 -4.673646f-5 2.8553897f-5 6.577486f-5 -3.8543207f-5 4.0060764f-5 0.0001953378 -9.2235525f-5 -8.571982f-6 1.4440342f-5 -0.00013854646; 2.1408754f-5 9.593646f-5 -4.738496f-5 4.732052f-5 -0.00024824127 -4.732115f-5 2.6718411f-5 -8.263985f-5 1.5497177f-5 -8.5198546f-5 -4.1833344f-5 -3.14811f-5 0.00010269098 -5.6399032f-5 1.7421926f-5 2.3837554f-6 8.1492464f-5 -0.000119179356 6.3751904f-5 2.5058574f-5 -8.191238f-5 6.434593f-5 3.4300912f-5 9.4067254f-5 0.00011397646 3.534332f-5 -0.00016860459 -0.00018563253 -6.822247f-5 -1.4889482f-5 0.00023276742 -2.7466636f-5; 6.616932f-5 1.28425345f-5 -0.0001042926 9.09377f-5 -0.00024482232 3.07259f-5 -0.00013715128 0.000101553676 -7.178563f-5 3.5842924f-5 -8.436059f-5 -0.00016869747 7.175845f-5 -9.25618f-5 -8.6544744f-5 -0.00011310014 -1.9110552f-5 0.00014068597 -3.46944f-5 2.8009728f-5 6.1295f-5 -1.4718792f-5 0.00011410012 4.691924f-5 1.5165745f-5 6.707154f-5 -6.649845f-5 -8.032728f-5 -4.362544f-5 -4.7473106f-5 -6.403594f-5 -0.00021157016], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-0.00022788436 -0.00015335342 -0.00011465894 0.00020399482 -0.00023230375 -0.00023374535 -5.701846f-5 2.2562022f-6 -3.3024367f-5 -8.095133f-5 -0.00013721343 -2.620743f-5 0.00015951965 -0.0001061864 0.000111545 1.7474338f-5 2.3434386f-5 -9.256927f-5 6.5034714f-5 -0.00015723762 2.8178138f-7 -4.313013f-5 7.1515926f-5 1.6893357f-5 5.1108167f-5 3.659688f-5 1.3686637f-5 1.3952281f-5 -6.235979f-5 -1.1992588f-5 -3.5462926f-5 7.7624674f-5; -9.5137206f-5 2.3902441f-5 -0.00012429585 0.00012397852 -2.834219f-5 1.251941f-6 1.3347854f-5 -3.714094f-5 8.1240454f-5 -9.8066725f-5 -0.00011307021 -3.4549306f-5 -8.919269f-5 -4.9514474f-5 -0.00014298115 -6.0232043f-5 -0.00020102633 4.9332135f-5 -0.00012386192 -7.3164238f-6 -0.00015395948 -0.0001424155 -5.7651367f-5 0.00011915152 -0.00019318277 -5.8882713f-5 9.0664114f-5 1.8740871f-5 0.00013420281 -5.03832f-5 -0.00023244778 7.6930075f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
-
-const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
-    Chain(
-        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
-        layer_2 = Dense(1 => 32, cos),  # 64 parameters
-        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
-        layer_4 = Dense(32 => 2),       # 66 parameters
-    ),
-)         # Total: 1_186 parameters,
-          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",q,[(a(),i("svg",B,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",V,[(a(),i("svg",b,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=h("where, ")),A("mjx-container",T,[(a(),i("svg",D,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=h(", ")),A("mjx-container",z,[(a(),i("svg",U,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=h(", and ")),A("mjx-container",P,[(a(),i("svg",K,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=h(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
-    χ, ϕ = u
-    p, M, e = ode_model_params
-
-    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
-    # it, however, in general, we should use \`st\` to store the state of the neural network.
-    y = 1 .+ nn_model([first(u)], nn_params)
-
-    numer = (1 + e * cos(χ))^2
-    denom = M * (p^(3 / 2))
-
-    χ̇ = (numer / denom) * y[1]
-    ϕ̇ = (numer / denom) * y[2]
-
-    return [χ̇, ϕ̇]
-end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
-waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    axislegend(ax, [[l1, s1], [l2, s2]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
-
-    fig
-end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
-
-function loss(θ)
-    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
-    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
-    return mseloss(pred_waveform, waveform)
-end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0006529859924286357
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
-
-function callback(θ, l)
-    push!(losses, l)
-    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
-    return false
-end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
-optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
-optprob = Optimization.OptimizationProblem(optf, params)
-res = Optimization.solve(
-    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
-    callback, maxiters=1000)
    retcode: Success
-u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [-8.733465620020983e-6; -0.00010797254071798166; -6.034535090281437e-6; 7.889705011615516e-5; -1.9253391656059496e-5; -4.612652992366777e-5; 7.144777919157657e-5; -8.932017226481967e-5; -4.71681050840083e-5; 9.798890823729545e-5; -2.1554476916162734e-5; 4.739751602749602e-5; 0.00016439423779959398; -4.360640014055303e-5; -6.190079147924201e-5; 6.240796210469526e-5; 0.00010571498569335307; 4.738614370580132e-5; 2.5381063096584814e-5; 4.914130477114876e-5; 0.00016239624528652472; 0.00013147806748750233; -0.00017642804596096446; -0.0001261118450201706; 3.079728458034808e-6; -3.787924651986792e-5; -1.16270905436827e-6; -0.00016794122348050335; 6.320179818424497e-5; 0.00012728852743741338; 9.064738696900327e-5; -1.5429377526735596e-5;;], bias = [-5.359601592163036e-18, -6.550829691992387e-17, -1.41806373185002e-17, 6.612883732914699e-17, -3.971281807812836e-17, 3.2571059060761017e-17, 9.58030628269383e-17, 1.3176812858269634e-16, 2.944634276201806e-17, 1.9336104666087008e-17, -5.330420267771681e-17, 7.372211719502685e-17, -2.7922867808978655e-17, -9.150352429637832e-18, -3.8438937777196597e-17, 9.00633053182541e-17, -1.5663529032065214e-17, 7.62186052855715e-18, -1.1978849530028129e-17, 3.9396538625362964e-17, 3.263723400316271e-16, -3.1675390001372084e-17, -1.3105712844388004e-16, -3.803273056227972e-17, -2.0332749183269495e-18, -1.2727593133643193e-17, -1.7718179375088544e-18, -2.8018370574615e-16, 7.587973597486665e-17, 8.609350097255151e-17, 7.4759154972248e-17, -9.260147643001434e-18]), layer_3 = (weight = [-2.81779129679041e-5 -3.9365283514297854e-5 -7.6574890913135e-5 0.00013857145044797106 -6.85615832181301e-7 -5.7781809077523993e-5 -5.151397503750668e-6 3.9835850698482504e-5 0.00012864310069319755 -6.512425803036386e-5 5.4659068196387454e-5 -6.377044970092395e-5 7.986889344950431e-5 -9.115820917456447e-5 -5.069790764052438e-5 -1.491042171137685e-6 -3.207358072783739e-5 -6.482248041235874e-5 2.0175230970680213e-5 9.132562358475261e-5 1.717424224382949e-5 6.390499133461278e-5 -4.389381406751195e-5 -0.00011969898237536553 -6.298581043181066e-5 1.9702146387616684e-5 -7.764463548183539e-6 1.9582808320888176e-6 7.794170328027556e-5 0.00010650963763085304 -8.729386077417963e-5 -3.0301974179265634e-5; 5.093000583673822e-5 -3.3695515683559355e-6 0.00010611892650576301 -9.076084677854146e-5 -6.206943215129883e-5 2.0946710374296105e-5 6.196243486574254e-5 -8.04957875332556e-5 9.118333548048649e-5 9.569926771309863e-5 -0.0002559040862068346 -8.457661750053847e-5 0.00011611161755373647 0.00015786205915009716 -0.00013252692440783356 -1.4725690907474887e-5 2.510067697768531e-6 5.846065107124997e-5 5.234997808674383e-5 -4.698632108539192e-5 -0.00024784683626402274 -4.0582605269986785e-5 -3.356815459286549e-6 7.707813494580927e-5 -5.3827805804582414e-5 -0.00012691746667751962 -0.00018019285778773963 3.0377035382311122e-5 -1.2278924605019604e-5 -0.0002268154333976734 0.00017366600661280175 -3.0415702711312987e-6; 0.00010137852786915331 -0.0001010015274014704 -2.427308391863947e-6 -4.770813849111468e-6 6.701661750316573e-5 0.00017136620426658485 -0.00019062613300871163 -1.067131212087478e-6 0.00012427559540715012 -3.5586695777104e-5 8.55602685978259e-5 -8.815592773796477e-7 -4.387152676013409e-5 0.00015637557789791322 -2.2594698567493444e-5 4.314996676743906e-5 -0.00011615114799592718 -1.5891253316160807e-5 -7.964016713326844e-5 1.0809817648757658e-5 3.317806676836575e-5 0.0001521374781068033 -0.0001245625624515501 4.1607805723239765e-5 -1.4163321612661705e-5 3.903749366049605e-5 -4.447955671007791e-5 -3.575690225357303e-5 7.252304767366444e-5 -0.00011933259684251707 -7.042072848076068e-5 -5.546812990455935e-5; -8.336042507189592e-5 0.00011739355553536533 -1.6887058029987743e-5 -9.929093054133669e-5 0.0001831156508497774 0.00016209405597428542 -0.0001248413506463285 1.6862267636589728e-5 -0.0001252187109120308 -2.3408347863458833e-5 9.515673021915105e-5 -6.6847330007470475e-6 5.598798028322887e-5 0.0001258904479410381 -3.764638737529744e-5 -0.00011733128181995121 5.898543471366949e-5 -3.217015242889768e-5 0.00016092859308364536 -0.00010215279022568149 -0.00013990668654778032 -0.00021372428879012443 0.00013839693141299363 -2.1802074718741762e-5 -5.540062523231966e-6 0.00017145255272871558 -6.449778871947236e-5 0.00010669620254439335 2.438082204853424e-5 -8.664751354708432e-5 -0.00010137273480986426 -0.00010135865583188165; 6.770358496073841e-6 7.97941223851332e-5 0.00010768450501077066 4.6960414138666134e-5 8.351425402965557e-5 5.775398577173693e-6 -7.493830581493888e-5 2.7759933246341794e-5 -5.564503463515998e-5 4.735090023190146e-5 6.739095023535343e-5 -1.759619100026308e-5 6.22388883044749e-5 -0.0001010620859318409 6.811957190679273e-5 -1.7852259233548545e-5 0.00014260215301932077 -3.112693467242631e-5 2.758543213589722e-5 -0.00021257582794474437 -2.0855234296848334e-5 -0.00015710228272780086 0.00011575050793055044 -9.060623652600814e-5 0.00012642943092091197 -6.715464643035007e-5 -2.3085242545998626e-5 0.00014910514200039036 0.0001751779540342266 0.0001241965704963434 -2.726808143903494e-5 0.0001073214201739075; 5.8766903805779695e-5 -0.00015708922904846402 2.6431680510403082e-5 -0.00010387897958676388 2.2030108141596797e-5 4.6587045347085305e-5 -0.00014493874363065735 7.219263176270675e-5 -2.457368094578844e-5 0.00013603266239701657 0.00013269154266058503 8.446827092743486e-5 9.409567252336606e-5 -0.00016357305315718972 1.966221140068937e-5 0.000159896391835762 -0.0001435019311767551 -1.580818565085058e-5 1.6059146276209827e-5 -3.5983315626392106e-5 1.4797095957732168e-5 1.8547740239996884e-5 4.254694619052582e-5 -9.158426286117389e-5 -0.00013229278005124026 -1.3918982346833022e-5 -1.9566823482047338e-5 -5.4902115095643754e-5 -3.019838908326368e-5 3.309413119754534e-5 0.00019555344448494636 9.743726519704837e-5; 6.582914752732503e-5 -1.9021242084925348e-5 -2.3916997494893138e-5 -8.26842650413899e-5 8.264597377353835e-5 0.00011346335045607408 0.00019279813673427136 5.6249428919618555e-5 -3.976422781044965e-5 -8.271452574910546e-5 6.101728390865519e-5 -7.841763622047724e-5 -1.2951023424746916e-5 0.00013066113856700814 -1.5206532995873412e-5 -0.0001237932582145024 4.110263503469658e-5 -6.578166980682285e-5 -1.7373078571754834e-7 -4.442866233796219e-5 -1.1633819413955027e-6 -0.00010926800937078438 -0.00018221662216738526 -0.00013746663066478056 0.0002607891957749194 -1.487281574293196e-5 1.9144042337643907e-5 -6.363366887590736e-5 -3.2120659713766265e-5 -0.00013503155682621117 -5.4511840832783175e-5 -0.00015212040929975444; 1.6406990034034946e-5 -1.9113927629197256e-5 0.00012204590195253912 -7.556622556946641e-5 -1.6685824158024457e-5 -1.590150866902443e-5 6.0559330740240846e-5 -1.5048051573805486e-5 -8.19364647038794e-5 -2.9995849836970368e-5 -1.038943053642793e-5 -3.10356369307312e-5 -0.00017250242643128175 3.37049280635499e-5 -4.3460542365913066e-5 -4.783470620238665e-5 2.2758766310549294e-6 5.0059814559306675e-6 -0.000156025118195224 2.3699043885450762e-5 3.2233576638856505e-5 -0.00012310168647743032 -2.8646086939959324e-5 5.1685634679624405e-5 0.00017521538422372607 -3.891999920667637e-5 0.00016759197052963493 4.498685695510351e-5 2.9358576575056893e-5 -9.714292507921566e-5 -8.786864492114564e-6 9.830446680937386e-5; 1.06843883722068e-6 -5.8245838158569814e-5 3.961602458283923e-5 5.7840384829517084e-5 2.6230438084068423e-5 -4.11123376116672e-6 -1.211492628945743e-5 9.190200187285233e-5 5.677242340888547e-5 -0.00011149531230776743 7.906633952651798e-5 2.2018106096849766e-5 -5.5081822506421394e-5 -9.197443728142482e-6 -6.653540215202865e-5 -6.85999624010108e-5 8.815267908639125e-5 2.2247449737817038e-5 -3.938655602260798e-5 -2.303216417650039e-5 -0.00015165226097897555 0.00014176119264001894 -0.0001215108062068825 2.5913533750184917e-5 -0.00015541871864562236 9.689767437074972e-5 -0.00012083387293833621 -5.919483039624893e-5 -2.0095083823350953e-5 1.9722208594516065e-5 1.845849662922129e-5 0.00016762771672355632; 2.9943606731133943e-5 -0.00021360929369762725 4.23057879547136e-5 -0.000148600576923457 1.602745740313698e-5 0.00013049759016069826 -0.00012181994518101749 0.00011504296869901559 -3.660772765357732e-5 0.0001379568291471869 1.5451092420729488e-5 6.206902051274912e-5 -4.913439463193834e-5 -5.633184466347801e-5 7.04915744594797e-5 -0.00016246069410186855 0.00020378697243217167 5.018367462838316e-5 -6.565380973885296e-6 2.513376400969248e-5 -0.00013978591434493068 6.94386997369688e-5 -5.751714181455874e-5 6.425686458755322e-5 -8.477288631326389e-5 2.7092237618801464e-6 -6.27312967677122e-5 -0.00011133964128162024 -0.00011893893517623317 -2.4912409773203008e-5 -6.552214310573785e-5 -1.6223006443002694e-5; 0.0001490338564274168 -3.518618352769437e-5 6.838436281599909e-5 1.806462575972689e-5 -1.2176147652786665e-5 4.807874035963595e-5 4.878401409079714e-6 2.5177831116016903e-5 -2.3934064234487803e-5 -0.0001705264202662507 -8.472366419096616e-5 2.6157609749355256e-5 3.1477708102315184e-5 -3.1985504325127225e-5 1.7296810323607505e-5 -6.5614860364583e-5 -6.457147349079619e-5 4.945808366232355e-5 -6.379848302981961e-5 0.00017259505154856278 -5.566240536224125e-5 -0.00013105830833025017 -2.3266429639777084e-5 0.00013214018879283523 5.176863312430556e-5 4.865734997496769e-5 0.00011115387503581927 2.226490694214039e-5 9.954913434050791e-6 -5.858894467179369e-5 -1.4106286403838673e-5 0.00011483243185852522; -4.141835410123147e-5 1.3913409077181379e-5 -0.00011940793950784414 -9.851428507103351e-5 7.357115651971301e-5 4.7400992173019836e-5 -6.785606996131011e-5 0.00019941878969108163 0.00012728528059811163 -0.00016363867501935345 -0.0001256172271838726 -1.6131280006125398e-5 -0.00019294186671469824 4.533782032335001e-6 4.713058120829044e-5 -0.00013652103547214534 -6.237496739926018e-5 -0.00019912244346200657 -0.00017180809112473725 2.6716919876302937e-5 -4.007740238886854e-5 -1.360023810559931e-5 -8.151377727482514e-5 5.214317030765395e-5 -1.983126998097565e-5 7.460925377231771e-5 0.00013041010066658793 -9.939551267745315e-5 8.682207231900662e-5 0.00013920795578303128 -5.427086028943335e-5 -0.0001590628689315487; -4.4683193628608677e-5 -5.462714667449189e-5 0.0001213522601021198 -4.510208869835473e-5 -3.475126774066839e-5 0.00010407246467291496 2.201067676576273e-5 -0.00013626279083912471 -2.2396877248705143e-5 -2.954669886169323e-5 0.0001530667696344408 5.707998788399833e-6 -3.319090588851249e-5 0.00020576113187342084 6.463542835188122e-5 -9.893293005081802e-5 8.663725478906726e-5 -0.00011787550734954773 -0.00014051892045319808 5.135184714088322e-5 -1.1269650811645857e-6 0.00011688509861339084 0.00013804465292119772 4.82593214102676e-5 -5.0914869426216386e-5 5.692928915939261e-6 -1.1633031461512262e-6 0.00010991657203627335 7.012377445040195e-6 -5.4912466076377036e-5 0.0001032820264655795 -2.207059602444299e-5; 3.8979493365287074e-5 0.00010581292208365888 4.484870642040789e-5 0.00028989690526936494 -2.1128604021259633e-5 -0.00010666924008551315 0.00022406709667843232 6.000646517344881e-5 -5.0462551189458834e-5 0.00013748680994462204 0.00016624314066571072 0.00016925391737454296 -7.03081657776027e-5 -8.645297002663145e-5 0.00010455398496555676 0.00012044191153103992 -0.00016730541192379013 4.827235734405655e-5 -0.00021465863919734152 0.00023091105158523368 -0.0001350471191472717 -2.8145244687099784e-5 8.933842236721743e-5 2.1200339870060362e-5 7.637753878908047e-6 -0.0001832826581449361 0.0001140298720201378 4.894151625187261e-5 8.410063510377457e-5 -9.038141233765218e-5 -0.0001559706923217688 -1.967892221732617e-5; -0.00017860869197075727 0.00016289136179357017 2.1552883805534698e-5 7.650577240955891e-5 -0.00015729013616119967 -4.9051138961393236e-5 0.00013334190803679445 8.897456642107079e-5 0.00011627310810244291 3.00154755280313e-5 -4.348845453668481e-6 -7.407840679998861e-5 3.351958217200008e-5 -9.550772451230695e-5 -6.280750462220311e-5 -3.885226546454502e-5 -5.6759801412865855e-5 4.452677590444851e-8 -2.860829740139632e-5 -8.249567112553275e-5 -8.704435244334782e-5 -1.5151074552430909e-5 -9.395606542684836e-6 -5.665910215948288e-5 3.152074564813159e-5 -1.2996369736091477e-5 -4.8611780260861976e-5 7.504250457378289e-5 -0.00013518759591871196 -7.636719023475917e-5 3.0250840666626814e-6 3.73406131587131e-5; -1.5228735603273347e-6 -0.00013971561548916717 -4.3014009690716666e-5 -1.3732278432340951e-5 0.0002871830221809317 8.435243961483699e-5 0.00015791237837018728 4.3380056183907456e-5 -4.04295495283931e-5 0.00013740570301504828 -0.00012764699914642993 -5.626892726669568e-5 3.8955284424237366e-5 0.00019921713118290995 -8.471329766709e-5 4.6631958128077746e-5 0.00012246462591772832 -1.701348139444165e-6 2.0177563954154805e-5 0.00017496838717105307 0.00012816098768577894 4.660376532935051e-6 9.724291174986106e-5 -0.0001476289906461012 -0.00013615125482120902 0.00021714670600684853 7.760427422715573e-6 -9.854168439435897e-5 -3.5386394131105624e-5 -0.00011540063843515129 -3.7346220608212875e-5 -0.00020943673717939665; -7.723347007773074e-5 0.00012642852888899184 -0.00013361183883167066 2.563478992310894e-5 -8.438698060122472e-5 -5.911473490773644e-5 -4.5789171421300255e-5 -7.400129928959714e-6 -6.326546499981479e-5 3.049368351281167e-5 -0.00010256551686684742 -0.00010149392838134362 0.00013105472825926016 -0.00019522991570188633 3.4176036566396015e-5 -0.00011024306368558402 -1.3398447121235416e-5 0.000101051720408621 -4.937291160236234e-5 0.00019615531127424067 -9.093289773220052e-5 0.0001726965907401498 9.911330343683025e-5 1.2376997434390403e-5 3.6345785334601886e-5 -0.00017203413372403965 -6.91730478174166e-5 -0.00010344155671547738 1.6594765471427673e-5 -0.00011501850022244405 -0.00020326738959187797 -0.00016199772144678218; -1.0492632138680406e-5 -1.8434707310424074e-5 3.441650694680668e-6 1.4894685552956632e-5 0.00016012509054921307 -1.530473586395223e-5 1.7829989662953562e-5 7.738736799869232e-5 2.4453803196178463e-5 -4.7624133951519924e-5 -1.5141768055814625e-5 -9.343188178223385e-6 0.00013970565392586398 -0.00013092193061534614 -0.00021145534359898768 0.00018979269130572327 3.271933652062365e-5 9.215454970903387e-5 0.00012066584800554172 -0.00010754331837456985 6.6663792108896785e-6 -7.77526884507643e-5 -6.519306138440145e-5 4.0988385951586975e-5 3.918204216237859e-5 3.647251920260201e-5 -4.6465652342208385e-5 0.00012628203599664346 5.7110655302891605e-6 -5.553913528497993e-5 -7.507118520375801e-5 -2.864719048598573e-5; -6.847838721998037e-5 -9.991352908374989e-5 -0.0001055645689118281 1.3994697838172852e-5 -0.0001805462516128645 -4.775208708426028e-5 -2.5409693080034265e-5 7.591687335918585e-6 0.00011534390159821088 -0.00015416799487840587 -2.0335547560276554e-5 -6.331477834438955e-5 0.00016125945057890732 -4.540443387642542e-5 1.9163630895748854e-5 6.950787883423187e-5 7.672932497396827e-5 -6.484157256610741e-5 -1.9393311049239834e-5 1.4185236978195102e-5 0.0001473621241532582 -8.95844632393345e-5 0.00018367340180727918 -2.6433709535667355e-5 6.585594747124718e-6 0.00010185551309430941 8.619813979774715e-5 -3.4717597825169725e-6 -3.997779913308183e-5 2.3616014914551076e-5 -2.63160809479066e-5 0.00022985375926535572; -0.0001559074115996235 3.2331194315476374e-5 -4.890794034731576e-6 -0.00011219705257667261 -3.5792185290342753e-5 -2.3049548510657626e-6 0.00020679663074566048 -2.06610983789208e-5 2.81375522695677e-6 -3.717862265170678e-5 -1.7452810343679628e-5 6.0096590541210817e-5 3.472123725236716e-5 -0.00017239979561986312 -2.5925883083114988e-5 -0.00013413560728644923 0.00015813822764335032 0.0001404479736172602 -0.00013199541345806668 6.1016347981198496e-5 8.263860589954412e-5 -0.00010763267701827329 0.00012680853623328007 -1.896853419589179e-5 4.379788887468307e-5 7.210932752946492e-5 5.486475501107193e-5 0.00014672021287855984 -7.48078199742359e-5 -5.188165951972205e-5 0.00016848296590046868 -5.266779763609969e-5; -0.00018211911149490352 0.00022923113944772652 -5.0249331915329287e-5 1.50120934186983e-5 8.91223164863911e-5 -9.50745459057368e-5 3.065978969163425e-5 -0.00015601891275662826 -4.408660673005281e-5 8.512935825921665e-5 -0.00012687509363684538 -4.773003886737635e-5 -9.324320920212499e-5 1.5834562210438314e-5 4.8913923156868715e-5 1.8481759317291004e-5 0.00023573439036328377 -4.9180890281496175e-5 -0.00011489574601808303 4.768967092895072e-6 0.00026578090844042306 -1.907459839356695e-5 -2.9712992481006527e-5 -4.080408391826853e-5 9.937422807626698e-5 0.00020834956310056393 1.2001217574943265e-5 -0.00012790382673200554 5.376274136426115e-5 4.0614478412968307e-5 6.384567611450428e-5 -0.000162398370529336; 6.825093374578964e-5 5.197363839830605e-7 -3.335581708591681e-5 4.980564996009565e-5 2.6537222857452056e-6 -8.809148925132326e-5 1.3807100240288665e-5 -4.1632666343442166e-5 0.000112948647492945 -0.00015741255696405715 0.0001046846657664584 -1.877159842473196e-5 -1.784083609884784e-5 -9.540280074355407e-5 -6.37152559101506e-5 -0.0001746790737706161 -0.000231588849364864 0.00024108002913641536 0.00010457094982490636 6.284408233119904e-5 -0.00015179132673431847 -6.077052306857798e-5 1.8313955715777306e-5 -5.211211167982715e-5 1.0136942601152516e-5 -1.0156773660068764e-5 -0.00011138957846918396 -0.0001394896032615441 8.460613686117172e-5 0.00014050759569733472 7.175846130909918e-6 0.0002610481817646952; -7.002370826366617e-5 -5.106787780813939e-5 0.0001432925769623622 -6.0367432029938036e-5 0.0002614535756439225 -9.752449346610743e-6 -0.00011116959054188046 1.4564485849501682e-5 5.4732353004606066e-5 0.0001050108826749296 -2.8776770210147576e-5 -0.00021153228722677796 -1.972661216849735e-5 -0.00017698574943633054 -7.08909951353186e-5 -0.00021033249636811404 0.00012432185625958184 3.8996232018785184e-5 -0.00011144529840375548 0.0001173811674963009 0.00026127356845255187 -6.956327838988016e-5 -1.3386972073830855e-5 -0.0001781363402696047 3.5561503449666485e-5 7.980289823909054e-5 -0.0001052690654031679 -7.726351164446e-5 4.9727589197669095e-5 2.7081604043003266e-5 2.1699660050313614e-5 0.00010321408494210545; 0.000208557849988628 0.00011113947141571466 4.709861174396069e-6 0.000137855812796677 1.8843950898789558e-5 9.677444643462762e-5 0.00011414959328836457 -4.9486002238647907e-5 -7.359415916236027e-6 -7.687753065833284e-5 -6.084503445303279e-5 -0.00012734968447470352 -9.817417128355986e-5 -0.00019467959179886857 3.383134492329044e-5 -0.00012810744635639062 -0.000124850436689979 -1.0283093500712565e-5 -0.0001018967840296145 0.00020154964761464605 -0.00010026951610920748 6.482763740355533e-5 -8.700202567847338e-5 0.00020038443210656232 5.832085764855876e-5 -1.9711788694204284e-5 -4.4581461109865615e-5 8.606037329305128e-5 3.70740747390149e-6 8.007586177559976e-5 1.0884558947925516e-5 -3.4412269882105496e-5; -7.422697903624872e-5 -7.489481735777707e-5 -0.00024892094601913496 3.70786051052727e-5 9.689584608618977e-5 1.0862010144555884e-5 -0.0002405210130246256 3.98886090397015e-5 -0.00030288317297621684 3.254935059427534e-5 -3.1010683001150635e-5 -8.407645013452556e-5 0.00018992414466535698 0.00017651922594719507 6.387575327149608e-5 6.626286399747263e-5 -5.6598257783694784e-5 -4.528130970815243e-5 -2.4962437732975195e-5 -6.458242800821592e-5 -7.741878603182283e-6 -8.642088009313119e-5 5.916942741803085e-5 0.00019641761669387614 7.140736620836752e-5 7.491552737970756e-5 0.0001599588262751778 -2.1445339866684955e-5 -0.0001259656733464354 1.5147155368109755e-5 -0.0002051364886438361 7.203133050548713e-5; -2.159497881946504e-5 -2.134217385318269e-5 0.00011487463569794437 -0.00021546796989567625 4.5991409648578535e-5 -0.00021722840239233896 5.4532623550054356e-5 5.8879156189621935e-6 -0.00015892369475621366 -0.00025069248141661675 0.00018308068473417853 2.750194624824808e-5 0.00012474154726659566 -3.933783438113124e-5 0.0001878828167595362 -1.2461244419524062e-6 -9.198580875235234e-5 1.8251677544339422e-5 0.00011640213568755016 1.5054546280606808e-5 0.0003478199629957018 0.0001561621153872714 -1.003409335099896e-5 0.00010481063298939619 -9.1638469087755e-5 2.6819804465232837e-6 0.00016895208471980533 5.70714088884325e-5 0.00010959616127949386 0.00014088963297254262 -7.130167279100299e-5 5.2245800985857894e-5; 6.238586386123643e-5 9.117573674690805e-5 0.0002045495991517601 0.0002084776267331782 0.0001809469019683818 -3.228003449814601e-5 -4.4114983462099916e-5 -0.00012658564926435023 -0.00015167921926669987 0.00010113434720940033 0.00010558566166244384 0.00012806956394731785 -5.997621817709457e-5 -1.2931565175423867e-5 -3.4507907253758794e-5 -6.619961029693845e-5 -6.625794892509532e-5 0.00010583802825229028 -0.00020346984936233685 0.00019539318484410414 -4.216923232501825e-5 0.00020606276550485159 -2.4115727934031092e-5 0.0001258139879830786 7.688773505313301e-5 -0.00010003003378708063 8.236302412504309e-5 -3.0223586941926334e-5 4.603920272009079e-5 -5.898235510879696e-5 -6.380496649651563e-7 -0.0001438044794446979; 0.0001531485357827098 -5.8804897098564516e-5 -5.292066480289389e-6 0.0001559643459313135 7.959446144919472e-5 0.00013301863861666608 -4.390488475148462e-6 7.376929340349078e-5 0.00012489618154173332 4.809717199346536e-5 0.00010628525694297254 -0.0001474218262319864 2.0827568614509572e-5 1.4280112618469538e-5 1.2789384409150318e-5 -0.000102658024961822 2.7062434919532456e-5 7.439761129679035e-6 -8.629096931148694e-5 1.694813897298262e-5 -1.990011344791509e-5 -1.0614390821541168e-5 -1.1935471893634369e-5 -2.9862820274541477e-5 -0.00010493927778046945 -5.5913045916933624e-5 -2.2774433209671578e-5 -0.00014706112790922307 -0.00017280460099758978 1.1295477515847536e-5 -7.577555891661407e-5 0.0001397165213984028; -7.245906754784296e-5 -0.00019401008040099318 -0.00020467261793479338 -8.61375986520706e-5 -1.811666816744006e-5 -0.0001308141393625719 -4.010882485695678e-5 0.00017401066424404033 -9.881015585657028e-5 -0.0001027414939987017 -7.136763024996941e-5 4.2205666153078866e-5 0.00011042508839943037 -0.00013644821893345413 5.977256359378199e-5 -3.1758045739991064e-6 5.8732520022523096e-5 5.9778820917328787e-5 0.00012411661436393897 -7.071601003795665e-5 -1.4847416702318032e-5 0.00021108418576964766 -7.310654773994394e-5 -0.00012905169870159122 0.0001685039575871451 -0.00017227842482194194 -4.357306471016276e-5 8.47789603479828e-5 -0.00011879198000920598 9.907540237105236e-6 -0.00017377811610154093 0.0001049144600013644; 3.421541212782677e-5 0.00017848797095547202 -1.1482518417602913e-5 -0.00010183441237705685 0.00024135885131153673 -0.00017500171135571485 -1.0641140321522063e-5 -3.1496500483909854e-5 4.25419125201117e-5 -4.2858675911986464e-5 0.00015603045129291171 -5.6242370661773574e-5 0.00011752773524101682 -0.00018053753639497483 -0.00011825127195725942 -7.634202530999464e-5 6.4720129409459e-6 0.0001642447746086802 6.444661219911101e-5 9.385125984744453e-5 0.00015255711285469946 -8.758773689184584e-6 -4.673478589786585e-5 2.855557021001359e-5 6.577653355970464e-5 -3.854153375533071e-5 4.006243715020314e-5 0.0001953394671397077 -9.223385178875912e-5 -8.570309106577303e-6 1.4442015057000899e-5 -0.0001385447835821711; 2.1407898503261834e-5 9.593560692391147e-5 -4.7385816896544564e-5 4.731966267225899e-5 -0.0002482421229337115 -4.732200674826759e-5 2.6717555857054658e-5 -8.264070190253982e-5 1.5496321184933076e-5 -8.519940153693118e-5 -4.183419939128277e-5 -3.148195598388567e-5 0.00010269012575972111 -5.639988800741287e-5 1.7421070270380343e-5 2.382899838799854e-6 8.149160869887804e-5 -0.00011918021179500511 6.375104870183805e-5 2.5057718931371288e-5 -8.191323710835339e-5 6.434507243337986e-5 3.430005684148854e-5 9.406639886091372e-5 0.00011397560168059937 3.534246509897024e-5 -0.00016860544262218825 -0.00018563338077856912 -6.822332640641795e-5 -1.4890337906226898e-5 0.00023276656004835546 -2.746749184499562e-5; 6.616636744536835e-5 1.2839582608552799e-5 -0.00010429555334314332 9.093474579812071e-5 -0.00024482527590054617 3.072294782567414e-5 -0.00013715422903816277 0.00010155072447913914 -7.178858288727115e-5 3.583997238264792e-5 -8.436354039633233e-5 -0.00016870042384299792 7.175549615013621e-5 -9.25647524364796e-5 -8.654769556061554e-5 -0.00011310309275915098 -1.9113503754948588e-5 0.0001406830145914539 -3.469735168084071e-5 2.800677645962824e-5 6.12920506445426e-5 -1.4721743924473833e-5 0.00011409716751033118 4.691628635000494e-5 1.516279314626611e-5 6.706858618952844e-5 -6.650139914756272e-5 -8.033022970820454e-5 -4.3628393201392015e-5 -4.747605783095499e-5 -6.403889494878001e-5 -0.0002115731164177468], bias = [-3.1191545595130843e-10, -2.0212426596476283e-9, 7.112152827835879e-11, -3.243830175005229e-10, 2.3623130303215395e-9, 4.857273311000907e-10, -1.6142140561750679e-9, -5.454064863858324e-10, -4.3685295932451826e-10, -1.5990931993431986e-9, 9.550369611522905e-10, -3.057664478431455e-9, 1.7217091029342412e-9, 2.7886926096957105e-9, -1.7295348785206703e-9, 1.7609525343764992e-9, -3.63288923365011e-9, 7.630274419652513e-10, 6.557459384335777e-10, 8.331787786529424e-10, 6.541732726041285e-10, -6.045774488692488e-10, -5.106411621077796e-10, 6.012910118557003e-10, -1.2392274956113632e-9, 3.1120056717862873e-9, 2.625460438220049e-9, 2.5090226537188095e-10, -2.6660078765665115e-9, 1.6730006732762932e-9, -8.555338492493487e-10, -2.951879758423589e-9]), layer_4 = (weight = [-0.0008930617061522622 -0.0008185306631164176 -0.0007798362903145225 -0.00046118253009551705 -0.000897480944822825 -0.0008989226899086836 -0.0007221957419533718 -0.000662921138863801 -0.0006982017109555845 -0.0007461286129488532 -0.0008023907532141756 -0.0006913845436001116 -0.000505657627539614 -0.0007713635499856448 -0.0005536322785323924 -0.0006477029338901213 -0.000641742644755058 -0.000757746606194418 -0.0006001426243246246 -0.0008224149504703233 -0.0006648955565676422 -0.0007083074698248665 -0.0005936614162959661 -0.0006482839826710184 -0.0006140691446771656 -0.0006285802307338592 -0.0006514905371313426 -0.0006512250660108998 -0.0007275369507994201 -0.0006771698666891329 -0.0007006402569624884 -0.0005875524598459974; 0.0001792947104920181 0.0002983343139061395 0.000150136063990197 0.00039841043602634435 0.0002460896645807945 0.00027568385565150753 0.00028777974423444203 0.00023729097376672882 0.0003556723695087773 0.00017636516606650494 0.00016136170006947375 0.0002398825145134938 0.00018523919873291076 0.00022491736075073136 0.00013145073567494362 0.0002141998426323734 7.340545978743833e-5 0.0003237640460638284 0.00015056999791557195 0.0002671154860365163 0.00012047243166020841 0.00013201640836588033 0.00021678054810414323 0.0003935834320404575 8.124913038213655e-5 0.00021554910660425 0.0003650959602604641 0.0002931727880915839 0.00040863465278771955 0.00022404868707168502 4.198412722765804e-5 0.0003513619042523084], bias = [-0.0006651773484053201, 0.0002744319173546264]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
-
-    lines!(ax, losses; linewidth=4, alpha=0.75)
-    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
-
-    fig
-end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
-waveform_nn_trained = first(compute_waveform(
-    dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
-    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
-        alpha=0.5, strokewidth=2, markersize=12)
-
-    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
-        position=:lb)
-
-    fig
-end
    Appendix ​
    julia
    using InteractiveUtils
-InteractiveUtils.versioninfo()
-
-if @isdefined(MLDataDevices)
-    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
-        println()
-        CUDA.versioninfo()
-    end
-
-    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
-        println()
-        AMDGPU.versioninfo()
-    end
-end
    Julia Version 1.10.6
-Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
-Build Info:
-  Official https://julialang.org/ release
-Platform Info:
-  OS: Linux (x86_64-linux-gnu)
-  CPU: 128 × AMD EPYC 7502 32-Core Processor
-  WORD_SIZE: 64
-  LIBM: libopenlibm
-  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
-Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
-Environment:
-  JULIA_CPU_THREADS = 16
-  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
-  JULIA_PKG_SERVER = 
-  JULIA_NUM_THREADS = 16
-  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
-  JULIA_PKG_PRECOMPILE_AUTO = 0
-  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const Z=t(e,[["render",X]]);export{S as __pageData,Z as default};
diff --git a/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.DNi4Y6LY.lean.js b/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.DNi4Y6LY.lean.js
deleted file mode 100644
index f299a14add..0000000000
--- a/previews/PR1000/assets/tutorials_advanced_1_GravitationalWaveForm.md.DNi4Y6LY.lean.js
+++ /dev/null
@@ -1,286 +0,0 @@
-import{_ as t,c as i,a2 as n,j as A,a as h,o as a}from"./chunks/framework.DjZDIZsN.js";const S=JSON.parse('{"title":"Training a Neural ODE to Model Gravitational Waveforms","description":"","frontmatter":{},"headers":[],"relativePath":"tutorials/advanced/1_GravitationalWaveForm.md","filePath":"tutorials/advanced/1_GravitationalWaveForm.md","lastUpdated":null}'),e={name:"tutorials/advanced/1_GravitationalWaveForm.md"},l={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},p={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"10.819ex",height:"1.658ex",role:"img",focusable:"false",viewBox:"0 -583 4782.1 733","aria-hidden":"true"},k={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},E={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},r={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},d={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.339ex"},xmlns:"http://www.w3.org/2000/svg",width:"2.008ex",height:"1.339ex",role:"img",focusable:"false",viewBox:"0 -442 887.6 592","aria-hidden":"true"},Q={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},o={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"24.527ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 10840.9 1000","aria-hidden":"true"},g={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},C={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},f={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},v={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},y={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},I={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},u={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},c={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},m={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},F={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"},q={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},B={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.117ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3587.6 1000","aria-hidden":"true"},V={class:"MathJax",jax:"SVG",display:"true",style:{direction:"ltr",display:"block","text-align":"center",margin:"1em 0",position:"relative"}},b={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.566ex"},xmlns:"http://www.w3.org/2000/svg",width:"8.049ex",height:"2.262ex",role:"img",focusable:"false",viewBox:"0 -750 3557.6 1000","aria-hidden":"true"},T={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},D={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.439ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.138ex",height:"1.439ex",role:"img",focusable:"false",viewBox:"0 -442 503 636","aria-hidden":"true"},z={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},U={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"0"},xmlns:"http://www.w3.org/2000/svg",width:"2.378ex",height:"1.545ex",role:"img",focusable:"false",viewBox:"0 -683 1051 683","aria-hidden":"true"},P={class:"MathJax",jax:"SVG",style:{direction:"ltr",position:"relative"}},K={style:{overflow:"visible","min-height":"1px","min-width":"1px","vertical-align":"-0.025ex"},xmlns:"http://www.w3.org/2000/svg",width:"1.054ex",height:"1.025ex",role:"img",focusable:"false",viewBox:"0 -442 466 453","aria-hidden":"true"};function X(R,s,L,w,j,W){return a(),i("div",null,[s[41]||(s[41]=n(`
    Training a Neural ODE to Model Gravitational Waveforms ​
    This code is adapted from Astroinformatics/ScientificMachineLearning
    The code has been minimally adapted from Keith et. al. 2021 which originally used Flux.jl
    Package Imports ​
    julia
    using Lux, ComponentArrays, LineSearches, OrdinaryDiffEqLowOrderRK, Optimization,
-      OptimizationOptimJL, Printf, Random, SciMLSensitivity
-using CairoMakie
    Define some Utility Functions ​
    Tip
    This section can be skipped. It defines functions to simulate the model, however, from a scientific machine learning perspective, isn't super relevant.
    `,7)),A("p",null,[s[6]||(s[6]=h("We need a very crude 2-body path. Assume the 1-body motion is a newtonian 2-body position vector ")),A("mjx-container",l,[(a(),i("svg",p,s[0]||(s[0]=[n('',1)]))),s[1]||(s[1]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"r"),A("mo",null,"="),A("msub",null,[A("mi",null,"r"),A("mn",null,"1")]),A("mo",null,"−"),A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[7]||(s[7]=h(" and use Newtonian formulas to get ")),A("mjx-container",k,[(a(),i("svg",E,s[2]||(s[2]=[n('',1)]))),s[3]||(s[3]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"1")])])],-1))]),s[8]||(s[8]=h(", ")),A("mjx-container",r,[(a(),i("svg",d,s[4]||(s[4]=[n('',1)]))),s[5]||(s[5]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("msub",null,[A("mi",null,"r"),A("mn",null,"2")])])],-1))]),s[9]||(s[9]=h(" (e.g. Theoretical Mechanics of Particles and Continua 4.3)"))]),s[42]||(s[42]=n(`
    julia
    function one2two(path, m₁, m₂)
-    M = m₁ + m₂
-    r₁ = m₂ / M .* path
-    r₂ = -m₁ / M .* path
-    return r₁, r₂
-end
    one2two (generic function with 1 method)
    `,2)),A("p",null,[s[12]||(s[12]=h("Next we define a function to perform the change of variables: ")),A("mjx-container",Q,[(a(),i("svg",o,s[10]||(s[10]=[n('',1)]))),s[11]||(s[11]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mo",{stretchy:"false"},"("),A("mi",null,"χ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"ϕ"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},"↦"),A("mo",{stretchy:"false"},"("),A("mi",null,"x"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",null,","),A("mi",null,"y"),A("mo",{stretchy:"false"},"("),A("mi",null,"t"),A("mo",{stretchy:"false"},")"),A("mo",{stretchy:"false"},")")])],-1))])]),s[43]||(s[43]=n(`
    julia
    @views function soln2orbit(soln, model_params=nothing)
-    @assert size(soln, 1) ∈ [2, 4] "size(soln,1) must be either 2 or 4"
-
-    if size(soln, 1) == 2
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-
-        @assert length(model_params)==3 "model_params must have length 3 when size(soln,2) = 2"
-        p, M, e = model_params
-    else
-        χ = soln[1, :]
-        ϕ = soln[2, :]
-        p = soln[3, :]
-        e = soln[4, :]
-    end
-
-    r = p ./ (1 .+ e .* cos.(χ))
-    x = r .* cos.(ϕ)
-    y = r .* sin.(ϕ)
-
-    orbit = vcat(x', y')
-    return orbit
-end
    soln2orbit (generic function with 2 methods)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d_dt(v::AbstractVector, dt)
-    a = -3 / 2 * v[1] + 2 * v[2] - 1 / 2 * v[3]
-    b = (v[3:end] .- v[1:(end - 2)]) / 2
-    c = 3 / 2 * v[end] - 2 * v[end - 1] + 1 / 2 * v[end - 2]
-    return [a; b; c] / dt
-end
    d_dt (generic function with 1 method)
    This function uses second-order one-sided difference stencils at the endpoints; see https://doi.org/10.1090/S0025-5718-1988-0935077-0
    julia
    function d2_dt2(v::AbstractVector, dt)
-    a = 2 * v[1] - 5 * v[2] + 4 * v[3] - v[4]
-    b = v[1:(end - 2)] .- 2 * v[2:(end - 1)] .+ v[3:end]
-    c = 2 * v[end] - 5 * v[end - 1] + 4 * v[end - 2] - v[end - 3]
-    return [a; b; c] / (dt^2)
-end
    d2_dt2 (generic function with 1 method)
    Now we define a function to compute the trace-free moment tensor from the orbit
    julia
    function orbit2tensor(orbit, component, mass=1)
-    x = orbit[1, :]
-    y = orbit[2, :]
-
-    Ixx = x .^ 2
-    Iyy = y .^ 2
-    Ixy = x .* y
-    trace = Ixx .+ Iyy
-
-    if component[1] == 1 && component[2] == 1
-        tmp = Ixx .- trace ./ 3
-    elseif component[1] == 2 && component[2] == 2
-        tmp = Iyy .- trace ./ 3
-    else
-        tmp = Ixy
-    end
-
-    return mass .* tmp
-end
-
-function h_22_quadrupole_components(dt, orbit, component, mass=1)
-    mtensor = orbit2tensor(orbit, component, mass)
-    mtensor_ddot = d2_dt2(mtensor, dt)
-    return 2 * mtensor_ddot
-end
-
-function h_22_quadrupole(dt, orbit, mass=1)
-    h11 = h_22_quadrupole_components(dt, orbit, (1, 1), mass)
-    h22 = h_22_quadrupole_components(dt, orbit, (2, 2), mass)
-    h12 = h_22_quadrupole_components(dt, orbit, (1, 2), mass)
-    return h11, h12, h22
-end
-
-function h_22_strain_one_body(dt::T, orbit) where {T}
-    h11, h12, h22 = h_22_quadrupole(dt, orbit)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-    h11_1, h12_1, h22_1 = h_22_quadrupole(dt, orbit1, mass1)
-    h11_2, h12_2, h22_2 = h_22_quadrupole(dt, orbit2, mass2)
-    h11 = h11_1 + h11_2
-    h12 = h12_1 + h12_2
-    h22 = h22_1 + h22_2
-    return h11, h12, h22
-end
-
-function h_22_strain_two_body(dt::T, orbit1, mass1, orbit2, mass2) where {T}
-    # compute (2,2) mode strain from orbits of BH 1 of mass1 and BH2 of mass 2
-
-    @assert abs(mass1 + mass2 - 1.0)<1e-12 "Masses do not sum to unity"
-
-    h11, h12, h22 = h_22_quadrupole_two_body(dt, orbit1, mass1, orbit2, mass2)
-
-    h₊ = h11 - h22
-    hₓ = T(2) * h12
-
-    scaling_const = √(T(π) / 5)
-    return scaling_const * h₊, -scaling_const * hₓ
-end
-
-function compute_waveform(dt::T, soln, mass_ratio, model_params=nothing) where {T}
-    @assert mass_ratio≤1 "mass_ratio must be <= 1"
-    @assert mass_ratio≥0 "mass_ratio must be non-negative"
-
-    orbit = soln2orbit(soln, model_params)
-    if mass_ratio > 0
-        m₂ = inv(T(1) + mass_ratio)
-        m₁ = mass_ratio * m₂
-
-        orbit₁, orbit₂ = one2two(orbit, m₁, m₂)
-        waveform = h_22_strain_two_body(dt, orbit₁, m₁, orbit₂, m₂)
-    else
-        waveform = h_22_strain_one_body(dt, orbit)
-    end
-    return waveform
-end
    compute_waveform (generic function with 2 methods)
    Simulating the True Model ​
    RelativisticOrbitModel defines system of odes which describes motion of point like particle in schwarzschild background, uses
    `,13)),A("mjx-container",g,[(a(),i("svg",C,s[13]||(s[13]=[n('',1)]))),s[14]||(s[14]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",f,[(a(),i("svg",v,s[15]||(s[15]=[n('',1)]))),s[16]||(s[16]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[23]||(s[23]=h("where, ")),A("mjx-container",y,[(a(),i("svg",I,s[17]||(s[17]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[18]||(s[18]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[24]||(s[24]=h(", ")),A("mjx-container",u,[(a(),i("svg",c,s[19]||(s[19]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[20]||(s[20]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[25]||(s[25]=h(", and ")),A("mjx-container",m,[(a(),i("svg",F,s[21]||(s[21]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[22]||(s[22]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[26]||(s[26]=h(" are constants"))]),s[44]||(s[44]=n(`
    julia
    function RelativisticOrbitModel(u, (p, M, e), t)
-    χ, ϕ = u
-
-    numer = (p - 2 - 2 * e * cos(χ)) * (1 + e * cos(χ))^2
-    denom = sqrt((p - 2)^2 - 4 * e^2)
-
-    χ̇ = numer * sqrt(p - 6 - 2 * e * cos(χ)) / (M * (p^2) * denom)
-    ϕ̇ = numer / (M * (p^(3 / 2)) * denom)
-
-    return [χ̇, ϕ̇]
-end
-
-mass_ratio = 0.0         # test particle
-u0 = Float64[π, 0.0]     # initial conditions
-datasize = 250
-tspan = (0.0f0, 6.0f4)   # timespace for GW waveform
-tsteps = range(tspan[1], tspan[2]; length=datasize)  # time at each timestep
-dt_data = tsteps[2] - tsteps[1]
-dt = 100.0
-const ode_model_params = [100.0, 1.0, 0.5]; # p, M, e
    Let's simulate the true model and plot the results using OrdinaryDiffEq.jl
    julia
    prob = ODEProblem(RelativisticOrbitModel, u0, tspan, ode_model_params)
-soln = Array(solve(prob, RK4(); saveat=tsteps, dt, adaptive=false))
-waveform = first(compute_waveform(dt_data, soln, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s = scatter!(ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5)
-
-    axislegend(ax, [[l, s]], ["Waveform Data"])
-
-    fig
-end
    Defiing a Neural Network Model ​
    Next, we define the neural network model that takes 1 input (time) and has two outputs. We'll make a function ODE_model that takes the initial conditions, neural network parameters and a time as inputs and returns the derivatives.
    It is typically never recommended to use globals but incase you do use them, make sure to mark them as const.
    We will deviate from the standard Neural Network initialization and use WeightInitializers.jl,
    julia
    const nn = Chain(Base.Fix1(fast_activation, cos),
-    Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32),
-    Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32))
-ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[-8.733466f-6; -0.00010797254; -6.034535f-6; 7.889705f-5; -1.9253392f-5; -4.612653f-5; 7.144778f-5; -8.932017f-5; -4.7168105f-5; 9.798891f-5; -2.1554477f-5; 4.7397516f-5; 0.00016439424; -4.36064f-5; -6.190079f-5; 6.240796f-5; 0.000105714986; 4.7386144f-5; 2.5381063f-5; 4.9141305f-5; 0.00016239625; 0.00013147807; -0.00017642805; -0.00012611185; 3.0797285f-6; -3.7879247f-5; -1.162709f-6; -0.00016794122; 6.32018f-5; 0.00012728853; 9.064739f-5; -1.5429378f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-2.8177601f-5 -3.936497f-5 -7.657458f-5 0.00013857176 -6.853039f-7 -5.7781497f-5 -5.1510856f-6 3.9836163f-5 0.00012864341 -6.5123946f-5 5.465938f-5 -6.377014f-5 7.9869205f-5 -9.11579f-5 -5.0697596f-5 -1.4907303f-6 -3.207327f-5 -6.482217f-5 2.0175543f-5 9.1325936f-5 1.7174554f-5 6.39053f-5 -4.3893502f-5 -0.00011969867 -6.29855f-5 1.9702458f-5 -7.764152f-6 1.9585927f-6 7.7942015f-5 0.00010650995 -8.729355f-5 -3.0301662f-5; 5.0932027f-5 -3.3675303f-6 0.00010612095 -9.0758826f-5 -6.206741f-5 2.0948732f-5 6.1964456f-5 -8.0493766f-5 9.118536f-5 9.570129f-5 -0.00025590206 -8.4574596f-5 0.00011611364 0.00015786408 -0.0001325249 -1.472367f-5 2.512089f-6 5.8462672f-5 5.2352f-5 -4.69843f-5 -0.00024784482 -4.0580584f-5 -3.3547942f-6 7.7080156f-5 -5.3825785f-5 -0.00012691545 -0.00018019084 3.0379057f-5 -1.2276903f-5 -0.00022681341 0.00017366803 -3.039549f-6; 0.00010137846 -0.0001010016 -2.4273795f-6 -4.770885f-6 6.701655f-5 0.00017136613 -0.0001906262 -1.0672023f-6 0.00012427552 -3.5586767f-5 8.55602f-5 -8.816304f-7 -4.3871598f-5 0.0001563755 -2.259477f-5 4.3149896f-5 -0.00011615122 -1.5891324f-5 -7.964024f-5 1.08097465f-5 3.3177996f-5 0.0001521374 -0.00012456263 4.1607735f-5 -1.4163393f-5 3.9037423f-5 -4.4479628f-5 -3.5756973f-5 7.252298f-5 -0.00011933267 -7.04208f-5 -5.54682f-5; -8.33601f-5 0.00011739388 -1.6886734f-5 -9.9290606f-5 0.00018311598 0.00016209438 -0.00012484103 1.6862592f-5 -0.00012521839 -2.3408023f-5 9.5157055f-5 -6.6844086f-6 5.5988305f-5 0.00012589077 -3.7646063f-5 -0.00011733096 5.898576f-5 -3.2169828f-5 0.00016092892 -0.000102152466 -0.00013990636 -0.00021372396 0.00013839726 -2.180175f-5 -5.539738f-6 0.00017145288 -6.4497464f-5 0.00010669653 2.4381146f-5 -8.664719f-5 -0.00010137241 -0.00010135833; 6.767996f-6 7.979176f-5 0.00010768214 4.6958052f-5 8.351189f-5 5.7730363f-6 -7.494067f-5 2.7757571f-5 -5.5647397f-5 4.7348538f-5 6.738859f-5 -1.7598553f-5 6.2236526f-5 -0.00010106445 6.811721f-5 -1.7854622f-5 0.00014259979 -3.1129297f-5 2.758307f-5 -0.00021257819 -2.0857597f-5 -0.00015710465 0.000115748146 -9.06086f-5 0.00012642707 -6.715701f-5 -2.3087605f-5 0.00014910278 0.00017517559 0.00012419421 -2.7270444f-5 0.00010731906; 5.8766418f-5 -0.00015708971 2.6431195f-5 -0.000103879465 2.2029622f-5 4.658656f-5 -0.00014493923 7.2192146f-5 -2.4574167f-5 0.00013603218 0.00013269106 8.4467785f-5 9.409519f-5 -0.00016357354 1.9661726f-5 0.0001598959 -0.00014350242 -1.5808671f-5 1.605866f-5 -3.59838f-5 1.479661f-5 1.8547255f-5 4.254646f-5 -9.158475f-5 -0.00013229327 -1.3919468f-5 -1.956731f-5 -5.49026f-5 -3.0198875f-5 3.3093645f-5 0.00019555296 9.743678f-5; 6.583076f-5 -1.9019628f-5 -2.3915383f-5 -8.268265f-5 8.264759f-5 0.000113464965 0.00019279975 5.6251043f-5 -3.9762614f-5 -8.271291f-5 6.1018898f-5 -7.841602f-5 -1.2949409f-5 0.00013066275 -1.5204919f-5 -0.00012379164 4.110425f-5 -6.5780056f-5 -1.7211657f-7 -4.4427048f-5 -1.1617677f-6 -0.000109266395 -0.00018221501 -0.00013746502 0.0002607908 -1.48712015f-5 1.9145657f-5 -6.3632055f-5 -3.2119045f-5 -0.00013502994 -5.4510227f-5 -0.0001521188; 1.6407535f-5 -1.9113382f-5 0.00012204645 -7.556568f-5 -1.6685279f-5 -1.5900963f-5 6.0559876f-5 -1.5047506f-5 -8.193592f-5 -2.9995304f-5 -1.0388885f-5 -3.103509f-5 -0.00017250188 3.3705473f-5 -4.3459997f-5 -4.783416f-5 2.276422f-6 5.006527f-6 -0.00015602457 2.369959f-5 3.2234122f-5 -0.00012310114 -2.8645542f-5 5.168618f-5 0.00017521593 -3.8919454f-5 0.00016759252 4.4987402f-5 2.9359122f-5 -9.714238f-5 -8.786319f-6 9.830501f-5; 1.0688757f-6 -5.82454f-5 3.961646f-5 5.784082f-5 2.6230875f-5 -4.110797f-6 -1.2114489f-5 9.190244f-5 5.677286f-5 -0.000111494875 7.906678f-5 2.2018543f-5 -5.5081386f-5 -9.197007f-6 -6.6534965f-5 -6.8599526f-5 8.8153116f-5 2.2247887f-5 -3.938612f-5 -2.3031727f-5 -0.00015165182 0.00014176163 -0.00012151037 2.591397f-5 -0.00015541828 9.689811f-5 -0.000120833436 -5.9194394f-5 -2.0094647f-5 1.9722645f-5 1.8458933f-5 0.00016762815; 2.9945206f-5 -0.0002136077 4.2307387f-5 -0.00014859898 1.6029056f-5 0.00013049919 -0.000121818346 0.00011504457 -3.660613f-5 0.00013795843 1.5452692f-5 6.207062f-5 -4.9132796f-5 -5.6330246f-5 7.049317f-5 -0.0001624591 0.00020378857 5.0185274f-5 -6.563782f-6 2.5135363f-5 -0.00013978432 6.94403f-5 -5.7515543f-5 6.4258464f-5 -8.477129f-5 2.7108229f-6 -6.27297f-5 -0.00011133804 -0.000118937336 -2.491081f-5 -6.5520544f-5 -1.6221407f-5; 0.0001490329 -3.518714f-5 6.838341f-5 1.806367f-5 -1.2177103f-5 4.8077785f-5 4.8774464f-6 2.5176876f-5 -2.393502f-5 -0.00017052738 -8.472462f-5 2.6156655f-5 3.1476753f-5 -3.198646f-5 1.7295855f-5 -6.5615815f-5 -6.457243f-5 4.945713f-5 -6.379944f-5 0.0001725941 -5.566336f-5 -0.00013105926 -2.3267385f-5 0.00013213923 5.1767678f-5 4.8656395f-5 0.00011115292 2.2263952f-5 9.953958f-6 -5.85899f-5 -1.4107241f-5 0.00011483148; -4.1415296f-5 1.3916467f-5 -0.00011940488 -9.851123f-5 7.3574214f-5 4.740405f-5 -6.785301f-5 0.00019942185 0.00012728834 -0.00016363562 -0.00012561417 -1.6128222f-5 -0.00019293881 4.5368397f-6 4.713364f-5 -0.00013651798 -6.237191f-5 -0.00019911939 -0.00017180503 2.6719978f-5 -4.0074345f-5 -1.359718f-5 -8.151072f-5 5.2146228f-5 -1.9828212f-5 7.461231f-5 0.00013041316 -9.9392455f-5 8.682513f-5 0.00013921101 -5.4267803f-5 -0.00015905981; -4.4684915f-5 -5.462887f-5 0.00012135054 -4.510381f-5 -3.475299f-5 0.00010407074 2.2008955f-5 -0.00013626451 -2.2398599f-5 -2.954842f-5 0.00015306505 5.706277f-6 -3.3192628f-5 0.00020575941 6.463371f-5 -9.893465f-5 8.663553f-5 -0.00011787723 -0.00014052064 5.1350125f-5 -1.1286868f-6 0.00011688338 0.00013804293 4.82576f-5 -5.091659f-5 5.691207f-6 -1.1650249f-6 0.00010991485 7.0106557f-6 -5.4914188f-5 0.000103280305 -2.2072318f-5; 3.8976705f-5 0.00010581013 4.4845918f-5 0.00028989412 -2.1131393f-5 -0.00010667203 0.00022406431 6.0003676f-5 -5.046534f-5 0.00013748402 0.00016624035 0.00016925113 -7.0310954f-5 -8.645576f-5 0.000104551196 0.00012043912 -0.0001673082 4.826957f-5 -0.00021466143 0.00023090826 -0.00013504991 -2.8148033f-5 8.9335634f-5 2.1197551f-5 7.634965f-6 -0.00018328545 0.00011402708 4.8938728f-5 8.409785f-5 -9.03842f-5 -0.00015597348 -1.9681711f-5; -0.00017860696 0.00016289309 2.1554613f-5 7.65075f-5 -0.0001572884 -4.904941f-5 0.00013334364 8.8976296f-5 0.00011627484 3.0017205f-5 -4.347116f-6 -7.407668f-5 3.352131f-5 -9.5505995f-5 -6.2805775f-5 -3.8850536f-5 -5.6758072f-5 4.625631f-8 -2.8606568f-5 -8.249394f-5 -8.704262f-5 -1.5149345f-5 -9.393877f-6 -5.6657373f-5 3.1522475f-5 -1.299464f-5 -4.861005f-5 7.5044234f-5 -0.00013518587 -7.636546f-5 3.0268136f-6 3.7342343f-5; -1.5246345f-6 -0.00013971738 -4.301577f-5 -1.3734039f-5 0.00028718126 8.435068f-5 0.00015791062 4.3378295f-5 -4.043131f-5 0.00013740394 -0.00012764876 -5.627069f-5 3.8953523f-5 0.00019921537 -8.471506f-5 4.6630197f-5 0.00012246286 -1.7031091f-6 2.0175803f-5 0.00017496663 0.00012815923 4.6586156f-6 9.724115f-5 -0.00014763075 -0.00013615302 0.00021714495 7.7586665f-6 -9.8543445f-5 -3.5388155f-5 -0.0001154024 -3.734798f-5 -0.0002094385; -7.722984f-5 0.00012643216 -0.0001336082 2.5638423f-5 -8.438335f-5 -5.9111102f-5 -4.578554f-5 -7.396497f-6 -6.326183f-5 3.0497316f-5 -0.000102561884 -0.000101490295 0.00013105836 -0.00019522628 3.417967f-5 -0.00011023943 -1.3394814f-5 0.00010105535 -4.936928f-5 0.00019615894 -9.0929265f-5 0.00017270022 9.9116936f-5 1.238063f-5 3.634942f-5 -0.0001720305 -6.9169415f-5 -0.000103437924 1.6598398f-5 -0.00011501487 -0.00020326376 -0.00016199409; -1.0493395f-5 -1.843547f-5 3.4408877f-6 1.48939225f-5 0.00016012433 -1.5305499f-5 1.7829227f-5 7.7386605f-5 2.445304f-5 -4.7624897f-5 -1.5142531f-5 -9.343951f-6 0.00013970489 -0.0001309227 -0.0002114561 0.00018979193 3.2718573f-5 9.215379f-5 0.000120665085 -0.00010754408 6.665616f-6 -7.775345f-5 -6.5193824f-5 4.0987623f-5 3.918128f-5 3.6471756f-5 -4.6466415f-5 0.00012628127 5.7103025f-6 -5.55399f-5 -7.507195f-5 -2.8647954f-5; -6.847904f-5 -9.9914185f-5 -0.000105565225 1.3994042f-5 -0.00018054691 -4.7752743f-5 -2.5410349f-5 7.5910316f-6 0.000115343246 -0.00015416865 -2.0336203f-5 -6.3315434f-5 0.0001612588 -4.540509f-5 1.9162975f-5 6.950722f-5 7.672867f-5 -6.484223f-5 -1.9393967f-5 1.4184581f-5 0.00014736147 -8.958512f-5 0.00018367275 -2.6434365f-5 6.584939f-6 0.00010185486 8.6197484f-5 -3.4724155f-6 -3.9978455f-5 2.361536f-5 -2.6316737f-5 0.0002298531; -0.00015590824 3.233036f-5 -4.891627f-6 -0.000112197886 -3.579302f-5 -2.305788f-6 0.0002067958 -2.0661932f-5 2.812922f-6 -3.7179456f-5 -1.7453644f-5 6.0095757f-5 3.4720404f-5 -0.00017240063 -2.5926716f-5 -0.00013413644 0.0001581374 0.00014044714 -0.00013199625 6.1015515f-5 8.263777f-5 -0.00010763351 0.0001268077 -1.8969367f-5 4.3797056f-5 7.2108494f-5 5.4863922f-5 0.00014671938 -7.480865f-5 -5.1882493f-5 0.00016848213 -5.266863f-5; -0.00018211977 0.00022923049 -5.0249986f-5 1.5011439f-5 8.912166f-5 -9.50752f-5 3.0659136f-5 -0.00015601957 -4.408726f-5 8.5128704f-5 -0.00012687575 -4.7730693f-5 -9.324386f-5 1.5833908f-5 4.891327f-5 1.8481105f-5 0.00023573374 -4.9181544f-5 -0.0001148964 4.768313f-6 0.00026578025 -1.9075253f-5 -2.9713647f-5 -4.0804738f-5 9.9373574f-5 0.00020834891 1.2000563f-5 -0.00012790448 5.3762087f-5 4.0613824f-5 6.384502f-5 -0.00016239902; 6.825154f-5 5.2034096f-7 -3.3355213f-5 4.9806255f-5 2.6543269f-6 -8.8090885f-5 1.3807705f-5 -4.163206f-5 0.00011294925 -0.00015741195 0.00010468527 -1.8770994f-5 -1.7840232f-5 -9.5402196f-5 -6.371465f-5 -0.00017467847 -0.00023158824 0.00024108063 0.000104571554 6.284469f-5 -0.00015179072 -6.076992f-5 1.831456f-5 -5.2111507f-5 1.0137547f-5 -1.0156169f-5 -0.000111388974 -0.000139489 8.460674f-5 0.0001405082 7.1764507f-6 0.0002610488; -7.00232f-5 -5.1067367f-5 0.00014329309 -6.036692f-5 0.0002614541 -9.751939f-6 -0.00011116908 1.45649965f-5 5.4732864f-5 0.00010501139 -2.877626f-5 -0.00021153178 -1.9726102f-5 -0.00017698524 -7.0890484f-5 -0.00021033199 0.00012432237 3.8996743f-5 -0.00011144479 0.00011738168 0.00026127408 -6.956277f-5 -1.3386461f-5 -0.00017813583 3.5562014f-5 7.980341f-5 -0.000105268555 -7.7263f-5 4.97281f-5 2.7082115f-5 2.170017f-5 0.000103214596; 0.00020855725 0.00011113887 4.70926f-6 0.00013785521 1.884335f-5 9.6773845f-5 0.00011414899 -4.9486604f-5 -7.360017f-6 -7.687813f-5 -6.0845636f-5 -0.00012735029 -9.817477f-5 -0.0001946802 3.3830744f-5 -0.00012810805 -0.00012485104 -1.0283695f-5 -0.000101897385 0.00020154905 -0.00010027012 6.4827036f-5 -8.700263f-5 0.00020038383 5.8320256f-5 -1.971239f-5 -4.4582062f-5 8.605977f-5 3.7068062f-6 8.007526f-5 1.0883958f-5 -3.441287f-5; -7.422574f-5 -7.489358f-5 -0.0002489197 3.7079844f-5 9.6897085f-5 1.0863249f-5 -0.00024051977 3.988985f-5 -0.00030288193 3.255059f-5 -3.1009444f-5 -8.407521f-5 0.00018992538 0.00017652047 6.387699f-5 6.62641f-5 -5.659702f-5 -4.528007f-5 -2.4961199f-5 -6.458119f-5 -7.740639f-6 -8.641964f-5 5.9170667f-5 0.00019641886 7.1408605f-5 7.491677f-5 0.00015996007 -2.14441f-5 -0.00012596443 1.5148395f-5 -0.00020513525 7.203257f-5; -2.159809f-5 -2.1345286f-5 0.000114871524 -0.00021547108 4.5988298f-5 -0.00021723151 5.452951f-5 5.8848036f-6 -0.0001589268 -0.0002506956 0.00018307757 2.7498834f-5 0.00012473844 -3.9340946f-5 0.0001878797 -1.2492364f-6 -9.198892f-5 1.8248566f-5 0.000116399024 1.5051434f-5 0.00034781685 0.000156159 -1.0037205f-5 0.00010480752 -9.164158f-5 2.6788684f-6 0.00016894897 5.7068297f-5 0.00010959305 0.00014088652 -7.1304785f-5 5.224269f-5; 6.238324f-5 9.117311f-5 0.00020454697 0.000208475 0.00018094428 -3.228266f-5 -4.411761f-5 -0.00012658827 -0.00015168184 0.00010113172 0.000105583036 0.00012806694 -5.9978844f-5 -1.2934191f-5 -3.4510533f-5 -6.6202236f-5 -6.6260574f-5 0.0001058354 -0.00020347247 0.00019539056 -4.2171858f-5 0.00020606014 -2.4118353f-5 0.00012581136 7.688511f-5 -0.00010003266 8.23604f-5 -3.0226212f-5 4.6036577f-5 -5.898498f-5 -6.406751f-7 -0.0001438071; 0.00015314828 -5.8805148f-5 -5.2923174f-6 0.0001559641 7.959421f-5 0.00013301839 -4.3907394f-6 7.376904f-5 0.00012489593 4.809692f-5 0.000106285006 -0.00014742208 2.0827318f-5 1.4279862f-5 1.27891335f-5 -0.000102658276 2.7062184f-5 7.4395102f-6 -8.629122f-5 1.6947888f-5 -1.9900364f-5 -1.0614642f-5 -1.1935723f-5 -2.9863071f-5 -0.00010493953 -5.5913297f-5 -2.2774684f-5 -0.00014706138 -0.00017280485 1.1295227f-5 -7.577581f-5 0.00013971627; -7.24564f-5 -0.00019400741 -0.00020466995 -8.613493f-5 -1.8114002f-5 -0.00013081147 -4.010616f-5 0.00017401333 -9.880749f-5 -0.00010273883 -7.1364964f-5 4.2208332f-5 0.000110427754 -0.00013644555 5.977523f-5 -3.1731386f-6 5.8735186f-5 5.9781487f-5 0.00012411928 -7.0713344f-5 -1.4844751f-5 0.00021108685 -7.310388f-5 -0.00012904903 0.00016850662 -0.00017227576 -4.35704f-5 8.4781626f-5 -0.000118789314 9.910206f-6 -0.00017377545 0.000104917126; 3.421374f-5 0.0001784863 -1.1484191f-5 -0.000101836085 0.00024135718 -0.00017500338 -1.0642813f-5 -3.1498173f-5 4.254024f-5 -4.286035f-5 0.00015602878 -5.6244044f-5 0.00011752606 -0.00018053921 -0.000118252945 -7.63437f-5 6.47034f-6 0.0001642431 6.444494f-5 9.384959f-5 0.00015255544 -8.760447f-6 -4.673646f-5 2.8553897f-5 6.577486f-5 -3.8543207f-5 4.0060764f-5 0.0001953378 -9.2235525f-5 -8.571982f-6 1.4440342f-5 -0.00013854646; 2.1408754f-5 9.593646f-5 -4.738496f-5 4.732052f-5 -0.00024824127 -4.732115f-5 2.6718411f-5 -8.263985f-5 1.5497177f-5 -8.5198546f-5 -4.1833344f-5 -3.14811f-5 0.00010269098 -5.6399032f-5 1.7421926f-5 2.3837554f-6 8.1492464f-5 -0.000119179356 6.3751904f-5 2.5058574f-5 -8.191238f-5 6.434593f-5 3.4300912f-5 9.4067254f-5 0.00011397646 3.534332f-5 -0.00016860459 -0.00018563253 -6.822247f-5 -1.4889482f-5 0.00023276742 -2.7466636f-5; 6.616932f-5 1.28425345f-5 -0.0001042926 9.09377f-5 -0.00024482232 3.07259f-5 -0.00013715128 0.000101553676 -7.178563f-5 3.5842924f-5 -8.436059f-5 -0.00016869747 7.175845f-5 -9.25618f-5 -8.6544744f-5 -0.00011310014 -1.9110552f-5 0.00014068597 -3.46944f-5 2.8009728f-5 6.1295f-5 -1.4718792f-5 0.00011410012 4.691924f-5 1.5165745f-5 6.707154f-5 -6.649845f-5 -8.032728f-5 -4.362544f-5 -4.7473106f-5 -6.403594f-5 -0.00021157016], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-0.00022788436 -0.00015335342 -0.00011465894 0.00020399482 -0.00023230375 -0.00023374535 -5.701846f-5 2.2562022f-6 -3.3024367f-5 -8.095133f-5 -0.00013721343 -2.620743f-5 0.00015951965 -0.0001061864 0.000111545 1.7474338f-5 2.3434386f-5 -9.256927f-5 6.5034714f-5 -0.00015723762 2.8178138f-7 -4.313013f-5 7.1515926f-5 1.6893357f-5 5.1108167f-5 3.659688f-5 1.3686637f-5 1.3952281f-5 -6.235979f-5 -1.1992588f-5 -3.5462926f-5 7.7624674f-5; -9.5137206f-5 2.3902441f-5 -0.00012429585 0.00012397852 -2.834219f-5 1.251941f-6 1.3347854f-5 -3.714094f-5 8.1240454f-5 -9.8066725f-5 -0.00011307021 -3.4549306f-5 -8.919269f-5 -4.9514474f-5 -0.00014298115 -6.0232043f-5 -0.00020102633 4.9332135f-5 -0.00012386192 -7.3164238f-6 -0.00015395948 -0.0001424155 -5.7651367f-5 0.00011915152 -0.00019318277 -5.8882713f-5 9.0664114f-5 1.8740871f-5 0.00013420281 -5.03832f-5 -0.00023244778 7.6930075f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))
    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64
    julia
    const params = ComponentArray(ps |> f64)
-
-const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
-    Chain(
-        layer_1 = WrappedFunction(Base.Fix1{typeof(LuxLib.API.fast_activation), typeof(cos)}(LuxLib.API.fast_activation, cos)),
-        layer_2 = Dense(1 => 32, cos),  # 64 parameters
-        layer_3 = Dense(32 => 32, cos),  # 1_056 parameters
-        layer_4 = Dense(32 => 2),       # 66 parameters
-    ),
-)         # Total: 1_186 parameters,
-          #        plus 0 states.
    Now we define a system of odes which describes motion of point like particle with Newtonian physics, uses
    `,14)),A("mjx-container",q,[(a(),i("svg",B,s[27]||(s[27]=[n('',1)]))),s[28]||(s[28]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"1"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"χ")])],-1))]),A("mjx-container",V,[(a(),i("svg",b,s[29]||(s[29]=[n('',1)]))),s[30]||(s[30]=A("mjx-assistive-mml",{unselectable:"on",display:"block",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",overflow:"hidden",width:"100%"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML",display:"block"},[A("mi",null,"u"),A("mo",{stretchy:"false"},"["),A("mn",null,"2"),A("mo",{stretchy:"false"},"]"),A("mo",null,"="),A("mi",null,"ϕ")])],-1))]),A("p",null,[s[37]||(s[37]=h("where, ")),A("mjx-container",T,[(a(),i("svg",D,s[31]||(s[31]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D45D",d:"M23 287Q24 290 25 295T30 317T40 348T55 381T75 411T101 433T134 442Q209 442 230 378L240 387Q302 442 358 442Q423 442 460 395T497 281Q497 173 421 82T249 -10Q227 -10 210 -4Q199 1 187 11T168 28L161 36Q160 35 139 -51T118 -138Q118 -144 126 -145T163 -148H188Q194 -155 194 -157T191 -175Q188 -187 185 -190T172 -194Q170 -194 161 -194T127 -193T65 -192Q-5 -192 -24 -194H-32Q-39 -187 -39 -183Q-37 -156 -26 -148H-6Q28 -147 33 -136Q36 -130 94 103T155 350Q156 355 156 364Q156 405 131 405Q109 405 94 377T71 316T59 280Q57 278 43 278H29Q23 284 23 287ZM178 102Q200 26 252 26Q282 26 310 49T356 107Q374 141 392 215T411 325V331Q411 405 350 405Q339 405 328 402T306 393T286 380T269 365T254 350T243 336T235 326L232 322Q232 321 229 308T218 264T204 212Q178 106 178 102Z",style:{"stroke-width":"3"}})])])],-1)]))),s[32]||(s[32]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"p")])],-1))]),s[38]||(s[38]=h(", ")),A("mjx-container",z,[(a(),i("svg",U,s[33]||(s[33]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D440",d:"M289 629Q289 635 232 637Q208 637 201 638T194 648Q194 649 196 659Q197 662 198 666T199 671T201 676T203 679T207 681T212 683T220 683T232 684Q238 684 262 684T307 683Q386 683 398 683T414 678Q415 674 451 396L487 117L510 154Q534 190 574 254T662 394Q837 673 839 675Q840 676 842 678T846 681L852 683H948Q965 683 988 683T1017 684Q1051 684 1051 673Q1051 668 1048 656T1045 643Q1041 637 1008 637Q968 636 957 634T939 623Q936 618 867 340T797 59Q797 55 798 54T805 50T822 48T855 46H886Q892 37 892 35Q892 19 885 5Q880 0 869 0Q864 0 828 1T736 2Q675 2 644 2T609 1Q592 1 592 11Q592 13 594 25Q598 41 602 43T625 46Q652 46 685 49Q699 52 704 61Q706 65 742 207T813 490T848 631L654 322Q458 10 453 5Q451 4 449 3Q444 0 433 0Q418 0 415 7Q413 11 374 317L335 624L267 354Q200 88 200 79Q206 46 272 46H282Q288 41 289 37T286 19Q282 3 278 1Q274 0 267 0Q265 0 255 0T221 1T157 2Q127 2 95 1T58 0Q43 0 39 2T35 11Q35 13 38 25T43 40Q45 46 65 46Q135 46 154 86Q158 92 223 354T289 629Z",style:{"stroke-width":"3"}})])])],-1)]))),s[34]||(s[34]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"M")])],-1))]),s[39]||(s[39]=h(", and ")),A("mjx-container",P,[(a(),i("svg",K,s[35]||(s[35]=[A("g",{stroke:"currentColor",fill:"currentColor","stroke-width":"0",transform:"scale(1,-1)"},[A("g",{"data-mml-node":"math"},[A("g",{"data-mml-node":"mi"},[A("path",{"data-c":"1D452",d:"M39 168Q39 225 58 272T107 350T174 402T244 433T307 442H310Q355 442 388 420T421 355Q421 265 310 237Q261 224 176 223Q139 223 138 221Q138 219 132 186T125 128Q125 81 146 54T209 26T302 45T394 111Q403 121 406 121Q410 121 419 112T429 98T420 82T390 55T344 24T281 -1T205 -11Q126 -11 83 42T39 168ZM373 353Q367 405 305 405Q272 405 244 391T199 357T170 316T154 280T149 261Q149 260 169 260Q282 260 327 284T373 353Z",style:{"stroke-width":"3"}})])])],-1)]))),s[36]||(s[36]=A("mjx-assistive-mml",{unselectable:"on",display:"inline",style:{top:"0px",left:"0px",clip:"rect(1px, 1px, 1px, 1px)","-webkit-touch-callout":"none","-webkit-user-select":"none","-khtml-user-select":"none","-moz-user-select":"none","-ms-user-select":"none","user-select":"none",position:"absolute",padding:"1px 0px 0px 0px",border:"0px",display:"block",width:"auto",overflow:"hidden"}},[A("math",{xmlns:"http://www.w3.org/1998/Math/MathML"},[A("mi",null,"e")])],-1))]),s[40]||(s[40]=h(" are constants"))]),s[45]||(s[45]=n(`
    julia
    function ODE_model(u, nn_params, t)
-    χ, ϕ = u
-    p, M, e = ode_model_params
-
-    # In this example we know that \`st\` is am empty NamedTuple hence we can safely ignore
-    # it, however, in general, we should use \`st\` to store the state of the neural network.
-    y = 1 .+ nn_model([first(u)], nn_params)
-
-    numer = (1 + e * cos(χ))^2
-    denom = M * (p^(3 / 2))
-
-    χ̇ = (numer / denom) * y[1]
-    ϕ̇ = (numer / denom) * y[2]
-
-    return [χ̇, ϕ̇]
-end
    ODE_model (generic function with 1 method)
    Let us now simulate the neural network model and plot the results. We'll use the untrained neural network parameters to simulate the model.
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, params)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=params, saveat=tsteps, dt, adaptive=false))
-waveform_nn = first(compute_waveform(dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, markersize=12, alpha=0.5, strokewidth=2)
-
-    axislegend(ax, [[l1, s1], [l2, s2]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
-
-    fig
-end
    Setting Up for Training the Neural Network ​
    Next, we define the objective (loss) function to be minimized when training the neural differential equations.
    julia
    const mseloss = MSELoss()
-
-function loss(θ)
-    pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
-    pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
-    return mseloss(pred_waveform, waveform)
-end
    loss (generic function with 1 method)
    Warmup the loss function
    julia
    loss(params)
    0.0006529859924286357
    Now let us define a callback function to store the loss over time
    julia
    const losses = Float64[]
-
-function callback(θ, l)
-    push!(losses, l)
-    @printf "Training \\t Iteration: %5d \\t Loss: %.10f\\n" θ.iter l
-    return false
-end
    callback (generic function with 1 method)
    Training the Neural Network ​
    Training uses the BFGS optimizers. This seems to give good results because the Newtonian model seems to give a very good initial guess
    julia
    adtype = Optimization.AutoZygote()
-optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)
-optprob = Optimization.OptimizationProblem(optf, params)
-res = Optimization.solve(
-    optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
-    callback, maxiters=1000)
    retcode: Success
-u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [-8.733465620020983e-6; -0.00010797254071798166; -6.034535090281437e-6; 7.889705011615516e-5; -1.9253391656059496e-5; -4.612652992366777e-5; 7.144777919157657e-5; -8.932017226481967e-5; -4.71681050840083e-5; 9.798890823729545e-5; -2.1554476916162734e-5; 4.739751602749602e-5; 0.00016439423779959398; -4.360640014055303e-5; -6.190079147924201e-5; 6.240796210469526e-5; 0.00010571498569335307; 4.738614370580132e-5; 2.5381063096584814e-5; 4.914130477114876e-5; 0.00016239624528652472; 0.00013147806748750233; -0.00017642804596096446; -0.0001261118450201706; 3.079728458034808e-6; -3.787924651986792e-5; -1.16270905436827e-6; -0.00016794122348050335; 6.320179818424497e-5; 0.00012728852743741338; 9.064738696900327e-5; -1.5429377526735596e-5;;], bias = [-5.359601592163036e-18, -6.550829691992387e-17, -1.41806373185002e-17, 6.612883732914699e-17, -3.971281807812836e-17, 3.2571059060761017e-17, 9.58030628269383e-17, 1.3176812858269634e-16, 2.944634276201806e-17, 1.9336104666087008e-17, -5.330420267771681e-17, 7.372211719502685e-17, -2.7922867808978655e-17, -9.150352429637832e-18, -3.8438937777196597e-17, 9.00633053182541e-17, -1.5663529032065214e-17, 7.62186052855715e-18, -1.1978849530028129e-17, 3.9396538625362964e-17, 3.263723400316271e-16, -3.1675390001372084e-17, -1.3105712844388004e-16, -3.803273056227972e-17, -2.0332749183269495e-18, -1.2727593133643193e-17, -1.7718179375088544e-18, -2.8018370574615e-16, 7.587973597486665e-17, 8.609350097255151e-17, 7.4759154972248e-17, -9.260147643001434e-18]), layer_3 = (weight = [-2.81779129679041e-5 -3.9365283514297854e-5 -7.6574890913135e-5 0.00013857145044797106 -6.85615832181301e-7 -5.7781809077523993e-5 -5.151397503750668e-6 3.9835850698482504e-5 0.00012864310069319755 -6.512425803036386e-5 5.4659068196387454e-5 -6.377044970092395e-5 7.986889344950431e-5 -9.115820917456447e-5 -5.069790764052438e-5 -1.491042171137685e-6 -3.207358072783739e-5 -6.482248041235874e-5 2.0175230970680213e-5 9.132562358475261e-5 1.717424224382949e-5 6.390499133461278e-5 -4.389381406751195e-5 -0.00011969898237536553 -6.298581043181066e-5 1.9702146387616684e-5 -7.764463548183539e-6 1.9582808320888176e-6 7.794170328027556e-5 0.00010650963763085304 -8.729386077417963e-5 -3.0301974179265634e-5; 5.093000583673822e-5 -3.3695515683559355e-6 0.00010611892650576301 -9.076084677854146e-5 -6.206943215129883e-5 2.0946710374296105e-5 6.196243486574254e-5 -8.04957875332556e-5 9.118333548048649e-5 9.569926771309863e-5 -0.0002559040862068346 -8.457661750053847e-5 0.00011611161755373647 0.00015786205915009716 -0.00013252692440783356 -1.4725690907474887e-5 2.510067697768531e-6 5.846065107124997e-5 5.234997808674383e-5 -4.698632108539192e-5 -0.00024784683626402274 -4.0582605269986785e-5 -3.356815459286549e-6 7.707813494580927e-5 -5.3827805804582414e-5 -0.00012691746667751962 -0.00018019285778773963 3.0377035382311122e-5 -1.2278924605019604e-5 -0.0002268154333976734 0.00017366600661280175 -3.0415702711312987e-6; 0.00010137852786915331 -0.0001010015274014704 -2.427308391863947e-6 -4.770813849111468e-6 6.701661750316573e-5 0.00017136620426658485 -0.00019062613300871163 -1.067131212087478e-6 0.00012427559540715012 -3.5586695777104e-5 8.55602685978259e-5 -8.815592773796477e-7 -4.387152676013409e-5 0.00015637557789791322 -2.2594698567493444e-5 4.314996676743906e-5 -0.00011615114799592718 -1.5891253316160807e-5 -7.964016713326844e-5 1.0809817648757658e-5 3.317806676836575e-5 0.0001521374781068033 -0.0001245625624515501 4.1607805723239765e-5 -1.4163321612661705e-5 3.903749366049605e-5 -4.447955671007791e-5 -3.575690225357303e-5 7.252304767366444e-5 -0.00011933259684251707 -7.042072848076068e-5 -5.546812990455935e-5; -8.336042507189592e-5 0.00011739355553536533 -1.6887058029987743e-5 -9.929093054133669e-5 0.0001831156508497774 0.00016209405597428542 -0.0001248413506463285 1.6862267636589728e-5 -0.0001252187109120308 -2.3408347863458833e-5 9.515673021915105e-5 -6.6847330007470475e-6 5.598798028322887e-5 0.0001258904479410381 -3.764638737529744e-5 -0.00011733128181995121 5.898543471366949e-5 -3.217015242889768e-5 0.00016092859308364536 -0.00010215279022568149 -0.00013990668654778032 -0.00021372428879012443 0.00013839693141299363 -2.1802074718741762e-5 -5.540062523231966e-6 0.00017145255272871558 -6.449778871947236e-5 0.00010669620254439335 2.438082204853424e-5 -8.664751354708432e-5 -0.00010137273480986426 -0.00010135865583188165; 6.770358496073841e-6 7.97941223851332e-5 0.00010768450501077066 4.6960414138666134e-5 8.351425402965557e-5 5.775398577173693e-6 -7.493830581493888e-5 2.7759933246341794e-5 -5.564503463515998e-5 4.735090023190146e-5 6.739095023535343e-5 -1.759619100026308e-5 6.22388883044749e-5 -0.0001010620859318409 6.811957190679273e-5 -1.7852259233548545e-5 0.00014260215301932077 -3.112693467242631e-5 2.758543213589722e-5 -0.00021257582794474437 -2.0855234296848334e-5 -0.00015710228272780086 0.00011575050793055044 -9.060623652600814e-5 0.00012642943092091197 -6.715464643035007e-5 -2.3085242545998626e-5 0.00014910514200039036 0.0001751779540342266 0.0001241965704963434 -2.726808143903494e-5 0.0001073214201739075; 5.8766903805779695e-5 -0.00015708922904846402 2.6431680510403082e-5 -0.00010387897958676388 2.2030108141596797e-5 4.6587045347085305e-5 -0.00014493874363065735 7.219263176270675e-5 -2.457368094578844e-5 0.00013603266239701657 0.00013269154266058503 8.446827092743486e-5 9.409567252336606e-5 -0.00016357305315718972 1.966221140068937e-5 0.000159896391835762 -0.0001435019311767551 -1.580818565085058e-5 1.6059146276209827e-5 -3.5983315626392106e-5 1.4797095957732168e-5 1.8547740239996884e-5 4.254694619052582e-5 -9.158426286117389e-5 -0.00013229278005124026 -1.3918982346833022e-5 -1.9566823482047338e-5 -5.4902115095643754e-5 -3.019838908326368e-5 3.309413119754534e-5 0.00019555344448494636 9.743726519704837e-5; 6.582914752732503e-5 -1.9021242084925348e-5 -2.3916997494893138e-5 -8.26842650413899e-5 8.264597377353835e-5 0.00011346335045607408 0.00019279813673427136 5.6249428919618555e-5 -3.976422781044965e-5 -8.271452574910546e-5 6.101728390865519e-5 -7.841763622047724e-5 -1.2951023424746916e-5 0.00013066113856700814 -1.5206532995873412e-5 -0.0001237932582145024 4.110263503469658e-5 -6.578166980682285e-5 -1.7373078571754834e-7 -4.442866233796219e-5 -1.1633819413955027e-6 -0.00010926800937078438 -0.00018221662216738526 -0.00013746663066478056 0.0002607891957749194 -1.487281574293196e-5 1.9144042337643907e-5 -6.363366887590736e-5 -3.2120659713766265e-5 -0.00013503155682621117 -5.4511840832783175e-5 -0.00015212040929975444; 1.6406990034034946e-5 -1.9113927629197256e-5 0.00012204590195253912 -7.556622556946641e-5 -1.6685824158024457e-5 -1.590150866902443e-5 6.0559330740240846e-5 -1.5048051573805486e-5 -8.19364647038794e-5 -2.9995849836970368e-5 -1.038943053642793e-5 -3.10356369307312e-5 -0.00017250242643128175 3.37049280635499e-5 -4.3460542365913066e-5 -4.783470620238665e-5 2.2758766310549294e-6 5.0059814559306675e-6 -0.000156025118195224 2.3699043885450762e-5 3.2233576638856505e-5 -0.00012310168647743032 -2.8646086939959324e-5 5.1685634679624405e-5 0.00017521538422372607 -3.891999920667637e-5 0.00016759197052963493 4.498685695510351e-5 2.9358576575056893e-5 -9.714292507921566e-5 -8.786864492114564e-6 9.830446680937386e-5; 1.06843883722068e-6 -5.8245838158569814e-5 3.961602458283923e-5 5.7840384829517084e-5 2.6230438084068423e-5 -4.11123376116672e-6 -1.211492628945743e-5 9.190200187285233e-5 5.677242340888547e-5 -0.00011149531230776743 7.906633952651798e-5 2.2018106096849766e-5 -5.5081822506421394e-5 -9.197443728142482e-6 -6.653540215202865e-5 -6.85999624010108e-5 8.815267908639125e-5 2.2247449737817038e-5 -3.938655602260798e-5 -2.303216417650039e-5 -0.00015165226097897555 0.00014176119264001894 -0.0001215108062068825 2.5913533750184917e-5 -0.00015541871864562236 9.689767437074972e-5 -0.00012083387293833621 -5.919483039624893e-5 -2.0095083823350953e-5 1.9722208594516065e-5 1.845849662922129e-5 0.00016762771672355632; 2.9943606731133943e-5 -0.00021360929369762725 4.23057879547136e-5 -0.000148600576923457 1.602745740313698e-5 0.00013049759016069826 -0.00012181994518101749 0.00011504296869901559 -3.660772765357732e-5 0.0001379568291471869 1.5451092420729488e-5 6.206902051274912e-5 -4.913439463193834e-5 -5.633184466347801e-5 7.04915744594797e-5 -0.00016246069410186855 0.00020378697243217167 5.018367462838316e-5 -6.565380973885296e-6 2.513376400969248e-5 -0.00013978591434493068 6.94386997369688e-5 -5.751714181455874e-5 6.425686458755322e-5 -8.477288631326389e-5 2.7092237618801464e-6 -6.27312967677122e-5 -0.00011133964128162024 -0.00011893893517623317 -2.4912409773203008e-5 -6.552214310573785e-5 -1.6223006443002694e-5; 0.0001490338564274168 -3.518618352769437e-5 6.838436281599909e-5 1.806462575972689e-5 -1.2176147652786665e-5 4.807874035963595e-5 4.878401409079714e-6 2.5177831116016903e-5 -2.3934064234487803e-5 -0.0001705264202662507 -8.472366419096616e-5 2.6157609749355256e-5 3.1477708102315184e-5 -3.1985504325127225e-5 1.7296810323607505e-5 -6.5614860364583e-5 -6.457147349079619e-5 4.945808366232355e-5 -6.379848302981961e-5 0.00017259505154856278 -5.566240536224125e-5 -0.00013105830833025017 -2.3266429639777084e-5 0.00013214018879283523 5.176863312430556e-5 4.865734997496769e-5 0.00011115387503581927 2.226490694214039e-5 9.954913434050791e-6 -5.858894467179369e-5 -1.4106286403838673e-5 0.00011483243185852522; -4.141835410123147e-5 1.3913409077181379e-5 -0.00011940793950784414 -9.851428507103351e-5 7.357115651971301e-5 4.7400992173019836e-5 -6.785606996131011e-5 0.00019941878969108163 0.00012728528059811163 -0.00016363867501935345 -0.0001256172271838726 -1.6131280006125398e-5 -0.00019294186671469824 4.533782032335001e-6 4.713058120829044e-5 -0.00013652103547214534 -6.237496739926018e-5 -0.00019912244346200657 -0.00017180809112473725 2.6716919876302937e-5 -4.007740238886854e-5 -1.360023810559931e-5 -8.151377727482514e-5 5.214317030765395e-5 -1.983126998097565e-5 7.460925377231771e-5 0.00013041010066658793 -9.939551267745315e-5 8.682207231900662e-5 0.00013920795578303128 -5.427086028943335e-5 -0.0001590628689315487; -4.4683193628608677e-5 -5.462714667449189e-5 0.0001213522601021198 -4.510208869835473e-5 -3.475126774066839e-5 0.00010407246467291496 2.201067676576273e-5 -0.00013626279083912471 -2.2396877248705143e-5 -2.954669886169323e-5 0.0001530667696344408 5.707998788399833e-6 -3.319090588851249e-5 0.00020576113187342084 6.463542835188122e-5 -9.893293005081802e-5 8.663725478906726e-5 -0.00011787550734954773 -0.00014051892045319808 5.135184714088322e-5 -1.1269650811645857e-6 0.00011688509861339084 0.00013804465292119772 4.82593214102676e-5 -5.0914869426216386e-5 5.692928915939261e-6 -1.1633031461512262e-6 0.00010991657203627335 7.012377445040195e-6 -5.4912466076377036e-5 0.0001032820264655795 -2.207059602444299e-5; 3.8979493365287074e-5 0.00010581292208365888 4.484870642040789e-5 0.00028989690526936494 -2.1128604021259633e-5 -0.00010666924008551315 0.00022406709667843232 6.000646517344881e-5 -5.0462551189458834e-5 0.00013748680994462204 0.00016624314066571072 0.00016925391737454296 -7.03081657776027e-5 -8.645297002663145e-5 0.00010455398496555676 0.00012044191153103992 -0.00016730541192379013 4.827235734405655e-5 -0.00021465863919734152 0.00023091105158523368 -0.0001350471191472717 -2.8145244687099784e-5 8.933842236721743e-5 2.1200339870060362e-5 7.637753878908047e-6 -0.0001832826581449361 0.0001140298720201378 4.894151625187261e-5 8.410063510377457e-5 -9.038141233765218e-5 -0.0001559706923217688 -1.967892221732617e-5; -0.00017860869197075727 0.00016289136179357017 2.1552883805534698e-5 7.650577240955891e-5 -0.00015729013616119967 -4.9051138961393236e-5 0.00013334190803679445 8.897456642107079e-5 0.00011627310810244291 3.00154755280313e-5 -4.348845453668481e-6 -7.407840679998861e-5 3.351958217200008e-5 -9.550772451230695e-5 -6.280750462220311e-5 -3.885226546454502e-5 -5.6759801412865855e-5 4.452677590444851e-8 -2.860829740139632e-5 -8.249567112553275e-5 -8.704435244334782e-5 -1.5151074552430909e-5 -9.395606542684836e-6 -5.665910215948288e-5 3.152074564813159e-5 -1.2996369736091477e-5 -4.8611780260861976e-5 7.504250457378289e-5 -0.00013518759591871196 -7.636719023475917e-5 3.0250840666626814e-6 3.73406131587131e-5; -1.5228735603273347e-6 -0.00013971561548916717 -4.3014009690716666e-5 -1.3732278432340951e-5 0.0002871830221809317 8.435243961483699e-5 0.00015791237837018728 4.3380056183907456e-5 -4.04295495283931e-5 0.00013740570301504828 -0.00012764699914642993 -5.626892726669568e-5 3.8955284424237366e-5 0.00019921713118290995 -8.471329766709e-5 4.6631958128077746e-5 0.00012246462591772832 -1.701348139444165e-6 2.0177563954154805e-5 0.00017496838717105307 0.00012816098768577894 4.660376532935051e-6 9.724291174986106e-5 -0.0001476289906461012 -0.00013615125482120902 0.00021714670600684853 7.760427422715573e-6 -9.854168439435897e-5 -3.5386394131105624e-5 -0.00011540063843515129 -3.7346220608212875e-5 -0.00020943673717939665; -7.723347007773074e-5 0.00012642852888899184 -0.00013361183883167066 2.563478992310894e-5 -8.438698060122472e-5 -5.911473490773644e-5 -4.5789171421300255e-5 -7.400129928959714e-6 -6.326546499981479e-5 3.049368351281167e-5 -0.00010256551686684742 -0.00010149392838134362 0.00013105472825926016 -0.00019522991570188633 3.4176036566396015e-5 -0.00011024306368558402 -1.3398447121235416e-5 0.000101051720408621 -4.937291160236234e-5 0.00019615531127424067 -9.093289773220052e-5 0.0001726965907401498 9.911330343683025e-5 1.2376997434390403e-5 3.6345785334601886e-5 -0.00017203413372403965 -6.91730478174166e-5 -0.00010344155671547738 1.6594765471427673e-5 -0.00011501850022244405 -0.00020326738959187797 -0.00016199772144678218; -1.0492632138680406e-5 -1.8434707310424074e-5 3.441650694680668e-6 1.4894685552956632e-5 0.00016012509054921307 -1.530473586395223e-5 1.7829989662953562e-5 7.738736799869232e-5 2.4453803196178463e-5 -4.7624133951519924e-5 -1.5141768055814625e-5 -9.343188178223385e-6 0.00013970565392586398 -0.00013092193061534614 -0.00021145534359898768 0.00018979269130572327 3.271933652062365e-5 9.215454970903387e-5 0.00012066584800554172 -0.00010754331837456985 6.6663792108896785e-6 -7.77526884507643e-5 -6.519306138440145e-5 4.0988385951586975e-5 3.918204216237859e-5 3.647251920260201e-5 -4.6465652342208385e-5 0.00012628203599664346 5.7110655302891605e-6 -5.553913528497993e-5 -7.507118520375801e-5 -2.864719048598573e-5; -6.847838721998037e-5 -9.991352908374989e-5 -0.0001055645689118281 1.3994697838172852e-5 -0.0001805462516128645 -4.775208708426028e-5 -2.5409693080034265e-5 7.591687335918585e-6 0.00011534390159821088 -0.00015416799487840587 -2.0335547560276554e-5 -6.331477834438955e-5 0.00016125945057890732 -4.540443387642542e-5 1.9163630895748854e-5 6.950787883423187e-5 7.672932497396827e-5 -6.484157256610741e-5 -1.9393311049239834e-5 1.4185236978195102e-5 0.0001473621241532582 -8.95844632393345e-5 0.00018367340180727918 -2.6433709535667355e-5 6.585594747124718e-6 0.00010185551309430941 8.619813979774715e-5 -3.4717597825169725e-6 -3.997779913308183e-5 2.3616014914551076e-5 -2.63160809479066e-5 0.00022985375926535572; -0.0001559074115996235 3.2331194315476374e-5 -4.890794034731576e-6 -0.00011219705257667261 -3.5792185290342753e-5 -2.3049548510657626e-6 0.00020679663074566048 -2.06610983789208e-5 2.81375522695677e-6 -3.717862265170678e-5 -1.7452810343679628e-5 6.0096590541210817e-5 3.472123725236716e-5 -0.00017239979561986312 -2.5925883083114988e-5 -0.00013413560728644923 0.00015813822764335032 0.0001404479736172602 -0.00013199541345806668 6.1016347981198496e-5 8.263860589954412e-5 -0.00010763267701827329 0.00012680853623328007 -1.896853419589179e-5 4.379788887468307e-5 7.210932752946492e-5 5.486475501107193e-5 0.00014672021287855984 -7.48078199742359e-5 -5.188165951972205e-5 0.00016848296590046868 -5.266779763609969e-5; -0.00018211911149490352 0.00022923113944772652 -5.0249331915329287e-5 1.50120934186983e-5 8.91223164863911e-5 -9.50745459057368e-5 3.065978969163425e-5 -0.00015601891275662826 -4.408660673005281e-5 8.512935825921665e-5 -0.00012687509363684538 -4.773003886737635e-5 -9.324320920212499e-5 1.5834562210438314e-5 4.8913923156868715e-5 1.8481759317291004e-5 0.00023573439036328377 -4.9180890281496175e-5 -0.00011489574601808303 4.768967092895072e-6 0.00026578090844042306 -1.907459839356695e-5 -2.9712992481006527e-5 -4.080408391826853e-5 9.937422807626698e-5 0.00020834956310056393 1.2001217574943265e-5 -0.00012790382673200554 5.376274136426115e-5 4.0614478412968307e-5 6.384567611450428e-5 -0.000162398370529336; 6.825093374578964e-5 5.197363839830605e-7 -3.335581708591681e-5 4.980564996009565e-5 2.6537222857452056e-6 -8.809148925132326e-5 1.3807100240288665e-5 -4.1632666343442166e-5 0.000112948647492945 -0.00015741255696405715 0.0001046846657664584 -1.877159842473196e-5 -1.784083609884784e-5 -9.540280074355407e-5 -6.37152559101506e-5 -0.0001746790737706161 -0.000231588849364864 0.00024108002913641536 0.00010457094982490636 6.284408233119904e-5 -0.00015179132673431847 -6.077052306857798e-5 1.8313955715777306e-5 -5.211211167982715e-5 1.0136942601152516e-5 -1.0156773660068764e-5 -0.00011138957846918396 -0.0001394896032615441 8.460613686117172e-5 0.00014050759569733472 7.175846130909918e-6 0.0002610481817646952; -7.002370826366617e-5 -5.106787780813939e-5 0.0001432925769623622 -6.0367432029938036e-5 0.0002614535756439225 -9.752449346610743e-6 -0.00011116959054188046 1.4564485849501682e-5 5.4732353004606066e-5 0.0001050108826749296 -2.8776770210147576e-5 -0.00021153228722677796 -1.972661216849735e-5 -0.00017698574943633054 -7.08909951353186e-5 -0.00021033249636811404 0.00012432185625958184 3.8996232018785184e-5 -0.00011144529840375548 0.0001173811674963009 0.00026127356845255187 -6.956327838988016e-5 -1.3386972073830855e-5 -0.0001781363402696047 3.5561503449666485e-5 7.980289823909054e-5 -0.0001052690654031679 -7.726351164446e-5 4.9727589197669095e-5 2.7081604043003266e-5 2.1699660050313614e-5 0.00010321408494210545; 0.000208557849988628 0.00011113947141571466 4.709861174396069e-6 0.000137855812796677 1.8843950898789558e-5 9.677444643462762e-5 0.00011414959328836457 -4.9486002238647907e-5 -7.359415916236027e-6 -7.687753065833284e-5 -6.084503445303279e-5 -0.00012734968447470352 -9.817417128355986e-5 -0.00019467959179886857 3.383134492329044e-5 -0.00012810744635639062 -0.000124850436689979 -1.0283093500712565e-5 -0.0001018967840296145 0.00020154964761464605 -0.00010026951610920748 6.482763740355533e-5 -8.700202567847338e-5 0.00020038443210656232 5.832085764855876e-5 -1.9711788694204284e-5 -4.4581461109865615e-5 8.606037329305128e-5 3.70740747390149e-6 8.007586177559976e-5 1.0884558947925516e-5 -3.4412269882105496e-5; -7.422697903624872e-5 -7.489481735777707e-5 -0.00024892094601913496 3.70786051052727e-5 9.689584608618977e-5 1.0862010144555884e-5 -0.0002405210130246256 3.98886090397015e-5 -0.00030288317297621684 3.254935059427534e-5 -3.1010683001150635e-5 -8.407645013452556e-5 0.00018992414466535698 0.00017651922594719507 6.387575327149608e-5 6.626286399747263e-5 -5.6598257783694784e-5 -4.528130970815243e-5 -2.4962437732975195e-5 -6.458242800821592e-5 -7.741878603182283e-6 -8.642088009313119e-5 5.916942741803085e-5 0.00019641761669387614 7.140736620836752e-5 7.491552737970756e-5 0.0001599588262751778 -2.1445339866684955e-5 -0.0001259656733464354 1.5147155368109755e-5 -0.0002051364886438361 7.203133050548713e-5; -2.159497881946504e-5 -2.134217385318269e-5 0.00011487463569794437 -0.00021546796989567625 4.5991409648578535e-5 -0.00021722840239233896 5.4532623550054356e-5 5.8879156189621935e-6 -0.00015892369475621366 -0.00025069248141661675 0.00018308068473417853 2.750194624824808e-5 0.00012474154726659566 -3.933783438113124e-5 0.0001878828167595362 -1.2461244419524062e-6 -9.198580875235234e-5 1.8251677544339422e-5 0.00011640213568755016 1.5054546280606808e-5 0.0003478199629957018 0.0001561621153872714 -1.003409335099896e-5 0.00010481063298939619 -9.1638469087755e-5 2.6819804465232837e-6 0.00016895208471980533 5.70714088884325e-5 0.00010959616127949386 0.00014088963297254262 -7.130167279100299e-5 5.2245800985857894e-5; 6.238586386123643e-5 9.117573674690805e-5 0.0002045495991517601 0.0002084776267331782 0.0001809469019683818 -3.228003449814601e-5 -4.4114983462099916e-5 -0.00012658564926435023 -0.00015167921926669987 0.00010113434720940033 0.00010558566166244384 0.00012806956394731785 -5.997621817709457e-5 -1.2931565175423867e-5 -3.4507907253758794e-5 -6.619961029693845e-5 -6.625794892509532e-5 0.00010583802825229028 -0.00020346984936233685 0.00019539318484410414 -4.216923232501825e-5 0.00020606276550485159 -2.4115727934031092e-5 0.0001258139879830786 7.688773505313301e-5 -0.00010003003378708063 8.236302412504309e-5 -3.0223586941926334e-5 4.603920272009079e-5 -5.898235510879696e-5 -6.380496649651563e-7 -0.0001438044794446979; 0.0001531485357827098 -5.8804897098564516e-5 -5.292066480289389e-6 0.0001559643459313135 7.959446144919472e-5 0.00013301863861666608 -4.390488475148462e-6 7.376929340349078e-5 0.00012489618154173332 4.809717199346536e-5 0.00010628525694297254 -0.0001474218262319864 2.0827568614509572e-5 1.4280112618469538e-5 1.2789384409150318e-5 -0.000102658024961822 2.7062434919532456e-5 7.439761129679035e-6 -8.629096931148694e-5 1.694813897298262e-5 -1.990011344791509e-5 -1.0614390821541168e-5 -1.1935471893634369e-5 -2.9862820274541477e-5 -0.00010493927778046945 -5.5913045916933624e-5 -2.2774433209671578e-5 -0.00014706112790922307 -0.00017280460099758978 1.1295477515847536e-5 -7.577555891661407e-5 0.0001397165213984028; -7.245906754784296e-5 -0.00019401008040099318 -0.00020467261793479338 -8.61375986520706e-5 -1.811666816744006e-5 -0.0001308141393625719 -4.010882485695678e-5 0.00017401066424404033 -9.881015585657028e-5 -0.0001027414939987017 -7.136763024996941e-5 4.2205666153078866e-5 0.00011042508839943037 -0.00013644821893345413 5.977256359378199e-5 -3.1758045739991064e-6 5.8732520022523096e-5 5.9778820917328787e-5 0.00012411661436393897 -7.071601003795665e-5 -1.4847416702318032e-5 0.00021108418576964766 -7.310654773994394e-5 -0.00012905169870159122 0.0001685039575871451 -0.00017227842482194194 -4.357306471016276e-5 8.47789603479828e-5 -0.00011879198000920598 9.907540237105236e-6 -0.00017377811610154093 0.0001049144600013644; 3.421541212782677e-5 0.00017848797095547202 -1.1482518417602913e-5 -0.00010183441237705685 0.00024135885131153673 -0.00017500171135571485 -1.0641140321522063e-5 -3.1496500483909854e-5 4.25419125201117e-5 -4.2858675911986464e-5 0.00015603045129291171 -5.6242370661773574e-5 0.00011752773524101682 -0.00018053753639497483 -0.00011825127195725942 -7.634202530999464e-5 6.4720129409459e-6 0.0001642447746086802 6.444661219911101e-5 9.385125984744453e-5 0.00015255711285469946 -8.758773689184584e-6 -4.673478589786585e-5 2.855557021001359e-5 6.577653355970464e-5 -3.854153375533071e-5 4.006243715020314e-5 0.0001953394671397077 -9.223385178875912e-5 -8.570309106577303e-6 1.4442015057000899e-5 -0.0001385447835821711; 2.1407898503261834e-5 9.593560692391147e-5 -4.7385816896544564e-5 4.731966267225899e-5 -0.0002482421229337115 -4.732200674826759e-5 2.6717555857054658e-5 -8.264070190253982e-5 1.5496321184933076e-5 -8.519940153693118e-5 -4.183419939128277e-5 -3.148195598388567e-5 0.00010269012575972111 -5.639988800741287e-5 1.7421070270380343e-5 2.382899838799854e-6 8.149160869887804e-5 -0.00011918021179500511 6.375104870183805e-5 2.5057718931371288e-5 -8.191323710835339e-5 6.434507243337986e-5 3.430005684148854e-5 9.406639886091372e-5 0.00011397560168059937 3.534246509897024e-5 -0.00016860544262218825 -0.00018563338077856912 -6.822332640641795e-5 -1.4890337906226898e-5 0.00023276656004835546 -2.746749184499562e-5; 6.616636744536835e-5 1.2839582608552799e-5 -0.00010429555334314332 9.093474579812071e-5 -0.00024482527590054617 3.072294782567414e-5 -0.00013715422903816277 0.00010155072447913914 -7.178858288727115e-5 3.583997238264792e-5 -8.436354039633233e-5 -0.00016870042384299792 7.175549615013621e-5 -9.25647524364796e-5 -8.654769556061554e-5 -0.00011310309275915098 -1.9113503754948588e-5 0.0001406830145914539 -3.469735168084071e-5 2.800677645962824e-5 6.12920506445426e-5 -1.4721743924473833e-5 0.00011409716751033118 4.691628635000494e-5 1.516279314626611e-5 6.706858618952844e-5 -6.650139914756272e-5 -8.033022970820454e-5 -4.3628393201392015e-5 -4.747605783095499e-5 -6.403889494878001e-5 -0.0002115731164177468], bias = [-3.1191545595130843e-10, -2.0212426596476283e-9, 7.112152827835879e-11, -3.243830175005229e-10, 2.3623130303215395e-9, 4.857273311000907e-10, -1.6142140561750679e-9, -5.454064863858324e-10, -4.3685295932451826e-10, -1.5990931993431986e-9, 9.550369611522905e-10, -3.057664478431455e-9, 1.7217091029342412e-9, 2.7886926096957105e-9, -1.7295348785206703e-9, 1.7609525343764992e-9, -3.63288923365011e-9, 7.630274419652513e-10, 6.557459384335777e-10, 8.331787786529424e-10, 6.541732726041285e-10, -6.045774488692488e-10, -5.106411621077796e-10, 6.012910118557003e-10, -1.2392274956113632e-9, 3.1120056717862873e-9, 2.625460438220049e-9, 2.5090226537188095e-10, -2.6660078765665115e-9, 1.6730006732762932e-9, -8.555338492493487e-10, -2.951879758423589e-9]), layer_4 = (weight = [-0.0008930617061522622 -0.0008185306631164176 -0.0007798362903145225 -0.00046118253009551705 -0.000897480944822825 -0.0008989226899086836 -0.0007221957419533718 -0.000662921138863801 -0.0006982017109555845 -0.0007461286129488532 -0.0008023907532141756 -0.0006913845436001116 -0.000505657627539614 -0.0007713635499856448 -0.0005536322785323924 -0.0006477029338901213 -0.000641742644755058 -0.000757746606194418 -0.0006001426243246246 -0.0008224149504703233 -0.0006648955565676422 -0.0007083074698248665 -0.0005936614162959661 -0.0006482839826710184 -0.0006140691446771656 -0.0006285802307338592 -0.0006514905371313426 -0.0006512250660108998 -0.0007275369507994201 -0.0006771698666891329 -0.0007006402569624884 -0.0005875524598459974; 0.0001792947104920181 0.0002983343139061395 0.000150136063990197 0.00039841043602634435 0.0002460896645807945 0.00027568385565150753 0.00028777974423444203 0.00023729097376672882 0.0003556723695087773 0.00017636516606650494 0.00016136170006947375 0.0002398825145134938 0.00018523919873291076 0.00022491736075073136 0.00013145073567494362 0.0002141998426323734 7.340545978743833e-5 0.0003237640460638284 0.00015056999791557195 0.0002671154860365163 0.00012047243166020841 0.00013201640836588033 0.00021678054810414323 0.0003935834320404575 8.124913038213655e-5 0.00021554910660425 0.0003650959602604641 0.0002931727880915839 0.00040863465278771955 0.00022404868707168502 4.198412722765804e-5 0.0003513619042523084], bias = [-0.0006651773484053201, 0.0002744319173546264]))
    Visualizing the Results ​
    Let us now plot the loss over time
    julia
    begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
-
-    lines!(ax, losses; linewidth=4, alpha=0.75)
-    scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
-
-    fig
-end
    Finally let us visualize the results
    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
-soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
-waveform_nn_trained = first(compute_waveform(
-    dt_data, soln_nn, mass_ratio, ode_model_params))
-
-begin
-    fig = Figure()
-    ax = CairoMakie.Axis(fig[1, 1]; xlabel="Time", ylabel="Waveform")
-
-    l1 = lines!(ax, tsteps, waveform; linewidth=2, alpha=0.75)
-    s1 = scatter!(
-        ax, tsteps, waveform; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l2 = lines!(ax, tsteps, waveform_nn; linewidth=2, alpha=0.75)
-    s2 = scatter!(
-        ax, tsteps, waveform_nn; marker=:circle, alpha=0.5, strokewidth=2, markersize=12)
-
-    l3 = lines!(ax, tsteps, waveform_nn_trained; linewidth=2, alpha=0.75)
-    s3 = scatter!(ax, tsteps, waveform_nn_trained; marker=:circle,
-        alpha=0.5, strokewidth=2, markersize=12)
-
-    axislegend(ax, [[l1, s1], [l2, s2], [l3, s3]],
-        ["Waveform Data", "Waveform Neural Net (Untrained)", "Waveform Neural Net"];
-        position=:lb)
-
-    fig
-end
    Appendix ​
    julia
    using InteractiveUtils
-InteractiveUtils.versioninfo()
-
-if @isdefined(MLDataDevices)
-    if @isdefined(CUDA) && MLDataDevices.functional(CUDADevice)
-        println()
-        CUDA.versioninfo()
-    end
-
-    if @isdefined(AMDGPU) && MLDataDevices.functional(AMDGPUDevice)
-        println()
-        AMDGPU.versioninfo()
-    end
-end
    Julia Version 1.10.6
-Commit 67dffc4a8ae (2024-10-28 12:23 UTC)
-Build Info:
-  Official https://julialang.org/ release
-Platform Info:
-  OS: Linux (x86_64-linux-gnu)
-  CPU: 128 × AMD EPYC 7502 32-Core Processor
-  WORD_SIZE: 64
-  LIBM: libopenlibm
-  LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
-Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
-Environment:
-  JULIA_CPU_THREADS = 16
-  JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
-  JULIA_PKG_SERVER = 
-  JULIA_NUM_THREADS = 16
-  JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
-  JULIA_PKG_PRECOMPILE_AUTO = 0
-  JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,31))])}const Z=t(e,[["render",X]]);export{S as __pageData,Z as default};
diff --git a/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.CQMmi5pL.js b/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.C0fFR2Ju.js
similarity index 67%
rename from previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.CQMmi5pL.js
rename to previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.C0fFR2Ju.js
index f363446281..60c46e48f2 100644
--- a/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.CQMmi5pL.js
+++ b/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.C0fFR2Ju.js
@@ -97,455 +97,455 @@ import{_ as a,c as n,a2 as i,o as p}from"./chunks/framework.DjZDIZsN.js";const r
     return (train_state.parameters, train_state.states) |> cpu_device()
 end
 
-ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.63922
-Epoch [  1]: Loss 0.59006
-Epoch [  1]: Loss 0.56046
-Epoch [  1]: Loss 0.53851
-Epoch [  1]: Loss 0.51878
-Epoch [  1]: Loss 0.50837
-Epoch [  1]: Loss 0.47988
-Validation: Loss 0.46056 Accuracy 1.00000
-Validation: Loss 0.46788 Accuracy 1.00000
-Epoch [  2]: Loss 0.47123
-Epoch [  2]: Loss 0.44911
-Epoch [  2]: Loss 0.44093
-Epoch [  2]: Loss 0.43190
-Epoch [  2]: Loss 0.41116
-Epoch [  2]: Loss 0.39899
-Epoch [  2]: Loss 0.42438
-Validation: Loss 0.36182 Accuracy 1.00000
-Validation: Loss 0.37078 Accuracy 1.00000
-Epoch [  3]: Loss 0.37969
-Epoch [  3]: Loss 0.36815
-Epoch [  3]: Loss 0.34446
-Epoch [  3]: Loss 0.34307
-Epoch [  3]: Loss 0.31119
-Epoch [  3]: Loss 0.30866
-Epoch [  3]: Loss 0.29978
-Validation: Loss 0.27641 Accuracy 1.00000
-Validation: Loss 0.28617 Accuracy 1.00000
-Epoch [  4]: Loss 0.29006
-Epoch [  4]: Loss 0.27774
-Epoch [  4]: Loss 0.26425
-Epoch [  4]: Loss 0.26136
-Epoch [  4]: Loss 0.25087
-Epoch [  4]: Loss 0.23835
-Epoch [  4]: Loss 0.21375
-Validation: Loss 0.20609 Accuracy 1.00000
-Validation: Loss 0.21591 Accuracy 1.00000
-Epoch [  5]: Loss 0.20882
-Epoch [  5]: Loss 0.22010
-Epoch [  5]: Loss 0.20382
-Epoch [  5]: Loss 0.19422
-Epoch [  5]: Loss 0.18539
-Epoch [  5]: Loss 0.17881
-Epoch [  5]: Loss 0.16316
-Validation: Loss 0.15119 Accuracy 1.00000
-Validation: Loss 0.16004 Accuracy 1.00000
-Epoch [  6]: Loss 0.16827
-Epoch [  6]: Loss 0.15449
-Epoch [  6]: Loss 0.15191
-Epoch [  6]: Loss 0.13655
-Epoch [  6]: Loss 0.13572
-Epoch [  6]: Loss 0.13289
-Epoch [  6]: Loss 0.13403
-Validation: Loss 0.10993 Accuracy 1.00000
-Validation: Loss 0.11702 Accuracy 1.00000
-Epoch [  7]: Loss 0.11842
-Epoch [  7]: Loss 0.12253
-Epoch [  7]: Loss 0.09597
-Epoch [  7]: Loss 0.11399
-Epoch [  7]: Loss 0.09454
-Epoch [  7]: Loss 0.09874
-Epoch [  7]: Loss 0.08202
-Validation: Loss 0.07828 Accuracy 1.00000
-Validation: Loss 0.08338 Accuracy 1.00000
-Epoch [  8]: Loss 0.08202
-Epoch [  8]: Loss 0.08031
-Epoch [  8]: Loss 0.08125
-Epoch [  8]: Loss 0.07767
-Epoch [  8]: Loss 0.07224
-Epoch [  8]: Loss 0.05991
-Epoch [  8]: Loss 0.06217
-Validation: Loss 0.05471 Accuracy 1.00000
-Validation: Loss 0.05807 Accuracy 1.00000
-Epoch [  9]: Loss 0.06353
-Epoch [  9]: Loss 0.05217
-Epoch [  9]: Loss 0.05646
-Epoch [  9]: Loss 0.05128
-Epoch [  9]: Loss 0.05138
-Epoch [  9]: Loss 0.04732
-Epoch [  9]: Loss 0.04093
-Validation: Loss 0.04107 Accuracy 1.00000
-Validation: Loss 0.04349 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04294
-Epoch [ 10]: Loss 0.04586
-Epoch [ 10]: Loss 0.04159
-Epoch [ 10]: Loss 0.04004
-Epoch [ 10]: Loss 0.03786
-Epoch [ 10]: Loss 0.03902
-Epoch [ 10]: Loss 0.03803
-Validation: Loss 0.03334 Accuracy 1.00000
-Validation: Loss 0.03532 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03516
-Epoch [ 11]: Loss 0.03884
-Epoch [ 11]: Loss 0.03437
-Epoch [ 11]: Loss 0.03213
-Epoch [ 11]: Loss 0.03224
-Epoch [ 11]: Loss 0.03263
-Epoch [ 11]: Loss 0.02999
-Validation: Loss 0.02831 Accuracy 1.00000
-Validation: Loss 0.03002 Accuracy 1.00000
-Epoch [ 12]: Loss 0.02805
-Epoch [ 12]: Loss 0.03113
-Epoch [ 12]: Loss 0.02716
-Epoch [ 12]: Loss 0.03118
-Epoch [ 12]: Loss 0.03009
-Epoch [ 12]: Loss 0.02828
-Epoch [ 12]: Loss 0.02706
-Validation: Loss 0.02463 Accuracy 1.00000
-Validation: Loss 0.02615 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02656
-Epoch [ 13]: Loss 0.02839
-Epoch [ 13]: Loss 0.02328
-Epoch [ 13]: Loss 0.02764
-Epoch [ 13]: Loss 0.02471
-Epoch [ 13]: Loss 0.02318
-Epoch [ 13]: Loss 0.02648
-Validation: Loss 0.02177 Accuracy 1.00000
-Validation: Loss 0.02313 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02412
-Epoch [ 14]: Loss 0.02352
-Epoch [ 14]: Loss 0.02210
-Epoch [ 14]: Loss 0.02238
-Epoch [ 14]: Loss 0.02261
-Epoch [ 14]: Loss 0.02210
-Epoch [ 14]: Loss 0.02290
-Validation: Loss 0.01947 Accuracy 1.00000
-Validation: Loss 0.02070 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02160
-Epoch [ 15]: Loss 0.02149
-Epoch [ 15]: Loss 0.02041
-Epoch [ 15]: Loss 0.02102
-Epoch [ 15]: Loss 0.01964
-Epoch [ 15]: Loss 0.01899
-Epoch [ 15]: Loss 0.01997
-Validation: Loss 0.01755 Accuracy 1.00000
-Validation: Loss 0.01868 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01773
-Epoch [ 16]: Loss 0.01748
-Epoch [ 16]: Loss 0.01898
-Epoch [ 16]: Loss 0.01945
-Epoch [ 16]: Loss 0.01935
-Epoch [ 16]: Loss 0.01908
-Epoch [ 16]: Loss 0.01547
-Validation: Loss 0.01594 Accuracy 1.00000
-Validation: Loss 0.01699 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01776
-Epoch [ 17]: Loss 0.01820
-Epoch [ 17]: Loss 0.01682
-Epoch [ 17]: Loss 0.01693
-Epoch [ 17]: Loss 0.01560
-Epoch [ 17]: Loss 0.01681
-Epoch [ 17]: Loss 0.01475
-Validation: Loss 0.01456 Accuracy 1.00000
-Validation: Loss 0.01553 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01625
-Epoch [ 18]: Loss 0.01666
-Epoch [ 18]: Loss 0.01450
-Epoch [ 18]: Loss 0.01508
-Epoch [ 18]: Loss 0.01564
-Epoch [ 18]: Loss 0.01561
-Epoch [ 18]: Loss 0.01295
-Validation: Loss 0.01337 Accuracy 1.00000
-Validation: Loss 0.01428 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01438
-Epoch [ 19]: Loss 0.01278
-Epoch [ 19]: Loss 0.01462
-Epoch [ 19]: Loss 0.01433
-Epoch [ 19]: Loss 0.01527
-Epoch [ 19]: Loss 0.01456
-Epoch [ 19]: Loss 0.01358
-Validation: Loss 0.01235 Accuracy 1.00000
-Validation: Loss 0.01320 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01343
-Epoch [ 20]: Loss 0.01354
-Epoch [ 20]: Loss 0.01340
-Epoch [ 20]: Loss 0.01300
-Epoch [ 20]: Loss 0.01298
-Epoch [ 20]: Loss 0.01360
-Epoch [ 20]: Loss 0.01105
-Validation: Loss 0.01143 Accuracy 1.00000
-Validation: Loss 0.01222 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01269
-Epoch [ 21]: Loss 0.01341
-Epoch [ 21]: Loss 0.01266
-Epoch [ 21]: Loss 0.01082
-Epoch [ 21]: Loss 0.01322
-Epoch [ 21]: Loss 0.01105
-Epoch [ 21]: Loss 0.01117
+ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.63034
+Epoch [  1]: Loss 0.60553
+Epoch [  1]: Loss 0.56538
+Epoch [  1]: Loss 0.54630
+Epoch [  1]: Loss 0.50381
+Epoch [  1]: Loss 0.50074
+Epoch [  1]: Loss 0.47526
+Validation: Loss 0.46276 Accuracy 1.00000
+Validation: Loss 0.47391 Accuracy 1.00000
+Epoch [  2]: Loss 0.47023
+Epoch [  2]: Loss 0.44691
+Epoch [  2]: Loss 0.43783
+Epoch [  2]: Loss 0.43551
+Epoch [  2]: Loss 0.41303
+Epoch [  2]: Loss 0.39960
+Epoch [  2]: Loss 0.38515
+Validation: Loss 0.36395 Accuracy 1.00000
+Validation: Loss 0.37779 Accuracy 1.00000
+Epoch [  3]: Loss 0.37114
+Epoch [  3]: Loss 0.34915
+Epoch [  3]: Loss 0.36132
+Epoch [  3]: Loss 0.33572
+Epoch [  3]: Loss 0.31298
+Epoch [  3]: Loss 0.31052
+Epoch [  3]: Loss 0.30406
+Validation: Loss 0.27789 Accuracy 1.00000
+Validation: Loss 0.29328 Accuracy 1.00000
+Epoch [  4]: Loss 0.28022
+Epoch [  4]: Loss 0.27531
+Epoch [  4]: Loss 0.26726
+Epoch [  4]: Loss 0.26451
+Epoch [  4]: Loss 0.23373
+Epoch [  4]: Loss 0.24550
+Epoch [  4]: Loss 0.21351
+Validation: Loss 0.20780 Accuracy 1.00000
+Validation: Loss 0.22336 Accuracy 1.00000
+Epoch [  5]: Loss 0.22156
+Epoch [  5]: Loss 0.20017
+Epoch [  5]: Loss 0.19203
+Epoch [  5]: Loss 0.20301
+Epoch [  5]: Loss 0.18432
+Epoch [  5]: Loss 0.16845
+Epoch [  5]: Loss 0.18846
+Validation: Loss 0.15293 Accuracy 1.00000
+Validation: Loss 0.16724 Accuracy 1.00000
+Epoch [  6]: Loss 0.16454
+Epoch [  6]: Loss 0.15258
+Epoch [  6]: Loss 0.14479
+Epoch [  6]: Loss 0.13681
+Epoch [  6]: Loss 0.13567
+Epoch [  6]: Loss 0.12978
+Epoch [  6]: Loss 0.14006
+Validation: Loss 0.11109 Accuracy 1.00000
+Validation: Loss 0.12287 Accuracy 1.00000
+Epoch [  7]: Loss 0.12029
+Epoch [  7]: Loss 0.10976
+Epoch [  7]: Loss 0.11662
+Epoch [  7]: Loss 0.09244
+Epoch [  7]: Loss 0.10328
+Epoch [  7]: Loss 0.08922
+Epoch [  7]: Loss 0.08470
+Validation: Loss 0.07890 Accuracy 1.00000
+Validation: Loss 0.08760 Accuracy 1.00000
+Epoch [  8]: Loss 0.07588
+Epoch [  8]: Loss 0.07787
+Epoch [  8]: Loss 0.07559
+Epoch [  8]: Loss 0.07404
+Epoch [  8]: Loss 0.07644
+Epoch [  8]: Loss 0.06436
+Epoch [  8]: Loss 0.05506
+Validation: Loss 0.05515 Accuracy 1.00000
+Validation: Loss 0.06097 Accuracy 1.00000
+Epoch [  9]: Loss 0.05574
+Epoch [  9]: Loss 0.05409
+Epoch [  9]: Loss 0.05341
+Epoch [  9]: Loss 0.05125
+Epoch [  9]: Loss 0.05115
+Epoch [  9]: Loss 0.04846
+Epoch [  9]: Loss 0.04370
+Validation: Loss 0.04143 Accuracy 1.00000
+Validation: Loss 0.04557 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04284
+Epoch [ 10]: Loss 0.04111
+Epoch [ 10]: Loss 0.03779
+Epoch [ 10]: Loss 0.03960
+Epoch [ 10]: Loss 0.04298
+Epoch [ 10]: Loss 0.03869
+Epoch [ 10]: Loss 0.03524
+Validation: Loss 0.03367 Accuracy 1.00000
+Validation: Loss 0.03705 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03339
+Epoch [ 11]: Loss 0.03459
+Epoch [ 11]: Loss 0.03342
+Epoch [ 11]: Loss 0.03407
+Epoch [ 11]: Loss 0.03388
+Epoch [ 11]: Loss 0.03202
+Epoch [ 11]: Loss 0.02971
+Validation: Loss 0.02860 Accuracy 1.00000
+Validation: Loss 0.03154 Accuracy 1.00000
+Epoch [ 12]: Loss 0.02859
+Epoch [ 12]: Loss 0.02878
+Epoch [ 12]: Loss 0.02993
+Epoch [ 12]: Loss 0.02963
+Epoch [ 12]: Loss 0.02878
+Epoch [ 12]: Loss 0.02608
+Epoch [ 12]: Loss 0.03050
+Validation: Loss 0.02489 Accuracy 1.00000
+Validation: Loss 0.02750 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02694
+Epoch [ 13]: Loss 0.02671
+Epoch [ 13]: Loss 0.02622
+Epoch [ 13]: Loss 0.02467
+Epoch [ 13]: Loss 0.02467
+Epoch [ 13]: Loss 0.02212
+Epoch [ 13]: Loss 0.02449
+Validation: Loss 0.02201 Accuracy 1.00000
+Validation: Loss 0.02435 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02324
+Epoch [ 14]: Loss 0.02266
+Epoch [ 14]: Loss 0.02391
+Epoch [ 14]: Loss 0.02036
+Epoch [ 14]: Loss 0.02121
+Epoch [ 14]: Loss 0.02264
+Epoch [ 14]: Loss 0.02342
+Validation: Loss 0.01970 Accuracy 1.00000
+Validation: Loss 0.02183 Accuracy 1.00000
+Epoch [ 15]: Loss 0.02103
+Epoch [ 15]: Loss 0.02069
+Epoch [ 15]: Loss 0.02057
+Epoch [ 15]: Loss 0.01919
+Epoch [ 15]: Loss 0.01962
+Epoch [ 15]: Loss 0.01935
+Epoch [ 15]: Loss 0.02118
+Validation: Loss 0.01777 Accuracy 1.00000
+Validation: Loss 0.01973 Accuracy 1.00000
+Epoch [ 16]: Loss 0.02003
+Epoch [ 16]: Loss 0.01873
+Epoch [ 16]: Loss 0.01931
+Epoch [ 16]: Loss 0.01905
+Epoch [ 16]: Loss 0.01646
+Epoch [ 16]: Loss 0.01608
+Epoch [ 16]: Loss 0.01732
+Validation: Loss 0.01614 Accuracy 1.00000
+Validation: Loss 0.01795 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01741
+Epoch [ 17]: Loss 0.01719
+Epoch [ 17]: Loss 0.01619
+Epoch [ 17]: Loss 0.01722
+Epoch [ 17]: Loss 0.01639
+Epoch [ 17]: Loss 0.01571
+Epoch [ 17]: Loss 0.01460
+Validation: Loss 0.01474 Accuracy 1.00000
+Validation: Loss 0.01642 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01682
+Epoch [ 18]: Loss 0.01484
+Epoch [ 18]: Loss 0.01554
+Epoch [ 18]: Loss 0.01390
+Epoch [ 18]: Loss 0.01602
+Epoch [ 18]: Loss 0.01499
+Epoch [ 18]: Loss 0.01172
+Validation: Loss 0.01352 Accuracy 1.00000
+Validation: Loss 0.01510 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01519
+Epoch [ 19]: Loss 0.01364
+Epoch [ 19]: Loss 0.01484
+Epoch [ 19]: Loss 0.01413
+Epoch [ 19]: Loss 0.01383
+Epoch [ 19]: Loss 0.01316
+Epoch [ 19]: Loss 0.01076
+Validation: Loss 0.01246 Accuracy 1.00000
+Validation: Loss 0.01394 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01428
+Epoch [ 20]: Loss 0.01335
+Epoch [ 20]: Loss 0.01268
+Epoch [ 20]: Loss 0.01281
+Epoch [ 20]: Loss 0.01328
+Epoch [ 20]: Loss 0.01177
+Epoch [ 20]: Loss 0.01040
+Validation: Loss 0.01150 Accuracy 1.00000
+Validation: Loss 0.01288 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01271
+Epoch [ 21]: Loss 0.01233
+Epoch [ 21]: Loss 0.01054
+Epoch [ 21]: Loss 0.01321
+Epoch [ 21]: Loss 0.01203
+Epoch [ 21]: Loss 0.01177
+Epoch [ 21]: Loss 0.00776
 Validation: Loss 0.01057 Accuracy 1.00000
-Validation: Loss 0.01130 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01172
-Epoch [ 22]: Loss 0.01148
-Epoch [ 22]: Loss 0.01044
-Epoch [ 22]: Loss 0.01167
-Epoch [ 22]: Loss 0.01094
-Epoch [ 22]: Loss 0.01203
-Epoch [ 22]: Loss 0.00936
-Validation: Loss 0.00968 Accuracy 1.00000
-Validation: Loss 0.01034 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01106
-Epoch [ 23]: Loss 0.01014
-Epoch [ 23]: Loss 0.01062
-Epoch [ 23]: Loss 0.00968
-Epoch [ 23]: Loss 0.00951
-Epoch [ 23]: Loss 0.01021
-Epoch [ 23]: Loss 0.01148
-Validation: Loss 0.00868 Accuracy 1.00000
-Validation: Loss 0.00925 Accuracy 1.00000
-Epoch [ 24]: Loss 0.01007
-Epoch [ 24]: Loss 0.00907
-Epoch [ 24]: Loss 0.00987
-Epoch [ 24]: Loss 0.00865
-Epoch [ 24]: Loss 0.00877
-Epoch [ 24]: Loss 0.00827
-Epoch [ 24]: Loss 0.00907
-Validation: Loss 0.00770 Accuracy 1.00000
-Validation: Loss 0.00819 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00904
-Epoch [ 25]: Loss 0.00849
-Epoch [ 25]: Loss 0.00762
-Epoch [ 25]: Loss 0.00839
-Epoch [ 25]: Loss 0.00697
-Epoch [ 25]: Loss 0.00816
-Epoch [ 25]: Loss 0.00820
-Validation: Loss 0.00699 Accuracy 1.00000
-Validation: Loss 0.00742 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.61964
-Epoch [  1]: Loss 0.60676
-Epoch [  1]: Loss 0.57950
-Epoch [  1]: Loss 0.54075
-Epoch [  1]: Loss 0.52232
-Epoch [  1]: Loss 0.49256
-Epoch [  1]: Loss 0.48370
-Validation: Loss 0.45461 Accuracy 1.00000
-Validation: Loss 0.46667 Accuracy 1.00000
-Epoch [  2]: Loss 0.46845
-Epoch [  2]: Loss 0.45738
-Epoch [  2]: Loss 0.43860
-Epoch [  2]: Loss 0.43470
-Epoch [  2]: Loss 0.41848
-Epoch [  2]: Loss 0.39610
-Epoch [  2]: Loss 0.38415
-Validation: Loss 0.35426 Accuracy 1.00000
-Validation: Loss 0.36856 Accuracy 1.00000
-Epoch [  3]: Loss 0.35825
-Epoch [  3]: Loss 0.35951
-Epoch [  3]: Loss 0.36279
-Epoch [  3]: Loss 0.32950
-Epoch [  3]: Loss 0.31807
-Epoch [  3]: Loss 0.32480
-Epoch [  3]: Loss 0.30269
-Validation: Loss 0.26718 Accuracy 1.00000
-Validation: Loss 0.28308 Accuracy 1.00000
-Epoch [  4]: Loss 0.27659
-Epoch [  4]: Loss 0.28368
-Epoch [  4]: Loss 0.25963
-Epoch [  4]: Loss 0.26311
-Epoch [  4]: Loss 0.25752
-Epoch [  4]: Loss 0.23747
-Epoch [  4]: Loss 0.21984
-Validation: Loss 0.19750 Accuracy 1.00000
-Validation: Loss 0.21342 Accuracy 1.00000
-Epoch [  5]: Loss 0.21820
-Epoch [  5]: Loss 0.20098
-Epoch [  5]: Loss 0.20661
-Epoch [  5]: Loss 0.19340
-Epoch [  5]: Loss 0.18270
-Epoch [  5]: Loss 0.18410
-Epoch [  5]: Loss 0.18297
-Validation: Loss 0.14377 Accuracy 1.00000
-Validation: Loss 0.15836 Accuracy 1.00000
-Epoch [  6]: Loss 0.16764
-Epoch [  6]: Loss 0.14448
-Epoch [  6]: Loss 0.14964
-Epoch [  6]: Loss 0.13646
-Epoch [  6]: Loss 0.15055
-Epoch [  6]: Loss 0.13269
-Epoch [  6]: Loss 0.13151
-Validation: Loss 0.10387 Accuracy 1.00000
-Validation: Loss 0.11588 Accuracy 1.00000
-Epoch [  7]: Loss 0.12214
-Epoch [  7]: Loss 0.12336
-Epoch [  7]: Loss 0.10728
-Epoch [  7]: Loss 0.10316
-Epoch [  7]: Loss 0.09760
-Epoch [  7]: Loss 0.09230
-Epoch [  7]: Loss 0.08866
-Validation: Loss 0.07395 Accuracy 1.00000
-Validation: Loss 0.08278 Accuracy 1.00000
-Epoch [  8]: Loss 0.08347
-Epoch [  8]: Loss 0.08922
-Epoch [  8]: Loss 0.07872
-Epoch [  8]: Loss 0.07093
-Epoch [  8]: Loss 0.06854
-Epoch [  8]: Loss 0.06745
-Epoch [  8]: Loss 0.05531
-Validation: Loss 0.05181 Accuracy 1.00000
-Validation: Loss 0.05771 Accuracy 1.00000
-Epoch [  9]: Loss 0.06019
-Epoch [  9]: Loss 0.05877
-Epoch [  9]: Loss 0.05510
-Epoch [  9]: Loss 0.05308
-Epoch [  9]: Loss 0.04923
-Epoch [  9]: Loss 0.04534
-Epoch [  9]: Loss 0.04535
-Validation: Loss 0.03873 Accuracy 1.00000
-Validation: Loss 0.04297 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04256
-Epoch [ 10]: Loss 0.04386
-Epoch [ 10]: Loss 0.04479
-Epoch [ 10]: Loss 0.03830
-Epoch [ 10]: Loss 0.03930
-Epoch [ 10]: Loss 0.03758
-Epoch [ 10]: Loss 0.03751
-Validation: Loss 0.03130 Accuracy 1.00000
-Validation: Loss 0.03479 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03591
-Epoch [ 11]: Loss 0.03614
-Epoch [ 11]: Loss 0.03522
-Epoch [ 11]: Loss 0.03385
-Epoch [ 11]: Loss 0.03280
-Epoch [ 11]: Loss 0.03028
-Epoch [ 11]: Loss 0.02854
-Validation: Loss 0.02646 Accuracy 1.00000
-Validation: Loss 0.02951 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03022
-Epoch [ 12]: Loss 0.03157
-Epoch [ 12]: Loss 0.02827
-Epoch [ 12]: Loss 0.02888
-Epoch [ 12]: Loss 0.02807
-Epoch [ 12]: Loss 0.02731
-Epoch [ 12]: Loss 0.02753
-Validation: Loss 0.02297 Accuracy 1.00000
-Validation: Loss 0.02568 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02879
-Epoch [ 13]: Loss 0.02518
-Epoch [ 13]: Loss 0.02496
-Epoch [ 13]: Loss 0.02617
-Epoch [ 13]: Loss 0.02322
-Epoch [ 13]: Loss 0.02429
-Epoch [ 13]: Loss 0.02511
-Validation: Loss 0.02027 Accuracy 1.00000
-Validation: Loss 0.02272 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02328
-Epoch [ 14]: Loss 0.02450
-Epoch [ 14]: Loss 0.02156
-Epoch [ 14]: Loss 0.02238
-Epoch [ 14]: Loss 0.02300
-Epoch [ 14]: Loss 0.02170
-Epoch [ 14]: Loss 0.01950
-Validation: Loss 0.01812 Accuracy 1.00000
-Validation: Loss 0.02035 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02289
-Epoch [ 15]: Loss 0.02111
-Epoch [ 15]: Loss 0.01972
-Epoch [ 15]: Loss 0.02013
-Epoch [ 15]: Loss 0.02020
-Epoch [ 15]: Loss 0.01836
-Epoch [ 15]: Loss 0.01899
-Validation: Loss 0.01635 Accuracy 1.00000
-Validation: Loss 0.01839 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01956
-Epoch [ 16]: Loss 0.01711
-Epoch [ 16]: Loss 0.01998
-Epoch [ 16]: Loss 0.01812
-Epoch [ 16]: Loss 0.01900
-Epoch [ 16]: Loss 0.01763
-Epoch [ 16]: Loss 0.01537
-Validation: Loss 0.01484 Accuracy 1.00000
-Validation: Loss 0.01674 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01888
-Epoch [ 17]: Loss 0.01663
-Epoch [ 17]: Loss 0.01522
-Epoch [ 17]: Loss 0.01668
-Epoch [ 17]: Loss 0.01708
-Epoch [ 17]: Loss 0.01657
-Epoch [ 17]: Loss 0.01637
-Validation: Loss 0.01354 Accuracy 1.00000
-Validation: Loss 0.01531 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01558
-Epoch [ 18]: Loss 0.01553
-Epoch [ 18]: Loss 0.01548
-Epoch [ 18]: Loss 0.01486
-Epoch [ 18]: Loss 0.01595
-Epoch [ 18]: Loss 0.01505
-Epoch [ 18]: Loss 0.01557
-Validation: Loss 0.01239 Accuracy 1.00000
-Validation: Loss 0.01405 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01565
-Epoch [ 19]: Loss 0.01480
-Epoch [ 19]: Loss 0.01289
-Epoch [ 19]: Loss 0.01431
-Epoch [ 19]: Loss 0.01376
-Epoch [ 19]: Loss 0.01384
-Epoch [ 19]: Loss 0.01330
-Validation: Loss 0.01137 Accuracy 1.00000
-Validation: Loss 0.01291 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01301
-Epoch [ 20]: Loss 0.01286
-Epoch [ 20]: Loss 0.01358
-Epoch [ 20]: Loss 0.01280
-Epoch [ 20]: Loss 0.01276
-Epoch [ 20]: Loss 0.01273
-Epoch [ 20]: Loss 0.01456
-Validation: Loss 0.01042 Accuracy 1.00000
 Validation: Loss 0.01185 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01172
-Epoch [ 21]: Loss 0.01329
-Epoch [ 21]: Loss 0.01186
-Epoch [ 21]: Loss 0.01169
-Epoch [ 21]: Loss 0.01179
-Epoch [ 21]: Loss 0.01095
-Epoch [ 21]: Loss 0.01284
-Validation: Loss 0.00947 Accuracy 1.00000
-Validation: Loss 0.01075 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01178
-Epoch [ 22]: Loss 0.01089
-Epoch [ 22]: Loss 0.01110
-Epoch [ 22]: Loss 0.01071
-Epoch [ 22]: Loss 0.01036
-Epoch [ 22]: Loss 0.00971
-Epoch [ 22]: Loss 0.01027
-Validation: Loss 0.00845 Accuracy 1.00000
-Validation: Loss 0.00955 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01146
-Epoch [ 23]: Loss 0.00902
-Epoch [ 23]: Loss 0.00849
-Epoch [ 23]: Loss 0.00903
-Epoch [ 23]: Loss 0.00886
-Epoch [ 23]: Loss 0.00972
-Epoch [ 23]: Loss 0.01066
-Validation: Loss 0.00751 Accuracy 1.00000
-Validation: Loss 0.00846 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00976
-Epoch [ 24]: Loss 0.00777
+Epoch [ 22]: Loss 0.01148
+Epoch [ 22]: Loss 0.01199
+Epoch [ 22]: Loss 0.01032
+Epoch [ 22]: Loss 0.01079
+Epoch [ 22]: Loss 0.01012
+Epoch [ 22]: Loss 0.01070
+Epoch [ 22]: Loss 0.01114
+Validation: Loss 0.00957 Accuracy 1.00000
+Validation: Loss 0.01072 Accuracy 1.00000
+Epoch [ 23]: Loss 0.01039
+Epoch [ 23]: Loss 0.00953
+Epoch [ 23]: Loss 0.01061
+Epoch [ 23]: Loss 0.00931
+Epoch [ 23]: Loss 0.00997
+Epoch [ 23]: Loss 0.00961
+Epoch [ 23]: Loss 0.00712
+Validation: Loss 0.00850 Accuracy 1.00000
+Validation: Loss 0.00949 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00893
+Epoch [ 24]: Loss 0.00878
+Epoch [ 24]: Loss 0.00866
+Epoch [ 24]: Loss 0.00924
+Epoch [ 24]: Loss 0.00824
 Epoch [ 24]: Loss 0.00825
-Epoch [ 24]: Loss 0.00881
-Epoch [ 24]: Loss 0.00796
-Epoch [ 24]: Loss 0.00826
-Epoch [ 24]: Loss 0.00787
-Validation: Loss 0.00682 Accuracy 1.00000
-Validation: Loss 0.00765 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00840
-Epoch [ 25]: Loss 0.00781
-Epoch [ 25]: Loss 0.00786
-Epoch [ 25]: Loss 0.00724
-Epoch [ 25]: Loss 0.00762
-Epoch [ 25]: Loss 0.00726
+Epoch [ 24]: Loss 0.00878
+Validation: Loss 0.00763 Accuracy 1.00000
+Validation: Loss 0.00849 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00802
+Epoch [ 25]: Loss 0.00754
+Epoch [ 25]: Loss 0.00771
+Epoch [ 25]: Loss 0.00777
+Epoch [ 25]: Loss 0.00775
+Epoch [ 25]: Loss 0.00812
+Epoch [ 25]: Loss 0.00833
+Validation: Loss 0.00700 Accuracy 1.00000
+Validation: Loss 0.00776 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.62243
+Epoch [  1]: Loss 0.61271
+Epoch [  1]: Loss 0.55580
+Epoch [  1]: Loss 0.53213
+Epoch [  1]: Loss 0.52075
+Epoch [  1]: Loss 0.49787
+Epoch [  1]: Loss 0.47505
+Validation: Loss 0.47777 Accuracy 1.00000
+Validation: Loss 0.46300 Accuracy 1.00000
+Epoch [  2]: Loss 0.46806
+Epoch [  2]: Loss 0.44811
+Epoch [  2]: Loss 0.44868
+Epoch [  2]: Loss 0.42514
+Epoch [  2]: Loss 0.41207
+Epoch [  2]: Loss 0.38999
+Epoch [  2]: Loss 0.40689
+Validation: Loss 0.38313 Accuracy 1.00000
+Validation: Loss 0.36497 Accuracy 1.00000
+Epoch [  3]: Loss 0.37506
+Epoch [  3]: Loss 0.35458
+Epoch [  3]: Loss 0.34745
+Epoch [  3]: Loss 0.32494
+Epoch [  3]: Loss 0.31280
+Epoch [  3]: Loss 0.32159
+Epoch [  3]: Loss 0.31559
+Validation: Loss 0.29940 Accuracy 1.00000
+Validation: Loss 0.27966 Accuracy 1.00000
+Epoch [  4]: Loss 0.27458
+Epoch [  4]: Loss 0.28234
+Epoch [  4]: Loss 0.26623
+Epoch [  4]: Loss 0.24585
+Epoch [  4]: Loss 0.24220
+Epoch [  4]: Loss 0.25250
+Epoch [  4]: Loss 0.22705
+Validation: Loss 0.22949 Accuracy 1.00000
+Validation: Loss 0.20991 Accuracy 1.00000
+Epoch [  5]: Loss 0.20868
+Epoch [  5]: Loss 0.20777
+Epoch [  5]: Loss 0.21327
+Epoch [  5]: Loss 0.18019
+Epoch [  5]: Loss 0.18150
+Epoch [  5]: Loss 0.17559
+Epoch [  5]: Loss 0.20257
+Validation: Loss 0.17268 Accuracy 1.00000
+Validation: Loss 0.15497 Accuracy 1.00000
+Epoch [  6]: Loss 0.15827
+Epoch [  6]: Loss 0.15591
+Epoch [  6]: Loss 0.14491
+Epoch [  6]: Loss 0.13292
+Epoch [  6]: Loss 0.14914
+Epoch [  6]: Loss 0.12443
+Epoch [  6]: Loss 0.13808
+Validation: Loss 0.12726 Accuracy 1.00000
+Validation: Loss 0.11293 Accuracy 1.00000
+Epoch [  7]: Loss 0.12103
+Epoch [  7]: Loss 0.11242
+Epoch [  7]: Loss 0.10596
+Epoch [  7]: Loss 0.09875
+Epoch [  7]: Loss 0.09495
+Epoch [  7]: Loss 0.09276
+Epoch [  7]: Loss 0.10884
+Validation: Loss 0.09070 Accuracy 1.00000
+Validation: Loss 0.08028 Accuracy 1.00000
+Epoch [  8]: Loss 0.09406
+Epoch [  8]: Loss 0.08342
+Epoch [  8]: Loss 0.07543
+Epoch [  8]: Loss 0.06123
+Epoch [  8]: Loss 0.06837
+Epoch [  8]: Loss 0.06419
+Epoch [  8]: Loss 0.05732
+Validation: Loss 0.06270 Accuracy 1.00000
+Validation: Loss 0.05587 Accuracy 1.00000
+Epoch [  9]: Loss 0.05821
+Epoch [  9]: Loss 0.05601
+Epoch [  9]: Loss 0.05212
+Epoch [  9]: Loss 0.05552
+Epoch [  9]: Loss 0.04673
+Epoch [  9]: Loss 0.04686
+Epoch [  9]: Loss 0.03952
+Validation: Loss 0.04691 Accuracy 1.00000
+Validation: Loss 0.04198 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04253
+Epoch [ 10]: Loss 0.04115
+Epoch [ 10]: Loss 0.04191
+Epoch [ 10]: Loss 0.04000
+Epoch [ 10]: Loss 0.03686
+Epoch [ 10]: Loss 0.03965
+Epoch [ 10]: Loss 0.03798
+Validation: Loss 0.03820 Accuracy 1.00000
+Validation: Loss 0.03413 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03729
+Epoch [ 11]: Loss 0.03196
+Epoch [ 11]: Loss 0.03405
+Epoch [ 11]: Loss 0.03641
+Epoch [ 11]: Loss 0.03278
+Epoch [ 11]: Loss 0.02875
+Epoch [ 11]: Loss 0.03003
+Validation: Loss 0.03254 Accuracy 1.00000
+Validation: Loss 0.02899 Accuracy 1.00000
+Epoch [ 12]: Loss 0.03042
+Epoch [ 12]: Loss 0.02825
+Epoch [ 12]: Loss 0.02816
+Epoch [ 12]: Loss 0.03091
+Epoch [ 12]: Loss 0.02586
+Epoch [ 12]: Loss 0.02826
+Epoch [ 12]: Loss 0.02898
+Validation: Loss 0.02840 Accuracy 1.00000
+Validation: Loss 0.02524 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02510
+Epoch [ 13]: Loss 0.02628
+Epoch [ 13]: Loss 0.02558
+Epoch [ 13]: Loss 0.02468
+Epoch [ 13]: Loss 0.02447
+Epoch [ 13]: Loss 0.02516
+Epoch [ 13]: Loss 0.02303
+Validation: Loss 0.02517 Accuracy 1.00000
+Validation: Loss 0.02233 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02156
+Epoch [ 14]: Loss 0.02310
+Epoch [ 14]: Loss 0.02294
+Epoch [ 14]: Loss 0.02135
+Epoch [ 14]: Loss 0.02244
+Epoch [ 14]: Loss 0.02250
+Epoch [ 14]: Loss 0.02294
+Validation: Loss 0.02259 Accuracy 1.00000
+Validation: Loss 0.01999 Accuracy 1.00000
+Epoch [ 15]: Loss 0.01831
+Epoch [ 15]: Loss 0.01957
+Epoch [ 15]: Loss 0.01967
+Epoch [ 15]: Loss 0.02107
+Epoch [ 15]: Loss 0.01948
+Epoch [ 15]: Loss 0.02159
+Epoch [ 15]: Loss 0.02296
+Validation: Loss 0.02044 Accuracy 1.00000
+Validation: Loss 0.01805 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01883
+Epoch [ 16]: Loss 0.01874
+Epoch [ 16]: Loss 0.01780
+Epoch [ 16]: Loss 0.01667
+Epoch [ 16]: Loss 0.01866
+Epoch [ 16]: Loss 0.01859
+Epoch [ 16]: Loss 0.01752
+Validation: Loss 0.01859 Accuracy 1.00000
+Validation: Loss 0.01638 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01578
+Epoch [ 17]: Loss 0.01836
+Epoch [ 17]: Loss 0.01503
+Epoch [ 17]: Loss 0.01757
+Epoch [ 17]: Loss 0.01687
+Epoch [ 17]: Loss 0.01556
+Epoch [ 17]: Loss 0.01723
+Validation: Loss 0.01700 Accuracy 1.00000
+Validation: Loss 0.01495 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01577
+Epoch [ 18]: Loss 0.01588
+Epoch [ 18]: Loss 0.01479
+Epoch [ 18]: Loss 0.01476
+Epoch [ 18]: Loss 0.01442
+Epoch [ 18]: Loss 0.01556
+Epoch [ 18]: Loss 0.01408
+Validation: Loss 0.01562 Accuracy 1.00000
+Validation: Loss 0.01370 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01447
+Epoch [ 19]: Loss 0.01459
+Epoch [ 19]: Loss 0.01301
+Epoch [ 19]: Loss 0.01425
+Epoch [ 19]: Loss 0.01311
+Epoch [ 19]: Loss 0.01418
+Epoch [ 19]: Loss 0.01348
+Validation: Loss 0.01438 Accuracy 1.00000
+Validation: Loss 0.01260 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01398
+Epoch [ 20]: Loss 0.01408
+Epoch [ 20]: Loss 0.01339
+Epoch [ 20]: Loss 0.01272
+Epoch [ 20]: Loss 0.01167
+Epoch [ 20]: Loss 0.01151
+Epoch [ 20]: Loss 0.01086
+Validation: Loss 0.01319 Accuracy 1.00000
+Validation: Loss 0.01155 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01245
+Epoch [ 21]: Loss 0.01062
+Epoch [ 21]: Loss 0.01189
+Epoch [ 21]: Loss 0.01221
+Epoch [ 21]: Loss 0.01134
+Epoch [ 21]: Loss 0.01185
+Epoch [ 21]: Loss 0.01080
+Validation: Loss 0.01194 Accuracy 1.00000
+Validation: Loss 0.01047 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01159
+Epoch [ 22]: Loss 0.01187
+Epoch [ 22]: Loss 0.01073
+Epoch [ 22]: Loss 0.00970
+Epoch [ 22]: Loss 0.01003
+Epoch [ 22]: Loss 0.00966
+Epoch [ 22]: Loss 0.00883
+Validation: Loss 0.01057 Accuracy 1.00000
+Validation: Loss 0.00929 Accuracy 1.00000
+Epoch [ 23]: Loss 0.00981
+Epoch [ 23]: Loss 0.01000
+Epoch [ 23]: Loss 0.00948
+Epoch [ 23]: Loss 0.00895
+Epoch [ 23]: Loss 0.00924
+Epoch [ 23]: Loss 0.00850
+Epoch [ 23]: Loss 0.00887
+Validation: Loss 0.00937 Accuracy 1.00000
+Validation: Loss 0.00828 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00856
+Epoch [ 24]: Loss 0.00821
+Epoch [ 24]: Loss 0.00878
+Epoch [ 24]: Loss 0.00829
+Epoch [ 24]: Loss 0.00788
+Epoch [ 24]: Loss 0.00851
+Epoch [ 24]: Loss 0.00775
+Validation: Loss 0.00852 Accuracy 1.00000
+Validation: Loss 0.00755 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00766
+Epoch [ 25]: Loss 0.00806
+Epoch [ 25]: Loss 0.00790
 Epoch [ 25]: Loss 0.00775
-Validation: Loss 0.00631 Accuracy 1.00000
-Validation: Loss 0.00707 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
+Epoch [ 25]: Loss 0.00713
+Epoch [ 25]: Loss 0.00745
+Epoch [ 25]: Loss 0.00773
+Validation: Loss 0.00790 Accuracy 1.00000
+Validation: Loss 0.00701 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
  :ps_trained
  :st_trained
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
diff --git a/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.CQMmi5pL.lean.js b/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.C0fFR2Ju.lean.js
similarity index 67%
rename from previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.CQMmi5pL.lean.js
rename to previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.C0fFR2Ju.lean.js
index f363446281..60c46e48f2 100644
--- a/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.CQMmi5pL.lean.js
+++ b/previews/PR1000/assets/tutorials_beginner_3_SimpleRNN.md.C0fFR2Ju.lean.js
@@ -97,455 +97,455 @@ import{_ as a,c as n,a2 as i,o as p}from"./chunks/framework.DjZDIZsN.js";const r
     return (train_state.parameters, train_state.states) |> cpu_device()
 end
 
-ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.63922
-Epoch [  1]: Loss 0.59006
-Epoch [  1]: Loss 0.56046
-Epoch [  1]: Loss 0.53851
-Epoch [  1]: Loss 0.51878
-Epoch [  1]: Loss 0.50837
-Epoch [  1]: Loss 0.47988
-Validation: Loss 0.46056 Accuracy 1.00000
-Validation: Loss 0.46788 Accuracy 1.00000
-Epoch [  2]: Loss 0.47123
-Epoch [  2]: Loss 0.44911
-Epoch [  2]: Loss 0.44093
-Epoch [  2]: Loss 0.43190
-Epoch [  2]: Loss 0.41116
-Epoch [  2]: Loss 0.39899
-Epoch [  2]: Loss 0.42438
-Validation: Loss 0.36182 Accuracy 1.00000
-Validation: Loss 0.37078 Accuracy 1.00000
-Epoch [  3]: Loss 0.37969
-Epoch [  3]: Loss 0.36815
-Epoch [  3]: Loss 0.34446
-Epoch [  3]: Loss 0.34307
-Epoch [  3]: Loss 0.31119
-Epoch [  3]: Loss 0.30866
-Epoch [  3]: Loss 0.29978
-Validation: Loss 0.27641 Accuracy 1.00000
-Validation: Loss 0.28617 Accuracy 1.00000
-Epoch [  4]: Loss 0.29006
-Epoch [  4]: Loss 0.27774
-Epoch [  4]: Loss 0.26425
-Epoch [  4]: Loss 0.26136
-Epoch [  4]: Loss 0.25087
-Epoch [  4]: Loss 0.23835
-Epoch [  4]: Loss 0.21375
-Validation: Loss 0.20609 Accuracy 1.00000
-Validation: Loss 0.21591 Accuracy 1.00000
-Epoch [  5]: Loss 0.20882
-Epoch [  5]: Loss 0.22010
-Epoch [  5]: Loss 0.20382
-Epoch [  5]: Loss 0.19422
-Epoch [  5]: Loss 0.18539
-Epoch [  5]: Loss 0.17881
-Epoch [  5]: Loss 0.16316
-Validation: Loss 0.15119 Accuracy 1.00000
-Validation: Loss 0.16004 Accuracy 1.00000
-Epoch [  6]: Loss 0.16827
-Epoch [  6]: Loss 0.15449
-Epoch [  6]: Loss 0.15191
-Epoch [  6]: Loss 0.13655
-Epoch [  6]: Loss 0.13572
-Epoch [  6]: Loss 0.13289
-Epoch [  6]: Loss 0.13403
-Validation: Loss 0.10993 Accuracy 1.00000
-Validation: Loss 0.11702 Accuracy 1.00000
-Epoch [  7]: Loss 0.11842
-Epoch [  7]: Loss 0.12253
-Epoch [  7]: Loss 0.09597
-Epoch [  7]: Loss 0.11399
-Epoch [  7]: Loss 0.09454
-Epoch [  7]: Loss 0.09874
-Epoch [  7]: Loss 0.08202
-Validation: Loss 0.07828 Accuracy 1.00000
-Validation: Loss 0.08338 Accuracy 1.00000
-Epoch [  8]: Loss 0.08202
-Epoch [  8]: Loss 0.08031
-Epoch [  8]: Loss 0.08125
-Epoch [  8]: Loss 0.07767
-Epoch [  8]: Loss 0.07224
-Epoch [  8]: Loss 0.05991
-Epoch [  8]: Loss 0.06217
-Validation: Loss 0.05471 Accuracy 1.00000
-Validation: Loss 0.05807 Accuracy 1.00000
-Epoch [  9]: Loss 0.06353
-Epoch [  9]: Loss 0.05217
-Epoch [  9]: Loss 0.05646
-Epoch [  9]: Loss 0.05128
-Epoch [  9]: Loss 0.05138
-Epoch [  9]: Loss 0.04732
-Epoch [  9]: Loss 0.04093
-Validation: Loss 0.04107 Accuracy 1.00000
-Validation: Loss 0.04349 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04294
-Epoch [ 10]: Loss 0.04586
-Epoch [ 10]: Loss 0.04159
-Epoch [ 10]: Loss 0.04004
-Epoch [ 10]: Loss 0.03786
-Epoch [ 10]: Loss 0.03902
-Epoch [ 10]: Loss 0.03803
-Validation: Loss 0.03334 Accuracy 1.00000
-Validation: Loss 0.03532 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03516
-Epoch [ 11]: Loss 0.03884
-Epoch [ 11]: Loss 0.03437
-Epoch [ 11]: Loss 0.03213
-Epoch [ 11]: Loss 0.03224
-Epoch [ 11]: Loss 0.03263
-Epoch [ 11]: Loss 0.02999
-Validation: Loss 0.02831 Accuracy 1.00000
-Validation: Loss 0.03002 Accuracy 1.00000
-Epoch [ 12]: Loss 0.02805
-Epoch [ 12]: Loss 0.03113
-Epoch [ 12]: Loss 0.02716
-Epoch [ 12]: Loss 0.03118
-Epoch [ 12]: Loss 0.03009
-Epoch [ 12]: Loss 0.02828
-Epoch [ 12]: Loss 0.02706
-Validation: Loss 0.02463 Accuracy 1.00000
-Validation: Loss 0.02615 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02656
-Epoch [ 13]: Loss 0.02839
-Epoch [ 13]: Loss 0.02328
-Epoch [ 13]: Loss 0.02764
-Epoch [ 13]: Loss 0.02471
-Epoch [ 13]: Loss 0.02318
-Epoch [ 13]: Loss 0.02648
-Validation: Loss 0.02177 Accuracy 1.00000
-Validation: Loss 0.02313 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02412
-Epoch [ 14]: Loss 0.02352
-Epoch [ 14]: Loss 0.02210
-Epoch [ 14]: Loss 0.02238
-Epoch [ 14]: Loss 0.02261
-Epoch [ 14]: Loss 0.02210
-Epoch [ 14]: Loss 0.02290
-Validation: Loss 0.01947 Accuracy 1.00000
-Validation: Loss 0.02070 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02160
-Epoch [ 15]: Loss 0.02149
-Epoch [ 15]: Loss 0.02041
-Epoch [ 15]: Loss 0.02102
-Epoch [ 15]: Loss 0.01964
-Epoch [ 15]: Loss 0.01899
-Epoch [ 15]: Loss 0.01997
-Validation: Loss 0.01755 Accuracy 1.00000
-Validation: Loss 0.01868 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01773
-Epoch [ 16]: Loss 0.01748
-Epoch [ 16]: Loss 0.01898
-Epoch [ 16]: Loss 0.01945
-Epoch [ 16]: Loss 0.01935
-Epoch [ 16]: Loss 0.01908
-Epoch [ 16]: Loss 0.01547
-Validation: Loss 0.01594 Accuracy 1.00000
-Validation: Loss 0.01699 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01776
-Epoch [ 17]: Loss 0.01820
-Epoch [ 17]: Loss 0.01682
-Epoch [ 17]: Loss 0.01693
-Epoch [ 17]: Loss 0.01560
-Epoch [ 17]: Loss 0.01681
-Epoch [ 17]: Loss 0.01475
-Validation: Loss 0.01456 Accuracy 1.00000
-Validation: Loss 0.01553 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01625
-Epoch [ 18]: Loss 0.01666
-Epoch [ 18]: Loss 0.01450
-Epoch [ 18]: Loss 0.01508
-Epoch [ 18]: Loss 0.01564
-Epoch [ 18]: Loss 0.01561
-Epoch [ 18]: Loss 0.01295
-Validation: Loss 0.01337 Accuracy 1.00000
-Validation: Loss 0.01428 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01438
-Epoch [ 19]: Loss 0.01278
-Epoch [ 19]: Loss 0.01462
-Epoch [ 19]: Loss 0.01433
-Epoch [ 19]: Loss 0.01527
-Epoch [ 19]: Loss 0.01456
-Epoch [ 19]: Loss 0.01358
-Validation: Loss 0.01235 Accuracy 1.00000
-Validation: Loss 0.01320 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01343
-Epoch [ 20]: Loss 0.01354
-Epoch [ 20]: Loss 0.01340
-Epoch [ 20]: Loss 0.01300
-Epoch [ 20]: Loss 0.01298
-Epoch [ 20]: Loss 0.01360
-Epoch [ 20]: Loss 0.01105
-Validation: Loss 0.01143 Accuracy 1.00000
-Validation: Loss 0.01222 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01269
-Epoch [ 21]: Loss 0.01341
-Epoch [ 21]: Loss 0.01266
-Epoch [ 21]: Loss 0.01082
-Epoch [ 21]: Loss 0.01322
-Epoch [ 21]: Loss 0.01105
-Epoch [ 21]: Loss 0.01117
+ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.63034
+Epoch [  1]: Loss 0.60553
+Epoch [  1]: Loss 0.56538
+Epoch [  1]: Loss 0.54630
+Epoch [  1]: Loss 0.50381
+Epoch [  1]: Loss 0.50074
+Epoch [  1]: Loss 0.47526
+Validation: Loss 0.46276 Accuracy 1.00000
+Validation: Loss 0.47391 Accuracy 1.00000
+Epoch [  2]: Loss 0.47023
+Epoch [  2]: Loss 0.44691
+Epoch [  2]: Loss 0.43783
+Epoch [  2]: Loss 0.43551
+Epoch [  2]: Loss 0.41303
+Epoch [  2]: Loss 0.39960
+Epoch [  2]: Loss 0.38515
+Validation: Loss 0.36395 Accuracy 1.00000
+Validation: Loss 0.37779 Accuracy 1.00000
+Epoch [  3]: Loss 0.37114
+Epoch [  3]: Loss 0.34915
+Epoch [  3]: Loss 0.36132
+Epoch [  3]: Loss 0.33572
+Epoch [  3]: Loss 0.31298
+Epoch [  3]: Loss 0.31052
+Epoch [  3]: Loss 0.30406
+Validation: Loss 0.27789 Accuracy 1.00000
+Validation: Loss 0.29328 Accuracy 1.00000
+Epoch [  4]: Loss 0.28022
+Epoch [  4]: Loss 0.27531
+Epoch [  4]: Loss 0.26726
+Epoch [  4]: Loss 0.26451
+Epoch [  4]: Loss 0.23373
+Epoch [  4]: Loss 0.24550
+Epoch [  4]: Loss 0.21351
+Validation: Loss 0.20780 Accuracy 1.00000
+Validation: Loss 0.22336 Accuracy 1.00000
+Epoch [  5]: Loss 0.22156
+Epoch [  5]: Loss 0.20017
+Epoch [  5]: Loss 0.19203
+Epoch [  5]: Loss 0.20301
+Epoch [  5]: Loss 0.18432
+Epoch [  5]: Loss 0.16845
+Epoch [  5]: Loss 0.18846
+Validation: Loss 0.15293 Accuracy 1.00000
+Validation: Loss 0.16724 Accuracy 1.00000
+Epoch [  6]: Loss 0.16454
+Epoch [  6]: Loss 0.15258
+Epoch [  6]: Loss 0.14479
+Epoch [  6]: Loss 0.13681
+Epoch [  6]: Loss 0.13567
+Epoch [  6]: Loss 0.12978
+Epoch [  6]: Loss 0.14006
+Validation: Loss 0.11109 Accuracy 1.00000
+Validation: Loss 0.12287 Accuracy 1.00000
+Epoch [  7]: Loss 0.12029
+Epoch [  7]: Loss 0.10976
+Epoch [  7]: Loss 0.11662
+Epoch [  7]: Loss 0.09244
+Epoch [  7]: Loss 0.10328
+Epoch [  7]: Loss 0.08922
+Epoch [  7]: Loss 0.08470
+Validation: Loss 0.07890 Accuracy 1.00000
+Validation: Loss 0.08760 Accuracy 1.00000
+Epoch [  8]: Loss 0.07588
+Epoch [  8]: Loss 0.07787
+Epoch [  8]: Loss 0.07559
+Epoch [  8]: Loss 0.07404
+Epoch [  8]: Loss 0.07644
+Epoch [  8]: Loss 0.06436
+Epoch [  8]: Loss 0.05506
+Validation: Loss 0.05515 Accuracy 1.00000
+Validation: Loss 0.06097 Accuracy 1.00000
+Epoch [  9]: Loss 0.05574
+Epoch [  9]: Loss 0.05409
+Epoch [  9]: Loss 0.05341
+Epoch [  9]: Loss 0.05125
+Epoch [  9]: Loss 0.05115
+Epoch [  9]: Loss 0.04846
+Epoch [  9]: Loss 0.04370
+Validation: Loss 0.04143 Accuracy 1.00000
+Validation: Loss 0.04557 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04284
+Epoch [ 10]: Loss 0.04111
+Epoch [ 10]: Loss 0.03779
+Epoch [ 10]: Loss 0.03960
+Epoch [ 10]: Loss 0.04298
+Epoch [ 10]: Loss 0.03869
+Epoch [ 10]: Loss 0.03524
+Validation: Loss 0.03367 Accuracy 1.00000
+Validation: Loss 0.03705 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03339
+Epoch [ 11]: Loss 0.03459
+Epoch [ 11]: Loss 0.03342
+Epoch [ 11]: Loss 0.03407
+Epoch [ 11]: Loss 0.03388
+Epoch [ 11]: Loss 0.03202
+Epoch [ 11]: Loss 0.02971
+Validation: Loss 0.02860 Accuracy 1.00000
+Validation: Loss 0.03154 Accuracy 1.00000
+Epoch [ 12]: Loss 0.02859
+Epoch [ 12]: Loss 0.02878
+Epoch [ 12]: Loss 0.02993
+Epoch [ 12]: Loss 0.02963
+Epoch [ 12]: Loss 0.02878
+Epoch [ 12]: Loss 0.02608
+Epoch [ 12]: Loss 0.03050
+Validation: Loss 0.02489 Accuracy 1.00000
+Validation: Loss 0.02750 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02694
+Epoch [ 13]: Loss 0.02671
+Epoch [ 13]: Loss 0.02622
+Epoch [ 13]: Loss 0.02467
+Epoch [ 13]: Loss 0.02467
+Epoch [ 13]: Loss 0.02212
+Epoch [ 13]: Loss 0.02449
+Validation: Loss 0.02201 Accuracy 1.00000
+Validation: Loss 0.02435 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02324
+Epoch [ 14]: Loss 0.02266
+Epoch [ 14]: Loss 0.02391
+Epoch [ 14]: Loss 0.02036
+Epoch [ 14]: Loss 0.02121
+Epoch [ 14]: Loss 0.02264
+Epoch [ 14]: Loss 0.02342
+Validation: Loss 0.01970 Accuracy 1.00000
+Validation: Loss 0.02183 Accuracy 1.00000
+Epoch [ 15]: Loss 0.02103
+Epoch [ 15]: Loss 0.02069
+Epoch [ 15]: Loss 0.02057
+Epoch [ 15]: Loss 0.01919
+Epoch [ 15]: Loss 0.01962
+Epoch [ 15]: Loss 0.01935
+Epoch [ 15]: Loss 0.02118
+Validation: Loss 0.01777 Accuracy 1.00000
+Validation: Loss 0.01973 Accuracy 1.00000
+Epoch [ 16]: Loss 0.02003
+Epoch [ 16]: Loss 0.01873
+Epoch [ 16]: Loss 0.01931
+Epoch [ 16]: Loss 0.01905
+Epoch [ 16]: Loss 0.01646
+Epoch [ 16]: Loss 0.01608
+Epoch [ 16]: Loss 0.01732
+Validation: Loss 0.01614 Accuracy 1.00000
+Validation: Loss 0.01795 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01741
+Epoch [ 17]: Loss 0.01719
+Epoch [ 17]: Loss 0.01619
+Epoch [ 17]: Loss 0.01722
+Epoch [ 17]: Loss 0.01639
+Epoch [ 17]: Loss 0.01571
+Epoch [ 17]: Loss 0.01460
+Validation: Loss 0.01474 Accuracy 1.00000
+Validation: Loss 0.01642 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01682
+Epoch [ 18]: Loss 0.01484
+Epoch [ 18]: Loss 0.01554
+Epoch [ 18]: Loss 0.01390
+Epoch [ 18]: Loss 0.01602
+Epoch [ 18]: Loss 0.01499
+Epoch [ 18]: Loss 0.01172
+Validation: Loss 0.01352 Accuracy 1.00000
+Validation: Loss 0.01510 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01519
+Epoch [ 19]: Loss 0.01364
+Epoch [ 19]: Loss 0.01484
+Epoch [ 19]: Loss 0.01413
+Epoch [ 19]: Loss 0.01383
+Epoch [ 19]: Loss 0.01316
+Epoch [ 19]: Loss 0.01076
+Validation: Loss 0.01246 Accuracy 1.00000
+Validation: Loss 0.01394 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01428
+Epoch [ 20]: Loss 0.01335
+Epoch [ 20]: Loss 0.01268
+Epoch [ 20]: Loss 0.01281
+Epoch [ 20]: Loss 0.01328
+Epoch [ 20]: Loss 0.01177
+Epoch [ 20]: Loss 0.01040
+Validation: Loss 0.01150 Accuracy 1.00000
+Validation: Loss 0.01288 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01271
+Epoch [ 21]: Loss 0.01233
+Epoch [ 21]: Loss 0.01054
+Epoch [ 21]: Loss 0.01321
+Epoch [ 21]: Loss 0.01203
+Epoch [ 21]: Loss 0.01177
+Epoch [ 21]: Loss 0.00776
 Validation: Loss 0.01057 Accuracy 1.00000
-Validation: Loss 0.01130 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01172
-Epoch [ 22]: Loss 0.01148
-Epoch [ 22]: Loss 0.01044
-Epoch [ 22]: Loss 0.01167
-Epoch [ 22]: Loss 0.01094
-Epoch [ 22]: Loss 0.01203
-Epoch [ 22]: Loss 0.00936
-Validation: Loss 0.00968 Accuracy 1.00000
-Validation: Loss 0.01034 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01106
-Epoch [ 23]: Loss 0.01014
-Epoch [ 23]: Loss 0.01062
-Epoch [ 23]: Loss 0.00968
-Epoch [ 23]: Loss 0.00951
-Epoch [ 23]: Loss 0.01021
-Epoch [ 23]: Loss 0.01148
-Validation: Loss 0.00868 Accuracy 1.00000
-Validation: Loss 0.00925 Accuracy 1.00000
-Epoch [ 24]: Loss 0.01007
-Epoch [ 24]: Loss 0.00907
-Epoch [ 24]: Loss 0.00987
-Epoch [ 24]: Loss 0.00865
-Epoch [ 24]: Loss 0.00877
-Epoch [ 24]: Loss 0.00827
-Epoch [ 24]: Loss 0.00907
-Validation: Loss 0.00770 Accuracy 1.00000
-Validation: Loss 0.00819 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00904
-Epoch [ 25]: Loss 0.00849
-Epoch [ 25]: Loss 0.00762
-Epoch [ 25]: Loss 0.00839
-Epoch [ 25]: Loss 0.00697
-Epoch [ 25]: Loss 0.00816
-Epoch [ 25]: Loss 0.00820
-Validation: Loss 0.00699 Accuracy 1.00000
-Validation: Loss 0.00742 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.61964
-Epoch [  1]: Loss 0.60676
-Epoch [  1]: Loss 0.57950
-Epoch [  1]: Loss 0.54075
-Epoch [  1]: Loss 0.52232
-Epoch [  1]: Loss 0.49256
-Epoch [  1]: Loss 0.48370
-Validation: Loss 0.45461 Accuracy 1.00000
-Validation: Loss 0.46667 Accuracy 1.00000
-Epoch [  2]: Loss 0.46845
-Epoch [  2]: Loss 0.45738
-Epoch [  2]: Loss 0.43860
-Epoch [  2]: Loss 0.43470
-Epoch [  2]: Loss 0.41848
-Epoch [  2]: Loss 0.39610
-Epoch [  2]: Loss 0.38415
-Validation: Loss 0.35426 Accuracy 1.00000
-Validation: Loss 0.36856 Accuracy 1.00000
-Epoch [  3]: Loss 0.35825
-Epoch [  3]: Loss 0.35951
-Epoch [  3]: Loss 0.36279
-Epoch [  3]: Loss 0.32950
-Epoch [  3]: Loss 0.31807
-Epoch [  3]: Loss 0.32480
-Epoch [  3]: Loss 0.30269
-Validation: Loss 0.26718 Accuracy 1.00000
-Validation: Loss 0.28308 Accuracy 1.00000
-Epoch [  4]: Loss 0.27659
-Epoch [  4]: Loss 0.28368
-Epoch [  4]: Loss 0.25963
-Epoch [  4]: Loss 0.26311
-Epoch [  4]: Loss 0.25752
-Epoch [  4]: Loss 0.23747
-Epoch [  4]: Loss 0.21984
-Validation: Loss 0.19750 Accuracy 1.00000
-Validation: Loss 0.21342 Accuracy 1.00000
-Epoch [  5]: Loss 0.21820
-Epoch [  5]: Loss 0.20098
-Epoch [  5]: Loss 0.20661
-Epoch [  5]: Loss 0.19340
-Epoch [  5]: Loss 0.18270
-Epoch [  5]: Loss 0.18410
-Epoch [  5]: Loss 0.18297
-Validation: Loss 0.14377 Accuracy 1.00000
-Validation: Loss 0.15836 Accuracy 1.00000
-Epoch [  6]: Loss 0.16764
-Epoch [  6]: Loss 0.14448
-Epoch [  6]: Loss 0.14964
-Epoch [  6]: Loss 0.13646
-Epoch [  6]: Loss 0.15055
-Epoch [  6]: Loss 0.13269
-Epoch [  6]: Loss 0.13151
-Validation: Loss 0.10387 Accuracy 1.00000
-Validation: Loss 0.11588 Accuracy 1.00000
-Epoch [  7]: Loss 0.12214
-Epoch [  7]: Loss 0.12336
-Epoch [  7]: Loss 0.10728
-Epoch [  7]: Loss 0.10316
-Epoch [  7]: Loss 0.09760
-Epoch [  7]: Loss 0.09230
-Epoch [  7]: Loss 0.08866
-Validation: Loss 0.07395 Accuracy 1.00000
-Validation: Loss 0.08278 Accuracy 1.00000
-Epoch [  8]: Loss 0.08347
-Epoch [  8]: Loss 0.08922
-Epoch [  8]: Loss 0.07872
-Epoch [  8]: Loss 0.07093
-Epoch [  8]: Loss 0.06854
-Epoch [  8]: Loss 0.06745
-Epoch [  8]: Loss 0.05531
-Validation: Loss 0.05181 Accuracy 1.00000
-Validation: Loss 0.05771 Accuracy 1.00000
-Epoch [  9]: Loss 0.06019
-Epoch [  9]: Loss 0.05877
-Epoch [  9]: Loss 0.05510
-Epoch [  9]: Loss 0.05308
-Epoch [  9]: Loss 0.04923
-Epoch [  9]: Loss 0.04534
-Epoch [  9]: Loss 0.04535
-Validation: Loss 0.03873 Accuracy 1.00000
-Validation: Loss 0.04297 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04256
-Epoch [ 10]: Loss 0.04386
-Epoch [ 10]: Loss 0.04479
-Epoch [ 10]: Loss 0.03830
-Epoch [ 10]: Loss 0.03930
-Epoch [ 10]: Loss 0.03758
-Epoch [ 10]: Loss 0.03751
-Validation: Loss 0.03130 Accuracy 1.00000
-Validation: Loss 0.03479 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03591
-Epoch [ 11]: Loss 0.03614
-Epoch [ 11]: Loss 0.03522
-Epoch [ 11]: Loss 0.03385
-Epoch [ 11]: Loss 0.03280
-Epoch [ 11]: Loss 0.03028
-Epoch [ 11]: Loss 0.02854
-Validation: Loss 0.02646 Accuracy 1.00000
-Validation: Loss 0.02951 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03022
-Epoch [ 12]: Loss 0.03157
-Epoch [ 12]: Loss 0.02827
-Epoch [ 12]: Loss 0.02888
-Epoch [ 12]: Loss 0.02807
-Epoch [ 12]: Loss 0.02731
-Epoch [ 12]: Loss 0.02753
-Validation: Loss 0.02297 Accuracy 1.00000
-Validation: Loss 0.02568 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02879
-Epoch [ 13]: Loss 0.02518
-Epoch [ 13]: Loss 0.02496
-Epoch [ 13]: Loss 0.02617
-Epoch [ 13]: Loss 0.02322
-Epoch [ 13]: Loss 0.02429
-Epoch [ 13]: Loss 0.02511
-Validation: Loss 0.02027 Accuracy 1.00000
-Validation: Loss 0.02272 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02328
-Epoch [ 14]: Loss 0.02450
-Epoch [ 14]: Loss 0.02156
-Epoch [ 14]: Loss 0.02238
-Epoch [ 14]: Loss 0.02300
-Epoch [ 14]: Loss 0.02170
-Epoch [ 14]: Loss 0.01950
-Validation: Loss 0.01812 Accuracy 1.00000
-Validation: Loss 0.02035 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02289
-Epoch [ 15]: Loss 0.02111
-Epoch [ 15]: Loss 0.01972
-Epoch [ 15]: Loss 0.02013
-Epoch [ 15]: Loss 0.02020
-Epoch [ 15]: Loss 0.01836
-Epoch [ 15]: Loss 0.01899
-Validation: Loss 0.01635 Accuracy 1.00000
-Validation: Loss 0.01839 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01956
-Epoch [ 16]: Loss 0.01711
-Epoch [ 16]: Loss 0.01998
-Epoch [ 16]: Loss 0.01812
-Epoch [ 16]: Loss 0.01900
-Epoch [ 16]: Loss 0.01763
-Epoch [ 16]: Loss 0.01537
-Validation: Loss 0.01484 Accuracy 1.00000
-Validation: Loss 0.01674 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01888
-Epoch [ 17]: Loss 0.01663
-Epoch [ 17]: Loss 0.01522
-Epoch [ 17]: Loss 0.01668
-Epoch [ 17]: Loss 0.01708
-Epoch [ 17]: Loss 0.01657
-Epoch [ 17]: Loss 0.01637
-Validation: Loss 0.01354 Accuracy 1.00000
-Validation: Loss 0.01531 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01558
-Epoch [ 18]: Loss 0.01553
-Epoch [ 18]: Loss 0.01548
-Epoch [ 18]: Loss 0.01486
-Epoch [ 18]: Loss 0.01595
-Epoch [ 18]: Loss 0.01505
-Epoch [ 18]: Loss 0.01557
-Validation: Loss 0.01239 Accuracy 1.00000
-Validation: Loss 0.01405 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01565
-Epoch [ 19]: Loss 0.01480
-Epoch [ 19]: Loss 0.01289
-Epoch [ 19]: Loss 0.01431
-Epoch [ 19]: Loss 0.01376
-Epoch [ 19]: Loss 0.01384
-Epoch [ 19]: Loss 0.01330
-Validation: Loss 0.01137 Accuracy 1.00000
-Validation: Loss 0.01291 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01301
-Epoch [ 20]: Loss 0.01286
-Epoch [ 20]: Loss 0.01358
-Epoch [ 20]: Loss 0.01280
-Epoch [ 20]: Loss 0.01276
-Epoch [ 20]: Loss 0.01273
-Epoch [ 20]: Loss 0.01456
-Validation: Loss 0.01042 Accuracy 1.00000
 Validation: Loss 0.01185 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01172
-Epoch [ 21]: Loss 0.01329
-Epoch [ 21]: Loss 0.01186
-Epoch [ 21]: Loss 0.01169
-Epoch [ 21]: Loss 0.01179
-Epoch [ 21]: Loss 0.01095
-Epoch [ 21]: Loss 0.01284
-Validation: Loss 0.00947 Accuracy 1.00000
-Validation: Loss 0.01075 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01178
-Epoch [ 22]: Loss 0.01089
-Epoch [ 22]: Loss 0.01110
-Epoch [ 22]: Loss 0.01071
-Epoch [ 22]: Loss 0.01036
-Epoch [ 22]: Loss 0.00971
-Epoch [ 22]: Loss 0.01027
-Validation: Loss 0.00845 Accuracy 1.00000
-Validation: Loss 0.00955 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01146
-Epoch [ 23]: Loss 0.00902
-Epoch [ 23]: Loss 0.00849
-Epoch [ 23]: Loss 0.00903
-Epoch [ 23]: Loss 0.00886
-Epoch [ 23]: Loss 0.00972
-Epoch [ 23]: Loss 0.01066
-Validation: Loss 0.00751 Accuracy 1.00000
-Validation: Loss 0.00846 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00976
-Epoch [ 24]: Loss 0.00777
+Epoch [ 22]: Loss 0.01148
+Epoch [ 22]: Loss 0.01199
+Epoch [ 22]: Loss 0.01032
+Epoch [ 22]: Loss 0.01079
+Epoch [ 22]: Loss 0.01012
+Epoch [ 22]: Loss 0.01070
+Epoch [ 22]: Loss 0.01114
+Validation: Loss 0.00957 Accuracy 1.00000
+Validation: Loss 0.01072 Accuracy 1.00000
+Epoch [ 23]: Loss 0.01039
+Epoch [ 23]: Loss 0.00953
+Epoch [ 23]: Loss 0.01061
+Epoch [ 23]: Loss 0.00931
+Epoch [ 23]: Loss 0.00997
+Epoch [ 23]: Loss 0.00961
+Epoch [ 23]: Loss 0.00712
+Validation: Loss 0.00850 Accuracy 1.00000
+Validation: Loss 0.00949 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00893
+Epoch [ 24]: Loss 0.00878
+Epoch [ 24]: Loss 0.00866
+Epoch [ 24]: Loss 0.00924
+Epoch [ 24]: Loss 0.00824
 Epoch [ 24]: Loss 0.00825
-Epoch [ 24]: Loss 0.00881
-Epoch [ 24]: Loss 0.00796
-Epoch [ 24]: Loss 0.00826
-Epoch [ 24]: Loss 0.00787
-Validation: Loss 0.00682 Accuracy 1.00000
-Validation: Loss 0.00765 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00840
-Epoch [ 25]: Loss 0.00781
-Epoch [ 25]: Loss 0.00786
-Epoch [ 25]: Loss 0.00724
-Epoch [ 25]: Loss 0.00762
-Epoch [ 25]: Loss 0.00726
+Epoch [ 24]: Loss 0.00878
+Validation: Loss 0.00763 Accuracy 1.00000
+Validation: Loss 0.00849 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00802
+Epoch [ 25]: Loss 0.00754
+Epoch [ 25]: Loss 0.00771
+Epoch [ 25]: Loss 0.00777
+Epoch [ 25]: Loss 0.00775
+Epoch [ 25]: Loss 0.00812
+Epoch [ 25]: Loss 0.00833
+Validation: Loss 0.00700 Accuracy 1.00000
+Validation: Loss 0.00776 Accuracy 1.00000
    We can also train the compact model with the exact same code!
    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.62243
+Epoch [  1]: Loss 0.61271
+Epoch [  1]: Loss 0.55580
+Epoch [  1]: Loss 0.53213
+Epoch [  1]: Loss 0.52075
+Epoch [  1]: Loss 0.49787
+Epoch [  1]: Loss 0.47505
+Validation: Loss 0.47777 Accuracy 1.00000
+Validation: Loss 0.46300 Accuracy 1.00000
+Epoch [  2]: Loss 0.46806
+Epoch [  2]: Loss 0.44811
+Epoch [  2]: Loss 0.44868
+Epoch [  2]: Loss 0.42514
+Epoch [  2]: Loss 0.41207
+Epoch [  2]: Loss 0.38999
+Epoch [  2]: Loss 0.40689
+Validation: Loss 0.38313 Accuracy 1.00000
+Validation: Loss 0.36497 Accuracy 1.00000
+Epoch [  3]: Loss 0.37506
+Epoch [  3]: Loss 0.35458
+Epoch [  3]: Loss 0.34745
+Epoch [  3]: Loss 0.32494
+Epoch [  3]: Loss 0.31280
+Epoch [  3]: Loss 0.32159
+Epoch [  3]: Loss 0.31559
+Validation: Loss 0.29940 Accuracy 1.00000
+Validation: Loss 0.27966 Accuracy 1.00000
+Epoch [  4]: Loss 0.27458
+Epoch [  4]: Loss 0.28234
+Epoch [  4]: Loss 0.26623
+Epoch [  4]: Loss 0.24585
+Epoch [  4]: Loss 0.24220
+Epoch [  4]: Loss 0.25250
+Epoch [  4]: Loss 0.22705
+Validation: Loss 0.22949 Accuracy 1.00000
+Validation: Loss 0.20991 Accuracy 1.00000
+Epoch [  5]: Loss 0.20868
+Epoch [  5]: Loss 0.20777
+Epoch [  5]: Loss 0.21327
+Epoch [  5]: Loss 0.18019
+Epoch [  5]: Loss 0.18150
+Epoch [  5]: Loss 0.17559
+Epoch [  5]: Loss 0.20257
+Validation: Loss 0.17268 Accuracy 1.00000
+Validation: Loss 0.15497 Accuracy 1.00000
+Epoch [  6]: Loss 0.15827
+Epoch [  6]: Loss 0.15591
+Epoch [  6]: Loss 0.14491
+Epoch [  6]: Loss 0.13292
+Epoch [  6]: Loss 0.14914
+Epoch [  6]: Loss 0.12443
+Epoch [  6]: Loss 0.13808
+Validation: Loss 0.12726 Accuracy 1.00000
+Validation: Loss 0.11293 Accuracy 1.00000
+Epoch [  7]: Loss 0.12103
+Epoch [  7]: Loss 0.11242
+Epoch [  7]: Loss 0.10596
+Epoch [  7]: Loss 0.09875
+Epoch [  7]: Loss 0.09495
+Epoch [  7]: Loss 0.09276
+Epoch [  7]: Loss 0.10884
+Validation: Loss 0.09070 Accuracy 1.00000
+Validation: Loss 0.08028 Accuracy 1.00000
+Epoch [  8]: Loss 0.09406
+Epoch [  8]: Loss 0.08342
+Epoch [  8]: Loss 0.07543
+Epoch [  8]: Loss 0.06123
+Epoch [  8]: Loss 0.06837
+Epoch [  8]: Loss 0.06419
+Epoch [  8]: Loss 0.05732
+Validation: Loss 0.06270 Accuracy 1.00000
+Validation: Loss 0.05587 Accuracy 1.00000
+Epoch [  9]: Loss 0.05821
+Epoch [  9]: Loss 0.05601
+Epoch [  9]: Loss 0.05212
+Epoch [  9]: Loss 0.05552
+Epoch [  9]: Loss 0.04673
+Epoch [  9]: Loss 0.04686
+Epoch [  9]: Loss 0.03952
+Validation: Loss 0.04691 Accuracy 1.00000
+Validation: Loss 0.04198 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04253
+Epoch [ 10]: Loss 0.04115
+Epoch [ 10]: Loss 0.04191
+Epoch [ 10]: Loss 0.04000
+Epoch [ 10]: Loss 0.03686
+Epoch [ 10]: Loss 0.03965
+Epoch [ 10]: Loss 0.03798
+Validation: Loss 0.03820 Accuracy 1.00000
+Validation: Loss 0.03413 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03729
+Epoch [ 11]: Loss 0.03196
+Epoch [ 11]: Loss 0.03405
+Epoch [ 11]: Loss 0.03641
+Epoch [ 11]: Loss 0.03278
+Epoch [ 11]: Loss 0.02875
+Epoch [ 11]: Loss 0.03003
+Validation: Loss 0.03254 Accuracy 1.00000
+Validation: Loss 0.02899 Accuracy 1.00000
+Epoch [ 12]: Loss 0.03042
+Epoch [ 12]: Loss 0.02825
+Epoch [ 12]: Loss 0.02816
+Epoch [ 12]: Loss 0.03091
+Epoch [ 12]: Loss 0.02586
+Epoch [ 12]: Loss 0.02826
+Epoch [ 12]: Loss 0.02898
+Validation: Loss 0.02840 Accuracy 1.00000
+Validation: Loss 0.02524 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02510
+Epoch [ 13]: Loss 0.02628
+Epoch [ 13]: Loss 0.02558
+Epoch [ 13]: Loss 0.02468
+Epoch [ 13]: Loss 0.02447
+Epoch [ 13]: Loss 0.02516
+Epoch [ 13]: Loss 0.02303
+Validation: Loss 0.02517 Accuracy 1.00000
+Validation: Loss 0.02233 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02156
+Epoch [ 14]: Loss 0.02310
+Epoch [ 14]: Loss 0.02294
+Epoch [ 14]: Loss 0.02135
+Epoch [ 14]: Loss 0.02244
+Epoch [ 14]: Loss 0.02250
+Epoch [ 14]: Loss 0.02294
+Validation: Loss 0.02259 Accuracy 1.00000
+Validation: Loss 0.01999 Accuracy 1.00000
+Epoch [ 15]: Loss 0.01831
+Epoch [ 15]: Loss 0.01957
+Epoch [ 15]: Loss 0.01967
+Epoch [ 15]: Loss 0.02107
+Epoch [ 15]: Loss 0.01948
+Epoch [ 15]: Loss 0.02159
+Epoch [ 15]: Loss 0.02296
+Validation: Loss 0.02044 Accuracy 1.00000
+Validation: Loss 0.01805 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01883
+Epoch [ 16]: Loss 0.01874
+Epoch [ 16]: Loss 0.01780
+Epoch [ 16]: Loss 0.01667
+Epoch [ 16]: Loss 0.01866
+Epoch [ 16]: Loss 0.01859
+Epoch [ 16]: Loss 0.01752
+Validation: Loss 0.01859 Accuracy 1.00000
+Validation: Loss 0.01638 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01578
+Epoch [ 17]: Loss 0.01836
+Epoch [ 17]: Loss 0.01503
+Epoch [ 17]: Loss 0.01757
+Epoch [ 17]: Loss 0.01687
+Epoch [ 17]: Loss 0.01556
+Epoch [ 17]: Loss 0.01723
+Validation: Loss 0.01700 Accuracy 1.00000
+Validation: Loss 0.01495 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01577
+Epoch [ 18]: Loss 0.01588
+Epoch [ 18]: Loss 0.01479
+Epoch [ 18]: Loss 0.01476
+Epoch [ 18]: Loss 0.01442
+Epoch [ 18]: Loss 0.01556
+Epoch [ 18]: Loss 0.01408
+Validation: Loss 0.01562 Accuracy 1.00000
+Validation: Loss 0.01370 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01447
+Epoch [ 19]: Loss 0.01459
+Epoch [ 19]: Loss 0.01301
+Epoch [ 19]: Loss 0.01425
+Epoch [ 19]: Loss 0.01311
+Epoch [ 19]: Loss 0.01418
+Epoch [ 19]: Loss 0.01348
+Validation: Loss 0.01438 Accuracy 1.00000
+Validation: Loss 0.01260 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01398
+Epoch [ 20]: Loss 0.01408
+Epoch [ 20]: Loss 0.01339
+Epoch [ 20]: Loss 0.01272
+Epoch [ 20]: Loss 0.01167
+Epoch [ 20]: Loss 0.01151
+Epoch [ 20]: Loss 0.01086
+Validation: Loss 0.01319 Accuracy 1.00000
+Validation: Loss 0.01155 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01245
+Epoch [ 21]: Loss 0.01062
+Epoch [ 21]: Loss 0.01189
+Epoch [ 21]: Loss 0.01221
+Epoch [ 21]: Loss 0.01134
+Epoch [ 21]: Loss 0.01185
+Epoch [ 21]: Loss 0.01080
+Validation: Loss 0.01194 Accuracy 1.00000
+Validation: Loss 0.01047 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01159
+Epoch [ 22]: Loss 0.01187
+Epoch [ 22]: Loss 0.01073
+Epoch [ 22]: Loss 0.00970
+Epoch [ 22]: Loss 0.01003
+Epoch [ 22]: Loss 0.00966
+Epoch [ 22]: Loss 0.00883
+Validation: Loss 0.01057 Accuracy 1.00000
+Validation: Loss 0.00929 Accuracy 1.00000
+Epoch [ 23]: Loss 0.00981
+Epoch [ 23]: Loss 0.01000
+Epoch [ 23]: Loss 0.00948
+Epoch [ 23]: Loss 0.00895
+Epoch [ 23]: Loss 0.00924
+Epoch [ 23]: Loss 0.00850
+Epoch [ 23]: Loss 0.00887
+Validation: Loss 0.00937 Accuracy 1.00000
+Validation: Loss 0.00828 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00856
+Epoch [ 24]: Loss 0.00821
+Epoch [ 24]: Loss 0.00878
+Epoch [ 24]: Loss 0.00829
+Epoch [ 24]: Loss 0.00788
+Epoch [ 24]: Loss 0.00851
+Epoch [ 24]: Loss 0.00775
+Validation: Loss 0.00852 Accuracy 1.00000
+Validation: Loss 0.00755 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00766
+Epoch [ 25]: Loss 0.00806
+Epoch [ 25]: Loss 0.00790
 Epoch [ 25]: Loss 0.00775
-Validation: Loss 0.00631 Accuracy 1.00000
-Validation: Loss 0.00707 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
+Epoch [ 25]: Loss 0.00713
+Epoch [ 25]: Loss 0.00745
+Epoch [ 25]: Loss 0.00773
+Validation: Loss 0.00790 Accuracy 1.00000
+Validation: Loss 0.00701 Accuracy 1.00000
    Saving the Model ​
    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.
    julia
    @save "trained_model.jld2" ps_trained st_trained
    Let's try loading the model
    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
  :ps_trained
  :st_trained
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
diff --git a/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.CL_BpnYq.js b/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.4OROSnOm.js
similarity index 93%
rename from previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.CL_BpnYq.js
rename to previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.4OROSnOm.js
index 35c4028386..d7f2084a28 100644
--- a/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.CL_BpnYq.js
+++ b/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.4OROSnOm.js
@@ -90,25 +90,25 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g
     end
 
     return tr_acc, te_acc
-end
    train (generic function with 1 method)
    Finally Training the Model ​
    First we will train the Lux model
    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 105.39s 	 Training Accuracy: 22.72% 	 Test Accuracy: 19.00%
-[ 2/10] 	 Time 97.47s 	 Training Accuracy: 47.39% 	 Test Accuracy: 46.00%
-[ 3/10] 	 Time 99.44s 	 Training Accuracy: 60.33% 	 Test Accuracy: 61.00%
-[ 4/10] 	 Time 99.45s 	 Training Accuracy: 69.56% 	 Test Accuracy: 65.50%
-[ 5/10] 	 Time 107.39s 	 Training Accuracy: 74.89% 	 Test Accuracy: 71.50%
-[ 6/10] 	 Time 104.03s 	 Training Accuracy: 78.22% 	 Test Accuracy: 73.50%
-[ 7/10] 	 Time 106.84s 	 Training Accuracy: 80.00% 	 Test Accuracy: 76.50%
-[ 8/10] 	 Time 104.29s 	 Training Accuracy: 83.00% 	 Test Accuracy: 80.00%
-[ 9/10] 	 Time 102.81s 	 Training Accuracy: 84.78% 	 Test Accuracy: 81.00%
-[10/10] 	 Time 108.25s 	 Training Accuracy: 86.22% 	 Test Accuracy: 83.50%
    Now we will train the SimpleChains model
    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.76s 	 Training Accuracy: 32.89% 	 Test Accuracy: 29.00%
-[ 2/10] 	 Time 17.69s 	 Training Accuracy: 44.83% 	 Test Accuracy: 42.00%
-[ 3/10] 	 Time 17.70s 	 Training Accuracy: 58.11% 	 Test Accuracy: 53.00%
-[ 4/10] 	 Time 17.70s 	 Training Accuracy: 65.56% 	 Test Accuracy: 62.50%
-[ 5/10] 	 Time 17.70s 	 Training Accuracy: 73.06% 	 Test Accuracy: 67.00%
-[ 6/10] 	 Time 17.70s 	 Training Accuracy: 78.50% 	 Test Accuracy: 76.50%
-[ 7/10] 	 Time 17.71s 	 Training Accuracy: 82.17% 	 Test Accuracy: 79.00%
-[ 8/10] 	 Time 17.70s 	 Training Accuracy: 83.44% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 17.70s 	 Training Accuracy: 85.89% 	 Test Accuracy: 85.00%
-[10/10] 	 Time 17.70s 	 Training Accuracy: 86.94% 	 Test Accuracy: 83.50%
    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.
    Appendix ​
    julia
    using InteractiveUtils
+end
    train (generic function with 1 method)
    Finally Training the Model ​
    First we will train the Lux model
    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 49.05s 	 Training Accuracy: 21.61% 	 Test Accuracy: 16.00%
+[ 2/10] 	 Time 37.82s 	 Training Accuracy: 46.17% 	 Test Accuracy: 43.50%
+[ 3/10] 	 Time 45.59s 	 Training Accuracy: 59.50% 	 Test Accuracy: 60.00%
+[ 4/10] 	 Time 60.41s 	 Training Accuracy: 69.22% 	 Test Accuracy: 65.50%
+[ 5/10] 	 Time 77.59s 	 Training Accuracy: 74.11% 	 Test Accuracy: 71.00%
+[ 6/10] 	 Time 78.28s 	 Training Accuracy: 78.11% 	 Test Accuracy: 75.00%
+[ 7/10] 	 Time 73.89s 	 Training Accuracy: 80.22% 	 Test Accuracy: 79.50%
+[ 8/10] 	 Time 82.82s 	 Training Accuracy: 82.39% 	 Test Accuracy: 79.50%
+[ 9/10] 	 Time 64.69s 	 Training Accuracy: 85.06% 	 Test Accuracy: 83.50%
+[10/10] 	 Time 72.24s 	 Training Accuracy: 86.56% 	 Test Accuracy: 83.50%
    Now we will train the SimpleChains model
    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.81s 	 Training Accuracy: 30.56% 	 Test Accuracy: 26.00%
+[ 2/10] 	 Time 17.57s 	 Training Accuracy: 47.06% 	 Test Accuracy: 45.00%
+[ 3/10] 	 Time 17.58s 	 Training Accuracy: 59.72% 	 Test Accuracy: 51.50%
+[ 4/10] 	 Time 17.54s 	 Training Accuracy: 67.61% 	 Test Accuracy: 64.00%
+[ 5/10] 	 Time 17.53s 	 Training Accuracy: 75.22% 	 Test Accuracy: 72.50%
+[ 6/10] 	 Time 17.54s 	 Training Accuracy: 80.22% 	 Test Accuracy: 73.50%
+[ 7/10] 	 Time 17.56s 	 Training Accuracy: 81.17% 	 Test Accuracy: 81.00%
+[ 8/10] 	 Time 17.51s 	 Training Accuracy: 85.61% 	 Test Accuracy: 83.50%
+[ 9/10] 	 Time 17.51s 	 Training Accuracy: 86.33% 	 Test Accuracy: 83.50%
+[10/10] 	 Time 17.51s 	 Training Accuracy: 87.33% 	 Test Accuracy: 85.50%
    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.CL_BpnYq.lean.js b/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.4OROSnOm.lean.js
similarity index 93%
rename from previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.CL_BpnYq.lean.js
rename to previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.4OROSnOm.lean.js
index 35c4028386..d7f2084a28 100644
--- a/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.CL_BpnYq.lean.js
+++ b/previews/PR1000/assets/tutorials_beginner_4_SimpleChains.md.4OROSnOm.lean.js
@@ -90,25 +90,25 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const g
     end
 
     return tr_acc, te_acc
-end
    train (generic function with 1 method)
    Finally Training the Model ​
    First we will train the Lux model
    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 105.39s 	 Training Accuracy: 22.72% 	 Test Accuracy: 19.00%
-[ 2/10] 	 Time 97.47s 	 Training Accuracy: 47.39% 	 Test Accuracy: 46.00%
-[ 3/10] 	 Time 99.44s 	 Training Accuracy: 60.33% 	 Test Accuracy: 61.00%
-[ 4/10] 	 Time 99.45s 	 Training Accuracy: 69.56% 	 Test Accuracy: 65.50%
-[ 5/10] 	 Time 107.39s 	 Training Accuracy: 74.89% 	 Test Accuracy: 71.50%
-[ 6/10] 	 Time 104.03s 	 Training Accuracy: 78.22% 	 Test Accuracy: 73.50%
-[ 7/10] 	 Time 106.84s 	 Training Accuracy: 80.00% 	 Test Accuracy: 76.50%
-[ 8/10] 	 Time 104.29s 	 Training Accuracy: 83.00% 	 Test Accuracy: 80.00%
-[ 9/10] 	 Time 102.81s 	 Training Accuracy: 84.78% 	 Test Accuracy: 81.00%
-[10/10] 	 Time 108.25s 	 Training Accuracy: 86.22% 	 Test Accuracy: 83.50%
    Now we will train the SimpleChains model
    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.76s 	 Training Accuracy: 32.89% 	 Test Accuracy: 29.00%
-[ 2/10] 	 Time 17.69s 	 Training Accuracy: 44.83% 	 Test Accuracy: 42.00%
-[ 3/10] 	 Time 17.70s 	 Training Accuracy: 58.11% 	 Test Accuracy: 53.00%
-[ 4/10] 	 Time 17.70s 	 Training Accuracy: 65.56% 	 Test Accuracy: 62.50%
-[ 5/10] 	 Time 17.70s 	 Training Accuracy: 73.06% 	 Test Accuracy: 67.00%
-[ 6/10] 	 Time 17.70s 	 Training Accuracy: 78.50% 	 Test Accuracy: 76.50%
-[ 7/10] 	 Time 17.71s 	 Training Accuracy: 82.17% 	 Test Accuracy: 79.00%
-[ 8/10] 	 Time 17.70s 	 Training Accuracy: 83.44% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 17.70s 	 Training Accuracy: 85.89% 	 Test Accuracy: 85.00%
-[10/10] 	 Time 17.70s 	 Training Accuracy: 86.94% 	 Test Accuracy: 83.50%
    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.
    Appendix ​
    julia
    using InteractiveUtils
+end
    train (generic function with 1 method)
    Finally Training the Model ​
    First we will train the Lux model
    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 49.05s 	 Training Accuracy: 21.61% 	 Test Accuracy: 16.00%
+[ 2/10] 	 Time 37.82s 	 Training Accuracy: 46.17% 	 Test Accuracy: 43.50%
+[ 3/10] 	 Time 45.59s 	 Training Accuracy: 59.50% 	 Test Accuracy: 60.00%
+[ 4/10] 	 Time 60.41s 	 Training Accuracy: 69.22% 	 Test Accuracy: 65.50%
+[ 5/10] 	 Time 77.59s 	 Training Accuracy: 74.11% 	 Test Accuracy: 71.00%
+[ 6/10] 	 Time 78.28s 	 Training Accuracy: 78.11% 	 Test Accuracy: 75.00%
+[ 7/10] 	 Time 73.89s 	 Training Accuracy: 80.22% 	 Test Accuracy: 79.50%
+[ 8/10] 	 Time 82.82s 	 Training Accuracy: 82.39% 	 Test Accuracy: 79.50%
+[ 9/10] 	 Time 64.69s 	 Training Accuracy: 85.06% 	 Test Accuracy: 83.50%
+[10/10] 	 Time 72.24s 	 Training Accuracy: 86.56% 	 Test Accuracy: 83.50%
    Now we will train the SimpleChains model
    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.81s 	 Training Accuracy: 30.56% 	 Test Accuracy: 26.00%
+[ 2/10] 	 Time 17.57s 	 Training Accuracy: 47.06% 	 Test Accuracy: 45.00%
+[ 3/10] 	 Time 17.58s 	 Training Accuracy: 59.72% 	 Test Accuracy: 51.50%
+[ 4/10] 	 Time 17.54s 	 Training Accuracy: 67.61% 	 Test Accuracy: 64.00%
+[ 5/10] 	 Time 17.53s 	 Training Accuracy: 75.22% 	 Test Accuracy: 72.50%
+[ 6/10] 	 Time 17.54s 	 Training Accuracy: 80.22% 	 Test Accuracy: 73.50%
+[ 7/10] 	 Time 17.56s 	 Training Accuracy: 81.17% 	 Test Accuracy: 81.00%
+[ 8/10] 	 Time 17.51s 	 Training Accuracy: 85.61% 	 Test Accuracy: 83.50%
+[ 9/10] 	 Time 17.51s 	 Training Accuracy: 86.33% 	 Test Accuracy: 83.50%
+[10/10] 	 Time 17.51s 	 Training Accuracy: 87.33% 	 Test Accuracy: 85.50%
    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.DqNEeGSX.js b/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.BdxKZnAQ.js
similarity index 60%
rename from previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.DqNEeGSX.js
rename to previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.BdxKZnAQ.js
index 682b28cf61..ad40ecb315 100644
--- a/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.DqNEeGSX.js
+++ b/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.BdxKZnAQ.js
@@ -80,31 +80,31 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.DjZDIZsN.js";const d
     return StatefulLuxLayer{true}(model, res.u, smodel.st)
 end
 
-trained_model = train_model(dataloader)
    Iteration:     1, Loss: 1.133634e-01
-Iteration:    26, Loss: 1.063336e-01
-Iteration:    51, Loss: 2.078427e-01
-Iteration:    76, Loss: 2.046876e-01
-Iteration:     1, Loss: 1.987609e-01
-Iteration:     1, Loss: 2.268675e-02
-Iteration:    26, Loss: 3.252327e-02
-Iteration:    51, Loss: 2.324151e-02
-Iteration:    76, Loss: 2.137157e-02
-Iteration:   101, Loss: 1.924721e-02
-Iteration:   126, Loss: 1.698606e-02
-Iteration:   151, Loss: 1.479297e-02
-Iteration:   176, Loss: 1.256829e-02
-Iteration:   201, Loss: 1.779426e-02
-Iteration:   226, Loss: 1.126407e-02
-Iteration:   251, Loss: 9.760977e-03
-Iteration:   276, Loss: 8.756177e-03
-Iteration:   301, Loss: 1.514340e-02
-Iteration:   326, Loss: 9.525211e-03
-Iteration:   351, Loss: 8.062281e-03
-Iteration:   376, Loss: 7.160148e-03
-Iteration:   401, Loss: 6.379412e-03
-Iteration:   426, Loss: 1.824650e-02
-Iteration:   451, Loss: 8.544287e-03
-Iteration:   476, Loss: 6.913106e-03
    Plotting the results ​
    julia
    dudt(u, p, t) = trained_model(u, p)
+trained_model = train_model(dataloader)
    Iteration:     1, Loss: 1.894389e-01
+Iteration:    26, Loss: 2.794352e-02
+Iteration:    51, Loss: 1.110121e-01
+Iteration:    76, Loss: 2.543880e-01
+Iteration:     1, Loss: 3.234872e-01
+Iteration:     1, Loss: 2.432619e-02
+Iteration:    26, Loss: 3.241556e-02
+Iteration:    51, Loss: 2.396435e-02
+Iteration:    76, Loss: 2.210484e-02
+Iteration:   101, Loss: 2.054718e-02
+Iteration:   126, Loss: 1.922303e-02
+Iteration:   151, Loss: 1.797500e-02
+Iteration:   176, Loss: 1.671544e-02
+Iteration:   201, Loss: 1.529878e-02
+Iteration:   226, Loss: 1.391988e-02
+Iteration:   251, Loss: 1.273907e-02
+Iteration:   276, Loss: 1.162500e-02
+Iteration:   301, Loss: 2.115832e-02
+Iteration:   326, Loss: 1.171498e-02
+Iteration:   351, Loss: 9.864081e-03
+Iteration:   376, Loss: 8.805853e-03
+Iteration:   401, Loss: 7.982390e-03
+Iteration:   426, Loss: 1.730291e-02
+Iteration:   451, Loss: 1.051912e-02
+Iteration:   476, Loss: 8.427156e-03
    Plotting the results ​
    julia
    dudt(u, p, t) = trained_model(u, p)
 prob = ODEProblem(dudt, gdev(u0), (tspan[1], tspan[2]), trained_model.ps)
 sol = solve(prob, Tsit5(); saveat=t)
 pred = convert(AbstractArray, sol) |> cdev
@@ -118,7 +118,7 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.DjZDIZsN.js";const d
     lines!(ax, t, pred[2, :]; label=L"\\hat{u}_2(t)", color=:red, linewidth=4)
     axislegend(ax; position=:lt)
     fig
-end
    Appendix ​
    julia
    using InteractiveUtils
+end
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.DqNEeGSX.lean.js b/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.BdxKZnAQ.lean.js
similarity index 60%
rename from previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.DqNEeGSX.lean.js
rename to previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.BdxKZnAQ.lean.js
index 682b28cf61..ad40ecb315 100644
--- a/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.DqNEeGSX.lean.js
+++ b/previews/PR1000/assets/tutorials_beginner_5_OptimizationIntegration.md.BdxKZnAQ.lean.js
@@ -80,31 +80,31 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.DjZDIZsN.js";const d
     return StatefulLuxLayer{true}(model, res.u, smodel.st)
 end
 
-trained_model = train_model(dataloader)
    Iteration:     1, Loss: 1.133634e-01
-Iteration:    26, Loss: 1.063336e-01
-Iteration:    51, Loss: 2.078427e-01
-Iteration:    76, Loss: 2.046876e-01
-Iteration:     1, Loss: 1.987609e-01
-Iteration:     1, Loss: 2.268675e-02
-Iteration:    26, Loss: 3.252327e-02
-Iteration:    51, Loss: 2.324151e-02
-Iteration:    76, Loss: 2.137157e-02
-Iteration:   101, Loss: 1.924721e-02
-Iteration:   126, Loss: 1.698606e-02
-Iteration:   151, Loss: 1.479297e-02
-Iteration:   176, Loss: 1.256829e-02
-Iteration:   201, Loss: 1.779426e-02
-Iteration:   226, Loss: 1.126407e-02
-Iteration:   251, Loss: 9.760977e-03
-Iteration:   276, Loss: 8.756177e-03
-Iteration:   301, Loss: 1.514340e-02
-Iteration:   326, Loss: 9.525211e-03
-Iteration:   351, Loss: 8.062281e-03
-Iteration:   376, Loss: 7.160148e-03
-Iteration:   401, Loss: 6.379412e-03
-Iteration:   426, Loss: 1.824650e-02
-Iteration:   451, Loss: 8.544287e-03
-Iteration:   476, Loss: 6.913106e-03
    Plotting the results ​
    julia
    dudt(u, p, t) = trained_model(u, p)
+trained_model = train_model(dataloader)
    Iteration:     1, Loss: 1.894389e-01
+Iteration:    26, Loss: 2.794352e-02
+Iteration:    51, Loss: 1.110121e-01
+Iteration:    76, Loss: 2.543880e-01
+Iteration:     1, Loss: 3.234872e-01
+Iteration:     1, Loss: 2.432619e-02
+Iteration:    26, Loss: 3.241556e-02
+Iteration:    51, Loss: 2.396435e-02
+Iteration:    76, Loss: 2.210484e-02
+Iteration:   101, Loss: 2.054718e-02
+Iteration:   126, Loss: 1.922303e-02
+Iteration:   151, Loss: 1.797500e-02
+Iteration:   176, Loss: 1.671544e-02
+Iteration:   201, Loss: 1.529878e-02
+Iteration:   226, Loss: 1.391988e-02
+Iteration:   251, Loss: 1.273907e-02
+Iteration:   276, Loss: 1.162500e-02
+Iteration:   301, Loss: 2.115832e-02
+Iteration:   326, Loss: 1.171498e-02
+Iteration:   351, Loss: 9.864081e-03
+Iteration:   376, Loss: 8.805853e-03
+Iteration:   401, Loss: 7.982390e-03
+Iteration:   426, Loss: 1.730291e-02
+Iteration:   451, Loss: 1.051912e-02
+Iteration:   476, Loss: 8.427156e-03
    Plotting the results ​
    julia
    dudt(u, p, t) = trained_model(u, p)
 prob = ODEProblem(dudt, gdev(u0), (tspan[1], tspan[2]), trained_model.ps)
 sol = solve(prob, Tsit5(); saveat=t)
 pred = convert(AbstractArray, sol) |> cdev
@@ -118,7 +118,7 @@ import{_ as s,c as i,a2 as a,o as n}from"./chunks/framework.DjZDIZsN.js";const d
     lines!(ax, t, pred[2, :]; label=L"\\hat{u}_2(t)", color=:red, linewidth=4)
     axislegend(ax; position=:lt)
     fig
-end
    Appendix ​
    julia
    using InteractiveUtils
+end
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.ByrCK2Fs.js b/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.BxeK2d6-.js
similarity index 95%
rename from previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.ByrCK2Fs.js
rename to previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.BxeK2d6-.js
index 6187a34c8e..6bc34ca756 100644
--- a/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.ByrCK2Fs.js
+++ b/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.BxeK2d6-.js
@@ -107,47 +107,47 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
     end
 end
 
-train(NeuralODECompact)
    [1/9]	Time 118.1406s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.7178s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5016s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
-[4/9]	Time 0.6758s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
-[5/9]	Time 0.4877s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
-[6/9]	Time 0.6598s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
-[7/9]	Time 0.4953s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
-[8/9]	Time 0.4947s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
-[9/9]	Time 0.7741s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 35.6246s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.5096s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5355s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
-[4/9]	Time 0.8174s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
-[5/9]	Time 0.5219s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
-[6/9]	Time 0.5240s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
-[7/9]	Time 0.8928s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
-[8/9]	Time 0.5157s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
-[9/9]	Time 0.5063s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%
    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).
    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 44.6107s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4743s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
-[3/9]	Time 0.5032s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
-[4/9]	Time 0.7259s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
-[5/9]	Time 0.5070s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
-[6/9]	Time 0.5044s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
-[7/9]	Time 0.7081s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
-[8/9]	Time 0.4944s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
-[9/9]	Time 0.5164s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%
    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.
    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 102.5954s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 21.1668s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
-[3/9]	Time 19.6983s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
-[4/9]	Time 22.0427s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
-[5/9]	Time 16.8381s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
-[6/9]	Time 20.4067s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 13.9161s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
-[8/9]	Time 14.8513s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
-[9/9]	Time 14.7005s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%
    For completeness, let's also test out discrete sensitivities!
    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 51.9802s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 27.0563s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
-[3/9]	Time 30.0257s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
-[4/9]	Time 32.3402s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
-[5/9]	Time 28.1834s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
-[6/9]	Time 36.2859s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 32.9532s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
-[8/9]	Time 27.0422s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
-[9/9]	Time 27.9113s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%
    Alternate Implementation using Stateful Layer ​
    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.
    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
+train(NeuralODECompact)
    [1/9]	Time 136.7023s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5255s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
+[3/9]	Time 0.7318s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
+[4/9]	Time 0.5021s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
+[5/9]	Time 0.5467s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
+[6/9]	Time 0.8045s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
+[7/9]	Time 0.5118s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
+[8/9]	Time 0.5154s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
+[9/9]	Time 0.9002s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 38.9844s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5079s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
+[3/9]	Time 0.5245s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
+[4/9]	Time 0.7894s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
+[5/9]	Time 0.5193s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
+[6/9]	Time 0.5215s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
+[7/9]	Time 0.7914s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
+[8/9]	Time 0.5144s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
+[9/9]	Time 0.5315s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%
    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).
    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 44.4626s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5993s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
+[3/9]	Time 0.5022s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
+[4/9]	Time 0.5087s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
+[5/9]	Time 0.7353s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
+[6/9]	Time 0.4898s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
+[7/9]	Time 0.4922s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
+[8/9]	Time 0.4996s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
+[9/9]	Time 0.4869s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%
    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.
    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 101.8574s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 13.8755s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
+[3/9]	Time 10.5309s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
+[4/9]	Time 10.5302s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
+[5/9]	Time 12.7074s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
+[6/9]	Time 14.3102s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 9.5009s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
+[8/9]	Time 14.1295s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
+[9/9]	Time 7.7721s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%
    For completeness, let's also test out discrete sensitivities!
    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 43.2785s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 24.6150s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
+[3/9]	Time 27.3617s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
+[4/9]	Time 20.8862s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
+[5/9]	Time 23.0043s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
+[6/9]	Time 20.1717s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 22.9598s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
+[8/9]	Time 19.8321s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
+[9/9]	Time 18.7652s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%
    Alternate Implementation using Stateful Layer ​
    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.
    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
        Lux.AbstractLuxWrapperLayer{:model}
     model::M
     solver::So
@@ -165,20 +165,20 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
     dudt(u, p, t) = st_model(u, p)
     prob = ODEProblem{false}(ODEFunction{false}(dudt), x, n.tspan, ps)
     return solve(prob, n.solver; n.kwargs...), st_model.st
-end
    Train the new Stateful Neural ODE ​
    julia
    train(StatefulNeuralODE)
    [1/9]	Time 40.6684s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4853s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
-[3/9]	Time 0.5115s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
-[4/9]	Time 0.5330s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
-[5/9]	Time 0.4983s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
-[6/9]	Time 0.4850s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
-[7/9]	Time 0.5257s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
-[8/9]	Time 0.6022s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
-[9/9]	Time 0.8892s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%
    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.
    Type Stability ​
    julia
    model, ps, st = create_model(NeuralODE)
+end
    Train the new Stateful Neural ODE ​
    julia
    train(StatefulNeuralODE)
    [1/9]	Time 40.9347s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4784s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
+[3/9]	Time 0.7823s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
+[4/9]	Time 0.4966s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
+[5/9]	Time 0.4866s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
+[6/9]	Time 0.5181s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
+[7/9]	Time 0.8805s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
+[8/9]	Time 0.4999s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4981s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%
    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.
    Type Stability ​
    julia
    model, ps, st = create_model(NeuralODE)
 
 model_stateful, ps_stateful, st_stateful = create_model(StatefulNeuralODE)
 
 x = gpu_device()(ones(Float32, 28, 28, 1, 3));
    NeuralODE is not type stable due to the boxing of st
    julia
    @code_warntype model(x, ps, st)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -188,7 +188,7 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
 1 ─ %1 = Base.getproperty(c, :layers)::@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}
 │   %2 = Lux.applychain(%1, x, ps, st)::TUPLE{CUDA.CUARRAY{FLOAT32, 2, CUDA.DEVICEMEMORY}, NAMEDTUPLE{(:LAYER_1, :LAYER_2, :LAYER_3, :LAYER_4, :LAYER_5), <:TUPLE{@NAMEDTUPLE{}, @NAMEDTUPLE{}, ANY, @NAMEDTUPLE{}, @NAMEDTUPLE{}}}}
 └──      return %2
    We avoid the problem entirely by using StatefulNeuralODE
    julia
    @code_warntype model_stateful(x, ps_stateful, st_stateful)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".StatefulNeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".StatefulNeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -200,7 +200,7 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
 └──      return %2
    Note, that we still recommend using this layer internally and not exposing this as the default API to the users.
    Finally checking the compact model
    julia
    model_compact, ps_compact, st_compact = create_model(NeuralODECompact)
 
 @code_warntype model_compact(x, ps_compact, st_compact)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.CompactLuxLayer{:₋₋₋no_special_dispatch₋₋₋, Main.var"##225".var"#2#3", Nothing, @NamedTuple{model::Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}}, Lux.CompactMacroImpl.ValueStorage{@NamedTuple{}, @NamedTuple{solver::Returns{OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}}, tspan::Returns{Tuple{Float32, Float32}}}}, Tuple{Tuple{Symbol}, Tuple{Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(model = ViewAxis(1:540, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))),)), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{model::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, solver::OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, tspan::Tuple{Float32, Float32}, ₋₋₋kwargs₋₋₋::@NamedTuple{kwargs::Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.CompactLuxLayer{:₋₋₋no_special_dispatch₋₋₋, Main.var"##225".var"#2#3", Nothing, @NamedTuple{model::Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}}, Lux.CompactMacroImpl.ValueStorage{@NamedTuple{}, @NamedTuple{solver::Returns{OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}}, tspan::Returns{Tuple{Float32, Float32}}}}, Tuple{Tuple{Symbol}, Tuple{Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -269,4 +269,4 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.889 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default};
+  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.920 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default};
diff --git a/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.ByrCK2Fs.lean.js b/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.BxeK2d6-.lean.js
similarity index 95%
rename from previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.ByrCK2Fs.lean.js
rename to previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.BxeK2d6-.lean.js
index 6187a34c8e..6bc34ca756 100644
--- a/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.ByrCK2Fs.lean.js
+++ b/previews/PR1000/assets/tutorials_intermediate_1_NeuralODE.md.BxeK2d6-.lean.js
@@ -107,47 +107,47 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
     end
 end
 
-train(NeuralODECompact)
    [1/9]	Time 118.1406s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.7178s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5016s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
-[4/9]	Time 0.6758s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
-[5/9]	Time 0.4877s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
-[6/9]	Time 0.6598s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
-[7/9]	Time 0.4953s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
-[8/9]	Time 0.4947s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
-[9/9]	Time 0.7741s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 35.6246s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.5096s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5355s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
-[4/9]	Time 0.8174s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
-[5/9]	Time 0.5219s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
-[6/9]	Time 0.5240s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
-[7/9]	Time 0.8928s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
-[8/9]	Time 0.5157s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
-[9/9]	Time 0.5063s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%
    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).
    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 44.6107s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4743s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
-[3/9]	Time 0.5032s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
-[4/9]	Time 0.7259s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
-[5/9]	Time 0.5070s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
-[6/9]	Time 0.5044s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
-[7/9]	Time 0.7081s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
-[8/9]	Time 0.4944s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
-[9/9]	Time 0.5164s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%
    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.
    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 102.5954s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 21.1668s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
-[3/9]	Time 19.6983s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
-[4/9]	Time 22.0427s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
-[5/9]	Time 16.8381s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
-[6/9]	Time 20.4067s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 13.9161s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
-[8/9]	Time 14.8513s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
-[9/9]	Time 14.7005s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%
    For completeness, let's also test out discrete sensitivities!
    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 51.9802s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 27.0563s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
-[3/9]	Time 30.0257s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
-[4/9]	Time 32.3402s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
-[5/9]	Time 28.1834s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
-[6/9]	Time 36.2859s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 32.9532s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
-[8/9]	Time 27.0422s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
-[9/9]	Time 27.9113s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%
    Alternate Implementation using Stateful Layer ​
    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.
    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
+train(NeuralODECompact)
    [1/9]	Time 136.7023s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5255s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
+[3/9]	Time 0.7318s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
+[4/9]	Time 0.5021s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
+[5/9]	Time 0.5467s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
+[6/9]	Time 0.8045s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
+[7/9]	Time 0.5118s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
+[8/9]	Time 0.5154s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
+[9/9]	Time 0.9002s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 38.9844s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5079s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
+[3/9]	Time 0.5245s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
+[4/9]	Time 0.7894s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
+[5/9]	Time 0.5193s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
+[6/9]	Time 0.5215s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
+[7/9]	Time 0.7914s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
+[8/9]	Time 0.5144s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
+[9/9]	Time 0.5315s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%
    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).
    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 44.4626s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5993s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
+[3/9]	Time 0.5022s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
+[4/9]	Time 0.5087s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
+[5/9]	Time 0.7353s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
+[6/9]	Time 0.4898s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
+[7/9]	Time 0.4922s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
+[8/9]	Time 0.4996s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
+[9/9]	Time 0.4869s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%
    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.
    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 101.8574s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 13.8755s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
+[3/9]	Time 10.5309s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
+[4/9]	Time 10.5302s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
+[5/9]	Time 12.7074s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
+[6/9]	Time 14.3102s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 9.5009s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
+[8/9]	Time 14.1295s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
+[9/9]	Time 7.7721s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%
    For completeness, let's also test out discrete sensitivities!
    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 43.2785s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 24.6150s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
+[3/9]	Time 27.3617s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
+[4/9]	Time 20.8862s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
+[5/9]	Time 23.0043s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
+[6/9]	Time 20.1717s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 22.9598s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
+[8/9]	Time 19.8321s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
+[9/9]	Time 18.7652s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%
    Alternate Implementation using Stateful Layer ​
    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.
    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
        Lux.AbstractLuxWrapperLayer{:model}
     model::M
     solver::So
@@ -165,20 +165,20 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
     dudt(u, p, t) = st_model(u, p)
     prob = ODEProblem{false}(ODEFunction{false}(dudt), x, n.tspan, ps)
     return solve(prob, n.solver; n.kwargs...), st_model.st
-end
    Train the new Stateful Neural ODE ​
    julia
    train(StatefulNeuralODE)
    [1/9]	Time 40.6684s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4853s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
-[3/9]	Time 0.5115s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
-[4/9]	Time 0.5330s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
-[5/9]	Time 0.4983s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
-[6/9]	Time 0.4850s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
-[7/9]	Time 0.5257s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
-[8/9]	Time 0.6022s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
-[9/9]	Time 0.8892s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%
    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.
    Type Stability ​
    julia
    model, ps, st = create_model(NeuralODE)
+end
    Train the new Stateful Neural ODE ​
    julia
    train(StatefulNeuralODE)
    [1/9]	Time 40.9347s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4784s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
+[3/9]	Time 0.7823s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
+[4/9]	Time 0.4966s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
+[5/9]	Time 0.4866s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
+[6/9]	Time 0.5181s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
+[7/9]	Time 0.8805s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
+[8/9]	Time 0.4999s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4981s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%
    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.
    Type Stability ​
    julia
    model, ps, st = create_model(NeuralODE)
 
 model_stateful, ps_stateful, st_stateful = create_model(StatefulNeuralODE)
 
 x = gpu_device()(ones(Float32, 28, 28, 1, 3));
    NeuralODE is not type stable due to the boxing of st
    julia
    @code_warntype model(x, ps, st)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -188,7 +188,7 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
 1 ─ %1 = Base.getproperty(c, :layers)::@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}
 │   %2 = Lux.applychain(%1, x, ps, st)::TUPLE{CUDA.CUARRAY{FLOAT32, 2, CUDA.DEVICEMEMORY}, NAMEDTUPLE{(:LAYER_1, :LAYER_2, :LAYER_3, :LAYER_4, :LAYER_5), <:TUPLE{@NAMEDTUPLE{}, @NAMEDTUPLE{}, ANY, @NAMEDTUPLE{}, @NAMEDTUPLE{}}}}
 └──      return %2
    We avoid the problem entirely by using StatefulNeuralODE
    julia
    @code_warntype model_stateful(x, ps_stateful, st_stateful)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".StatefulNeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".StatefulNeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -200,7 +200,7 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
 └──      return %2
    Note, that we still recommend using this layer internally and not exposing this as the default API to the users.
    Finally checking the compact model
    julia
    model_compact, ps_compact, st_compact = create_model(NeuralODECompact)
 
 @code_warntype model_compact(x, ps_compact, st_compact)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.CompactLuxLayer{:₋₋₋no_special_dispatch₋₋₋, Main.var"##225".var"#2#3", Nothing, @NamedTuple{model::Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}}, Lux.CompactMacroImpl.ValueStorage{@NamedTuple{}, @NamedTuple{solver::Returns{OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}}, tspan::Returns{Tuple{Float32, Float32}}}}, Tuple{Tuple{Symbol}, Tuple{Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(model = ViewAxis(1:540, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))),)), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{model::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, solver::OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, tspan::Tuple{Float32, Float32}, ₋₋₋kwargs₋₋₋::@NamedTuple{kwargs::Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.CompactLuxLayer{:₋₋₋no_special_dispatch₋₋₋, Main.var"##225".var"#2#3", Nothing, @NamedTuple{model::Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}}, Lux.CompactMacroImpl.ValueStorage{@NamedTuple{}, @NamedTuple{solver::Returns{OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}}, tspan::Returns{Tuple{Float32, Float32}}}}, Tuple{Tuple{Symbol}, Tuple{Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -269,4 +269,4 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const E
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.889 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default};
+  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.920 GiB / 4.750 GiB available)
    This page was generated using Literate.jl.
    `,62)]))}const o=i(e,[["render",l]]);export{E as __pageData,o as default};
diff --git a/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.DS1q_HRo.js b/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.BKUcq4eS.js
similarity index 99%
rename from previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.DS1q_HRo.js
rename to previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.BKUcq4eS.js
index d1ad5939ba..e8cff3ff7a 100644
--- a/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.DS1q_HRo.js
+++ b/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.BKUcq4eS.js
@@ -85,8 +85,8 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.DjZDIZsN.js";
 Iterations        = 1:1:5000
 Number of chains  = 1
 Samples per chain = 5000
-Wall duration     = 26.66 seconds
-Compute duration  = 26.66 seconds
+Wall duration     = 20.32 seconds
+Compute duration  = 20.32 seconds
 parameters        = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20]
 internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size
 
@@ -94,26 +94,26 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.DjZDIZsN.js";
       parameters      mean       std      mcse   ess_bulk   ess_tail      rhat   ess_per_sec
           Symbol   Float64   Float64   Float64    Float64    Float64   Float64       Float64
 
-   parameters[1]    2.0965    4.4644    1.3270    12.0887    23.0787    1.3335        0.4534
-   parameters[2]    0.0843    0.3952    0.0510    79.9413    35.2287    1.0240        2.9983
-   parameters[3]    4.9916    1.9230    0.4456    19.3935    81.5852    1.2170        0.7274
-   parameters[4]    0.3356    2.3096    0.6117    14.3653    24.1405    1.2067        0.5388
-   parameters[5]    5.0569    2.3192    0.6141    15.0672    34.9402    1.1058        0.5651
-   parameters[6]    0.7127    1.2191    0.2991    21.7200    21.7891    1.1550        0.8146
-   parameters[7]    1.7149    3.8638    1.1143    12.4420    21.7560    1.1439        0.4667
-   parameters[8]    0.3690    1.2835    0.2555    26.1914    29.6846    1.0246        0.9823
-   parameters[9]   -0.4968    2.2271    0.6133    14.1406    24.5983    1.2648        0.5304
-  parameters[10]    0.0842    2.1828    0.5840    14.0865    21.8689    1.1832        0.5283
-  parameters[11]   -1.0288    1.5663    0.3628    18.3294    27.7789    1.0536        0.6875
-  parameters[12]   -4.1763    1.7426    0.3705    23.0782    28.1565    1.0633        0.8656
-  parameters[13]    3.1846    1.4791    0.3401    19.4472    53.5564    1.0444        0.7294
-  parameters[14]    2.7199    1.9547    0.5178    14.3490    46.8984    1.3048        0.5382
-  parameters[15]   -2.0613    1.4937    0.3727    16.3035    39.2852    1.0885        0.6115
-  parameters[16]   -2.9853    1.4059    0.2557    31.3669    31.3956    1.0012        1.1765
-  parameters[17]   -2.4061    2.6897    0.7370    15.6752    20.0204    1.0733        0.5879
-  parameters[18]   -5.3040    1.1943    0.1791    44.9414    68.7121    1.0867        1.6856
-  parameters[19]   -5.1706    2.2709    0.5991    17.6788    18.3886    1.0540        0.6631
-  parameters[20]   -5.1303    1.3008    0.2366    30.4880    60.9517    1.0163        1.1435
+   parameters[1]    2.0965    4.4644    1.3270    12.0887    23.0787    1.3335        0.5950
+   parameters[2]    0.0843    0.3952    0.0510    79.9413    35.2287    1.0240        3.9345
+   parameters[3]    4.9916    1.9230    0.4456    19.3935    81.5852    1.2170        0.9545
+   parameters[4]    0.3356    2.3096    0.6117    14.3653    24.1405    1.2067        0.7070
+   parameters[5]    5.0569    2.3192    0.6141    15.0672    34.9402    1.1058        0.7416
+   parameters[6]    0.7127    1.2191    0.2991    21.7200    21.7891    1.1550        1.0690
+   parameters[7]    1.7149    3.8638    1.1143    12.4420    21.7560    1.1439        0.6124
+   parameters[8]    0.3690    1.2835    0.2555    26.1914    29.6846    1.0246        1.2891
+   parameters[9]   -0.4968    2.2271    0.6133    14.1406    24.5983    1.2648        0.6960
+  parameters[10]    0.0842    2.1828    0.5840    14.0865    21.8689    1.1832        0.6933
+  parameters[11]   -1.0288    1.5663    0.3628    18.3294    27.7789    1.0536        0.9021
+  parameters[12]   -4.1763    1.7426    0.3705    23.0782    28.1565    1.0633        1.1359
+  parameters[13]    3.1846    1.4791    0.3401    19.4472    53.5564    1.0444        0.9571
+  parameters[14]    2.7199    1.9547    0.5178    14.3490    46.8984    1.3048        0.7062
+  parameters[15]   -2.0613    1.4937    0.3727    16.3035    39.2852    1.0885        0.8024
+  parameters[16]   -2.9853    1.4059    0.2557    31.3669    31.3956    1.0012        1.5438
+  parameters[17]   -2.4061    2.6897    0.7370    15.6752    20.0204    1.0733        0.7715
+  parameters[18]   -5.3040    1.1943    0.1791    44.9414    68.7121    1.0867        2.2119
+  parameters[19]   -5.1706    2.2709    0.5991    17.6788    18.3886    1.0540        0.8701
+  parameters[20]   -5.1303    1.3008    0.2366    30.4880    60.9517    1.0163        1.5005
 
 Quantiles
       parameters      2.5%     25.0%     50.0%     75.0%     97.5%
@@ -195,12 +195,12 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.DjZDIZsN.js";
   WORD_SIZE: 64
   LIBM: libopenlibm
   LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
-Threads: 128 default, 0 interactive, 64 GC (on 128 virtual cores)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
 Environment:
-  JULIA_CPU_THREADS = 128
+  JULIA_CPU_THREADS = 16
   JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
   JULIA_PKG_SERVER = 
-  JULIA_NUM_THREADS = 128
+  JULIA_NUM_THREADS = 16
   JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
   JULIA_PKG_PRECOMPILE_AUTO = 0
   JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,16))])}const I=t(p,[["render",l]]);export{B as __pageData,I as default};
diff --git a/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.DS1q_HRo.lean.js b/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.BKUcq4eS.lean.js
similarity index 99%
rename from previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.DS1q_HRo.lean.js
rename to previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.BKUcq4eS.lean.js
index d1ad5939ba..e8cff3ff7a 100644
--- a/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.DS1q_HRo.lean.js
+++ b/previews/PR1000/assets/tutorials_intermediate_2_BayesianNN.md.BKUcq4eS.lean.js
@@ -85,8 +85,8 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.DjZDIZsN.js";
 Iterations        = 1:1:5000
 Number of chains  = 1
 Samples per chain = 5000
-Wall duration     = 26.66 seconds
-Compute duration  = 26.66 seconds
+Wall duration     = 20.32 seconds
+Compute duration  = 20.32 seconds
 parameters        = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20]
 internals         = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size
 
@@ -94,26 +94,26 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.DjZDIZsN.js";
       parameters      mean       std      mcse   ess_bulk   ess_tail      rhat   ess_per_sec
           Symbol   Float64   Float64   Float64    Float64    Float64   Float64       Float64
 
-   parameters[1]    2.0965    4.4644    1.3270    12.0887    23.0787    1.3335        0.4534
-   parameters[2]    0.0843    0.3952    0.0510    79.9413    35.2287    1.0240        2.9983
-   parameters[3]    4.9916    1.9230    0.4456    19.3935    81.5852    1.2170        0.7274
-   parameters[4]    0.3356    2.3096    0.6117    14.3653    24.1405    1.2067        0.5388
-   parameters[5]    5.0569    2.3192    0.6141    15.0672    34.9402    1.1058        0.5651
-   parameters[6]    0.7127    1.2191    0.2991    21.7200    21.7891    1.1550        0.8146
-   parameters[7]    1.7149    3.8638    1.1143    12.4420    21.7560    1.1439        0.4667
-   parameters[8]    0.3690    1.2835    0.2555    26.1914    29.6846    1.0246        0.9823
-   parameters[9]   -0.4968    2.2271    0.6133    14.1406    24.5983    1.2648        0.5304
-  parameters[10]    0.0842    2.1828    0.5840    14.0865    21.8689    1.1832        0.5283
-  parameters[11]   -1.0288    1.5663    0.3628    18.3294    27.7789    1.0536        0.6875
-  parameters[12]   -4.1763    1.7426    0.3705    23.0782    28.1565    1.0633        0.8656
-  parameters[13]    3.1846    1.4791    0.3401    19.4472    53.5564    1.0444        0.7294
-  parameters[14]    2.7199    1.9547    0.5178    14.3490    46.8984    1.3048        0.5382
-  parameters[15]   -2.0613    1.4937    0.3727    16.3035    39.2852    1.0885        0.6115
-  parameters[16]   -2.9853    1.4059    0.2557    31.3669    31.3956    1.0012        1.1765
-  parameters[17]   -2.4061    2.6897    0.7370    15.6752    20.0204    1.0733        0.5879
-  parameters[18]   -5.3040    1.1943    0.1791    44.9414    68.7121    1.0867        1.6856
-  parameters[19]   -5.1706    2.2709    0.5991    17.6788    18.3886    1.0540        0.6631
-  parameters[20]   -5.1303    1.3008    0.2366    30.4880    60.9517    1.0163        1.1435
+   parameters[1]    2.0965    4.4644    1.3270    12.0887    23.0787    1.3335        0.5950
+   parameters[2]    0.0843    0.3952    0.0510    79.9413    35.2287    1.0240        3.9345
+   parameters[3]    4.9916    1.9230    0.4456    19.3935    81.5852    1.2170        0.9545
+   parameters[4]    0.3356    2.3096    0.6117    14.3653    24.1405    1.2067        0.7070
+   parameters[5]    5.0569    2.3192    0.6141    15.0672    34.9402    1.1058        0.7416
+   parameters[6]    0.7127    1.2191    0.2991    21.7200    21.7891    1.1550        1.0690
+   parameters[7]    1.7149    3.8638    1.1143    12.4420    21.7560    1.1439        0.6124
+   parameters[8]    0.3690    1.2835    0.2555    26.1914    29.6846    1.0246        1.2891
+   parameters[9]   -0.4968    2.2271    0.6133    14.1406    24.5983    1.2648        0.6960
+  parameters[10]    0.0842    2.1828    0.5840    14.0865    21.8689    1.1832        0.6933
+  parameters[11]   -1.0288    1.5663    0.3628    18.3294    27.7789    1.0536        0.9021
+  parameters[12]   -4.1763    1.7426    0.3705    23.0782    28.1565    1.0633        1.1359
+  parameters[13]    3.1846    1.4791    0.3401    19.4472    53.5564    1.0444        0.9571
+  parameters[14]    2.7199    1.9547    0.5178    14.3490    46.8984    1.3048        0.7062
+  parameters[15]   -2.0613    1.4937    0.3727    16.3035    39.2852    1.0885        0.8024
+  parameters[16]   -2.9853    1.4059    0.2557    31.3669    31.3956    1.0012        1.5438
+  parameters[17]   -2.4061    2.6897    0.7370    15.6752    20.0204    1.0733        0.7715
+  parameters[18]   -5.3040    1.1943    0.1791    44.9414    68.7121    1.0867        2.2119
+  parameters[19]   -5.1706    2.2709    0.5991    17.6788    18.3886    1.0540        0.8701
+  parameters[20]   -5.1303    1.3008    0.2366    30.4880    60.9517    1.0163        1.5005
 
 Quantiles
       parameters      2.5%     25.0%     50.0%     75.0%     97.5%
@@ -195,12 +195,12 @@ import{_ as t,c as a,a2 as i,j as A,o as n}from"./chunks/framework.DjZDIZsN.js";
   WORD_SIZE: 64
   LIBM: libopenlibm
   LLVM: libLLVM-15.0.7 (ORCJIT, znver2)
-Threads: 128 default, 0 interactive, 64 GC (on 128 virtual cores)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
 Environment:
-  JULIA_CPU_THREADS = 128
+  JULIA_CPU_THREADS = 16
   JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6
   JULIA_PKG_SERVER = 
-  JULIA_NUM_THREADS = 128
+  JULIA_NUM_THREADS = 16
   JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
   JULIA_PKG_PRECOMPILE_AUTO = 0
   JULIA_DEBUG = Literate
    This page was generated using Literate.jl.
    `,16))])}const I=t(p,[["render",l]]);export{B as __pageData,I as default};
diff --git a/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.C2PwwltN.js b/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.BoHQxIel.js
similarity index 83%
rename from previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.C2PwwltN.js
rename to previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.BoHQxIel.js
index 943afa5a69..2ed81f2e05 100644
--- a/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.C2PwwltN.js
+++ b/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.BoHQxIel.js
@@ -107,109 +107,109 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const d
     return test_acc_list
 end
 
-test_acc_list = train()
    [  1/ 50]	       MNIST	Time 67.59538s	Training Accuracy: 60.45%	Test Accuracy: 56.25%
-[  1/ 50]	FashionMNIST	Time 0.02886s	Training Accuracy: 53.81%	Test Accuracy: 46.88%
-[  2/ 50]	       MNIST	Time 0.02877s	Training Accuracy: 69.43%	Test Accuracy: 62.50%
-[  2/ 50]	FashionMNIST	Time 0.02784s	Training Accuracy: 62.50%	Test Accuracy: 50.00%
-[  3/ 50]	       MNIST	Time 0.02909s	Training Accuracy: 78.22%	Test Accuracy: 65.62%
-[  3/ 50]	FashionMNIST	Time 0.02795s	Training Accuracy: 62.21%	Test Accuracy: 50.00%
-[  4/ 50]	       MNIST	Time 0.02895s	Training Accuracy: 76.76%	Test Accuracy: 68.75%
-[  4/ 50]	FashionMNIST	Time 0.02354s	Training Accuracy: 62.21%	Test Accuracy: 46.88%
-[  5/ 50]	       MNIST	Time 0.02153s	Training Accuracy: 83.40%	Test Accuracy: 78.12%
-[  5/ 50]	FashionMNIST	Time 0.02135s	Training Accuracy: 69.73%	Test Accuracy: 59.38%
-[  6/ 50]	       MNIST	Time 0.02083s	Training Accuracy: 85.74%	Test Accuracy: 75.00%
-[  6/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 73.93%	Test Accuracy: 62.50%
-[  7/ 50]	       MNIST	Time 0.02118s	Training Accuracy: 91.02%	Test Accuracy: 75.00%
-[  7/ 50]	FashionMNIST	Time 0.02260s	Training Accuracy: 68.65%	Test Accuracy: 53.12%
-[  8/ 50]	       MNIST	Time 0.02231s	Training Accuracy: 91.99%	Test Accuracy: 78.12%
-[  8/ 50]	FashionMNIST	Time 0.03982s	Training Accuracy: 73.14%	Test Accuracy: 62.50%
-[  9/ 50]	       MNIST	Time 0.02182s	Training Accuracy: 94.14%	Test Accuracy: 84.38%
-[  9/ 50]	FashionMNIST	Time 0.02077s	Training Accuracy: 77.93%	Test Accuracy: 56.25%
-[ 10/ 50]	       MNIST	Time 0.02068s	Training Accuracy: 95.70%	Test Accuracy: 84.38%
-[ 10/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 79.20%	Test Accuracy: 65.62%
-[ 11/ 50]	       MNIST	Time 0.02058s	Training Accuracy: 97.17%	Test Accuracy: 78.12%
-[ 11/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 82.42%	Test Accuracy: 68.75%
-[ 12/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 97.85%	Test Accuracy: 78.12%
-[ 12/ 50]	FashionMNIST	Time 0.02021s	Training Accuracy: 84.08%	Test Accuracy: 71.88%
-[ 13/ 50]	       MNIST	Time 0.02726s	Training Accuracy: 97.95%	Test Accuracy: 78.12%
-[ 13/ 50]	FashionMNIST	Time 0.02063s	Training Accuracy: 82.23%	Test Accuracy: 71.88%
-[ 14/ 50]	       MNIST	Time 0.02042s	Training Accuracy: 98.63%	Test Accuracy: 81.25%
-[ 14/ 50]	FashionMNIST	Time 0.02058s	Training Accuracy: 83.40%	Test Accuracy: 65.62%
-[ 15/ 50]	       MNIST	Time 0.02022s	Training Accuracy: 99.22%	Test Accuracy: 78.12%
-[ 15/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 84.08%	Test Accuracy: 68.75%
-[ 16/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 99.61%	Test Accuracy: 78.12%
-[ 16/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 85.35%	Test Accuracy: 68.75%
-[ 17/ 50]	       MNIST	Time 0.02028s	Training Accuracy: 99.80%	Test Accuracy: 78.12%
-[ 17/ 50]	FashionMNIST	Time 0.02797s	Training Accuracy: 86.13%	Test Accuracy: 68.75%
-[ 18/ 50]	       MNIST	Time 0.02152s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 18/ 50]	FashionMNIST	Time 0.02059s	Training Accuracy: 86.23%	Test Accuracy: 65.62%
-[ 19/ 50]	       MNIST	Time 0.02069s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 19/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 86.62%	Test Accuracy: 68.75%
-[ 20/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 20/ 50]	FashionMNIST	Time 0.02016s	Training Accuracy: 87.79%	Test Accuracy: 68.75%
-[ 21/ 50]	       MNIST	Time 0.02172s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 21/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 88.57%	Test Accuracy: 68.75%
-[ 22/ 50]	       MNIST	Time 0.02040s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 22/ 50]	FashionMNIST	Time 0.02024s	Training Accuracy: 88.38%	Test Accuracy: 65.62%
-[ 23/ 50]	       MNIST	Time 0.02088s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 23/ 50]	FashionMNIST	Time 0.02052s	Training Accuracy: 87.89%	Test Accuracy: 68.75%
-[ 24/ 50]	       MNIST	Time 0.02050s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 24/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 88.96%	Test Accuracy: 62.50%
-[ 25/ 50]	       MNIST	Time 0.02047s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 25/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 90.53%	Test Accuracy: 65.62%
-[ 26/ 50]	       MNIST	Time 0.02916s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 26/ 50]	FashionMNIST	Time 0.02016s	Training Accuracy: 91.31%	Test Accuracy: 65.62%
-[ 27/ 50]	       MNIST	Time 0.02073s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 27/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 91.50%	Test Accuracy: 62.50%
-[ 28/ 50]	       MNIST	Time 0.02008s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 28/ 50]	FashionMNIST	Time 0.02012s	Training Accuracy: 92.48%	Test Accuracy: 68.75%
-[ 29/ 50]	       MNIST	Time 0.02081s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 29/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 93.07%	Test Accuracy: 65.62%
-[ 30/ 50]	       MNIST	Time 0.02097s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 30/ 50]	FashionMNIST	Time 0.02821s	Training Accuracy: 92.97%	Test Accuracy: 68.75%
-[ 31/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 31/ 50]	FashionMNIST	Time 0.02067s	Training Accuracy: 93.65%	Test Accuracy: 65.62%
-[ 32/ 50]	       MNIST	Time 0.02055s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 32/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 93.85%	Test Accuracy: 68.75%
-[ 33/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 33/ 50]	FashionMNIST	Time 0.02084s	Training Accuracy: 94.53%	Test Accuracy: 68.75%
-[ 34/ 50]	       MNIST	Time 0.02110s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 34/ 50]	FashionMNIST	Time 0.02081s	Training Accuracy: 94.34%	Test Accuracy: 68.75%
-[ 35/ 50]	       MNIST	Time 0.02813s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 35/ 50]	FashionMNIST	Time 0.02068s	Training Accuracy: 94.43%	Test Accuracy: 71.88%
-[ 36/ 50]	       MNIST	Time 0.02078s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 36/ 50]	FashionMNIST	Time 0.02239s	Training Accuracy: 95.02%	Test Accuracy: 75.00%
-[ 37/ 50]	       MNIST	Time 0.02039s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 37/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 95.21%	Test Accuracy: 75.00%
-[ 38/ 50]	       MNIST	Time 0.02081s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 38/ 50]	FashionMNIST	Time 0.02029s	Training Accuracy: 95.70%	Test Accuracy: 75.00%
-[ 39/ 50]	       MNIST	Time 0.02027s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 39/ 50]	FashionMNIST	Time 0.02033s	Training Accuracy: 95.61%	Test Accuracy: 71.88%
-[ 40/ 50]	       MNIST	Time 0.02076s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 40/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 95.90%	Test Accuracy: 71.88%
-[ 41/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 41/ 50]	FashionMNIST	Time 0.02013s	Training Accuracy: 96.39%	Test Accuracy: 75.00%
-[ 42/ 50]	       MNIST	Time 0.02089s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 42/ 50]	FashionMNIST	Time 0.02078s	Training Accuracy: 96.88%	Test Accuracy: 75.00%
-[ 43/ 50]	       MNIST	Time 0.02095s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 43/ 50]	FashionMNIST	Time 0.02954s	Training Accuracy: 96.48%	Test Accuracy: 65.62%
-[ 44/ 50]	       MNIST	Time 0.02079s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 44/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 95.90%	Test Accuracy: 68.75%
-[ 45/ 50]	       MNIST	Time 0.02045s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
-[ 45/ 50]	FashionMNIST	Time 0.02060s	Training Accuracy: 95.90%	Test Accuracy: 71.88%
-[ 46/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
-[ 46/ 50]	FashionMNIST	Time 0.02057s	Training Accuracy: 95.41%	Test Accuracy: 71.88%
-[ 47/ 50]	       MNIST	Time 0.02223s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 47/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 96.48%	Test Accuracy: 71.88%
-[ 48/ 50]	       MNIST	Time 0.02729s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 48/ 50]	FashionMNIST	Time 0.02069s	Training Accuracy: 96.68%	Test Accuracy: 71.88%
-[ 49/ 50]	       MNIST	Time 0.02059s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 49/ 50]	FashionMNIST	Time 0.02063s	Training Accuracy: 96.58%	Test Accuracy: 75.00%
-[ 50/ 50]	       MNIST	Time 0.02060s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 50/ 50]	FashionMNIST	Time 0.02066s	Training Accuracy: 96.00%	Test Accuracy: 71.88%
+test_acc_list = train()
    [  1/ 50]	       MNIST	Time 80.69589s	Training Accuracy: 55.27%	Test Accuracy: 56.25%
+[  1/ 50]	FashionMNIST	Time 0.03641s	Training Accuracy: 32.81%	Test Accuracy: 21.88%
+[  2/ 50]	       MNIST	Time 0.03428s	Training Accuracy: 67.09%	Test Accuracy: 71.88%
+[  2/ 50]	FashionMNIST	Time 0.05600s	Training Accuracy: 52.25%	Test Accuracy: 46.88%
+[  3/ 50]	       MNIST	Time 0.03427s	Training Accuracy: 77.34%	Test Accuracy: 59.38%
+[  3/ 50]	FashionMNIST	Time 0.06616s	Training Accuracy: 63.48%	Test Accuracy: 59.38%
+[  4/ 50]	       MNIST	Time 0.02669s	Training Accuracy: 78.81%	Test Accuracy: 65.62%
+[  4/ 50]	FashionMNIST	Time 0.02622s	Training Accuracy: 65.92%	Test Accuracy: 59.38%
+[  5/ 50]	       MNIST	Time 0.02200s	Training Accuracy: 82.91%	Test Accuracy: 71.88%
+[  5/ 50]	FashionMNIST	Time 0.02199s	Training Accuracy: 67.97%	Test Accuracy: 59.38%
+[  6/ 50]	       MNIST	Time 0.05453s	Training Accuracy: 84.47%	Test Accuracy: 59.38%
+[  6/ 50]	FashionMNIST	Time 0.04688s	Training Accuracy: 71.19%	Test Accuracy: 59.38%
+[  7/ 50]	       MNIST	Time 0.04077s	Training Accuracy: 88.87%	Test Accuracy: 71.88%
+[  7/ 50]	FashionMNIST	Time 0.04811s	Training Accuracy: 71.48%	Test Accuracy: 75.00%
+[  8/ 50]	       MNIST	Time 0.05305s	Training Accuracy: 90.23%	Test Accuracy: 71.88%
+[  8/ 50]	FashionMNIST	Time 0.02038s	Training Accuracy: 74.71%	Test Accuracy: 65.62%
+[  9/ 50]	       MNIST	Time 0.03395s	Training Accuracy: 93.65%	Test Accuracy: 78.12%
+[  9/ 50]	FashionMNIST	Time 0.02593s	Training Accuracy: 75.98%	Test Accuracy: 68.75%
+[ 10/ 50]	       MNIST	Time 0.02077s	Training Accuracy: 95.21%	Test Accuracy: 78.12%
+[ 10/ 50]	FashionMNIST	Time 0.02246s	Training Accuracy: 81.25%	Test Accuracy: 71.88%
+[ 11/ 50]	       MNIST	Time 0.02136s	Training Accuracy: 97.07%	Test Accuracy: 78.12%
+[ 11/ 50]	FashionMNIST	Time 0.02048s	Training Accuracy: 81.54%	Test Accuracy: 65.62%
+[ 12/ 50]	       MNIST	Time 0.02282s	Training Accuracy: 97.07%	Test Accuracy: 81.25%
+[ 12/ 50]	FashionMNIST	Time 0.02859s	Training Accuracy: 78.12%	Test Accuracy: 62.50%
+[ 13/ 50]	       MNIST	Time 0.02120s	Training Accuracy: 98.05%	Test Accuracy: 81.25%
+[ 13/ 50]	FashionMNIST	Time 0.02173s	Training Accuracy: 82.13%	Test Accuracy: 71.88%
+[ 14/ 50]	       MNIST	Time 0.02161s	Training Accuracy: 99.02%	Test Accuracy: 81.25%
+[ 14/ 50]	FashionMNIST	Time 0.02257s	Training Accuracy: 84.38%	Test Accuracy: 62.50%
+[ 15/ 50]	       MNIST	Time 0.02224s	Training Accuracy: 99.32%	Test Accuracy: 81.25%
+[ 15/ 50]	FashionMNIST	Time 0.02504s	Training Accuracy: 84.47%	Test Accuracy: 62.50%
+[ 16/ 50]	       MNIST	Time 0.02212s	Training Accuracy: 99.41%	Test Accuracy: 81.25%
+[ 16/ 50]	FashionMNIST	Time 0.04436s	Training Accuracy: 85.64%	Test Accuracy: 62.50%
+[ 17/ 50]	       MNIST	Time 0.02359s	Training Accuracy: 99.80%	Test Accuracy: 81.25%
+[ 17/ 50]	FashionMNIST	Time 0.02177s	Training Accuracy: 86.52%	Test Accuracy: 59.38%
+[ 18/ 50]	       MNIST	Time 0.02864s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
+[ 18/ 50]	FashionMNIST	Time 0.02119s	Training Accuracy: 87.99%	Test Accuracy: 62.50%
+[ 19/ 50]	       MNIST	Time 0.02161s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 19/ 50]	FashionMNIST	Time 0.02909s	Training Accuracy: 87.70%	Test Accuracy: 62.50%
+[ 20/ 50]	       MNIST	Time 0.02218s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 20/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 88.96%	Test Accuracy: 62.50%
+[ 21/ 50]	       MNIST	Time 0.02941s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 21/ 50]	FashionMNIST	Time 0.03479s	Training Accuracy: 88.87%	Test Accuracy: 62.50%
+[ 22/ 50]	       MNIST	Time 0.02223s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 22/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 89.06%	Test Accuracy: 62.50%
+[ 23/ 50]	       MNIST	Time 0.02614s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 23/ 50]	FashionMNIST	Time 0.02191s	Training Accuracy: 89.45%	Test Accuracy: 62.50%
+[ 24/ 50]	       MNIST	Time 0.02288s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 24/ 50]	FashionMNIST	Time 0.02745s	Training Accuracy: 89.94%	Test Accuracy: 59.38%
+[ 25/ 50]	       MNIST	Time 0.02736s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 25/ 50]	FashionMNIST	Time 0.02957s	Training Accuracy: 90.43%	Test Accuracy: 65.62%
+[ 26/ 50]	       MNIST	Time 0.02499s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 26/ 50]	FashionMNIST	Time 0.02296s	Training Accuracy: 91.02%	Test Accuracy: 62.50%
+[ 27/ 50]	       MNIST	Time 0.03058s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 27/ 50]	FashionMNIST	Time 0.02179s	Training Accuracy: 90.92%	Test Accuracy: 65.62%
+[ 28/ 50]	       MNIST	Time 0.03407s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 28/ 50]	FashionMNIST	Time 0.01977s	Training Accuracy: 91.60%	Test Accuracy: 65.62%
+[ 29/ 50]	       MNIST	Time 0.02877s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 29/ 50]	FashionMNIST	Time 0.02133s	Training Accuracy: 91.31%	Test Accuracy: 62.50%
+[ 30/ 50]	       MNIST	Time 0.02732s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 30/ 50]	FashionMNIST	Time 0.02149s	Training Accuracy: 91.41%	Test Accuracy: 68.75%
+[ 31/ 50]	       MNIST	Time 0.02000s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 31/ 50]	FashionMNIST	Time 0.02580s	Training Accuracy: 91.60%	Test Accuracy: 65.62%
+[ 32/ 50]	       MNIST	Time 0.03682s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 32/ 50]	FashionMNIST	Time 0.02041s	Training Accuracy: 92.29%	Test Accuracy: 65.62%
+[ 33/ 50]	       MNIST	Time 0.02557s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 33/ 50]	FashionMNIST	Time 0.02514s	Training Accuracy: 92.09%	Test Accuracy: 65.62%
+[ 34/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 34/ 50]	FashionMNIST	Time 0.02045s	Training Accuracy: 92.68%	Test Accuracy: 65.62%
+[ 35/ 50]	       MNIST	Time 0.02030s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 35/ 50]	FashionMNIST	Time 0.02601s	Training Accuracy: 92.87%	Test Accuracy: 65.62%
+[ 36/ 50]	       MNIST	Time 0.02066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 36/ 50]	FashionMNIST	Time 0.02098s	Training Accuracy: 93.16%	Test Accuracy: 68.75%
+[ 37/ 50]	       MNIST	Time 0.02270s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 37/ 50]	FashionMNIST	Time 0.02096s	Training Accuracy: 93.46%	Test Accuracy: 68.75%
+[ 38/ 50]	       MNIST	Time 0.02106s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 38/ 50]	FashionMNIST	Time 0.03050s	Training Accuracy: 93.75%	Test Accuracy: 68.75%
+[ 39/ 50]	       MNIST	Time 0.02782s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 39/ 50]	FashionMNIST	Time 0.02598s	Training Accuracy: 94.04%	Test Accuracy: 68.75%
+[ 40/ 50]	       MNIST	Time 0.02427s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 40/ 50]	FashionMNIST	Time 0.03184s	Training Accuracy: 94.24%	Test Accuracy: 68.75%
+[ 41/ 50]	       MNIST	Time 0.02034s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 41/ 50]	FashionMNIST	Time 0.02142s	Training Accuracy: 94.53%	Test Accuracy: 71.88%
+[ 42/ 50]	       MNIST	Time 0.02225s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 42/ 50]	FashionMNIST	Time 0.02270s	Training Accuracy: 94.43%	Test Accuracy: 71.88%
+[ 43/ 50]	       MNIST	Time 0.03666s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 43/ 50]	FashionMNIST	Time 0.02332s	Training Accuracy: 94.82%	Test Accuracy: 71.88%
+[ 44/ 50]	       MNIST	Time 0.03407s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 44/ 50]	FashionMNIST	Time 0.02114s	Training Accuracy: 95.31%	Test Accuracy: 75.00%
+[ 45/ 50]	       MNIST	Time 0.02362s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 45/ 50]	FashionMNIST	Time 0.02191s	Training Accuracy: 95.21%	Test Accuracy: 75.00%
+[ 46/ 50]	       MNIST	Time 0.02321s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 46/ 50]	FashionMNIST	Time 0.02938s	Training Accuracy: 95.31%	Test Accuracy: 75.00%
+[ 47/ 50]	       MNIST	Time 0.03179s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 47/ 50]	FashionMNIST	Time 0.04171s	Training Accuracy: 95.61%	Test Accuracy: 75.00%
+[ 48/ 50]	       MNIST	Time 0.02858s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 48/ 50]	FashionMNIST	Time 0.02325s	Training Accuracy: 95.70%	Test Accuracy: 75.00%
+[ 49/ 50]	       MNIST	Time 0.02226s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 49/ 50]	FashionMNIST	Time 0.02216s	Training Accuracy: 96.09%	Test Accuracy: 75.00%
+[ 50/ 50]	       MNIST	Time 0.02234s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 50/ 50]	FashionMNIST	Time 0.02797s	Training Accuracy: 96.00%	Test Accuracy: 75.00%
 
-[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[FINAL]	FashionMNIST	Training Accuracy: 96.00%	Test Accuracy: 71.88%
    Appendix ​
    julia
    using InteractiveUtils
+[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[FINAL]	FashionMNIST	Training Accuracy: 96.00%	Test Accuracy: 75.00%
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.C2PwwltN.lean.js b/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.BoHQxIel.lean.js
similarity index 83%
rename from previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.C2PwwltN.lean.js
rename to previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.BoHQxIel.lean.js
index 943afa5a69..2ed81f2e05 100644
--- a/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.C2PwwltN.lean.js
+++ b/previews/PR1000/assets/tutorials_intermediate_3_HyperNet.md.BoHQxIel.lean.js
@@ -107,109 +107,109 @@ import{_ as i,c as a,a2 as n,o as t}from"./chunks/framework.DjZDIZsN.js";const d
     return test_acc_list
 end
 
-test_acc_list = train()
    [  1/ 50]	       MNIST	Time 67.59538s	Training Accuracy: 60.45%	Test Accuracy: 56.25%
-[  1/ 50]	FashionMNIST	Time 0.02886s	Training Accuracy: 53.81%	Test Accuracy: 46.88%
-[  2/ 50]	       MNIST	Time 0.02877s	Training Accuracy: 69.43%	Test Accuracy: 62.50%
-[  2/ 50]	FashionMNIST	Time 0.02784s	Training Accuracy: 62.50%	Test Accuracy: 50.00%
-[  3/ 50]	       MNIST	Time 0.02909s	Training Accuracy: 78.22%	Test Accuracy: 65.62%
-[  3/ 50]	FashionMNIST	Time 0.02795s	Training Accuracy: 62.21%	Test Accuracy: 50.00%
-[  4/ 50]	       MNIST	Time 0.02895s	Training Accuracy: 76.76%	Test Accuracy: 68.75%
-[  4/ 50]	FashionMNIST	Time 0.02354s	Training Accuracy: 62.21%	Test Accuracy: 46.88%
-[  5/ 50]	       MNIST	Time 0.02153s	Training Accuracy: 83.40%	Test Accuracy: 78.12%
-[  5/ 50]	FashionMNIST	Time 0.02135s	Training Accuracy: 69.73%	Test Accuracy: 59.38%
-[  6/ 50]	       MNIST	Time 0.02083s	Training Accuracy: 85.74%	Test Accuracy: 75.00%
-[  6/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 73.93%	Test Accuracy: 62.50%
-[  7/ 50]	       MNIST	Time 0.02118s	Training Accuracy: 91.02%	Test Accuracy: 75.00%
-[  7/ 50]	FashionMNIST	Time 0.02260s	Training Accuracy: 68.65%	Test Accuracy: 53.12%
-[  8/ 50]	       MNIST	Time 0.02231s	Training Accuracy: 91.99%	Test Accuracy: 78.12%
-[  8/ 50]	FashionMNIST	Time 0.03982s	Training Accuracy: 73.14%	Test Accuracy: 62.50%
-[  9/ 50]	       MNIST	Time 0.02182s	Training Accuracy: 94.14%	Test Accuracy: 84.38%
-[  9/ 50]	FashionMNIST	Time 0.02077s	Training Accuracy: 77.93%	Test Accuracy: 56.25%
-[ 10/ 50]	       MNIST	Time 0.02068s	Training Accuracy: 95.70%	Test Accuracy: 84.38%
-[ 10/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 79.20%	Test Accuracy: 65.62%
-[ 11/ 50]	       MNIST	Time 0.02058s	Training Accuracy: 97.17%	Test Accuracy: 78.12%
-[ 11/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 82.42%	Test Accuracy: 68.75%
-[ 12/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 97.85%	Test Accuracy: 78.12%
-[ 12/ 50]	FashionMNIST	Time 0.02021s	Training Accuracy: 84.08%	Test Accuracy: 71.88%
-[ 13/ 50]	       MNIST	Time 0.02726s	Training Accuracy: 97.95%	Test Accuracy: 78.12%
-[ 13/ 50]	FashionMNIST	Time 0.02063s	Training Accuracy: 82.23%	Test Accuracy: 71.88%
-[ 14/ 50]	       MNIST	Time 0.02042s	Training Accuracy: 98.63%	Test Accuracy: 81.25%
-[ 14/ 50]	FashionMNIST	Time 0.02058s	Training Accuracy: 83.40%	Test Accuracy: 65.62%
-[ 15/ 50]	       MNIST	Time 0.02022s	Training Accuracy: 99.22%	Test Accuracy: 78.12%
-[ 15/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 84.08%	Test Accuracy: 68.75%
-[ 16/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 99.61%	Test Accuracy: 78.12%
-[ 16/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 85.35%	Test Accuracy: 68.75%
-[ 17/ 50]	       MNIST	Time 0.02028s	Training Accuracy: 99.80%	Test Accuracy: 78.12%
-[ 17/ 50]	FashionMNIST	Time 0.02797s	Training Accuracy: 86.13%	Test Accuracy: 68.75%
-[ 18/ 50]	       MNIST	Time 0.02152s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 18/ 50]	FashionMNIST	Time 0.02059s	Training Accuracy: 86.23%	Test Accuracy: 65.62%
-[ 19/ 50]	       MNIST	Time 0.02069s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 19/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 86.62%	Test Accuracy: 68.75%
-[ 20/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 20/ 50]	FashionMNIST	Time 0.02016s	Training Accuracy: 87.79%	Test Accuracy: 68.75%
-[ 21/ 50]	       MNIST	Time 0.02172s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 21/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 88.57%	Test Accuracy: 68.75%
-[ 22/ 50]	       MNIST	Time 0.02040s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 22/ 50]	FashionMNIST	Time 0.02024s	Training Accuracy: 88.38%	Test Accuracy: 65.62%
-[ 23/ 50]	       MNIST	Time 0.02088s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 23/ 50]	FashionMNIST	Time 0.02052s	Training Accuracy: 87.89%	Test Accuracy: 68.75%
-[ 24/ 50]	       MNIST	Time 0.02050s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 24/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 88.96%	Test Accuracy: 62.50%
-[ 25/ 50]	       MNIST	Time 0.02047s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 25/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 90.53%	Test Accuracy: 65.62%
-[ 26/ 50]	       MNIST	Time 0.02916s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 26/ 50]	FashionMNIST	Time 0.02016s	Training Accuracy: 91.31%	Test Accuracy: 65.62%
-[ 27/ 50]	       MNIST	Time 0.02073s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 27/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 91.50%	Test Accuracy: 62.50%
-[ 28/ 50]	       MNIST	Time 0.02008s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 28/ 50]	FashionMNIST	Time 0.02012s	Training Accuracy: 92.48%	Test Accuracy: 68.75%
-[ 29/ 50]	       MNIST	Time 0.02081s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 29/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 93.07%	Test Accuracy: 65.62%
-[ 30/ 50]	       MNIST	Time 0.02097s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 30/ 50]	FashionMNIST	Time 0.02821s	Training Accuracy: 92.97%	Test Accuracy: 68.75%
-[ 31/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 31/ 50]	FashionMNIST	Time 0.02067s	Training Accuracy: 93.65%	Test Accuracy: 65.62%
-[ 32/ 50]	       MNIST	Time 0.02055s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 32/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 93.85%	Test Accuracy: 68.75%
-[ 33/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 33/ 50]	FashionMNIST	Time 0.02084s	Training Accuracy: 94.53%	Test Accuracy: 68.75%
-[ 34/ 50]	       MNIST	Time 0.02110s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 34/ 50]	FashionMNIST	Time 0.02081s	Training Accuracy: 94.34%	Test Accuracy: 68.75%
-[ 35/ 50]	       MNIST	Time 0.02813s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 35/ 50]	FashionMNIST	Time 0.02068s	Training Accuracy: 94.43%	Test Accuracy: 71.88%
-[ 36/ 50]	       MNIST	Time 0.02078s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 36/ 50]	FashionMNIST	Time 0.02239s	Training Accuracy: 95.02%	Test Accuracy: 75.00%
-[ 37/ 50]	       MNIST	Time 0.02039s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 37/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 95.21%	Test Accuracy: 75.00%
-[ 38/ 50]	       MNIST	Time 0.02081s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 38/ 50]	FashionMNIST	Time 0.02029s	Training Accuracy: 95.70%	Test Accuracy: 75.00%
-[ 39/ 50]	       MNIST	Time 0.02027s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 39/ 50]	FashionMNIST	Time 0.02033s	Training Accuracy: 95.61%	Test Accuracy: 71.88%
-[ 40/ 50]	       MNIST	Time 0.02076s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 40/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 95.90%	Test Accuracy: 71.88%
-[ 41/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 41/ 50]	FashionMNIST	Time 0.02013s	Training Accuracy: 96.39%	Test Accuracy: 75.00%
-[ 42/ 50]	       MNIST	Time 0.02089s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 42/ 50]	FashionMNIST	Time 0.02078s	Training Accuracy: 96.88%	Test Accuracy: 75.00%
-[ 43/ 50]	       MNIST	Time 0.02095s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 43/ 50]	FashionMNIST	Time 0.02954s	Training Accuracy: 96.48%	Test Accuracy: 65.62%
-[ 44/ 50]	       MNIST	Time 0.02079s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 44/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 95.90%	Test Accuracy: 68.75%
-[ 45/ 50]	       MNIST	Time 0.02045s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
-[ 45/ 50]	FashionMNIST	Time 0.02060s	Training Accuracy: 95.90%	Test Accuracy: 71.88%
-[ 46/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
-[ 46/ 50]	FashionMNIST	Time 0.02057s	Training Accuracy: 95.41%	Test Accuracy: 71.88%
-[ 47/ 50]	       MNIST	Time 0.02223s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 47/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 96.48%	Test Accuracy: 71.88%
-[ 48/ 50]	       MNIST	Time 0.02729s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 48/ 50]	FashionMNIST	Time 0.02069s	Training Accuracy: 96.68%	Test Accuracy: 71.88%
-[ 49/ 50]	       MNIST	Time 0.02059s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 49/ 50]	FashionMNIST	Time 0.02063s	Training Accuracy: 96.58%	Test Accuracy: 75.00%
-[ 50/ 50]	       MNIST	Time 0.02060s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 50/ 50]	FashionMNIST	Time 0.02066s	Training Accuracy: 96.00%	Test Accuracy: 71.88%
+test_acc_list = train()
    [  1/ 50]	       MNIST	Time 80.69589s	Training Accuracy: 55.27%	Test Accuracy: 56.25%
+[  1/ 50]	FashionMNIST	Time 0.03641s	Training Accuracy: 32.81%	Test Accuracy: 21.88%
+[  2/ 50]	       MNIST	Time 0.03428s	Training Accuracy: 67.09%	Test Accuracy: 71.88%
+[  2/ 50]	FashionMNIST	Time 0.05600s	Training Accuracy: 52.25%	Test Accuracy: 46.88%
+[  3/ 50]	       MNIST	Time 0.03427s	Training Accuracy: 77.34%	Test Accuracy: 59.38%
+[  3/ 50]	FashionMNIST	Time 0.06616s	Training Accuracy: 63.48%	Test Accuracy: 59.38%
+[  4/ 50]	       MNIST	Time 0.02669s	Training Accuracy: 78.81%	Test Accuracy: 65.62%
+[  4/ 50]	FashionMNIST	Time 0.02622s	Training Accuracy: 65.92%	Test Accuracy: 59.38%
+[  5/ 50]	       MNIST	Time 0.02200s	Training Accuracy: 82.91%	Test Accuracy: 71.88%
+[  5/ 50]	FashionMNIST	Time 0.02199s	Training Accuracy: 67.97%	Test Accuracy: 59.38%
+[  6/ 50]	       MNIST	Time 0.05453s	Training Accuracy: 84.47%	Test Accuracy: 59.38%
+[  6/ 50]	FashionMNIST	Time 0.04688s	Training Accuracy: 71.19%	Test Accuracy: 59.38%
+[  7/ 50]	       MNIST	Time 0.04077s	Training Accuracy: 88.87%	Test Accuracy: 71.88%
+[  7/ 50]	FashionMNIST	Time 0.04811s	Training Accuracy: 71.48%	Test Accuracy: 75.00%
+[  8/ 50]	       MNIST	Time 0.05305s	Training Accuracy: 90.23%	Test Accuracy: 71.88%
+[  8/ 50]	FashionMNIST	Time 0.02038s	Training Accuracy: 74.71%	Test Accuracy: 65.62%
+[  9/ 50]	       MNIST	Time 0.03395s	Training Accuracy: 93.65%	Test Accuracy: 78.12%
+[  9/ 50]	FashionMNIST	Time 0.02593s	Training Accuracy: 75.98%	Test Accuracy: 68.75%
+[ 10/ 50]	       MNIST	Time 0.02077s	Training Accuracy: 95.21%	Test Accuracy: 78.12%
+[ 10/ 50]	FashionMNIST	Time 0.02246s	Training Accuracy: 81.25%	Test Accuracy: 71.88%
+[ 11/ 50]	       MNIST	Time 0.02136s	Training Accuracy: 97.07%	Test Accuracy: 78.12%
+[ 11/ 50]	FashionMNIST	Time 0.02048s	Training Accuracy: 81.54%	Test Accuracy: 65.62%
+[ 12/ 50]	       MNIST	Time 0.02282s	Training Accuracy: 97.07%	Test Accuracy: 81.25%
+[ 12/ 50]	FashionMNIST	Time 0.02859s	Training Accuracy: 78.12%	Test Accuracy: 62.50%
+[ 13/ 50]	       MNIST	Time 0.02120s	Training Accuracy: 98.05%	Test Accuracy: 81.25%
+[ 13/ 50]	FashionMNIST	Time 0.02173s	Training Accuracy: 82.13%	Test Accuracy: 71.88%
+[ 14/ 50]	       MNIST	Time 0.02161s	Training Accuracy: 99.02%	Test Accuracy: 81.25%
+[ 14/ 50]	FashionMNIST	Time 0.02257s	Training Accuracy: 84.38%	Test Accuracy: 62.50%
+[ 15/ 50]	       MNIST	Time 0.02224s	Training Accuracy: 99.32%	Test Accuracy: 81.25%
+[ 15/ 50]	FashionMNIST	Time 0.02504s	Training Accuracy: 84.47%	Test Accuracy: 62.50%
+[ 16/ 50]	       MNIST	Time 0.02212s	Training Accuracy: 99.41%	Test Accuracy: 81.25%
+[ 16/ 50]	FashionMNIST	Time 0.04436s	Training Accuracy: 85.64%	Test Accuracy: 62.50%
+[ 17/ 50]	       MNIST	Time 0.02359s	Training Accuracy: 99.80%	Test Accuracy: 81.25%
+[ 17/ 50]	FashionMNIST	Time 0.02177s	Training Accuracy: 86.52%	Test Accuracy: 59.38%
+[ 18/ 50]	       MNIST	Time 0.02864s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
+[ 18/ 50]	FashionMNIST	Time 0.02119s	Training Accuracy: 87.99%	Test Accuracy: 62.50%
+[ 19/ 50]	       MNIST	Time 0.02161s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 19/ 50]	FashionMNIST	Time 0.02909s	Training Accuracy: 87.70%	Test Accuracy: 62.50%
+[ 20/ 50]	       MNIST	Time 0.02218s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 20/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 88.96%	Test Accuracy: 62.50%
+[ 21/ 50]	       MNIST	Time 0.02941s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 21/ 50]	FashionMNIST	Time 0.03479s	Training Accuracy: 88.87%	Test Accuracy: 62.50%
+[ 22/ 50]	       MNIST	Time 0.02223s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 22/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 89.06%	Test Accuracy: 62.50%
+[ 23/ 50]	       MNIST	Time 0.02614s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 23/ 50]	FashionMNIST	Time 0.02191s	Training Accuracy: 89.45%	Test Accuracy: 62.50%
+[ 24/ 50]	       MNIST	Time 0.02288s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 24/ 50]	FashionMNIST	Time 0.02745s	Training Accuracy: 89.94%	Test Accuracy: 59.38%
+[ 25/ 50]	       MNIST	Time 0.02736s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 25/ 50]	FashionMNIST	Time 0.02957s	Training Accuracy: 90.43%	Test Accuracy: 65.62%
+[ 26/ 50]	       MNIST	Time 0.02499s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 26/ 50]	FashionMNIST	Time 0.02296s	Training Accuracy: 91.02%	Test Accuracy: 62.50%
+[ 27/ 50]	       MNIST	Time 0.03058s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 27/ 50]	FashionMNIST	Time 0.02179s	Training Accuracy: 90.92%	Test Accuracy: 65.62%
+[ 28/ 50]	       MNIST	Time 0.03407s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 28/ 50]	FashionMNIST	Time 0.01977s	Training Accuracy: 91.60%	Test Accuracy: 65.62%
+[ 29/ 50]	       MNIST	Time 0.02877s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 29/ 50]	FashionMNIST	Time 0.02133s	Training Accuracy: 91.31%	Test Accuracy: 62.50%
+[ 30/ 50]	       MNIST	Time 0.02732s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 30/ 50]	FashionMNIST	Time 0.02149s	Training Accuracy: 91.41%	Test Accuracy: 68.75%
+[ 31/ 50]	       MNIST	Time 0.02000s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 31/ 50]	FashionMNIST	Time 0.02580s	Training Accuracy: 91.60%	Test Accuracy: 65.62%
+[ 32/ 50]	       MNIST	Time 0.03682s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 32/ 50]	FashionMNIST	Time 0.02041s	Training Accuracy: 92.29%	Test Accuracy: 65.62%
+[ 33/ 50]	       MNIST	Time 0.02557s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 33/ 50]	FashionMNIST	Time 0.02514s	Training Accuracy: 92.09%	Test Accuracy: 65.62%
+[ 34/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 34/ 50]	FashionMNIST	Time 0.02045s	Training Accuracy: 92.68%	Test Accuracy: 65.62%
+[ 35/ 50]	       MNIST	Time 0.02030s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 35/ 50]	FashionMNIST	Time 0.02601s	Training Accuracy: 92.87%	Test Accuracy: 65.62%
+[ 36/ 50]	       MNIST	Time 0.02066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 36/ 50]	FashionMNIST	Time 0.02098s	Training Accuracy: 93.16%	Test Accuracy: 68.75%
+[ 37/ 50]	       MNIST	Time 0.02270s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 37/ 50]	FashionMNIST	Time 0.02096s	Training Accuracy: 93.46%	Test Accuracy: 68.75%
+[ 38/ 50]	       MNIST	Time 0.02106s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 38/ 50]	FashionMNIST	Time 0.03050s	Training Accuracy: 93.75%	Test Accuracy: 68.75%
+[ 39/ 50]	       MNIST	Time 0.02782s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 39/ 50]	FashionMNIST	Time 0.02598s	Training Accuracy: 94.04%	Test Accuracy: 68.75%
+[ 40/ 50]	       MNIST	Time 0.02427s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 40/ 50]	FashionMNIST	Time 0.03184s	Training Accuracy: 94.24%	Test Accuracy: 68.75%
+[ 41/ 50]	       MNIST	Time 0.02034s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 41/ 50]	FashionMNIST	Time 0.02142s	Training Accuracy: 94.53%	Test Accuracy: 71.88%
+[ 42/ 50]	       MNIST	Time 0.02225s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 42/ 50]	FashionMNIST	Time 0.02270s	Training Accuracy: 94.43%	Test Accuracy: 71.88%
+[ 43/ 50]	       MNIST	Time 0.03666s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 43/ 50]	FashionMNIST	Time 0.02332s	Training Accuracy: 94.82%	Test Accuracy: 71.88%
+[ 44/ 50]	       MNIST	Time 0.03407s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 44/ 50]	FashionMNIST	Time 0.02114s	Training Accuracy: 95.31%	Test Accuracy: 75.00%
+[ 45/ 50]	       MNIST	Time 0.02362s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 45/ 50]	FashionMNIST	Time 0.02191s	Training Accuracy: 95.21%	Test Accuracy: 75.00%
+[ 46/ 50]	       MNIST	Time 0.02321s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 46/ 50]	FashionMNIST	Time 0.02938s	Training Accuracy: 95.31%	Test Accuracy: 75.00%
+[ 47/ 50]	       MNIST	Time 0.03179s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 47/ 50]	FashionMNIST	Time 0.04171s	Training Accuracy: 95.61%	Test Accuracy: 75.00%
+[ 48/ 50]	       MNIST	Time 0.02858s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 48/ 50]	FashionMNIST	Time 0.02325s	Training Accuracy: 95.70%	Test Accuracy: 75.00%
+[ 49/ 50]	       MNIST	Time 0.02226s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 49/ 50]	FashionMNIST	Time 0.02216s	Training Accuracy: 96.09%	Test Accuracy: 75.00%
+[ 50/ 50]	       MNIST	Time 0.02234s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 50/ 50]	FashionMNIST	Time 0.02797s	Training Accuracy: 96.00%	Test Accuracy: 75.00%
 
-[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[FINAL]	FashionMNIST	Training Accuracy: 96.00%	Test Accuracy: 71.88%
    Appendix ​
    julia
    using InteractiveUtils
+[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[FINAL]	FashionMNIST	Training Accuracy: 96.00%	Test Accuracy: 75.00%
    Appendix ​
    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
diff --git a/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.D4cij4Vv.js b/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.BZvwgXm0.js
similarity index 99%
rename from previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.D4cij4Vv.js
rename to previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.BZvwgXm0.js
index cf735625c2..a566e74328 100644
--- a/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.D4cij4Vv.js
+++ b/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.BZvwgXm0.js
@@ -147,16 +147,16 @@ import{_ as p,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.DjZDIZ
 trained_u = Lux.testmode(StatefulLuxLayer{true}(
     trained_model.model.u, trained_model.ps.u, trained_model.st.u))
    ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162476 (1.982162476) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505080 (0.598505080)
 Iteration: [  501 / 50000] 	 Loss: 0.040918160 (0.025583776) 	 Physics Loss: 0.000391877 (0.000269295) 	 Data Loss: 0.014243508 (0.009196416) 	 BC Loss: 0.026282774 (0.016118063)
 Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281426) 	 Physics Loss: 0.000071670 (0.000163182) 	 Data Loss: 0.007905648 (0.010876314) 	 BC Loss: 0.007363341 (0.014241929)
diff --git a/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.D4cij4Vv.lean.js b/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.BZvwgXm0.lean.js
similarity index 99%
rename from previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.D4cij4Vv.lean.js
rename to previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.BZvwgXm0.lean.js
index cf735625c2..a566e74328 100644
--- a/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.D4cij4Vv.lean.js
+++ b/previews/PR1000/assets/tutorials_intermediate_4_PINN2DPDE.md.BZvwgXm0.lean.js
@@ -147,16 +147,16 @@ import{_ as p,c as n,a2 as a,j as s,a as t,o as l}from"./chunks/framework.DjZDIZ
 trained_u = Lux.testmode(StatefulLuxLayer{true}(
     trained_model.model.u, trained_model.ps.u, trained_model.st.u))
    ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: \`Lag(T, b)\` is deprecated.  Use \`CircBuff(T,b,rev=true)\` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162476 (1.982162476) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505080 (0.598505080)
 Iteration: [  501 / 50000] 	 Loss: 0.040918160 (0.025583776) 	 Physics Loss: 0.000391877 (0.000269295) 	 Data Loss: 0.014243508 (0.009196416) 	 BC Loss: 0.026282774 (0.016118063)
 Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281426) 	 Physics Loss: 0.000071670 (0.000163182) 	 Data Loss: 0.007905648 (0.010876314) 	 BC Loss: 0.007363341 (0.014241929)
diff --git a/previews/PR1000/ecosystem.html b/previews/PR1000/ecosystem.html
index 7d7344ff29..29f5869caf 100644
--- a/previews/PR1000/ecosystem.html
+++ b/previews/PR1000/ecosystem.html
@@ -5,14 +5,14 @@
     
     Lux.jl Docs
     
-    
+    
     
     
-    
+    
     
-    
+    
     
-    
+    
     
     
     
@@ -27,8 +27,8 @@
     
   
   
-    
    Lux.jl Docs
    Appearance
    Ecosystem
    Frameworks Extending Lux.jl
    DiffEqFlux.jl
    DiffEqFlux.jl
    Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods
    SciMLSensitivity.jl
    SciMLSensitivity.jl
    A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.
    NeuralPDE.jl
    NeuralPDE.jl
    Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation
    NeuralLyapunov.jl
    NeuralLyapunov.jl
    A library for searching for neural Lyapunov functions in Julia
    DeepEquilibriumNetworks.jl
    DeepEquilibriumNetworks.jl
    Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence
    AbstractCosmologicalEmulators.jl
    AbstractCosmologicalEmulators.jl
    Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization
    ContinuousNormalizingFlows.jl
    ContinuousNormalizingFlows.jl
    Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia
    Sophon.jl
    Sophon.jl
    Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks
    DataDrivenDiffEq.jl
    DataDrivenDiffEq.jl
    Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization
    NeuralGraphPDE.jl
    NeuralGraphPDE.jl
    Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks
    Solaris.jl
    Solaris.jl
    Lightweight module for fusing physical and neural models
    Boltz.jl
    Boltz.jl
     Accelerate your ML research using pre-built Deep Learning Models with Lux
    GeometricMachineLearning.jl
    GeometricMachineLearning.jl
    Structure Preserving Machine Learning Models in Julia
    Automatic Differentiation
    Zygote.jl
    Zygote.jl
    Lux.jl default choice for AD
    Tracker.jl
    Tracker.jl
    Well tested and robust AD library (might fail on edge cases)
    ForwardDiff.jl
    ForwardDiff.jl
    For forward mode AD support
    ReverseDiff.jl
    ReverseDiff.jl
    Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)
    Enzyme.jl
    Enzyme.jl
    Experimental Support but will become the Future Default
    Data Manipulation, Data Loading & Datasets
    MLUtils.jl
    MLUtils.jl
    Utilities and abstractions for Machine Learning tasks
    MLDatasets.jl
    MLDatasets.jl
    Utility package for accessing common Machine Learning datasets in Julia
    Images.jl
    Images.jl
    An image library for Julia
    DataAugmentation.jl
    DataAugmentation.jl
    Flexible data augmentation library for machine and deep learning
    Neural Network Primitives
    NNlib.jl
    NNlib.jl
    Neural Network primitives with multiple backends
    LuxLib.jl
    LuxLib.jl
    Backend for Lux.jl
    Optimization
    Optimization.jl
    Optimization.jl
    Unified API for Optimization in Julia
    Optimisers.jl
    Optimisers.jl
    Optimisers.jl defines many standard optimisers and utilities for learning loops
    ParameterSchedulers.jl
    ParameterSchedulers.jl
    Common hyperparameter scheduling for ML
    Parameter Manipulation
    Functors.jl
    Functors.jl
    Parameterise all the things
    ComponentArrays.jl
    ComponentArrays.jl
    Arrays with arbitrarily nested named components
    Serialization
    Serialization.jl
    Serialization.jl
    Provides serialization of Julia objects
    JLD2.jl
    JLD2.jl
    HDF5-compatible file format in pure Julia
    Testing Utilities
    FiniteDiff.jl
    FiniteDiff.jl
    Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support
    FiniteDifferences.jl
    FiniteDifferences.jl
    High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)
    JET.jl
    JET.jl
    JET employs Julia's type inference system to detect potential bugs and type instabilities
    LuxTestUtils.jl
    LuxTestUtils.jl
    Collection of Functions useful for testing various packages in the Lux Ecosystem
    Training Visualization & Logging
    MLFlowClient.jl
    MLFlowClient.jl
    Julia client for MLFlow
    TensorBoardLogger.jl
    TensorBoardLogger.jl
    Easy peasy logging to TensorBoard with Julia
    Wandb.jl
    Wandb.jl
    Unofficial Julia bindings for logging experiments to wandb.ai
    
-    
+    
    Lux.jl Docs
    Appearance
    Ecosystem
    Frameworks Extending Lux.jl
    DiffEqFlux.jl
    DiffEqFlux.jl
    Universal neural differential equations with O(1) backprop, GPUs, and stiff+non-stiff DE solvers, demonstrating scientific machine learning (SciML) and physics-informed machine learning methods
    SciMLSensitivity.jl
    SciMLSensitivity.jl
    A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.
    NeuralPDE.jl
    NeuralPDE.jl
    Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation
    NeuralLyapunov.jl
    NeuralLyapunov.jl
    A library for searching for neural Lyapunov functions in Julia
    DeepEquilibriumNetworks.jl
    DeepEquilibriumNetworks.jl
    Implicit Layer Machine Learning via Deep Equilibrium Networks, O(1) backpropagation with accelerated convergence
    AbstractCosmologicalEmulators.jl
    AbstractCosmologicalEmulators.jl
    Repository containing the abstract interface to the emulators used in the CosmologicalEmulators organization
    ContinuousNormalizingFlows.jl
    ContinuousNormalizingFlows.jl
    Implementations of Infinitesimal Continuous Normalizing Flows Algorithms in Julia
    Sophon.jl
    Sophon.jl
    Efficient, Accurate, and Streamlined Training of Physics-Informed Neural Networks
    DataDrivenDiffEq.jl
    DataDrivenDiffEq.jl
    Data driven modeling and automated discovery of dynamical systems for the SciML Scientific Machine Learning organization
    NeuralGraphPDE.jl
    NeuralGraphPDE.jl
    Integrating Neural Ordinary Differential Equations, the Method of Lines, and Graph Neural Networks
    Solaris.jl
    Solaris.jl
    Lightweight module for fusing physical and neural models
    Boltz.jl
    Boltz.jl
     Accelerate your ML research using pre-built Deep Learning Models with Lux
    GeometricMachineLearning.jl
    GeometricMachineLearning.jl
    Structure Preserving Machine Learning Models in Julia
    Automatic Differentiation
    Zygote.jl
    Zygote.jl
    Lux.jl default choice for AD
    Tracker.jl
    Tracker.jl
    Well tested and robust AD library (might fail on edge cases)
    ForwardDiff.jl
    ForwardDiff.jl
    For forward mode AD support
    ReverseDiff.jl
    ReverseDiff.jl
    Tape based reverse mode AD (might fail on edge cases and doesn't work on GPU)
    Enzyme.jl
    Enzyme.jl
    Experimental Support but will become the Future Default
    Data Manipulation, Data Loading & Datasets
    MLUtils.jl
    MLUtils.jl
    Utilities and abstractions for Machine Learning tasks
    MLDatasets.jl
    MLDatasets.jl
    Utility package for accessing common Machine Learning datasets in Julia
    Images.jl
    Images.jl
    An image library for Julia
    DataAugmentation.jl
    DataAugmentation.jl
    Flexible data augmentation library for machine and deep learning
    Neural Network Primitives
    NNlib.jl
    NNlib.jl
    Neural Network primitives with multiple backends
    LuxLib.jl
    LuxLib.jl
    Backend for Lux.jl
    Optimization
    Optimization.jl
    Optimization.jl
    Unified API for Optimization in Julia
    Optimisers.jl
    Optimisers.jl
    Optimisers.jl defines many standard optimisers and utilities for learning loops
    ParameterSchedulers.jl
    ParameterSchedulers.jl
    Common hyperparameter scheduling for ML
    Parameter Manipulation
    Functors.jl
    Functors.jl
    Parameterise all the things
    ComponentArrays.jl
    ComponentArrays.jl
    Arrays with arbitrarily nested named components
    Serialization
    Serialization.jl
    Serialization.jl
    Provides serialization of Julia objects
    JLD2.jl
    JLD2.jl
    HDF5-compatible file format in pure Julia
    Testing Utilities
    FiniteDiff.jl
    FiniteDiff.jl
    Fast non-allocating calculations of gradients, Jacobians, and Hessians with sparsity support
    FiniteDifferences.jl
    FiniteDifferences.jl
    High accuracy derivatives, estimated via numerical finite differences (formerly FDM.jl)
    JET.jl
    JET.jl
    JET employs Julia's type inference system to detect potential bugs and type instabilities
    LuxTestUtils.jl
    LuxTestUtils.jl
    Collection of Functions useful for testing various packages in the Lux Ecosystem
    Training Visualization & Logging
    MLFlowClient.jl
    MLFlowClient.jl
    Julia client for MLFlow
    TensorBoardLogger.jl
    TensorBoardLogger.jl
    Easy peasy logging to TensorBoard with Julia
    Wandb.jl
    Wandb.jl
    Unofficial Julia bindings for logging experiments to wandb.ai
    
+    
     
   
 
\ No newline at end of file
diff --git a/previews/PR1000/hashmap.json b/previews/PR1000/hashmap.json
index 52e530ee53..9071256a31 100644
--- a/previews/PR1000/hashmap.json
+++ b/previews/PR1000/hashmap.json
@@ -1 +1 @@
-{"api_accelerator_support_mldatadevices.md":"CGqLoJfj","api_building_blocks_luxcore.md":"Tk3NWXme","api_building_blocks_luxlib.md":"DM8CkpLX","api_building_blocks_weightinitializers.md":"jFCbAP4G","api_lux_autodiff.md":"65bMeWBg","api_lux_contrib.md":"Cn9lWBAP","api_lux_distributed_utils.md":"DrmUj9oe","api_lux_interop.md":"xen3KeOd","api_lux_layers.md":"sNH-X5ys","api_lux_utilities.md":"DwOGkpib","api_testing_functionality_luxtestutils.md":"D4Wj3KkE","ecosystem.md":"C5I9rkF4","index.md":"aWoZd1IO","introduction_citation.md":"DX_ZBrI9","introduction_index.md":"-gkQ_DqZ","introduction_overview.md":"SfZYUku2","introduction_resources.md":"PqTIqUoo","introduction_updating_to_v1.md":"Cai1nhhv","manual_autodiff.md":"eRV3gJsF","manual_compiling_lux_models.md":"DiLOBsEm","manual_debugging.md":"L4FxOj9s","manual_dispatch_custom_input.md":"BycrtfJi","manual_distributed_utils.md":"DhFcLeMe","manual_freezing_model_parameters.md":"DQ5B11bP","manual_gpu_management.md":"B_AqyWxs","manual_interface.md":"DuHSPsxb","manual_migrate_from_flux.md":"B2ZBcJwH","manual_nested_autodiff.md":"CDY_fLt0","manual_nn_inside_gpu_kernels.md":"B7FGJ-OT","manual_performance_pitfalls.md":"D_ooezpN","manual_preferences.md":"FoFJ7Snr","manual_weight_initializers.md":"DnHa7Oz9","tutorials_advanced_1_gravitationalwaveform.md":"DNi4Y6LY","tutorials_beginner_1_basics.md":"Bccn_xgd","tutorials_beginner_2_polynomialfitting.md":"22IrMX1R","tutorials_beginner_3_simplernn.md":"CQMmi5pL","tutorials_beginner_4_simplechains.md":"CL_BpnYq","tutorials_beginner_5_optimizationintegration.md":"DqNEeGSX","tutorials_index.md":"BIap521k","tutorials_intermediate_1_neuralode.md":"ByrCK2Fs","tutorials_intermediate_2_bayesiannn.md":"DS1q_HRo","tutorials_intermediate_3_hypernet.md":"C2PwwltN","tutorials_intermediate_4_pinn2dpde.md":"D4cij4Vv"}
+{"api_accelerator_support_mldatadevices.md":"BWXp7_hs","api_building_blocks_luxcore.md":"D2jtdUfi","api_building_blocks_luxlib.md":"VqNljgWV","api_building_blocks_weightinitializers.md":"C7zduFBi","api_lux_autodiff.md":"kxYwKcE-","api_lux_contrib.md":"DGQu-ivG","api_lux_distributed_utils.md":"DUxpyBki","api_lux_interop.md":"CfzA7PpN","api_lux_layers.md":"mQAmD5HD","api_lux_utilities.md":"C5yiud75","api_testing_functionality_luxtestutils.md":"DpA6j4Xc","ecosystem.md":"2XMefkkh","index.md":"BEFx1SYc","introduction_citation.md":"DX_ZBrI9","introduction_index.md":"-gkQ_DqZ","introduction_overview.md":"SfZYUku2","introduction_resources.md":"PqTIqUoo","introduction_updating_to_v1.md":"Cai1nhhv","manual_autodiff.md":"eRV3gJsF","manual_compiling_lux_models.md":"DiLOBsEm","manual_debugging.md":"Cd8vyuD9","manual_dispatch_custom_input.md":"BrY2DpLL","manual_distributed_utils.md":"DhFcLeMe","manual_freezing_model_parameters.md":"Cvglf3j-","manual_gpu_management.md":"B_AqyWxs","manual_interface.md":"DuHSPsxb","manual_migrate_from_flux.md":"CdtVRtI7","manual_nested_autodiff.md":"BHR1mjww","manual_nn_inside_gpu_kernels.md":"B7FGJ-OT","manual_performance_pitfalls.md":"D_ooezpN","manual_preferences.md":"FoFJ7Snr","manual_weight_initializers.md":"pgcyJMH_","tutorials_advanced_1_gravitationalwaveform.md":"Bfi8AeMz","tutorials_beginner_1_basics.md":"Bccn_xgd","tutorials_beginner_2_polynomialfitting.md":"22IrMX1R","tutorials_beginner_3_simplernn.md":"C0fFR2Ju","tutorials_beginner_4_simplechains.md":"4OROSnOm","tutorials_beginner_5_optimizationintegration.md":"BdxKZnAQ","tutorials_index.md":"BIap521k","tutorials_intermediate_1_neuralode.md":"BxeK2d6-","tutorials_intermediate_2_bayesiannn.md":"BKUcq4eS","tutorials_intermediate_3_hypernet.md":"BoHQxIel","tutorials_intermediate_4_pinn2dpde.md":"BZvwgXm0"}
diff --git a/previews/PR1000/index.html b/previews/PR1000/index.html
index b761cdd1db..41a493c3a2 100644
--- a/previews/PR1000/index.html
+++ b/previews/PR1000/index.html
@@ -5,14 +5,14 @@
     
     Lux.jl Docs
     
-    
+    
     
     
-    
+    
     
-    
+    
     
-    
+    
     
     
     
@@ -29,13 +29,13 @@
   
     
    Lux.jl Docs
    Appearance
    LuxDL Docs
    Elegant & Performant Scientific Machine Learning in JuliaLang
    A Pure Julia Deep Learning Framework designed for Scientific Machine Learning
    Tutorials
    Ecosystem
    API Reference 📚
    View on GitHub
    Lux.jl
    🚀
    Fast & Extendible
    Lux.jl is written in Julia itself, making it extremely extendible. CUDA and AMDGPU are supported first-class, with experimental support for Metal Hardware.
    🧑‍🔬
    SciML ❤️ Lux
    Lux is the default choice for many SciML packages, including DiffEqFlux.jl, NeuralPDE.jl, and more.
    🧩
    Uniquely Composable
    Lux.jl natively supports Arbitrary Parameter Types, making it uniquely composable with other Julia packages (and even Non-Julia packages).
    🧪
    Well Tested
    Lux.jl tests every supported Automatic Differentiation Framework with every supported hardware backend against Finite Differences to prevent sneaky 🐛 in your code.
    How to Install Lux.jl? ​
    Its easy to install Lux.jl. Since Lux.jl is registered in the Julia General registry, you can simply run the following command in the Julia REPL:
    julia
    julia> using Pkg
 julia> Pkg.add("Lux")
    If you want to use the latest unreleased version of Lux.jl, you can run the following command: (in most cases the released version will be same as the version on github)
    julia
    julia> using Pkg
-julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
+julia> Pkg.add(url="https://github.com/LuxDL/Lux.jl")
    Want GPU Support? ​
    Install the following package(s):
    julia
    using Pkg
 Pkg.add("LuxCUDA")
 # or
 Pkg.add(["CUDA", "cuDNN"])
    julia
    using Pkg
 Pkg.add("AMDGPU")
    julia
    using Pkg
 Pkg.add("Metal")
    julia
    using Pkg
-Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
+Pkg.add("oneAPI")
    Run the following to access a device:
    julia
    using Lux, LuxCUDA
 
 const dev = gpu_device()
    julia
    using Lux, AMDGPU
 
@@ -44,7 +44,7 @@
 const dev = gpu_device()
    julia
    using Lux, oneAPI
 
 const dev = gpu_device()
    Want XLA Support? ​
    Install the following package:
    julia
    using Pkg;
-Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
+Pkg.add("Reactant")
    Run the following to access a device:
    julia
    using Reactant, Lux
 Reactant.set_default_backend("cpu") # default
 
 const dev = xla_device()
    julia
    using Reactant, Lux
@@ -54,7 +54,7 @@
 Reactant.set_default_backend("tpu")
 
 const dev = xla_device()
    
-    
+    
     
   
 
\ No newline at end of file
diff --git a/previews/PR1000/introduction/citation.html b/previews/PR1000/introduction/citation.html
index ce631a2a76..590a8a5389 100644
--- a/previews/PR1000/introduction/citation.html
+++ b/previews/PR1000/introduction/citation.html
@@ -5,12 +5,12 @@
     
     Citation | Lux.jl Docs
     
-    
+    
     
     
-    
+    
     
-    
+    
     
     
     
@@ -43,7 +43,7 @@
   year      = {2023},
   school    = {Massachusetts Institute of Technology}
 }
    
-    
+    
     
   
 
\ No newline at end of file
diff --git a/previews/PR1000/introduction/index.html b/previews/PR1000/introduction/index.html
index 377d0c5fb2..9411265ea4 100644
--- a/previews/PR1000/introduction/index.html
+++ b/previews/PR1000/introduction/index.html
@@ -5,12 +5,12 @@
     
     Getting Started | Lux.jl Docs
     
-    
+    
     
     
-    
+    
     
-    
+    
     
     
     
@@ -138,7 +138,7 @@
 Iteration: 0801 	 Loss: 0.000239130808
 Iteration: 0901 	 Loss: 0.00140763051
 Iteration: 1000 	 Loss: 0.000214067404

    Training with Optimization.jl

    If you are coming from the SciML ecosystem and want to use Optimization.jl, please refer to the Optimization.jl Tutorial.

    Additional Packages ​

    LuxDL hosts various packages that provide additional functionality for Lux.jl. All packages mentioned in this documentation are available via the Julia General Registry.

    You can install all those packages via import Pkg; Pkg.add(<package name>).

    GPU Support ​

    GPU Support for Lux.jl requires loading additional packages:

    - + \ No newline at end of file diff --git a/previews/PR1000/introduction/overview.html b/previews/PR1000/introduction/overview.html index e82b600684..d2412b6187 100644 --- a/previews/PR1000/introduction/overview.html +++ b/previews/PR1000/introduction/overview.html @@ -5,12 +5,12 @@ Why we wrote Lux? | Lux.jl Docs - + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Why we wrote Lux? ​

    Julia already has quite a few well established Neural Network Frameworks – Flux & KNet. However, certain design elements – Coupled Model and Parameters & Internal Mutations – associated with these frameworks make them less compiler and user friendly. Making changes to address these problems in the respective frameworks would be too disruptive for users. Here comes in Lux: a neural network framework built completely using pure functions to make it both compiler and autodiff friendly.

    Design Principles ​

    • Layers must be immutable – cannot store any parameter/state but rather store the information to construct them

    • Layers are pure functions

    • Layers return a Tuple containing the result and the updated state

    • Given same inputs the outputs must be same – yes this must hold true even for stochastic functions. Randomness must be controlled using rngs passed in the state.

    • Easily extensible

    • Extensive Testing – All layers and features are tested across all supported AD backends across all supported hardware backends.

    Why use Lux over Flux? ​

    • Neural Networks for SciML: For SciML Applications (Neural ODEs, Deep Equilibrium Models) solvers typically expect a monolithic parameter vector. Flux enables this via its destructure mechanism, but destructure comes with various edge cases and limitations. Lux forces users to make an explicit distinction between state variables and parameter variables to avoid these issues. Also, it comes battery-included for distributed training.

    • Sensible display of Custom Layers – Ever wanted to see Pytorch like Network printouts or wondered how to extend the pretty printing of Flux's layers? Lux handles all of that by default.

    • Truly immutable models - No unexpected internal mutations since all layers are implemented as pure functions. All layers are also deterministic given the parameters and state: if a layer is supposed to be stochastic (say Dropout), the state must contain a seed which is then updated after the function call.

    • Easy Parameter Manipulation – By separating parameter data and layer structures, Lux makes implementing WeightNorm, SpectralNorm, etc. downright trivial. Without this separation, it is much harder to pass such parameters around without mutations which AD systems don't like.

    • Wider AD Support – Lux has extensive support for most AD systems in julia, while Flux is mostly tied to Zygote (with some initial support for Enzyme).

    • Small Neural Networks on CPU – Lux is developed for training large neural networks. For smaller architectures, we recommend using SimpleChains.jl or even better use it in conjunction with Lux via ToSimpleChainsAdaptor.

    • Reliability – We have learned from the mistakes of the past with Flux and everything in our core framework is extensively tested, along with downstream CI to ensure that everything works as expected.

    Revising Previous Recommendation about Large Models

    Previously we recommended not using Lux for very large models. But we have been making a lot of head-way with Reactant.jl and it would be worthwhile to test larger models with Lux. See compiling Lux models for more information.

    - + \ No newline at end of file diff --git a/previews/PR1000/introduction/resources.html b/previews/PR1000/introduction/resources.html index a599c249eb..7228d06183 100644 --- a/previews/PR1000/introduction/resources.html +++ b/previews/PR1000/introduction/resources.html @@ -5,12 +5,12 @@ Resources to Get Started | Lux.jl Docs - + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Resources to Get Started ​

    • Go through the Quickstart Example.

    • Read the introductory tutorials on Julia and Lux.

    • Go through the examples sorted based on their complexity in the documentation.

    Have More Questions?

    For usage related questions, please use Github Discussions which allows questions and answers to be indexed. To report bugs use Github Issues or even better send in a Pull Request.

    - + \ No newline at end of file diff --git a/previews/PR1000/introduction/updating_to_v1.html b/previews/PR1000/introduction/updating_to_v1.html index c2ba892517..1fc0b156c1 100644 --- a/previews/PR1000/introduction/updating_to_v1.html +++ b/previews/PR1000/introduction/updating_to_v1.html @@ -5,12 +5,12 @@ Updating to Lux v1 | Lux.jl Docs - + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Updating to Lux v1 ​

    Lux v1 is a Major Release, mostly to signify the stability of the API. In this page, we list out a concrete set of changes that need to be made to your code to update to Lux v1. We also list out some new exciting features that were added as part of this release.

    LuxLib.jl ​

    Breaking Changes ​

    • Old deprecated API with keyword arguments has been removed. See the new docs in LuxLib API for more details.

    • Default for layernorm dims has been changed to exclude the batch dimension.

    New Major Features ​

    • Dense layers now support CUDA backend for Enzyme (starting v1.1). Wider support for other operations with Enzyme + CUDA is being actively worked on.

    LuxCore.jl ​

    Breaking Changes ​

    • AbstractExplicitLayer has been renamed to AbstractLuxLayer.

    • AbstractExplicitContainerLayer behaviour

      • This has been renamed to AbstractLuxContainerLayer.

      • Previously, AbstractExplicitContainerLayer{(:a,)} (i.e. singleton containers) would produce default initial parameters and states without wrapping them in a NamedTuple{(:a,)}. This was inconsistent with non-singleton containers, and was a source of confusion. With v we return (; a = <parameters>) and (; a = <states>) by default. See AbstractLuxWrapperLayer for a replacement of this functionality.

    • inputsize has been removed since it was ambiguous and not used anywhere.

    • Changes to outputsize:

      • Single argument version has been removed. See LuxCore.jl Pull Request 43 for more details on the rationale behind this change.

      • Fallback implementation has been moved to Lux.jl. (i.e. users using Lux shouldn't see a difference, but if Lux.jl isn't loaded, this function has error.)

        • Internally this uses a NilArray that is able to compute sizes without actually running the computation.

    • Functors and Setfield have been made into optional dependencies. Certain LuxCore functionality that rely on these functions, will throw an error if these packages are not loaded.

    New Major Features ​

    • Introduction of AbstractLuxWrapperLayer. This behaves exactly like the old singleton container. For example, the old AbstractExplicitContainerLayer{(:a,)} is equivalent to AbstractLuxWrapperLayer{:a}.

    WeightInitializers.jl ​

    This was a major release to signify the stability of the API. There were no breaking changes. We do support a wider range of RNG types, see Supported RNG Types for more details.

    MLDataDevices.jl ​

    This is the most aggressive change that was made. We renamed the LuxDeviceUtils.jl package to MLDataDevices.jl, to allow for non-Lux packages to use this shared device management abstraction.

    Deprecation of LuxDeviceUtils.jl

    This also marks the deprecation of the LuxDeviceUtils.jl package. We won't be making any updates to that package, including fixing any bugs. All users should switch to MLDataDevices.jl instead.

    Breaking Changes ​

    • Lux(___)Device objects have been renamed to (___)Device. For example, LuxCUDADevice has been renamed to CUDADevice.

    • Lux(___)Adaptor objects have been removed. The corresponding Device objects should be used directly instead.

    New Major Features ​

    • DeviceIterator provides a generalization of CUDA.CuIterator and works for all backends and more data types (using Functors.jl). MLUtils.DataLoader |> gdev now returns a DeviceIterator instead of being a no-op.

    Lux.jl ​

    Breaking Changes (Removed Functionality) ​

    • Direct reexport of NNlib has been removed. We reexport selected functionality from NNlib. Direactly load NNlib if you need to use the other functions.

    • Flattening of Chain layers has been removed, and the corresponding disable_optimizations kwarg has been removed.

    • Some layers overloaded Base.keys, these have been removed. These were mostly un-documented and weren't supposed to be used outside of the Lux.jl package.

    • Training.TrainState construction with rng has been removed.

    • Older versions of Preferences have been removed.

    • disable_stacktrace_truncation! has been removed. From Julia 1.9 onwards, stacktrace truncation is enabled by default.

    • Certain Experimental features were present outside the Lux.Experimental module. These have been removed, use them via Lux.Experimental instead. Run Julia with with depwarn as error and Lux v0.5 to see the deprecations.

    • Lux.Experimental.@layer_map is not longer needed and has been removed. The name of the variable prevents writing generic functions and is no longer pre-pended to the KeyPath. See the docstring of Lux.Experimental.layer_map for more details.

    • allow_fast_activation kwarg has been removed completely. Pass an anonymous function as the activation to prevent internal modivations to the activation function.

    Breaking Changes (Moved Functionality) ​

    • Lux.Experimental.Training has been moved to Lux.Training. We guarantee SemVar on this new module.

    • Lux.cpu and Lux.gpu have been removed. Use cpu_device and gpu_device instead.

    • Experimental.@compact can be directly used via @compact now.

    • Experimental.StatefulLuxLayer has been moved to Lux.StatefulLuxLayer.

    • st_fixed_path kwarg has been removed from Lux.StatefulLuxLayer, instead use it as StatefulLuxLayer{st_fixed_path}(...).

    • Strings as inputs to Lux.Experimental.layer_map and Lux.Experimental.@debug_mode are removed, use Functors.KeyPath instead.

    • CrossCor has been removed. Use Conv(args...; kwargs..., cross_correlation=true) instead.

    Breaking Changes (Changes in Defaults) ​

    • Conv and ConvTranspose use an initialization based on the activation function, taken from Pytorch. Pytorch assumes the activation function is leakyrelu to compute the gain, however, we compute the gain based on the activation function passed in to the layer.

    • Upsample now has an align_corners keyword argument, which defaults to false. Previously this was always true.

    • Dense and Bilinear have updated default initializations to align with the defaults from Pytorch. See the documentation for more details.

    • InstanceNorm now defaults to affine=false instead of affine=true.

    • Embedding now defaults to init_weight=rand32 instead of init_weight=randn32.

    • Recurrent Cells - RNNCell, LSTMCell, and GRUCell now have different default initializations. See the documentation for more details.

    New Features ​

    • InstanceNorm now supports tracking statistics.

    • RNNCell and LSTMCell add bias_ih and bias_hh to the parameters to align with Pytorch. Both are controlled using init_bias and use_bias.

    • ConvTranspose allows flipkernel=true via cross_correlation=true. This makes it efficient for MIOpen.

    • ConvTranspose now has an outpad keyword argument, which is used to increase the size of the output in the desired dimensions.

    • Pooling Layers based on lpnorm have been added – LPPool, GlobalLPPool, and AdaptiveLPPool.

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/autodiff.html b/previews/PR1000/manual/autodiff.html index a7020d53d5..f2f277de57 100644 --- a/previews/PR1000/manual/autodiff.html +++ b/previews/PR1000/manual/autodiff.html @@ -5,12 +5,12 @@ Automatic Differentiation | Lux.jl Docs - + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Automatic Differentiation ​

    Lux is not an AD package, but it composes well with most of the AD packages available in the Julia ecosystem. This document lists the current level of support for various AD packages in Lux. Additionally, we provide some convenience functions for working with AD.

    Overview ​

    AD PackageModeCPUGPUNested 2nd Order ADSupport Class
    ChainRules.jl[1]Reverse✔️✔️✔️Tier I
    Enzyme.jlReverse✔️❓[2]❓[2:1]Tier I[3]
    Zygote.jlReverse✔️✔️✔️Tier I
    ForwardDiff.jlForward✔️✔️✔️Tier I
    ReverseDiff.jlReverse✔️❌❌Tier II
    Tracker.jlReverse✔️✔️❌Tier II
    Mooncake.jlReverse❓[2:2]❌❌Tier III
    Diffractor.jlForward❓[2:3]❓[2:4]❓[2:5]Tier III

    Recommendations ​

    • For CPU Usacases:

      1. Use Zygote.jl for the best performance. This is the most reliable and fastest option for CPU for the time-being. (We are working on faster Enzyme support for CPU)

      2. Use Enzyme.jl, if there are mutations in the code and/or Zygote.jl fails.

      3. If Enzyme.jl fails for some reason, (open an issue and) try ReverseDiff.jl (possibly with compiled mode).

    • For GPU Usacases:

      1. Use Zygote.jl for the best performance. This is the most reliable and fastest option for GPU for the time-being. We are working on supporting Enzyme.jl for GPU as well.

    Support Class ​

    1. Tier I: These packages are fully supported and have been tested extensively. Often have special rules to enhance performance. Issues for these backends take the highest priority.

    2. Tier II: These packages are supported and extensively tested but often don't have the best performance. Issues against these backends are less critical, but we fix them when possible. (Some specific edge cases, especially with AMDGPU, are known to fail here)

    3. Tier III: We don't know if these packages currently work with Lux. We'd love to add tests for these backends, but currently these are not our priority.

    Footnotes ​

    1. Note that ChainRules.jl is not really an AD package, but we have first-class support for packages that use rrules. ↩︎

    2. This feature is supported downstream, but we don't extensively test it to ensure that it works with Lux. ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

    3. Currently Enzyme outperforms other AD packages in terms of CPU performance. However, there are some edge cases where it might not work with Lux. We are working on improving the compatibility. Please report any issues you encounter. ↩︎

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/compiling_lux_models.html b/previews/PR1000/manual/compiling_lux_models.html index d201cc8b3b..7726a1a339 100644 --- a/previews/PR1000/manual/compiling_lux_models.html +++ b/previews/PR1000/manual/compiling_lux_models.html @@ -5,12 +5,12 @@ Compiling Lux Models using Reactant.jl | Lux.jl Docs - + - + - + @@ -102,7 +102,7 @@ Iter: [ 800/1000] Loss: 0.03070261 Iter: [ 900/1000] Loss: 0.02143306 Iter: [1000/1000] Loss: 0.01542492 - + \ No newline at end of file diff --git a/previews/PR1000/manual/debugging.html b/previews/PR1000/manual/debugging.html index 956a88043c..848f0959d0 100644 --- a/previews/PR1000/manual/debugging.html +++ b/previews/PR1000/manual/debugging.html @@ -5,14 +5,14 @@ Debugging Lux Models | Lux.jl Docs - + - + - + - + @@ -69,7 +69,7 @@ [ Info: Input Type: Matrix{Float32} | Input Structure: (3, 2). [ Info: Running Layer: Dense(1 => 1) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2)! ┌ Error: Layer Dense(1 => 1) failed!! This layer is present at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2). -└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-10/julialang/lux-dot-jl/src/contrib/debug.jl:102 +└ @ Lux.Experimental /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/src/contrib/debug.jl:102 DimensionMismatch("A has shape (1, 1) but B has shape (3, 2)")

    See now we know that model.layers.layer_2.layers.layer_2 is the problematic layer. Let us fix that layer and see what happens:

    julia
    model = Chain(Dense(1 => 16, relu),
     Chain(
         Dense(16 => 3),  
@@ -147,7 +147,7 @@
 [ Info: Running Layer: BatchNorm(1, affine=true, track_stats=true) at location KeyPath(:model, :layers, :layer_3)!
 [ Info: Output Type: Matrix{Float32} | Output Structure: (1, 2).
 DomainError(Float32[NaN 0.0], "NaNs detected in pullback output (x)  of layer WrappedFunction(offending_layer) at location KeyPath(:model, :layers, :layer_2, :layers, :layer_2).")

    And there you go our debug layer prints that the problem is in WrappedFunction(offending_layer) at location model.layers.layer_2.layers.layer_2! Once we fix the pullback of the layer, we will fix the NaNs.

    Conclusion ​

    In this manual section, we have discussed tracking down errors in Lux models. We have covered tracking incorrect model specifications and NaNs in forward and backward passes. However, remember that this is an Experimental feature, and there might be edge cases that don't work correctly. If you find any such cases, please open an issue on GitHub!

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/dispatch_custom_input.html b/previews/PR1000/manual/dispatch_custom_input.html index 7c732ddaf0..6d9a967297 100644 --- a/previews/PR1000/manual/dispatch_custom_input.html +++ b/previews/PR1000/manual/dispatch_custom_input.html @@ -5,14 +5,14 @@ Dispatching on Custom Input Types | Lux.jl Docs - + - + - + - + @@ -65,11 +65,11 @@ Closest candidates are: apply(!Matched::AbstractLuxLayer, ::Any, ::Any, ::Any) - @ LuxCore ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/src/LuxCore.jl:154 + @ LuxCore /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/src/LuxCore.jl:154 apply(!Matched::AbstractLuxLayer, !Matched::Tracker.TrackedArray, ::Any, ::Any) - @ LuxCoreArrayInterfaceTrackerExt ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/ext/LuxCoreArrayInterfaceTrackerExt.jl:19 + @ LuxCoreArrayInterfaceTrackerExt /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/ext/LuxCoreArrayInterfaceTrackerExt.jl:19 apply(!Matched::AbstractLuxLayer, !Matched::ReverseDiff.TrackedArray, ::Any, ::Any) - @ LuxCoreArrayInterfaceReverseDiffExt ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxCore/IBKvY/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:20 + @ LuxCoreArrayInterfaceReverseDiffExt /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxCore/ext/LuxCoreArrayInterfaceReverseDiffExt.jl:20 ...

    Writing the Correct Dispatch Rules ​

    • Create a Custom Layer storing the time.
    julia
    struct ArrayAndTime{A <: AbstractArray, T <: Real}
     array::A
     time::T
@@ -88,7 +88,7 @@
 ps, st = Lux.setup(rng, model)
 
 model(xt, ps, st)[1]
    Main.ArrayAndTime{Matrix{Float32}, Float32}(Float32[0.33758628 -0.079208925], 10.0f0)
    - + \ No newline at end of file diff --git a/previews/PR1000/manual/distributed_utils.html b/previews/PR1000/manual/distributed_utils.html index c5dfb0a3c6..b23ac3d695 100644 --- a/previews/PR1000/manual/distributed_utils.html +++ b/previews/PR1000/manual/distributed_utils.html @@ -5,12 +5,12 @@ Distributed Data Parallel Training | Lux.jl Docs - + - + - + @@ -31,7 +31,7 @@ st = DistributedUtils.synchronize!!(backend, st)
    julia
    data = DistributedUtils.DistributedDataContainer(backend, data)
    • Wrap the optimizer in DistributedUtils.DistributedOptimizer to ensure that the optimizer is correctly synchronized across all processes before parameter updates. After initializing the state of the optimizer, synchronize the state across all processes.
    julia
    opt = DistributedUtils.DistributedOptimizer(backend, opt)
 opt_state = Optimisers.setup(opt, ps)
 opt_state = DistributedUtils.synchronize!!(backend, opt_state)
    • Finally change all logging and serialization code to trigger on local_rank(backend) == 0. This ensures that only the master process logs and serializes the model.

    Migration Guide from FluxMPI.jl ​

    Let's compare the changes we need to make wrt the FluxMPI.jl integration guide.

    1. FluxMPI.Init is now DistributedUtils.initialize.

    2. FluxMPI.synchronize!(x) needs to be changed to x_new = DistributedUtils.synchronize!!(backend, x).

    3. DistributedUtils.DistributedDataContainer, DistributedUtils.local_rank, and DistributedUtils.DistributedOptimizer need backend as the first input.

    And that's pretty much it!

    Removed Functionality ​

    1. FluxMPI.allreduce_gradients no longer exists. Previously this was needed when CUDA communication was flaky, with NCCL.jl this is no longer the case.

    2. FluxMPIFluxModel has been removed. DistributedUtils no longer works with Flux.

    Key Differences ​

    1. FluxMPI.synchronize! is now DistributedUtils.synchronize!! to highlight the fact that some of the inputs are not updated in-place.

    2. All of the functions now require a communication backend as input.

    3. We don't automatically determine if the MPI Implementation is CUDA or ROCM aware. See GPU-aware MPI for more information.

    4. Older (now non-existent) Lux.gpu implementations used to "just work" with FluxMPI.jl. We expect gpu_device to continue working as expected, however, we recommend using gpu_device after calling DistributedUtils.initialize to avoid any mismatch between the device set via DistributedUtils and the device stores in CUDADevice or AMDGPUDevice.

    Known Shortcomings ​

    1. Currently we don't run tests with CUDA or ROCM aware MPI, use those features at your own risk. We are working on adding tests for these features.

    2. AMDGPU support is mostly experimental and causes deadlocks in certain situations, this is being investigated. If you have a minimal reproducer for this, please open an issue.

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/freezing_model_parameters.html b/previews/PR1000/manual/freezing_model_parameters.html index bcddef9e02..dc0b0c37e0 100644 --- a/previews/PR1000/manual/freezing_model_parameters.html +++ b/previews/PR1000/manual/freezing_model_parameters.html @@ -5,14 +5,14 @@ Freezing Model Parameters | Lux.jl Docs - + - + - + - + @@ -46,7 +46,7 @@ model_frozen, ps_frozen, st_frozen = Lux.Experimental.layer_map(freeze_dense, model, ps, st) -model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
+model_frozen(x, ps_frozen, st_frozen)
    (Float32[0.6886741 -1.2361472], (layer_1 = (frozen_params = (weight = Float32[-0.028461456 -0.5999714 -0.3850993; -0.18860114 0.72428167 0.32322538; -0.965117 -0.4585489 -0.32623518; -0.86290836 -0.82805836 -0.7673453], bias = Float32[0.4216236, -0.4510427, -0.097253, 0.23325463]), states = NamedTuple()), layer_2 = (layer_1 = (frozen_params = (weight = Float32[-0.680748 0.1764085 0.34383082 0.6469914; -0.13819042 -0.109261915 -0.6143286 -0.21672015; -0.20881107 0.70390546 0.48137343 0.25662464; 0.38187847 0.05779423 -0.35181466 -0.096988946], bias = Float32[0.41246277, 0.4318977, -0.4305781, 0.3367505]), states = NamedTuple()), layer_2 = (rng = Random.Xoshiro(0x4fa3403dd074e603, 0x12c522b8034ae186, 0x8e0c3a65079041bb, 0x21617f7747d97206, 0x22a21880af5dc689), training = Val{true}()), layer_3 = (running_mean = Float32[0.01965834, 0.0, 0.0, 0.015937408], running_var = Float32[0.90772897, 0.9, 0.9, 0.90508], training = Val{true}())), layer_3 = (frozen_params = (weight = Float32[0.7794657 0.8337032 0.6323408 -0.18308182], bias = Float32[-0.27373654]), states = NamedTuple())))

    Freezing by Layer Name ​

    When the function in layer_map is called, the 4th argument is the name of the layer. For example, if you want to freeze the 1st layer inside the inner Chain. The name for this would be layer_2.layer_1.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
@@ -55,7 +55,7 @@
 function freeze_dense(d::Dense, ps, st, _)
     return Lux.Experimental.freeze(d, ps, st, (:weight, :bias))
 end
-freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
+freeze_dense(l, ps, st, _) = (l, ps, st)

    Freezing Part of the Parameters ​

    Instead of freezing all the parameters, we can simply specify (:weight,) to freeze only the weight parameter while training the bias parameter.

    julia
    
 function freeze_by_name(d, ps, st, name::KeyPath)
     name == KeyPath(:layer_2, :layer_1) &&
         return Lux.Experimental.freeze(d, ps, st, (:weight,))
@@ -78,7 +78,7 @@
 x = randn(rng, Float32, 3, 2)
 
 model_frozen(x, ps, st)
    (Float32[0.7429947 -1.2904677], (layer_1 = (layer_1 = NamedTuple(), layer_2 = NamedTuple()), layer_2 = (frozen_params = (layer_3 = NamedTuple(), layer_4 = (scale = Float32[1.0, 1.0, 1.0, 1.0], bias = Float32[0.0, 0.0, 0.0, 0.0])), states = (layer_3 = (rng = Random.TaskLocalRNG(), training = Val{true}()), layer_4 = (running_mean = Float32[0.0, 0.048522998, 0.0, 0.015937408], running_var = Float32[0.9, 0.9470896, 0.9, 0.90508], training = Val{true}()))), layer_3 = NamedTuple()))
    - + \ No newline at end of file diff --git a/previews/PR1000/manual/gpu_management.html b/previews/PR1000/manual/gpu_management.html index e72b4d118f..faf0220241 100644 --- a/previews/PR1000/manual/gpu_management.html +++ b/previews/PR1000/manual/gpu_management.html @@ -5,12 +5,12 @@ GPU Management | Lux.jl Docs - + - + - + @@ -54,7 +54,7 @@ end (x_gpu |> cdev) ≈ x_cpu
    true
    - + \ No newline at end of file diff --git a/previews/PR1000/manual/interface.html b/previews/PR1000/manual/interface.html index 616720fa19..5d13d3d7db 100644 --- a/previews/PR1000/manual/interface.html +++ b/previews/PR1000/manual/interface.html @@ -5,12 +5,12 @@ Lux Interface | Lux.jl Docs - + - + - + @@ -118,7 +118,7 @@ ps = DenseLayerParameters(ps_default.weight, ps_default.bias) println("Result with `DenseLayerParameters` parameters: ", first(d(x, ps, st)))
    Result with `DenseLayerParameters` parameters: Float32[0.23710957; 0.1003911; -0.57671577;;]

    The takeaway from this shouldn't be – lets define weird parameter types. Simply because you can do weird things like this doesn't mean you should, since it only leads to bugs.

    Instead this shows the flexibility you have for how your parameters can be structured.

    State Interface ​

    States are always type constrained to be NamedTuple. The structure of the input state must match that of the output state, i.e. keys(st_in) == keys(st_out). This doesn't imply that types of the input and output state match. To generate efficient code, we often do dispatch on the state, for example, Dropout, BatchNorm, etc.

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/migrate_from_flux.html b/previews/PR1000/manual/migrate_from_flux.html index 4b0203b1c9..014f301e1c 100644 --- a/previews/PR1000/manual/migrate_from_flux.html +++ b/previews/PR1000/manual/migrate_from_flux.html @@ -5,14 +5,14 @@ Migrating from Flux to Lux | Lux.jl Docs - + - + - + - + @@ -27,7 +27,7 @@ -
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Migrating from Flux to Lux ​

    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.

    julia
    using Lux, Random, NNlib, Zygote
+    
    Lux.jl Docs
    Appearance

     Trusted by 
    Scientific Computing
    SciML.ai
    Machine Learning
    Migrating from Flux to Lux ​
    For the core library layers like Dense, Conv, etc. we have intentionally kept the API very similar to Flux. In most cases, replacing using Flux with using Lux should be enough to get you started. We cover the additional changes that you will have to make in the following example.
    julia
    using Lux, Random, NNlib, Zygote
 
 model = Chain(Dense(2 => 4), BatchNorm(4, relu), Dense(4 => 2))
 rng = Random.default_rng()
@@ -47,7 +47,7 @@
 
 model(x)
 
-gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes A×B×x where A is not trainable and B is trainable.
    julia
    using Lux, Random, NNlib, Zygote
+gradient(model -> sum(model(x)), model)
    Implementing Custom Layers ​
    Flux and Lux operate under extremely different design philosophies regarding how layers should be implemented. A summary of the differences would be:
    • Flux stores everything in a single struct and relies on Functors.@functor and Flux.trainable to distinguish between trainable and non-trainable parameters.
    • Lux relies on the user to define Lux.initialparameters and Lux.initialstates to distinguish between trainable parameters (called "parameters") and non-trainable parameters (called "states"). Additionally, Lux layers define the model architecture, hence device transfer utilities like gpu_device, cpu_device, etc. cannot be applied on Lux layers, instead they need to be applied on the parameters and states.
    Let's work through a concrete example to demonstrate this. We will implement a very simple layer that computes A×B×x where A is not trainable and B is trainable.
    julia
    using Lux, Random, NNlib, Zygote
 
 struct LuxLinear <: Lux.AbstractLuxLayer
     init_A
@@ -85,7 +85,7 @@
 # Needed so that both `A` and `B` can be transferred between devices
 Flux.@functor FluxLinear
 
-(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
+(l::FluxLinear)(x) = l.A * l.B * x
    Now let us run the model.
    julia
    rng = Random.default_rng()
 model = LuxLinear(randn(rng, 2, 4), randn(rng, 4, 2))
 x = randn(rng, 2, 1)
 
@@ -102,7 +102,7 @@
 model(x)
 
 gradient(model -> sum(model(x)), model)
    To reiterate some important points:
    • Don't store mutables like Arrays inside a Lux Layer.
    • Parameters and States should be constructured inside the respective initial* functions.
    Certain Important Implementation Details ​
    Training/Inference Mode ​
    Flux supports a mode called :auto which automatically decides if the user is training the model or running inference. This is the default mode for Flux.BatchNorm, Flux.GroupNorm, Flux.Dropout, etc. Lux doesn't support this mode (specifically to keep code simple and do exactly what the user wants), hence our default mode is training. This can be changed using Lux.testmode.
    Can we still use Flux Layers? ​
    If you have Flux loaded in your code, you can use the function FromFluxAdaptor to automatically convert your model to Lux. Note that in case a native Lux counterpart isn't available, we fallback to using Optimisers.destructure.
    
-    
+    
     
   
 
\ No newline at end of file
diff --git a/previews/PR1000/manual/nested_autodiff.html b/previews/PR1000/manual/nested_autodiff.html
index e71c95ae9e..3bfc75b297 100644
--- a/previews/PR1000/manual/nested_autodiff.html
+++ b/previews/PR1000/manual/nested_autodiff.html
@@ -5,14 +5,14 @@
     
     Nested Automatic Differentiation | Lux.jl Docs
     
-    
+    
     
     
-    
+    
     
-    
+    
     
-    
+    
     
     
     
@@ -51,9 +51,9 @@
 
 println("∞-norm(∂x - ∂x_fd): ", norm(∂x .- ∂x_fd, Inf))
 println("∞-norm(∂ps - ∂ps_fd): ", norm(ComponentArray(∂ps) .- ∂ps_fd, Inf))
    ┌ Warning: `training` is set to `Val{true}()` but is not being used within an autodiff call (gradient, jacobian, etc...). This will be slow. If you are using a `Lux.jl` model, set it to inference (test) mode using `LuxCore.testmode`. Reliance on this behavior is discouraged, and is not guaranteed by Semantic Versioning, and might be removed without a deprecation cycle. It is recommended to fix this issue in your code.
-└ @ LuxLib.Utils ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxLib/I9RHW/src/utils.jl:309
+└ @ LuxLib.Utils /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxLib/src/utils.jl:309
 ┌ Warning: `training` is set to `Val{true}()` but is not being used within an autodiff call (gradient, jacobian, etc...). This will be slow. If you are using a `Lux.jl` model, set it to inference (test) mode using `LuxCore.testmode`. Reliance on this behavior is discouraged, and is not guaranteed by Semantic Versioning, and might be removed without a deprecation cycle. It is recommended to fix this issue in your code.
-└ @ LuxLib.Utils ~/.cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6/packages/LuxLib/I9RHW/src/utils.jl:309
+└ @ LuxLib.Utils /var/lib/buildkite-agent/builds/gpuci-6/julialang/lux-dot-jl/lib/LuxLib/src/utils.jl:309
 ∞-norm(∂x - ∂x_fd): 0.00046014786
 ∞-norm(∂ps - ∂ps_fd): 0.00068473816

    That's pretty good, of course you will have some error from the finite differences calculation.

    Using Batched Jacobian for Multiple Inputs ​

    Notice that in this example the Jacobian J consists on the full matrix of derivatives of smodel with respect the different inputs in x. In many cases, we are interested in computing the Jacobian with respect to each input individually, avoiding the unnecessary calculation of zero entries of the Jacobian. This can be achieved with batched_jacobian to parse the calculation of the Jacobian per each single input. Using the same example from the previous section:

    julia
    model = Chain(Dense(2 => 4, tanh), Dense(4 => 2))
 ps, st = Lux.setup(StableRNG(0), model)
@@ -146,7 +146,7 @@
 ∞-norm(∂ps using vjp): 0.0
 ∞-norm(∂x using full jacobian): 9.536743e-7
 ∞-norm(∂ps using full jacobian): 1.4305115e-6
    - + \ No newline at end of file diff --git a/previews/PR1000/manual/nn_inside_gpu_kernels.html b/previews/PR1000/manual/nn_inside_gpu_kernels.html index 465c09721b..28e96797ac 100644 --- a/previews/PR1000/manual/nn_inside_gpu_kernels.html +++ b/previews/PR1000/manual/nn_inside_gpu_kernels.html @@ -5,12 +5,12 @@ Neural Networks Inside GPU Kernels | Lux.jl Docs - + - + - + @@ -130,7 +130,7 @@ [2.390200114697191; 0.5267549745189349; -1.657670184695808; -1.7089496198123055;;] [2.1846486482317626; -0.031414255389526885; -1.3279041356366077; -1.6909446526419574;;] [1.3650193059617517; 0.5210742834996898; -0.7689272356710357; -0.6642563709240284;;] - + \ No newline at end of file diff --git a/previews/PR1000/manual/performance_pitfalls.html b/previews/PR1000/manual/performance_pitfalls.html index a5ab52e79c..0c0837d698 100644 --- a/previews/PR1000/manual/performance_pitfalls.html +++ b/previews/PR1000/manual/performance_pitfalls.html @@ -5,12 +5,12 @@ Performance Pitfalls & How to Catch Them | Lux.jl Docs - + - + - + @@ -55,7 +55,7 @@ # do some computation # ... end

    Here, X and y are on the gpu device gdev and the data transfer happens in the worker processes. Additionally, it behaves similar to CuIterator from CUDA.jl and eagerly frees the data after every iteration (this is device agnostic and works on all supported GPU backends).

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/preferences.html b/previews/PR1000/manual/preferences.html index b6cf6c1043..f489a763ed 100644 --- a/previews/PR1000/manual/preferences.html +++ b/previews/PR1000/manual/preferences.html @@ -5,12 +5,12 @@ Preferences for Lux.jl | Lux.jl Docs - + - + - + @@ -30,7 +30,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Preferences for Lux.jl ​

    How to set Preferences

    PreferenceTools.jl provides an interactive way to set preferences. First run the following command:

    julia
    using PreferenceTools

    Then in the pkg mode (press ] in the REPL), run the following command:

    julia
    pkg> preference add Lux <preference-name>=<value>
 pkg> preference add LuxLib <preference-name>=<value>
 pkg> preference add LuxCore <preference-name>=<value>

    Lux.jl relies on several preferences to make decision on how to run your code. Here is an exhaustive list of preferences that Lux.jl uses.

    Nested Automatic Differentiation ​

    1. automatic_nested_ad_switching - Set this to false to disable automatic switching of backends for nested automatic differentiation. See the manual section on nested automatic differentiation for more details.

    GPU-Aware MPI Support ​

    If you are using a custom MPI build that supports CUDA or ROCM, you can use the following preferences with Preferences.jl:

    1. cuda_aware_mpi - Set this to true if your MPI build is CUDA aware.

    2. rocm_aware_mpi - Set this to true if your MPI build is ROCM aware.

    By default, both of these preferences are set to false.

    GPU Backend Selection ​

    1. gpu_backend - Set this to bypass the automatic backend selection and use a specific gpu backend. Valid options are "cuda", "rocm", "metal", and "oneapi". This preference needs to be set for MLDataDevices package. It is recommended to use MLDataDevices.gpu_backend! to set this preference.

    Automatic Eltype Conversion ​

    1. eltype_mismatch_handling - Preference controlling what happens when layers get different eltypes as input. See the documentation on match_eltype for more details.

    Dispatch Doctor ​

    1. instability_check - Preference controlling the dispatch doctor. See the documentation on Lux.set_dispatch_doctor_preferences! for more details. The preferences need to be set for LuxCore and LuxLib packages. Both of them default to disable.

    • Setting the LuxCore preference sets the check at the level of LuxCore.apply. This essentially activates the dispatch doctor for all Lux layers.

    • Setting the LuxLib preference sets the check at the level of functional layer of Lux, for example, fused_dense_bias_activation. These functions are supposed to be type stable for common input types and can be used to guarantee type stability.

    Disabling Loop Vectorization / Octavian ​

    LoopVectorization.jl and Octavian.jl are optional dependencies that are used to accelerate certain CPU operations. However, these packages are tightly coupled with julia and might not work with all julia versions and systems. If these packages are loaded in any form LuxLib will use the optimized versions of the functions. But it might be desirable to disable these packages and use the default implementations instead. This can be done by setting the disable_loop_vectorization preference to true for LuxLib.

    - + \ No newline at end of file diff --git a/previews/PR1000/manual/weight_initializers.html b/previews/PR1000/manual/weight_initializers.html index b926b0d1c3..8d0b6ed4d9 100644 --- a/previews/PR1000/manual/weight_initializers.html +++ b/previews/PR1000/manual/weight_initializers.html @@ -5,14 +5,14 @@ Initializing Weights | Lux.jl Docs - + - + - + - + @@ -47,17 +47,17 @@ 0.486214 0.321506 -0.306641 0.145296 0.206476

    To generate weights directly on GPU, pass in a CUDA.RNG. For a complete list of supported RNG types, see Supported RNG Types.

    julia
    using LuxCUDA
 
 weights = kaiming_normal(CUDA.default_rng(), 2, 5)
    2×5 CuArray{Float32, 2, CUDA.DeviceMemory}:
-  0.108558   0.944269  -0.0671347   0.509523   0.00845341
- -0.279856  -0.507971  -0.178438   -0.112973  -1.44735

    You can also generate Complex Numbers:

    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
- 0.542428-1.17063im   -0.586636+0.809742im  …  -0.599194-0.446424im
- 0.407158+0.421916im   -1.24735+1.2547im        -0.71315+0.916298im

    Quick examples ​

    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.

    julia
    weights = init(rng, dims...)

    The rng is optional, if not specified a default one will be used.

    julia
    weights = init(dims...)

    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.

    julia
    weights_init = init(rng; kwargs...)
+ -1.24304    1.00968   -0.756944   -0.509345  -0.343497
+ -0.335454  -0.655999  -0.0236516   0.697709   0.9543

    You can also generate Complex Numbers:

    julia
    weights = kaiming_normal(CUDA.default_rng(), ComplexF32, 2, 5)
    2×5 CuArray{ComplexF32, 2, CUDA.DeviceMemory}:
+  1.51066-0.410399im    0.148313+0.795943im  …  -0.266121+0.532213im
+ 0.548101+0.283859im  -0.0162294-0.608428im      0.964663+0.16569im

    Quick examples ​

    The package is meant to be working with deep learning libraries such as (F)Lux. All the methods take as input the chosen rng type and the dimension for the array.

    julia
    weights = init(rng, dims...)

    The rng is optional, if not specified a default one will be used.

    julia
    weights = init(dims...)

    If there is the need to use keyword arguments the methods can be called with just the rng (optionally) and the keywords to get in return a function behaving like the two examples above.

    julia
    weights_init = init(rng; kwargs...)
 weights = weights_init(rng, dims...)
 
 # Or
 
 weights_init = init(; kwargs...)
 weights = weights_init(dims...)
    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm.html b/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm.html index 7dc3599383..a9be2d02f3 100644 --- a/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm.html +++ b/previews/PR1000/tutorials/advanced/1_GravitationalWaveForm.html @@ -5,14 +5,14 @@ Training a Neural ODE to Model Gravitational Waveforms | Lux.jl Docs - + - + - + - + @@ -184,7 +184,7 @@ Dense(1 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32), Dense(32 => 32, cos; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32), Dense(32 => 2; init_weight=truncated_normal(; std=1e-4), init_bias=zeros32)) -ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[-8.733466f-6; -0.00010797254; -6.034535f-6; 7.889705f-5; -1.9253392f-5; -4.612653f-5; 7.144778f-5; -8.932017f-5; -4.7168105f-5; 9.798891f-5; -2.1554477f-5; 4.7397516f-5; 0.00016439424; -4.36064f-5; -6.190079f-5; 6.240796f-5; 0.000105714986; 4.7386144f-5; 2.5381063f-5; 4.9141305f-5; 0.00016239625; 0.00013147807; -0.00017642805; -0.00012611185; 3.0797285f-6; -3.7879247f-5; -1.162709f-6; -0.00016794122; 6.32018f-5; 0.00012728853; 9.064739f-5; -1.5429378f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-2.8177601f-5 -3.936497f-5 -7.657458f-5 0.00013857176 -6.853039f-7 -5.7781497f-5 -5.1510856f-6 3.9836163f-5 0.00012864341 -6.5123946f-5 5.465938f-5 -6.377014f-5 7.9869205f-5 -9.11579f-5 -5.0697596f-5 -1.4907303f-6 -3.207327f-5 -6.482217f-5 2.0175543f-5 9.1325936f-5 1.7174554f-5 6.39053f-5 -4.3893502f-5 -0.00011969867 -6.29855f-5 1.9702458f-5 -7.764152f-6 1.9585927f-6 7.7942015f-5 0.00010650995 -8.729355f-5 -3.0301662f-5; 5.0932027f-5 -3.3675303f-6 0.00010612095 -9.0758826f-5 -6.206741f-5 2.0948732f-5 6.1964456f-5 -8.0493766f-5 9.118536f-5 9.570129f-5 -0.00025590206 -8.4574596f-5 0.00011611364 0.00015786408 -0.0001325249 -1.472367f-5 2.512089f-6 5.8462672f-5 5.2352f-5 -4.69843f-5 -0.00024784482 -4.0580584f-5 -3.3547942f-6 7.7080156f-5 -5.3825785f-5 -0.00012691545 -0.00018019084 3.0379057f-5 -1.2276903f-5 -0.00022681341 0.00017366803 -3.039549f-6; 0.00010137846 -0.0001010016 -2.4273795f-6 -4.770885f-6 6.701655f-5 0.00017136613 -0.0001906262 -1.0672023f-6 0.00012427552 -3.5586767f-5 8.55602f-5 -8.816304f-7 -4.3871598f-5 0.0001563755 -2.259477f-5 4.3149896f-5 -0.00011615122 -1.5891324f-5 -7.964024f-5 1.08097465f-5 3.3177996f-5 0.0001521374 -0.00012456263 4.1607735f-5 -1.4163393f-5 3.9037423f-5 -4.4479628f-5 -3.5756973f-5 7.252298f-5 -0.00011933267 -7.04208f-5 -5.54682f-5; -8.33601f-5 0.00011739388 -1.6886734f-5 -9.9290606f-5 0.00018311598 0.00016209438 -0.00012484103 1.6862592f-5 -0.00012521839 -2.3408023f-5 9.5157055f-5 -6.6844086f-6 5.5988305f-5 0.00012589077 -3.7646063f-5 -0.00011733096 5.898576f-5 -3.2169828f-5 0.00016092892 -0.000102152466 -0.00013990636 -0.00021372396 0.00013839726 -2.180175f-5 -5.539738f-6 0.00017145288 -6.4497464f-5 0.00010669653 2.4381146f-5 -8.664719f-5 -0.00010137241 -0.00010135833; 6.767996f-6 7.979176f-5 0.00010768214 4.6958052f-5 8.351189f-5 5.7730363f-6 -7.494067f-5 2.7757571f-5 -5.5647397f-5 4.7348538f-5 6.738859f-5 -1.7598553f-5 6.2236526f-5 -0.00010106445 6.811721f-5 -1.7854622f-5 0.00014259979 -3.1129297f-5 2.758307f-5 -0.00021257819 -2.0857597f-5 -0.00015710465 0.000115748146 -9.06086f-5 0.00012642707 -6.715701f-5 -2.3087605f-5 0.00014910278 0.00017517559 0.00012419421 -2.7270444f-5 0.00010731906; 5.8766418f-5 -0.00015708971 2.6431195f-5 -0.000103879465 2.2029622f-5 4.658656f-5 -0.00014493923 7.2192146f-5 -2.4574167f-5 0.00013603218 0.00013269106 8.4467785f-5 9.409519f-5 -0.00016357354 1.9661726f-5 0.0001598959 -0.00014350242 -1.5808671f-5 1.605866f-5 -3.59838f-5 1.479661f-5 1.8547255f-5 4.254646f-5 -9.158475f-5 -0.00013229327 -1.3919468f-5 -1.956731f-5 -5.49026f-5 -3.0198875f-5 3.3093645f-5 0.00019555296 9.743678f-5; 6.583076f-5 -1.9019628f-5 -2.3915383f-5 -8.268265f-5 8.264759f-5 0.000113464965 0.00019279975 5.6251043f-5 -3.9762614f-5 -8.271291f-5 6.1018898f-5 -7.841602f-5 -1.2949409f-5 0.00013066275 -1.5204919f-5 -0.00012379164 4.110425f-5 -6.5780056f-5 -1.7211657f-7 -4.4427048f-5 -1.1617677f-6 -0.000109266395 -0.00018221501 -0.00013746502 0.0002607908 -1.48712015f-5 1.9145657f-5 -6.3632055f-5 -3.2119045f-5 -0.00013502994 -5.4510227f-5 -0.0001521188; 1.6407535f-5 -1.9113382f-5 0.00012204645 -7.556568f-5 -1.6685279f-5 -1.5900963f-5 6.0559876f-5 -1.5047506f-5 -8.193592f-5 -2.9995304f-5 -1.0388885f-5 -3.103509f-5 -0.00017250188 3.3705473f-5 -4.3459997f-5 -4.783416f-5 2.276422f-6 5.006527f-6 -0.00015602457 2.369959f-5 3.2234122f-5 -0.00012310114 -2.8645542f-5 5.168618f-5 0.00017521593 -3.8919454f-5 0.00016759252 4.4987402f-5 2.9359122f-5 -9.714238f-5 -8.786319f-6 9.830501f-5; 1.0688757f-6 -5.82454f-5 3.961646f-5 5.784082f-5 2.6230875f-5 -4.110797f-6 -1.2114489f-5 9.190244f-5 5.677286f-5 -0.000111494875 7.906678f-5 2.2018543f-5 -5.5081386f-5 -9.197007f-6 -6.6534965f-5 -6.8599526f-5 8.8153116f-5 2.2247887f-5 -3.938612f-5 -2.3031727f-5 -0.00015165182 0.00014176163 -0.00012151037 2.591397f-5 -0.00015541828 9.689811f-5 -0.000120833436 -5.9194394f-5 -2.0094647f-5 1.9722645f-5 1.8458933f-5 0.00016762815; 2.9945206f-5 -0.0002136077 4.2307387f-5 -0.00014859898 1.6029056f-5 0.00013049919 -0.000121818346 0.00011504457 -3.660613f-5 0.00013795843 1.5452692f-5 6.207062f-5 -4.9132796f-5 -5.6330246f-5 7.049317f-5 -0.0001624591 0.00020378857 5.0185274f-5 -6.563782f-6 2.5135363f-5 -0.00013978432 6.94403f-5 -5.7515543f-5 6.4258464f-5 -8.477129f-5 2.7108229f-6 -6.27297f-5 -0.00011133804 -0.000118937336 -2.491081f-5 -6.5520544f-5 -1.6221407f-5; 0.0001490329 -3.518714f-5 6.838341f-5 1.806367f-5 -1.2177103f-5 4.8077785f-5 4.8774464f-6 2.5176876f-5 -2.393502f-5 -0.00017052738 -8.472462f-5 2.6156655f-5 3.1476753f-5 -3.198646f-5 1.7295855f-5 -6.5615815f-5 -6.457243f-5 4.945713f-5 -6.379944f-5 0.0001725941 -5.566336f-5 -0.00013105926 -2.3267385f-5 0.00013213923 5.1767678f-5 4.8656395f-5 0.00011115292 2.2263952f-5 9.953958f-6 -5.85899f-5 -1.4107241f-5 0.00011483148; -4.1415296f-5 1.3916467f-5 -0.00011940488 -9.851123f-5 7.3574214f-5 4.740405f-5 -6.785301f-5 0.00019942185 0.00012728834 -0.00016363562 -0.00012561417 -1.6128222f-5 -0.00019293881 4.5368397f-6 4.713364f-5 -0.00013651798 -6.237191f-5 -0.00019911939 -0.00017180503 2.6719978f-5 -4.0074345f-5 -1.359718f-5 -8.151072f-5 5.2146228f-5 -1.9828212f-5 7.461231f-5 0.00013041316 -9.9392455f-5 8.682513f-5 0.00013921101 -5.4267803f-5 -0.00015905981; -4.4684915f-5 -5.462887f-5 0.00012135054 -4.510381f-5 -3.475299f-5 0.00010407074 2.2008955f-5 -0.00013626451 -2.2398599f-5 -2.954842f-5 0.00015306505 5.706277f-6 -3.3192628f-5 0.00020575941 6.463371f-5 -9.893465f-5 8.663553f-5 -0.00011787723 -0.00014052064 5.1350125f-5 -1.1286868f-6 0.00011688338 0.00013804293 4.82576f-5 -5.091659f-5 5.691207f-6 -1.1650249f-6 0.00010991485 7.0106557f-6 -5.4914188f-5 0.000103280305 -2.2072318f-5; 3.8976705f-5 0.00010581013 4.4845918f-5 0.00028989412 -2.1131393f-5 -0.00010667203 0.00022406431 6.0003676f-5 -5.046534f-5 0.00013748402 0.00016624035 0.00016925113 -7.0310954f-5 -8.645576f-5 0.000104551196 0.00012043912 -0.0001673082 4.826957f-5 -0.00021466143 0.00023090826 -0.00013504991 -2.8148033f-5 8.9335634f-5 2.1197551f-5 7.634965f-6 -0.00018328545 0.00011402708 4.8938728f-5 8.409785f-5 -9.03842f-5 -0.00015597348 -1.9681711f-5; -0.00017860696 0.00016289309 2.1554613f-5 7.65075f-5 -0.0001572884 -4.904941f-5 0.00013334364 8.8976296f-5 0.00011627484 3.0017205f-5 -4.347116f-6 -7.407668f-5 3.352131f-5 -9.5505995f-5 -6.2805775f-5 -3.8850536f-5 -5.6758072f-5 4.625631f-8 -2.8606568f-5 -8.249394f-5 -8.704262f-5 -1.5149345f-5 -9.393877f-6 -5.6657373f-5 3.1522475f-5 -1.299464f-5 -4.861005f-5 7.5044234f-5 -0.00013518587 -7.636546f-5 3.0268136f-6 3.7342343f-5; -1.5246345f-6 -0.00013971738 -4.301577f-5 -1.3734039f-5 0.00028718126 8.435068f-5 0.00015791062 4.3378295f-5 -4.043131f-5 0.00013740394 -0.00012764876 -5.627069f-5 3.8953523f-5 0.00019921537 -8.471506f-5 4.6630197f-5 0.00012246286 -1.7031091f-6 2.0175803f-5 0.00017496663 0.00012815923 4.6586156f-6 9.724115f-5 -0.00014763075 -0.00013615302 0.00021714495 7.7586665f-6 -9.8543445f-5 -3.5388155f-5 -0.0001154024 -3.734798f-5 -0.0002094385; -7.722984f-5 0.00012643216 -0.0001336082 2.5638423f-5 -8.438335f-5 -5.9111102f-5 -4.578554f-5 -7.396497f-6 -6.326183f-5 3.0497316f-5 -0.000102561884 -0.000101490295 0.00013105836 -0.00019522628 3.417967f-5 -0.00011023943 -1.3394814f-5 0.00010105535 -4.936928f-5 0.00019615894 -9.0929265f-5 0.00017270022 9.9116936f-5 1.238063f-5 3.634942f-5 -0.0001720305 -6.9169415f-5 -0.000103437924 1.6598398f-5 -0.00011501487 -0.00020326376 -0.00016199409; -1.0493395f-5 -1.843547f-5 3.4408877f-6 1.48939225f-5 0.00016012433 -1.5305499f-5 1.7829227f-5 7.7386605f-5 2.445304f-5 -4.7624897f-5 -1.5142531f-5 -9.343951f-6 0.00013970489 -0.0001309227 -0.0002114561 0.00018979193 3.2718573f-5 9.215379f-5 0.000120665085 -0.00010754408 6.665616f-6 -7.775345f-5 -6.5193824f-5 4.0987623f-5 3.918128f-5 3.6471756f-5 -4.6466415f-5 0.00012628127 5.7103025f-6 -5.55399f-5 -7.507195f-5 -2.8647954f-5; -6.847904f-5 -9.9914185f-5 -0.000105565225 1.3994042f-5 -0.00018054691 -4.7752743f-5 -2.5410349f-5 7.5910316f-6 0.000115343246 -0.00015416865 -2.0336203f-5 -6.3315434f-5 0.0001612588 -4.540509f-5 1.9162975f-5 6.950722f-5 7.672867f-5 -6.484223f-5 -1.9393967f-5 1.4184581f-5 0.00014736147 -8.958512f-5 0.00018367275 -2.6434365f-5 6.584939f-6 0.00010185486 8.6197484f-5 -3.4724155f-6 -3.9978455f-5 2.361536f-5 -2.6316737f-5 0.0002298531; -0.00015590824 3.233036f-5 -4.891627f-6 -0.000112197886 -3.579302f-5 -2.305788f-6 0.0002067958 -2.0661932f-5 2.812922f-6 -3.7179456f-5 -1.7453644f-5 6.0095757f-5 3.4720404f-5 -0.00017240063 -2.5926716f-5 -0.00013413644 0.0001581374 0.00014044714 -0.00013199625 6.1015515f-5 8.263777f-5 -0.00010763351 0.0001268077 -1.8969367f-5 4.3797056f-5 7.2108494f-5 5.4863922f-5 0.00014671938 -7.480865f-5 -5.1882493f-5 0.00016848213 -5.266863f-5; -0.00018211977 0.00022923049 -5.0249986f-5 1.5011439f-5 8.912166f-5 -9.50752f-5 3.0659136f-5 -0.00015601957 -4.408726f-5 8.5128704f-5 -0.00012687575 -4.7730693f-5 -9.324386f-5 1.5833908f-5 4.891327f-5 1.8481105f-5 0.00023573374 -4.9181544f-5 -0.0001148964 4.768313f-6 0.00026578025 -1.9075253f-5 -2.9713647f-5 -4.0804738f-5 9.9373574f-5 0.00020834891 1.2000563f-5 -0.00012790448 5.3762087f-5 4.0613824f-5 6.384502f-5 -0.00016239902; 6.825154f-5 5.2034096f-7 -3.3355213f-5 4.9806255f-5 2.6543269f-6 -8.8090885f-5 1.3807705f-5 -4.163206f-5 0.00011294925 -0.00015741195 0.00010468527 -1.8770994f-5 -1.7840232f-5 -9.5402196f-5 -6.371465f-5 -0.00017467847 -0.00023158824 0.00024108063 0.000104571554 6.284469f-5 -0.00015179072 -6.076992f-5 1.831456f-5 -5.2111507f-5 1.0137547f-5 -1.0156169f-5 -0.000111388974 -0.000139489 8.460674f-5 0.0001405082 7.1764507f-6 0.0002610488; -7.00232f-5 -5.1067367f-5 0.00014329309 -6.036692f-5 0.0002614541 -9.751939f-6 -0.00011116908 1.45649965f-5 5.4732864f-5 0.00010501139 -2.877626f-5 -0.00021153178 -1.9726102f-5 -0.00017698524 -7.0890484f-5 -0.00021033199 0.00012432237 3.8996743f-5 -0.00011144479 0.00011738168 0.00026127408 -6.956277f-5 -1.3386461f-5 -0.00017813583 3.5562014f-5 7.980341f-5 -0.000105268555 -7.7263f-5 4.97281f-5 2.7082115f-5 2.170017f-5 0.000103214596; 0.00020855725 0.00011113887 4.70926f-6 0.00013785521 1.884335f-5 9.6773845f-5 0.00011414899 -4.9486604f-5 -7.360017f-6 -7.687813f-5 -6.0845636f-5 -0.00012735029 -9.817477f-5 -0.0001946802 3.3830744f-5 -0.00012810805 -0.00012485104 -1.0283695f-5 -0.000101897385 0.00020154905 -0.00010027012 6.4827036f-5 -8.700263f-5 0.00020038383 5.8320256f-5 -1.971239f-5 -4.4582062f-5 8.605977f-5 3.7068062f-6 8.007526f-5 1.0883958f-5 -3.441287f-5; -7.422574f-5 -7.489358f-5 -0.0002489197 3.7079844f-5 9.6897085f-5 1.0863249f-5 -0.00024051977 3.988985f-5 -0.00030288193 3.255059f-5 -3.1009444f-5 -8.407521f-5 0.00018992538 0.00017652047 6.387699f-5 6.62641f-5 -5.659702f-5 -4.528007f-5 -2.4961199f-5 -6.458119f-5 -7.740639f-6 -8.641964f-5 5.9170667f-5 0.00019641886 7.1408605f-5 7.491677f-5 0.00015996007 -2.14441f-5 -0.00012596443 1.5148395f-5 -0.00020513525 7.203257f-5; -2.159809f-5 -2.1345286f-5 0.000114871524 -0.00021547108 4.5988298f-5 -0.00021723151 5.452951f-5 5.8848036f-6 -0.0001589268 -0.0002506956 0.00018307757 2.7498834f-5 0.00012473844 -3.9340946f-5 0.0001878797 -1.2492364f-6 -9.198892f-5 1.8248566f-5 0.000116399024 1.5051434f-5 0.00034781685 0.000156159 -1.0037205f-5 0.00010480752 -9.164158f-5 2.6788684f-6 0.00016894897 5.7068297f-5 0.00010959305 0.00014088652 -7.1304785f-5 5.224269f-5; 6.238324f-5 9.117311f-5 0.00020454697 0.000208475 0.00018094428 -3.228266f-5 -4.411761f-5 -0.00012658827 -0.00015168184 0.00010113172 0.000105583036 0.00012806694 -5.9978844f-5 -1.2934191f-5 -3.4510533f-5 -6.6202236f-5 -6.6260574f-5 0.0001058354 -0.00020347247 0.00019539056 -4.2171858f-5 0.00020606014 -2.4118353f-5 0.00012581136 7.688511f-5 -0.00010003266 8.23604f-5 -3.0226212f-5 4.6036577f-5 -5.898498f-5 -6.406751f-7 -0.0001438071; 0.00015314828 -5.8805148f-5 -5.2923174f-6 0.0001559641 7.959421f-5 0.00013301839 -4.3907394f-6 7.376904f-5 0.00012489593 4.809692f-5 0.000106285006 -0.00014742208 2.0827318f-5 1.4279862f-5 1.27891335f-5 -0.000102658276 2.7062184f-5 7.4395102f-6 -8.629122f-5 1.6947888f-5 -1.9900364f-5 -1.0614642f-5 -1.1935723f-5 -2.9863071f-5 -0.00010493953 -5.5913297f-5 -2.2774684f-5 -0.00014706138 -0.00017280485 1.1295227f-5 -7.577581f-5 0.00013971627; -7.24564f-5 -0.00019400741 -0.00020466995 -8.613493f-5 -1.8114002f-5 -0.00013081147 -4.010616f-5 0.00017401333 -9.880749f-5 -0.00010273883 -7.1364964f-5 4.2208332f-5 0.000110427754 -0.00013644555 5.977523f-5 -3.1731386f-6 5.8735186f-5 5.9781487f-5 0.00012411928 -7.0713344f-5 -1.4844751f-5 0.00021108685 -7.310388f-5 -0.00012904903 0.00016850662 -0.00017227576 -4.35704f-5 8.4781626f-5 -0.000118789314 9.910206f-6 -0.00017377545 0.000104917126; 3.421374f-5 0.0001784863 -1.1484191f-5 -0.000101836085 0.00024135718 -0.00017500338 -1.0642813f-5 -3.1498173f-5 4.254024f-5 -4.286035f-5 0.00015602878 -5.6244044f-5 0.00011752606 -0.00018053921 -0.000118252945 -7.63437f-5 6.47034f-6 0.0001642431 6.444494f-5 9.384959f-5 0.00015255544 -8.760447f-6 -4.673646f-5 2.8553897f-5 6.577486f-5 -3.8543207f-5 4.0060764f-5 0.0001953378 -9.2235525f-5 -8.571982f-6 1.4440342f-5 -0.00013854646; 2.1408754f-5 9.593646f-5 -4.738496f-5 4.732052f-5 -0.00024824127 -4.732115f-5 2.6718411f-5 -8.263985f-5 1.5497177f-5 -8.5198546f-5 -4.1833344f-5 -3.14811f-5 0.00010269098 -5.6399032f-5 1.7421926f-5 2.3837554f-6 8.1492464f-5 -0.000119179356 6.3751904f-5 2.5058574f-5 -8.191238f-5 6.434593f-5 3.4300912f-5 9.4067254f-5 0.00011397646 3.534332f-5 -0.00016860459 -0.00018563253 -6.822247f-5 -1.4889482f-5 0.00023276742 -2.7466636f-5; 6.616932f-5 1.28425345f-5 -0.0001042926 9.09377f-5 -0.00024482232 3.07259f-5 -0.00013715128 0.000101553676 -7.178563f-5 3.5842924f-5 -8.436059f-5 -0.00016869747 7.175845f-5 -9.25618f-5 -8.6544744f-5 -0.00011310014 -1.9110552f-5 0.00014068597 -3.46944f-5 2.8009728f-5 6.1295f-5 -1.4718792f-5 0.00011410012 4.691924f-5 1.5165745f-5 6.707154f-5 -6.649845f-5 -8.032728f-5 -4.362544f-5 -4.7473106f-5 -6.403594f-5 -0.00021157016], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-0.00022788436 -0.00015335342 -0.00011465894 0.00020399482 -0.00023230375 -0.00023374535 -5.701846f-5 2.2562022f-6 -3.3024367f-5 -8.095133f-5 -0.00013721343 -2.620743f-5 0.00015951965 -0.0001061864 0.000111545 1.7474338f-5 2.3434386f-5 -9.256927f-5 6.5034714f-5 -0.00015723762 2.8178138f-7 -4.313013f-5 7.1515926f-5 1.6893357f-5 5.1108167f-5 3.659688f-5 1.3686637f-5 1.3952281f-5 -6.235979f-5 -1.1992588f-5 -3.5462926f-5 7.7624674f-5; -9.5137206f-5 2.3902441f-5 -0.00012429585 0.00012397852 -2.834219f-5 1.251941f-6 1.3347854f-5 -3.714094f-5 8.1240454f-5 -9.8066725f-5 -0.00011307021 -3.4549306f-5 -8.919269f-5 -4.9514474f-5 -0.00014298115 -6.0232043f-5 -0.00020102633 4.9332135f-5 -0.00012386192 -7.3164238f-6 -0.00015395948 -0.0001424155 -5.7651367f-5 0.00011915152 -0.00019318277 -5.8882713f-5 9.0664114f-5 1.8740871f-5 0.00013420281 -5.03832f-5 -0.00023244778 7.6930075f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))

    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64

    julia
    const params = ComponentArray(ps |> f64)
+ps, st = Lux.setup(Random.default_rng(), nn)
    ((layer_1 = NamedTuple(), layer_2 = (weight = Float32[-5.067828f-5; 0.00020338836; 1.4273281f-5; -4.4280547f-5; -4.2506847f-5; 7.748175f-5; 6.288575f-6; 3.7066755f-5; 0.00014335215; -0.00011426929; 0.0001108622; 8.886729f-5; -0.00023507916; -0.00026994065; -2.1895053f-6; -4.1372066f-5; 0.00014377595; -1.2547869f-5; 0.00015402514; -6.948349f-5; 9.75148f-7; -2.2759441f-5; 5.7592948f-5; -9.247271f-6; -7.653275f-5; -5.4306223f-5; -5.6007382f-5; 4.515377f-5; -9.931654f-5; 0.00011081037; -2.5950818f-5; -2.3206787f-5;;], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_3 = (weight = Float32[-6.4881f-5 5.1595423f-5 -0.00010973665 0.00011888758 -7.869763f-5 2.6731343f-5 -4.3378386f-5 -0.0001431499 2.6663824f-5 -0.00017339565 -2.6334403f-5 9.1598034f-5 -0.00015508423 -4.310734f-5 1.6837905f-5 1.7922774f-5 -7.000959f-5 -4.674141f-5 3.0365556f-5 -2.1681205f-6 2.4398596f-5 1.1775482f-5 9.9300596f-5 0.0001105504 9.565299f-6 -4.7076144f-5 9.590853f-5 -4.885141f-5 0.000120821234 -0.00022250146 -9.05917f-5 -3.2679833f-5; 0.00012063581 -6.609236f-5 -8.302315f-5 -1.954174f-5 -6.6735214f-5 -1.8397034f-6 -2.99155f-5 0.00013611194 -0.00010268538 7.048608f-5 -1.6720745f-5 7.6480974f-5 8.878988f-5 3.9338465f-5 -2.3043342f-5 1.4160479f-5 -3.0309327f-5 -4.2313906f-5 -5.78722f-5 -1.2361765f-5 0.00011966063 -8.0163416f-5 3.7680657f-5 -3.4606725f-5 -9.543387f-5 6.8428395f-5 1.155613f-5 -0.000115886956 -6.523307f-5 5.2380037f-5 0.00024232482 7.973879f-6; 8.224085f-5 -1.21905605f-5 2.7365168f-5 4.015555f-5 -4.8491387f-5 5.793325f-5 7.1500195f-5 7.0172675f-5 -8.642905f-5 1.9702306f-5 4.9804526f-5 9.729075f-5 9.6269556f-5 -9.517981f-5 -0.000108945256 -6.5701315f-5 -0.00014163546 0.00020532864 6.4517335f-5 -0.00010703346 2.3381364f-5 4.1316358f-5 -9.1240385f-5 2.8189028f-5 3.4416404f-5 8.883044f-5 -9.182712f-5 0.00014107842 5.8567588f-5 8.330891f-5 8.732947f-5 2.7254235f-5; 2.0640122f-5 -6.5564316f-5 -9.267124f-5 1.2782187f-5 -2.1939577f-5 4.8729362f-5 -0.0002059266 0.00019216529 -0.00010824198 -0.000106641404 -0.00011950945 4.0736777f-5 3.8151837f-5 1.5811183f-5 -5.0110833f-5 -1.12530515f-5 -9.231862f-5 0.00014269506 -9.049237f-5 0.0001928238 -8.382982f-5 -3.1066495f-6 1.396399f-5 3.5595588f-6 -0.0001678254 -5.0562965f-5 5.644873f-5 0.0001424172 0.00012270945 1.297231f-5 6.82423f-5 1.0279729f-5; -1.3927525f-5 -2.5807853f-5 -3.353609f-5 -0.0001509003 -5.633148f-5 2.3138953f-5 -9.295884f-5 -1.6964543f-5 8.622773f-6 -2.287433f-5 -9.8104494f-5 0.00019548782 -1.4208715f-5 4.2834552f-5 -0.00014059727 0.00012722421 7.491649f-5 -0.00013340326 -7.4468226f-5 6.701388f-7 7.389381f-6 7.241207f-6 0.0001258489 2.1600781f-5 -6.4029395f-5 9.2974806f-5 -0.00013893936 -2.0481652f-6 9.337237f-5 1.4617574f-6 -4.278323f-5 -4.6443496f-5; -9.862447f-6 -8.921982f-5 9.193483f-5 4.696958f-5 -7.964992f-6 -0.00015717202 1.1457437f-5 4.2228417f-5 1.0273722f-5 -0.0001314355 -6.983743f-5 3.2823362f-5 -2.2050655f-5 4.167267f-5 8.6878565f-5 0.00012557801 9.7389f-5 5.7838948f-5 0.00013859413 -5.4861524f-5 9.106602f-5 0.00010086939 0.00014309268 -1.2072048f-5 -5.2197997f-6 4.112339f-5 0.00015406324 -3.9871407f-5 5.1895193f-5 8.516548f-5 6.1933075f-5 -9.757812f-5; 0.000100224905 3.8019178f-5 -0.00011406402 0.00020252966 0.00012403002 -7.962413f-6 -7.4583564f-5 6.3843974f-5 0.00011404545 0.000169954 8.429819f-5 4.996831f-5 0.00011434548 0.00016386562 -6.057491f-5 -9.412545f-5 -0.00015873568 -6.482431f-5 3.091892f-5 -2.159468f-5 0.00012109969 -7.771308f-5 -1.8478474f-6 -9.604292f-6 -0.00010614015 -0.00013608659 -3.699296f-5 -9.4771065f-5 -0.000106533116 -1.4093011f-5 -0.00010621838 -1.599139f-5; -6.859555f-5 5.9557868f-5 0.00013538415 -5.856615f-5 -1.4068297f-6 -9.1875416f-5 4.9636124f-5 -2.6894352f-6 0.00010794412 3.837797f-5 0.00023890898 -0.00011218314 3.951066f-5 7.6334916f-5 -6.472669f-5 2.6779146f-5 1.4812273f-5 -8.1917766f-5 1.3783435f-5 -0.00013009476 0.00010798479 9.0076355f-5 -5.1770206f-5 -3.248121f-5 1.4516136f-5 7.808087f-5 -4.9713708f-5 -4.637024f-5 0.00012679571 7.44449f-5 -0.00014616112 -9.48301f-5; -5.843995f-5 -7.837018f-5 -0.00015722003 0.00013298752 0.0002058001 0.0001131393 -6.0574115f-5 6.8372574f-6 8.129709f-5 5.0710118f-5 0.00025877086 -0.00015811148 -2.5584899f-5 -0.00010511612 -0.00012036738 9.864298f-5 -4.5218094f-6 -0.00011798466 2.9166674f-6 0.00014341663 -5.8281497f-5 4.9651182f-5 -6.645968f-5 -0.0001173513 2.0992076f-5 -1.2117198f-5 -0.00013775482 -5.2643958f-5 -8.244777f-6 -6.679681f-5 -0.000112732036 2.2170127f-5; 1.8069002f-5 -8.4170046f-5 4.027615f-5 -9.122345f-5 1.6444257f-5 8.069179f-6 -4.988504f-5 0.00018525976 7.757772f-5 -0.000103358674 -5.7851914f-5 3.672725f-5 0.00014628147 0.00011684927 0.00014759402 0.00010566242 0.0001684325 7.3146985f-5 -0.00029925458 -9.248588f-5 -0.00010037117 2.386982f-5 -8.735626f-5 -0.00014760795 8.600574f-5 -0.00010550478 -1.5243796f-5 -2.6065078f-5 0.00014419388 9.088046f-5 4.867762f-5 4.917452f-5; -6.598519f-5 -6.0307575f-5 -7.217373f-5 -7.9992154f-5 -0.00020351214 8.730133f-5 3.311596f-5 -7.431629f-5 -6.286525f-6 -4.801173f-5 0.0001289165 4.1723186f-5 -8.892634f-5 0.00024919503 4.605369f-5 8.194185f-5 -4.1749663f-5 -4.7821406f-5 -7.79361f-5 -4.722093f-5 -5.6371475f-5 -2.9041228f-5 -4.2099673f-5 -4.402712f-5 -2.151333f-5 0.00010635854 -0.00018420393 7.879363f-5 6.56917f-5 -0.00014872105 -0.00025311403 0.00016434335; -3.2588643f-5 -6.82938f-5 -8.62755f-5 -2.488788f-5 -0.00010869599 -1.875883f-5 -5.000146f-5 0.00010093272 -0.00017555879 8.3838204f-5 -0.00019008624 -2.0509233f-5 9.822452f-5 -0.00014449273 -4.487349f-5 -0.0001129706 5.2817784f-5 0.00010259473 1.2052635f-5 0.0001414271 1.6518059f-5 0.000114337774 -9.06657f-5 3.4899254f-5 2.4568328f-5 3.0560605f-5 0.000109304965 -4.3656746f-5 5.3483916f-5 5.271978f-5 -6.0059614f-5 6.0993232f-5; 4.6849014f-5 0.00015922435 3.7304446f-5 0.00010522433 -1.7149052f-5 9.699794f-5 8.679175f-5 -5.980967f-5 0.000102687576 -0.00011774394 -9.3231676f-5 -7.073589f-5 9.4761046f-5 -2.0640855f-5 -0.00023659732 -0.00015139587 -7.254155f-5 0.00018710685 -7.661277f-5 -6.958926f-5 -0.00023375152 0.000113982176 6.118777f-5 -0.00014834842 -2.048975f-5 2.8104418f-5 2.5483473f-6 6.996796f-5 -9.3646326f-5 -0.00014223685 -0.00017213685 -0.00018663933; -0.00011808671 4.8037993f-5 1.6431522f-5 -0.00015378026 7.1910195f-5 -0.0001872632 -5.120849f-5 -0.00010370612 0.00029613348 3.4528523f-5 -7.646572f-5 5.0855837f-5 9.754051f-5 6.584373f-5 -4.716242f-5 -2.1120639f-5 -0.00011250277 8.575611f-5 -1.6247295f-5 -0.00027942262 1.3625263f-6 8.212421f-5 6.171591f-5 6.960599f-5 -1.3251189f-5 0.00010215443 0.0001165171 -2.5964511f-5 4.8122438f-5 4.640835f-5 0.00014826102 -7.550899f-5; -6.171692f-5 -2.2494147f-5 -0.00011052692 0.000108468135 -6.2695945f-5 -9.421945f-5 0.00014297807 3.1365198f-5 8.234304f-5 7.663826f-6 -0.000109010085 -8.211707f-6 -4.7824436f-5 -0.0001506865 -7.138279f-5 0.00012205401 0.00010132722 -2.9440233f-5 8.614374f-5 -3.23056f-5 5.3609398f-5 9.664525f-5 -1.596922f-5 -8.9840825f-5 5.949551f-5 1.0887881f-5 4.2916592f-5 -0.00010351466 0.00012603638 5.6517703f-5 6.545845f-5 0.00019466775; -6.0357146f-5 -3.048198f-5 1.9146099f-5 4.148812f-5 -2.4972449f-5 -9.497651f-5 -5.7457888f-5 0.00010517364 6.8643836f-5 -5.117873f-6 9.27461f-5 1.2731097f-5 2.0752132f-5 -1.23914115f-5 3.9276125f-7 1.6265903f-5 8.1468446f-5 9.052403f-5 0.00018212575 7.8478355f-5 1.3240985f-5 -6.920781f-5 1.9615041f-5 -0.00020374315 3.897066f-5 -0.0001280138 -6.9336143f-6 0.000103044265 0.000104789826 4.8267525f-6 6.81914f-5 -0.00012826166; -2.809343f-5 -3.0831015f-5 2.2828628f-5 8.898851f-5 1.2071551f-5 -3.803355f-5 -1.0437871f-5 5.422938f-5 4.867486f-5 -4.518063f-5 7.922069f-5 -0.00010169225 0.0001434924 -7.677246f-5 0.000111034875 -5.2173058f-5 1.2276159f-6 -2.8245557f-5 9.942706f-5 1.1442849f-5 1.463341f-6 -5.2849533f-5 -4.7457823f-5 -5.7791447f-5 -2.5199992f-5 2.5344176f-5 9.2255f-5 5.7867932f-5 -4.42645f-5 -0.00022660938 -0.00029660418 -9.220186f-5; 6.735818f-5 -4.176811f-5 -1.2232209f-5 -0.0001059559 5.754733f-5 -6.4020445f-5 -0.00014015102 0.00011234368 3.3669865f-5 -0.00012898138 -0.0001243019 7.9176105f-5 0.00021939976 -1.7735352f-6 1.3753903f-5 3.0978037f-5 1.261377f-5 3.2221094f-5 1.1791008f-5 0.00012344854 2.0633179f-5 -0.00019241248 3.0955096f-5 -4.4071912f-5 0.00013447642 0.00015082232 2.4748131f-5 7.190226f-5 5.500368f-5 -2.3531034f-6 0.000106698426 -0.00015557921; 0.00015004253 7.1710674f-5 7.18664f-5 -4.449272f-5 0.00010724826 -3.0034582f-5 -6.1318984f-5 -7.845766f-6 7.125385f-5 -1.1303766f-5 4.3301923f-5 1.4665254f-5 7.303844f-5 0.00012035262 3.3127606f-5 -1.6960375f-5 9.650369f-5 -0.000110265166 1.5607555f-5 -7.302818f-5 2.9097244f-5 -0.00014719421 1.1049523f-5 -2.7170749f-5 6.9879985f-5 -0.00017216924 -5.5424134f-5 0.00015734922 -2.3573297f-5 -9.351214f-6 -4.482439f-5 0.00015532046; 9.200195f-5 -7.894702f-5 6.909601f-5 6.397365f-5 -7.337188f-5 1.810738f-5 -1.3611387f-5 -0.00011573212 6.84041f-5 0.00016948365 -3.591823f-5 -0.00010666039 5.9029382f-5 -1.1825928f-5 5.6072877f-5 4.840527f-5 0.00019966012 -7.507182f-5 3.281313f-5 -0.00026297724 0.00013116715 -6.360724f-5 1.34186175f-5 -5.0054907f-5 -6.6365785f-5 -5.7354497f-5 -0.00027880043 5.8056135f-5 4.771379f-6 0.00023913936 -0.00027277894 0.00013627937; -3.8475675f-5 0.00013499877 -0.00011172844 -1.5792652f-5 1.7460216f-5 8.990932f-5 -0.00013876938 0.00010837003 2.5286476f-5 4.0916253f-5 -8.68343f-5 2.7274678f-5 -0.0002117695 -4.198623f-5 2.5651116f-5 -4.3864326f-5 0.00010112666 8.5957945f-5 -0.00011121263 4.618323f-5 -2.0187681f-5 0.00010101195 -6.718644f-5 0.0001685784 -6.6461784f-5 -8.18679f-5 5.8931157f-5 6.0350863f-5 0.000110743706 2.1969163f-5 8.4774925f-5 2.8408233f-5; 0.00015168458 -0.00015583177 9.526762f-7 3.8325594f-5 3.7966631f-6 -2.4170578f-5 -5.1147792f-5 -1.8475313f-5 2.11474f-6 2.0240308f-5 -0.00019447393 7.808402f-5 -1.9867588f-5 2.6745403f-5 2.3344088f-5 -7.562572f-5 3.0028186f-5 0.00020181146 0.0001235826 -4.559302f-5 0.00010591059 0.00026848647 -3.44617f-5 2.3216366f-5 8.163823f-5 -1.3018025f-5 0.00011558951 -6.941131f-5 -0.00019635934 -8.809247f-5 0.00015325508 -0.00014338388; 1.7577355f-5 -0.00011430754 -2.3156954f-5 -1.96567f-5 -0.000108622946 8.5745494f-5 -6.518311f-6 -4.53642f-5 -5.5961747f-5 0.00012402056 5.957273f-5 0.0001436606 -2.3027369f-5 -2.1112419f-5 1.8754206f-5 6.52711f-5 -6.917144f-5 0.00015894721 8.440349f-5 0.00015213025 4.88502f-7 -7.422043f-5 4.4104203f-5 -5.2935447f-5 3.839207f-5 -0.00026345407 -0.00027999925 -0.00010818485 0.000100323494 0.00013953423 -0.00015231865 9.962925f-5; 3.5571054f-5 -7.7061304f-5 -0.00021089091 0.000117353586 -8.080046f-6 0.00020298584 -7.7734745f-5 0.000111111476 0.00021550554 -0.00011431153 -5.765442f-5 -9.9738056f-5 -0.00012480488 0.00012828007 -7.215347f-5 -9.8592405f-5 -5.0616338f-5 6.370253f-5 -1.0569401f-5 -8.1941704f-5 -9.611313f-5 4.9776292f-5 7.01193f-5 -3.1117284f-5 -0.00011037468 -0.00011528408 -7.007478f-5 -0.0001132747 2.4702704f-5 -0.00013523488 0.000120989585 -0.0002573811; 0.00026173593 1.283452f-5 -1.7065893f-5 -0.000104993436 -1.4819496f-5 0.00016174733 5.1968495f-6 0.00012912843 1.9695099f-5 0.00014575227 6.3173524f-5 -5.2360924f-6 -5.6768888f-5 -0.00014571872 -0.0001785546 5.9773887f-5 -0.00018953496 6.7306304f-5 0.0001568174 7.134748f-5 6.8328976f-5 -3.3585914f-6 -5.0178074f-5 2.9536714f-5 1.9834733f-5 4.025718f-5 -0.00021762686 9.823387f-5 2.9596104f-5 -5.052526f-5 0.00011400497 0.0001491102; 3.034911f-5 0.00011196634 6.7014895f-5 -0.00013202667 -4.6789377f-5 8.572312f-7 9.169984f-5 -2.3223301f-5 -0.00010554397 8.7856f-5 5.0881303f-5 -9.156455f-5 -6.564096f-5 3.4504257f-5 -0.00016190331 -6.2145766f-5 1.9165122f-5 -7.877776f-5 -4.080841f-5 2.136334f-5 -7.858364f-5 -8.119439f-6 0.00013435273 4.4416214f-5 -0.00012370772 0.000101974394 -4.802835f-6 -0.00010022334 1.2407417f-5 -0.00015726443 -0.00010571935 -3.07934f-5; 8.026358f-5 -2.2175438f-5 -0.00012479624 -4.0228583f-6 -0.0002115195 2.9970723f-5 -4.1155407f-5 0.00014840579 5.4885195f-6 0.00029171017 0.00013645006 -0.00017074464 0.00018228586 -0.00011459783 3.7666694f-5 -1.6101341f-5 0.0001842996 -0.0001169029 -1.4438652f-5 -6.3499065f-5 -9.643319f-5 -0.00017683755 5.7190664f-5 -3.922391f-5 -3.1976626f-5 -2.109656f-5 -3.90291f-6 -0.00013517434 0.00016720014 -0.00013073278 8.5466956f-5 -1.526848f-5; -4.2852924f-5 -0.00015807984 -8.3653595f-6 2.5182915f-6 -0.00013412097 -9.557661f-5 8.1717204f-5 -5.246506f-5 -1.7424688f-6 9.216007f-5 -7.319666f-5 -0.00027948042 0.00015722163 3.1111696f-5 -0.00020530884 -5.4546465f-5 -6.29937f-6 -0.0001428365 3.9120237f-5 3.721716f-5 -6.6093584f-5 -1.8950981f-6 -9.6505835f-5 2.1517824f-5 5.098992f-5 3.5149867f-5 0.0001302061 0.00012702291 -7.683339f-5 0.00011238969 2.6854445f-6 4.7272497f-5; -4.8100424f-5 4.516353f-5 0.000111997026 5.204853f-6 -6.4593376f-5 0.00010572687 3.921466f-6 6.905398f-6 -0.00015121218 1.7533444f-5 -3.0009418f-5 -3.6821777f-5 -0.0001382609 -0.00013444631 -0.000118799144 -0.00011516807 -2.658559f-5 -1.0310642f-5 -0.00013526081 8.310802f-6 1.39192125f-5 4.0677514f-5 5.7376667f-5 -4.978068f-5 0.00015903164 3.915949f-8 -1.5798567f-5 5.3141244f-5 -0.00014180606 -0.0001386008 0.00013365413 0.00013656895; 0.00017168031 -2.2275335f-5 -9.818207f-5 -6.38929f-5 -9.4072755f-5 -3.0191156f-6 8.873516f-5 6.795865f-5 -0.000121237004 7.997962f-5 0.00013511072 0.00013809897 -2.5103287f-5 0.00012684942 -0.0001443395 -9.857521f-5 0.00017308586 4.2674477f-5 -1.6791339f-5 -8.397967f-5 1.0641746f-5 -5.0464263f-5 -0.0001410602 -2.9612102f-5 1.5865919f-5 6.131211f-5 -1.00230445f-5 -2.2032244f-5 1.0033375f-5 -0.00012724352 -1.5285226f-5 0.00014202898; -1.4121472f-5 -5.0044982f-5 7.915148f-5 0.00010705557 -4.692474f-5 4.8653448f-5 -6.4731226f-5 -7.822046f-5 -1.0700496f-5 2.0036923f-6 0.00012267969 3.603697f-7 1.9522387f-5 -0.00010675869 -0.00013979523 -4.262446f-5 -6.636593f-5 -2.5301544f-5 0.00017978139 8.460204f-5 -9.998411f-5 -0.0001354939 -9.499201f-6 -4.361895f-5 -0.00013717679 4.524789f-5 0.0001629717 7.41474f-5 -0.0001341515 0.00012045276 -8.077927f-5 1.8191115f-6; 8.464025f-5 -0.0001911386 -0.00013323688 -5.179797f-5 3.0535775f-5 -3.6069952f-5 5.7020614f-5 -6.241533f-5 -3.7568614f-5 -9.843058f-5 0.00024355052 -0.00013379584 -6.849716f-5 1.1521302f-6 -1.6933034f-5 6.922868f-5 0.00015571411 -8.287445f-5 4.693961f-5 0.00018321241 7.1558397f-6 -0.00023000097 -2.3251116f-6 9.687749f-5 8.636353f-5 8.079613f-5 6.684031f-5 4.8694303f-5 -7.006521f-6 -5.833168f-7 -4.4962962f-5 -6.118613f-6], bias = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), layer_4 = (weight = Float32[-0.00010431886 7.263726f-5 0.0001153885 2.3408782f-6 -0.00014803729 0.00017549854 -1.4270489f-5 9.409741f-5 -2.1240825f-5 0.00027308802 -2.6860275f-6 1.595533f-5 -8.200385f-5 0.00014178251 -1.6477046f-5 -9.237305f-5 -0.00011124783 1.087667f-5 0.00022355978 2.2964196f-5 -9.138082f-5 -3.8097627f-5 -7.44995f-5 -0.00022302984 -9.659961f-5 -0.00034645712 9.354396f-5 6.146441f-5 -9.705188f-5 -4.282779f-5 8.976634f-5 -4.020095f-5; 6.706114f-5 0.0002107104 1.2080838f-5 6.554694f-5 6.5647335f-5 3.832331f-5 -0.00016438964 -2.266784f-5 -7.713031f-5 -0.00018027052 6.2050116f-5 -5.3583874f-5 0.00016199157 -2.413137f-5 1.9695848f-5 0.000119893564 2.6232174f-5 -2.5299496f-5 7.291109f-5 -0.0002123351 0.0001043167 -3.4432705f-5 -9.066305f-5 -0.00017630673 -5.5120025f-5 0.00013201217 -0.00011529041 0.00013865373 5.0689f-5 8.1196495f-6 -0.00010693036 9.6504766f-5], bias = Float32[0.0, 0.0])), (layer_1 = NamedTuple(), layer_2 = NamedTuple(), layer_3 = NamedTuple(), layer_4 = NamedTuple()))

    Similar to most DL frameworks, Lux defaults to using Float32, however, in this case we need Float64

    julia
    const params = ComponentArray(ps |> f64)
 
 const nn_model = StatefulLuxLayer{true}(nn, nothing, st)
    StatefulLuxLayer{true}(
     Chain(
@@ -229,13 +229,13 @@
         ["Waveform Data", "Waveform Neural Net (Untrained)"]; position=:lb)
 
     fig
-end

    Setting Up for Training the Neural Network ​

    Next, we define the objective (loss) function to be minimized when training the neural differential equations.

    julia
    const mseloss = MSELoss()
+end

    Setting Up for Training the Neural Network ​

    Next, we define the objective (loss) function to be minimized when training the neural differential equations.

    julia
    const mseloss = MSELoss()
 
 function loss(θ)
     pred = Array(solve(prob_nn, RK4(); u0, p=θ, saveat=tsteps, dt, adaptive=false))
     pred_waveform = first(compute_waveform(dt_data, pred, mass_ratio, ode_model_params))
     return mseloss(pred_waveform, waveform)
-end
    loss (generic function with 1 method)

    Warmup the loss function

    julia
    loss(params)
    0.0006529859924286357

    Now let us define a callback function to store the loss over time

    julia
    const losses = Float64[]
+end
    loss (generic function with 1 method)

    Warmup the loss function

    julia
    loss(params)
    0.0007098617733391764

    Now let us define a callback function to store the loss over time

    julia
    const losses = Float64[]
 
 function callback(θ, l)
     push!(losses, l)
@@ -247,7 +247,7 @@
 res = Optimization.solve(
     optprob, BFGS(; initial_stepnorm=0.01, linesearch=LineSearches.BackTracking());
     callback, maxiters=1000)
    retcode: Success
-u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [-8.733465620020983e-6; -0.00010797254071798166; -6.034535090281437e-6; 7.889705011615516e-5; -1.9253391656059496e-5; -4.612652992366777e-5; 7.144777919157657e-5; -8.932017226481967e-5; -4.71681050840083e-5; 9.798890823729545e-5; -2.1554476916162734e-5; 4.739751602749602e-5; 0.00016439423779959398; -4.360640014055303e-5; -6.190079147924201e-5; 6.240796210469526e-5; 0.00010571498569335307; 4.738614370580132e-5; 2.5381063096584814e-5; 4.914130477114876e-5; 0.00016239624528652472; 0.00013147806748750233; -0.00017642804596096446; -0.0001261118450201706; 3.079728458034808e-6; -3.787924651986792e-5; -1.16270905436827e-6; -0.00016794122348050335; 6.320179818424497e-5; 0.00012728852743741338; 9.064738696900327e-5; -1.5429377526735596e-5;;], bias = [-5.359601592163036e-18, -6.550829691992387e-17, -1.41806373185002e-17, 6.612883732914699e-17, -3.971281807812836e-17, 3.2571059060761017e-17, 9.58030628269383e-17, 1.3176812858269634e-16, 2.944634276201806e-17, 1.9336104666087008e-17, -5.330420267771681e-17, 7.372211719502685e-17, -2.7922867808978655e-17, -9.150352429637832e-18, -3.8438937777196597e-17, 9.00633053182541e-17, -1.5663529032065214e-17, 7.62186052855715e-18, -1.1978849530028129e-17, 3.9396538625362964e-17, 3.263723400316271e-16, -3.1675390001372084e-17, -1.3105712844388004e-16, -3.803273056227972e-17, -2.0332749183269495e-18, -1.2727593133643193e-17, -1.7718179375088544e-18, -2.8018370574615e-16, 7.587973597486665e-17, 8.609350097255151e-17, 7.4759154972248e-17, -9.260147643001434e-18]), layer_3 = (weight = [-2.81779129679041e-5 -3.9365283514297854e-5 -7.6574890913135e-5 0.00013857145044797106 -6.85615832181301e-7 -5.7781809077523993e-5 -5.151397503750668e-6 3.9835850698482504e-5 0.00012864310069319755 -6.512425803036386e-5 5.4659068196387454e-5 -6.377044970092395e-5 7.986889344950431e-5 -9.115820917456447e-5 -5.069790764052438e-5 -1.491042171137685e-6 -3.207358072783739e-5 -6.482248041235874e-5 2.0175230970680213e-5 9.132562358475261e-5 1.717424224382949e-5 6.390499133461278e-5 -4.389381406751195e-5 -0.00011969898237536553 -6.298581043181066e-5 1.9702146387616684e-5 -7.764463548183539e-6 1.9582808320888176e-6 7.794170328027556e-5 0.00010650963763085304 -8.729386077417963e-5 -3.0301974179265634e-5; 5.093000583673822e-5 -3.3695515683559355e-6 0.00010611892650576301 -9.076084677854146e-5 -6.206943215129883e-5 2.0946710374296105e-5 6.196243486574254e-5 -8.04957875332556e-5 9.118333548048649e-5 9.569926771309863e-5 -0.0002559040862068346 -8.457661750053847e-5 0.00011611161755373647 0.00015786205915009716 -0.00013252692440783356 -1.4725690907474887e-5 2.510067697768531e-6 5.846065107124997e-5 5.234997808674383e-5 -4.698632108539192e-5 -0.00024784683626402274 -4.0582605269986785e-5 -3.356815459286549e-6 7.707813494580927e-5 -5.3827805804582414e-5 -0.00012691746667751962 -0.00018019285778773963 3.0377035382311122e-5 -1.2278924605019604e-5 -0.0002268154333976734 0.00017366600661280175 -3.0415702711312987e-6; 0.00010137852786915331 -0.0001010015274014704 -2.427308391863947e-6 -4.770813849111468e-6 6.701661750316573e-5 0.00017136620426658485 -0.00019062613300871163 -1.067131212087478e-6 0.00012427559540715012 -3.5586695777104e-5 8.55602685978259e-5 -8.815592773796477e-7 -4.387152676013409e-5 0.00015637557789791322 -2.2594698567493444e-5 4.314996676743906e-5 -0.00011615114799592718 -1.5891253316160807e-5 -7.964016713326844e-5 1.0809817648757658e-5 3.317806676836575e-5 0.0001521374781068033 -0.0001245625624515501 4.1607805723239765e-5 -1.4163321612661705e-5 3.903749366049605e-5 -4.447955671007791e-5 -3.575690225357303e-5 7.252304767366444e-5 -0.00011933259684251707 -7.042072848076068e-5 -5.546812990455935e-5; -8.336042507189592e-5 0.00011739355553536533 -1.6887058029987743e-5 -9.929093054133669e-5 0.0001831156508497774 0.00016209405597428542 -0.0001248413506463285 1.6862267636589728e-5 -0.0001252187109120308 -2.3408347863458833e-5 9.515673021915105e-5 -6.6847330007470475e-6 5.598798028322887e-5 0.0001258904479410381 -3.764638737529744e-5 -0.00011733128181995121 5.898543471366949e-5 -3.217015242889768e-5 0.00016092859308364536 -0.00010215279022568149 -0.00013990668654778032 -0.00021372428879012443 0.00013839693141299363 -2.1802074718741762e-5 -5.540062523231966e-6 0.00017145255272871558 -6.449778871947236e-5 0.00010669620254439335 2.438082204853424e-5 -8.664751354708432e-5 -0.00010137273480986426 -0.00010135865583188165; 6.770358496073841e-6 7.97941223851332e-5 0.00010768450501077066 4.6960414138666134e-5 8.351425402965557e-5 5.775398577173693e-6 -7.493830581493888e-5 2.7759933246341794e-5 -5.564503463515998e-5 4.735090023190146e-5 6.739095023535343e-5 -1.759619100026308e-5 6.22388883044749e-5 -0.0001010620859318409 6.811957190679273e-5 -1.7852259233548545e-5 0.00014260215301932077 -3.112693467242631e-5 2.758543213589722e-5 -0.00021257582794474437 -2.0855234296848334e-5 -0.00015710228272780086 0.00011575050793055044 -9.060623652600814e-5 0.00012642943092091197 -6.715464643035007e-5 -2.3085242545998626e-5 0.00014910514200039036 0.0001751779540342266 0.0001241965704963434 -2.726808143903494e-5 0.0001073214201739075; 5.8766903805779695e-5 -0.00015708922904846402 2.6431680510403082e-5 -0.00010387897958676388 2.2030108141596797e-5 4.6587045347085305e-5 -0.00014493874363065735 7.219263176270675e-5 -2.457368094578844e-5 0.00013603266239701657 0.00013269154266058503 8.446827092743486e-5 9.409567252336606e-5 -0.00016357305315718972 1.966221140068937e-5 0.000159896391835762 -0.0001435019311767551 -1.580818565085058e-5 1.6059146276209827e-5 -3.5983315626392106e-5 1.4797095957732168e-5 1.8547740239996884e-5 4.254694619052582e-5 -9.158426286117389e-5 -0.00013229278005124026 -1.3918982346833022e-5 -1.9566823482047338e-5 -5.4902115095643754e-5 -3.019838908326368e-5 3.309413119754534e-5 0.00019555344448494636 9.743726519704837e-5; 6.582914752732503e-5 -1.9021242084925348e-5 -2.3916997494893138e-5 -8.26842650413899e-5 8.264597377353835e-5 0.00011346335045607408 0.00019279813673427136 5.6249428919618555e-5 -3.976422781044965e-5 -8.271452574910546e-5 6.101728390865519e-5 -7.841763622047724e-5 -1.2951023424746916e-5 0.00013066113856700814 -1.5206532995873412e-5 -0.0001237932582145024 4.110263503469658e-5 -6.578166980682285e-5 -1.7373078571754834e-7 -4.442866233796219e-5 -1.1633819413955027e-6 -0.00010926800937078438 -0.00018221662216738526 -0.00013746663066478056 0.0002607891957749194 -1.487281574293196e-5 1.9144042337643907e-5 -6.363366887590736e-5 -3.2120659713766265e-5 -0.00013503155682621117 -5.4511840832783175e-5 -0.00015212040929975444; 1.6406990034034946e-5 -1.9113927629197256e-5 0.00012204590195253912 -7.556622556946641e-5 -1.6685824158024457e-5 -1.590150866902443e-5 6.0559330740240846e-5 -1.5048051573805486e-5 -8.19364647038794e-5 -2.9995849836970368e-5 -1.038943053642793e-5 -3.10356369307312e-5 -0.00017250242643128175 3.37049280635499e-5 -4.3460542365913066e-5 -4.783470620238665e-5 2.2758766310549294e-6 5.0059814559306675e-6 -0.000156025118195224 2.3699043885450762e-5 3.2233576638856505e-5 -0.00012310168647743032 -2.8646086939959324e-5 5.1685634679624405e-5 0.00017521538422372607 -3.891999920667637e-5 0.00016759197052963493 4.498685695510351e-5 2.9358576575056893e-5 -9.714292507921566e-5 -8.786864492114564e-6 9.830446680937386e-5; 1.06843883722068e-6 -5.8245838158569814e-5 3.961602458283923e-5 5.7840384829517084e-5 2.6230438084068423e-5 -4.11123376116672e-6 -1.211492628945743e-5 9.190200187285233e-5 5.677242340888547e-5 -0.00011149531230776743 7.906633952651798e-5 2.2018106096849766e-5 -5.5081822506421394e-5 -9.197443728142482e-6 -6.653540215202865e-5 -6.85999624010108e-5 8.815267908639125e-5 2.2247449737817038e-5 -3.938655602260798e-5 -2.303216417650039e-5 -0.00015165226097897555 0.00014176119264001894 -0.0001215108062068825 2.5913533750184917e-5 -0.00015541871864562236 9.689767437074972e-5 -0.00012083387293833621 -5.919483039624893e-5 -2.0095083823350953e-5 1.9722208594516065e-5 1.845849662922129e-5 0.00016762771672355632; 2.9943606731133943e-5 -0.00021360929369762725 4.23057879547136e-5 -0.000148600576923457 1.602745740313698e-5 0.00013049759016069826 -0.00012181994518101749 0.00011504296869901559 -3.660772765357732e-5 0.0001379568291471869 1.5451092420729488e-5 6.206902051274912e-5 -4.913439463193834e-5 -5.633184466347801e-5 7.04915744594797e-5 -0.00016246069410186855 0.00020378697243217167 5.018367462838316e-5 -6.565380973885296e-6 2.513376400969248e-5 -0.00013978591434493068 6.94386997369688e-5 -5.751714181455874e-5 6.425686458755322e-5 -8.477288631326389e-5 2.7092237618801464e-6 -6.27312967677122e-5 -0.00011133964128162024 -0.00011893893517623317 -2.4912409773203008e-5 -6.552214310573785e-5 -1.6223006443002694e-5; 0.0001490338564274168 -3.518618352769437e-5 6.838436281599909e-5 1.806462575972689e-5 -1.2176147652786665e-5 4.807874035963595e-5 4.878401409079714e-6 2.5177831116016903e-5 -2.3934064234487803e-5 -0.0001705264202662507 -8.472366419096616e-5 2.6157609749355256e-5 3.1477708102315184e-5 -3.1985504325127225e-5 1.7296810323607505e-5 -6.5614860364583e-5 -6.457147349079619e-5 4.945808366232355e-5 -6.379848302981961e-5 0.00017259505154856278 -5.566240536224125e-5 -0.00013105830833025017 -2.3266429639777084e-5 0.00013214018879283523 5.176863312430556e-5 4.865734997496769e-5 0.00011115387503581927 2.226490694214039e-5 9.954913434050791e-6 -5.858894467179369e-5 -1.4106286403838673e-5 0.00011483243185852522; -4.141835410123147e-5 1.3913409077181379e-5 -0.00011940793950784414 -9.851428507103351e-5 7.357115651971301e-5 4.7400992173019836e-5 -6.785606996131011e-5 0.00019941878969108163 0.00012728528059811163 -0.00016363867501935345 -0.0001256172271838726 -1.6131280006125398e-5 -0.00019294186671469824 4.533782032335001e-6 4.713058120829044e-5 -0.00013652103547214534 -6.237496739926018e-5 -0.00019912244346200657 -0.00017180809112473725 2.6716919876302937e-5 -4.007740238886854e-5 -1.360023810559931e-5 -8.151377727482514e-5 5.214317030765395e-5 -1.983126998097565e-5 7.460925377231771e-5 0.00013041010066658793 -9.939551267745315e-5 8.682207231900662e-5 0.00013920795578303128 -5.427086028943335e-5 -0.0001590628689315487; -4.4683193628608677e-5 -5.462714667449189e-5 0.0001213522601021198 -4.510208869835473e-5 -3.475126774066839e-5 0.00010407246467291496 2.201067676576273e-5 -0.00013626279083912471 -2.2396877248705143e-5 -2.954669886169323e-5 0.0001530667696344408 5.707998788399833e-6 -3.319090588851249e-5 0.00020576113187342084 6.463542835188122e-5 -9.893293005081802e-5 8.663725478906726e-5 -0.00011787550734954773 -0.00014051892045319808 5.135184714088322e-5 -1.1269650811645857e-6 0.00011688509861339084 0.00013804465292119772 4.82593214102676e-5 -5.0914869426216386e-5 5.692928915939261e-6 -1.1633031461512262e-6 0.00010991657203627335 7.012377445040195e-6 -5.4912466076377036e-5 0.0001032820264655795 -2.207059602444299e-5; 3.8979493365287074e-5 0.00010581292208365888 4.484870642040789e-5 0.00028989690526936494 -2.1128604021259633e-5 -0.00010666924008551315 0.00022406709667843232 6.000646517344881e-5 -5.0462551189458834e-5 0.00013748680994462204 0.00016624314066571072 0.00016925391737454296 -7.03081657776027e-5 -8.645297002663145e-5 0.00010455398496555676 0.00012044191153103992 -0.00016730541192379013 4.827235734405655e-5 -0.00021465863919734152 0.00023091105158523368 -0.0001350471191472717 -2.8145244687099784e-5 8.933842236721743e-5 2.1200339870060362e-5 7.637753878908047e-6 -0.0001832826581449361 0.0001140298720201378 4.894151625187261e-5 8.410063510377457e-5 -9.038141233765218e-5 -0.0001559706923217688 -1.967892221732617e-5; -0.00017860869197075727 0.00016289136179357017 2.1552883805534698e-5 7.650577240955891e-5 -0.00015729013616119967 -4.9051138961393236e-5 0.00013334190803679445 8.897456642107079e-5 0.00011627310810244291 3.00154755280313e-5 -4.348845453668481e-6 -7.407840679998861e-5 3.351958217200008e-5 -9.550772451230695e-5 -6.280750462220311e-5 -3.885226546454502e-5 -5.6759801412865855e-5 4.452677590444851e-8 -2.860829740139632e-5 -8.249567112553275e-5 -8.704435244334782e-5 -1.5151074552430909e-5 -9.395606542684836e-6 -5.665910215948288e-5 3.152074564813159e-5 -1.2996369736091477e-5 -4.8611780260861976e-5 7.504250457378289e-5 -0.00013518759591871196 -7.636719023475917e-5 3.0250840666626814e-6 3.73406131587131e-5; -1.5228735603273347e-6 -0.00013971561548916717 -4.3014009690716666e-5 -1.3732278432340951e-5 0.0002871830221809317 8.435243961483699e-5 0.00015791237837018728 4.3380056183907456e-5 -4.04295495283931e-5 0.00013740570301504828 -0.00012764699914642993 -5.626892726669568e-5 3.8955284424237366e-5 0.00019921713118290995 -8.471329766709e-5 4.6631958128077746e-5 0.00012246462591772832 -1.701348139444165e-6 2.0177563954154805e-5 0.00017496838717105307 0.00012816098768577894 4.660376532935051e-6 9.724291174986106e-5 -0.0001476289906461012 -0.00013615125482120902 0.00021714670600684853 7.760427422715573e-6 -9.854168439435897e-5 -3.5386394131105624e-5 -0.00011540063843515129 -3.7346220608212875e-5 -0.00020943673717939665; -7.723347007773074e-5 0.00012642852888899184 -0.00013361183883167066 2.563478992310894e-5 -8.438698060122472e-5 -5.911473490773644e-5 -4.5789171421300255e-5 -7.400129928959714e-6 -6.326546499981479e-5 3.049368351281167e-5 -0.00010256551686684742 -0.00010149392838134362 0.00013105472825926016 -0.00019522991570188633 3.4176036566396015e-5 -0.00011024306368558402 -1.3398447121235416e-5 0.000101051720408621 -4.937291160236234e-5 0.00019615531127424067 -9.093289773220052e-5 0.0001726965907401498 9.911330343683025e-5 1.2376997434390403e-5 3.6345785334601886e-5 -0.00017203413372403965 -6.91730478174166e-5 -0.00010344155671547738 1.6594765471427673e-5 -0.00011501850022244405 -0.00020326738959187797 -0.00016199772144678218; -1.0492632138680406e-5 -1.8434707310424074e-5 3.441650694680668e-6 1.4894685552956632e-5 0.00016012509054921307 -1.530473586395223e-5 1.7829989662953562e-5 7.738736799869232e-5 2.4453803196178463e-5 -4.7624133951519924e-5 -1.5141768055814625e-5 -9.343188178223385e-6 0.00013970565392586398 -0.00013092193061534614 -0.00021145534359898768 0.00018979269130572327 3.271933652062365e-5 9.215454970903387e-5 0.00012066584800554172 -0.00010754331837456985 6.6663792108896785e-6 -7.77526884507643e-5 -6.519306138440145e-5 4.0988385951586975e-5 3.918204216237859e-5 3.647251920260201e-5 -4.6465652342208385e-5 0.00012628203599664346 5.7110655302891605e-6 -5.553913528497993e-5 -7.507118520375801e-5 -2.864719048598573e-5; -6.847838721998037e-5 -9.991352908374989e-5 -0.0001055645689118281 1.3994697838172852e-5 -0.0001805462516128645 -4.775208708426028e-5 -2.5409693080034265e-5 7.591687335918585e-6 0.00011534390159821088 -0.00015416799487840587 -2.0335547560276554e-5 -6.331477834438955e-5 0.00016125945057890732 -4.540443387642542e-5 1.9163630895748854e-5 6.950787883423187e-5 7.672932497396827e-5 -6.484157256610741e-5 -1.9393311049239834e-5 1.4185236978195102e-5 0.0001473621241532582 -8.95844632393345e-5 0.00018367340180727918 -2.6433709535667355e-5 6.585594747124718e-6 0.00010185551309430941 8.619813979774715e-5 -3.4717597825169725e-6 -3.997779913308183e-5 2.3616014914551076e-5 -2.63160809479066e-5 0.00022985375926535572; -0.0001559074115996235 3.2331194315476374e-5 -4.890794034731576e-6 -0.00011219705257667261 -3.5792185290342753e-5 -2.3049548510657626e-6 0.00020679663074566048 -2.06610983789208e-5 2.81375522695677e-6 -3.717862265170678e-5 -1.7452810343679628e-5 6.0096590541210817e-5 3.472123725236716e-5 -0.00017239979561986312 -2.5925883083114988e-5 -0.00013413560728644923 0.00015813822764335032 0.0001404479736172602 -0.00013199541345806668 6.1016347981198496e-5 8.263860589954412e-5 -0.00010763267701827329 0.00012680853623328007 -1.896853419589179e-5 4.379788887468307e-5 7.210932752946492e-5 5.486475501107193e-5 0.00014672021287855984 -7.48078199742359e-5 -5.188165951972205e-5 0.00016848296590046868 -5.266779763609969e-5; -0.00018211911149490352 0.00022923113944772652 -5.0249331915329287e-5 1.50120934186983e-5 8.91223164863911e-5 -9.50745459057368e-5 3.065978969163425e-5 -0.00015601891275662826 -4.408660673005281e-5 8.512935825921665e-5 -0.00012687509363684538 -4.773003886737635e-5 -9.324320920212499e-5 1.5834562210438314e-5 4.8913923156868715e-5 1.8481759317291004e-5 0.00023573439036328377 -4.9180890281496175e-5 -0.00011489574601808303 4.768967092895072e-6 0.00026578090844042306 -1.907459839356695e-5 -2.9712992481006527e-5 -4.080408391826853e-5 9.937422807626698e-5 0.00020834956310056393 1.2001217574943265e-5 -0.00012790382673200554 5.376274136426115e-5 4.0614478412968307e-5 6.384567611450428e-5 -0.000162398370529336; 6.825093374578964e-5 5.197363839830605e-7 -3.335581708591681e-5 4.980564996009565e-5 2.6537222857452056e-6 -8.809148925132326e-5 1.3807100240288665e-5 -4.1632666343442166e-5 0.000112948647492945 -0.00015741255696405715 0.0001046846657664584 -1.877159842473196e-5 -1.784083609884784e-5 -9.540280074355407e-5 -6.37152559101506e-5 -0.0001746790737706161 -0.000231588849364864 0.00024108002913641536 0.00010457094982490636 6.284408233119904e-5 -0.00015179132673431847 -6.077052306857798e-5 1.8313955715777306e-5 -5.211211167982715e-5 1.0136942601152516e-5 -1.0156773660068764e-5 -0.00011138957846918396 -0.0001394896032615441 8.460613686117172e-5 0.00014050759569733472 7.175846130909918e-6 0.0002610481817646952; -7.002370826366617e-5 -5.106787780813939e-5 0.0001432925769623622 -6.0367432029938036e-5 0.0002614535756439225 -9.752449346610743e-6 -0.00011116959054188046 1.4564485849501682e-5 5.4732353004606066e-5 0.0001050108826749296 -2.8776770210147576e-5 -0.00021153228722677796 -1.972661216849735e-5 -0.00017698574943633054 -7.08909951353186e-5 -0.00021033249636811404 0.00012432185625958184 3.8996232018785184e-5 -0.00011144529840375548 0.0001173811674963009 0.00026127356845255187 -6.956327838988016e-5 -1.3386972073830855e-5 -0.0001781363402696047 3.5561503449666485e-5 7.980289823909054e-5 -0.0001052690654031679 -7.726351164446e-5 4.9727589197669095e-5 2.7081604043003266e-5 2.1699660050313614e-5 0.00010321408494210545; 0.000208557849988628 0.00011113947141571466 4.709861174396069e-6 0.000137855812796677 1.8843950898789558e-5 9.677444643462762e-5 0.00011414959328836457 -4.9486002238647907e-5 -7.359415916236027e-6 -7.687753065833284e-5 -6.084503445303279e-5 -0.00012734968447470352 -9.817417128355986e-5 -0.00019467959179886857 3.383134492329044e-5 -0.00012810744635639062 -0.000124850436689979 -1.0283093500712565e-5 -0.0001018967840296145 0.00020154964761464605 -0.00010026951610920748 6.482763740355533e-5 -8.700202567847338e-5 0.00020038443210656232 5.832085764855876e-5 -1.9711788694204284e-5 -4.4581461109865615e-5 8.606037329305128e-5 3.70740747390149e-6 8.007586177559976e-5 1.0884558947925516e-5 -3.4412269882105496e-5; -7.422697903624872e-5 -7.489481735777707e-5 -0.00024892094601913496 3.70786051052727e-5 9.689584608618977e-5 1.0862010144555884e-5 -0.0002405210130246256 3.98886090397015e-5 -0.00030288317297621684 3.254935059427534e-5 -3.1010683001150635e-5 -8.407645013452556e-5 0.00018992414466535698 0.00017651922594719507 6.387575327149608e-5 6.626286399747263e-5 -5.6598257783694784e-5 -4.528130970815243e-5 -2.4962437732975195e-5 -6.458242800821592e-5 -7.741878603182283e-6 -8.642088009313119e-5 5.916942741803085e-5 0.00019641761669387614 7.140736620836752e-5 7.491552737970756e-5 0.0001599588262751778 -2.1445339866684955e-5 -0.0001259656733464354 1.5147155368109755e-5 -0.0002051364886438361 7.203133050548713e-5; -2.159497881946504e-5 -2.134217385318269e-5 0.00011487463569794437 -0.00021546796989567625 4.5991409648578535e-5 -0.00021722840239233896 5.4532623550054356e-5 5.8879156189621935e-6 -0.00015892369475621366 -0.00025069248141661675 0.00018308068473417853 2.750194624824808e-5 0.00012474154726659566 -3.933783438113124e-5 0.0001878828167595362 -1.2461244419524062e-6 -9.198580875235234e-5 1.8251677544339422e-5 0.00011640213568755016 1.5054546280606808e-5 0.0003478199629957018 0.0001561621153872714 -1.003409335099896e-5 0.00010481063298939619 -9.1638469087755e-5 2.6819804465232837e-6 0.00016895208471980533 5.70714088884325e-5 0.00010959616127949386 0.00014088963297254262 -7.130167279100299e-5 5.2245800985857894e-5; 6.238586386123643e-5 9.117573674690805e-5 0.0002045495991517601 0.0002084776267331782 0.0001809469019683818 -3.228003449814601e-5 -4.4114983462099916e-5 -0.00012658564926435023 -0.00015167921926669987 0.00010113434720940033 0.00010558566166244384 0.00012806956394731785 -5.997621817709457e-5 -1.2931565175423867e-5 -3.4507907253758794e-5 -6.619961029693845e-5 -6.625794892509532e-5 0.00010583802825229028 -0.00020346984936233685 0.00019539318484410414 -4.216923232501825e-5 0.00020606276550485159 -2.4115727934031092e-5 0.0001258139879830786 7.688773505313301e-5 -0.00010003003378708063 8.236302412504309e-5 -3.0223586941926334e-5 4.603920272009079e-5 -5.898235510879696e-5 -6.380496649651563e-7 -0.0001438044794446979; 0.0001531485357827098 -5.8804897098564516e-5 -5.292066480289389e-6 0.0001559643459313135 7.959446144919472e-5 0.00013301863861666608 -4.390488475148462e-6 7.376929340349078e-5 0.00012489618154173332 4.809717199346536e-5 0.00010628525694297254 -0.0001474218262319864 2.0827568614509572e-5 1.4280112618469538e-5 1.2789384409150318e-5 -0.000102658024961822 2.7062434919532456e-5 7.439761129679035e-6 -8.629096931148694e-5 1.694813897298262e-5 -1.990011344791509e-5 -1.0614390821541168e-5 -1.1935471893634369e-5 -2.9862820274541477e-5 -0.00010493927778046945 -5.5913045916933624e-5 -2.2774433209671578e-5 -0.00014706112790922307 -0.00017280460099758978 1.1295477515847536e-5 -7.577555891661407e-5 0.0001397165213984028; -7.245906754784296e-5 -0.00019401008040099318 -0.00020467261793479338 -8.61375986520706e-5 -1.811666816744006e-5 -0.0001308141393625719 -4.010882485695678e-5 0.00017401066424404033 -9.881015585657028e-5 -0.0001027414939987017 -7.136763024996941e-5 4.2205666153078866e-5 0.00011042508839943037 -0.00013644821893345413 5.977256359378199e-5 -3.1758045739991064e-6 5.8732520022523096e-5 5.9778820917328787e-5 0.00012411661436393897 -7.071601003795665e-5 -1.4847416702318032e-5 0.00021108418576964766 -7.310654773994394e-5 -0.00012905169870159122 0.0001685039575871451 -0.00017227842482194194 -4.357306471016276e-5 8.47789603479828e-5 -0.00011879198000920598 9.907540237105236e-6 -0.00017377811610154093 0.0001049144600013644; 3.421541212782677e-5 0.00017848797095547202 -1.1482518417602913e-5 -0.00010183441237705685 0.00024135885131153673 -0.00017500171135571485 -1.0641140321522063e-5 -3.1496500483909854e-5 4.25419125201117e-5 -4.2858675911986464e-5 0.00015603045129291171 -5.6242370661773574e-5 0.00011752773524101682 -0.00018053753639497483 -0.00011825127195725942 -7.634202530999464e-5 6.4720129409459e-6 0.0001642447746086802 6.444661219911101e-5 9.385125984744453e-5 0.00015255711285469946 -8.758773689184584e-6 -4.673478589786585e-5 2.855557021001359e-5 6.577653355970464e-5 -3.854153375533071e-5 4.006243715020314e-5 0.0001953394671397077 -9.223385178875912e-5 -8.570309106577303e-6 1.4442015057000899e-5 -0.0001385447835821711; 2.1407898503261834e-5 9.593560692391147e-5 -4.7385816896544564e-5 4.731966267225899e-5 -0.0002482421229337115 -4.732200674826759e-5 2.6717555857054658e-5 -8.264070190253982e-5 1.5496321184933076e-5 -8.519940153693118e-5 -4.183419939128277e-5 -3.148195598388567e-5 0.00010269012575972111 -5.639988800741287e-5 1.7421070270380343e-5 2.382899838799854e-6 8.149160869887804e-5 -0.00011918021179500511 6.375104870183805e-5 2.5057718931371288e-5 -8.191323710835339e-5 6.434507243337986e-5 3.430005684148854e-5 9.406639886091372e-5 0.00011397560168059937 3.534246509897024e-5 -0.00016860544262218825 -0.00018563338077856912 -6.822332640641795e-5 -1.4890337906226898e-5 0.00023276656004835546 -2.746749184499562e-5; 6.616636744536835e-5 1.2839582608552799e-5 -0.00010429555334314332 9.093474579812071e-5 -0.00024482527590054617 3.072294782567414e-5 -0.00013715422903816277 0.00010155072447913914 -7.178858288727115e-5 3.583997238264792e-5 -8.436354039633233e-5 -0.00016870042384299792 7.175549615013621e-5 -9.25647524364796e-5 -8.654769556061554e-5 -0.00011310309275915098 -1.9113503754948588e-5 0.0001406830145914539 -3.469735168084071e-5 2.800677645962824e-5 6.12920506445426e-5 -1.4721743924473833e-5 0.00011409716751033118 4.691628635000494e-5 1.516279314626611e-5 6.706858618952844e-5 -6.650139914756272e-5 -8.033022970820454e-5 -4.3628393201392015e-5 -4.747605783095499e-5 -6.403889494878001e-5 -0.0002115731164177468], bias = [-3.1191545595130843e-10, -2.0212426596476283e-9, 7.112152827835879e-11, -3.243830175005229e-10, 2.3623130303215395e-9, 4.857273311000907e-10, -1.6142140561750679e-9, -5.454064863858324e-10, -4.3685295932451826e-10, -1.5990931993431986e-9, 9.550369611522905e-10, -3.057664478431455e-9, 1.7217091029342412e-9, 2.7886926096957105e-9, -1.7295348785206703e-9, 1.7609525343764992e-9, -3.63288923365011e-9, 7.630274419652513e-10, 6.557459384335777e-10, 8.331787786529424e-10, 6.541732726041285e-10, -6.045774488692488e-10, -5.106411621077796e-10, 6.012910118557003e-10, -1.2392274956113632e-9, 3.1120056717862873e-9, 2.625460438220049e-9, 2.5090226537188095e-10, -2.6660078765665115e-9, 1.6730006732762932e-9, -8.555338492493487e-10, -2.951879758423589e-9]), layer_4 = (weight = [-0.0008930617061522622 -0.0008185306631164176 -0.0007798362903145225 -0.00046118253009551705 -0.000897480944822825 -0.0008989226899086836 -0.0007221957419533718 -0.000662921138863801 -0.0006982017109555845 -0.0007461286129488532 -0.0008023907532141756 -0.0006913845436001116 -0.000505657627539614 -0.0007713635499856448 -0.0005536322785323924 -0.0006477029338901213 -0.000641742644755058 -0.000757746606194418 -0.0006001426243246246 -0.0008224149504703233 -0.0006648955565676422 -0.0007083074698248665 -0.0005936614162959661 -0.0006482839826710184 -0.0006140691446771656 -0.0006285802307338592 -0.0006514905371313426 -0.0006512250660108998 -0.0007275369507994201 -0.0006771698666891329 -0.0007006402569624884 -0.0005875524598459974; 0.0001792947104920181 0.0002983343139061395 0.000150136063990197 0.00039841043602634435 0.0002460896645807945 0.00027568385565150753 0.00028777974423444203 0.00023729097376672882 0.0003556723695087773 0.00017636516606650494 0.00016136170006947375 0.0002398825145134938 0.00018523919873291076 0.00022491736075073136 0.00013145073567494362 0.0002141998426323734 7.340545978743833e-5 0.0003237640460638284 0.00015056999791557195 0.0002671154860365163 0.00012047243166020841 0.00013201640836588033 0.00021678054810414323 0.0003935834320404575 8.124913038213655e-5 0.00021554910660425 0.0003650959602604641 0.0002931727880915839 0.00040863465278771955 0.00022404868707168502 4.198412722765804e-5 0.0003513619042523084], bias = [-0.0006651773484053201, 0.0002744319173546264]))

    Visualizing the Results ​

    Let us now plot the loss over time

    julia
    begin
+u: ComponentVector{Float64}(layer_1 = Float64[], layer_2 = (weight = [-5.067828169553457e-5; 0.0002033883647527833; 1.4273280612531504e-5; -4.428054671737328e-5; -4.250684651193247e-5; 7.748175266906924e-5; 6.288575150393261e-6; 3.7066754884979796e-5; 0.00014335215382743272; -0.00011426929268051497; 0.00011086220183643978; 8.886728755885818e-5; -0.0002350791619390836; -0.00026994064683101955; -2.1895052668672963e-6; -4.137206633454982e-5; 0.00014377594925456472; -1.254786911885069e-5; 0.0001540251396362168; -6.948348891458086e-5; 9.75148054747745e-7; -2.275944098073367e-5; 5.759294799648488e-5; -9.247271009369957e-6; -7.65327495172892e-5; -5.430622331914719e-5; -5.6007382227096824e-5; 4.515377077039578e-5; -9.931653767119681e-5; 0.00011081036791418321; -2.595081787143777e-5; -2.3206786863723692e-5;;], bias = [-5.310483371894584e-17, -3.1011765244248956e-17, 1.2287450168839055e-17, 3.141598635647141e-17, -4.114087861200215e-17, -1.087215296099596e-16, -5.37164552124023e-19, 4.2209422355381365e-17, 1.3438114389553043e-18, -3.747220297384205e-17, -1.5831630774748768e-17, 1.2461438073596785e-16, -1.4656832046366923e-16, 8.680182063003482e-17, -2.656384109725839e-18, -4.3500024256058437e-17, 7.393138650556651e-17, -1.2508691195987064e-17, 1.6063779055314245e-16, 2.5819551063395286e-17, 1.742596203254984e-18, -1.739224497562247e-18, -3.550440848984027e-17, -1.8956723722780767e-18, -7.511083048774223e-17, 1.7363671075209743e-18, -1.260939208188153e-17, 1.8950031747943242e-17, -6.621050923994785e-17, 2.2468062521676514e-16, -5.786119776188296e-17, -1.7438878875030686e-17]), layer_3 = (weight = [-6.488304392744379e-5 5.1593381391560795e-5 -0.00010973868961057376 0.00011888554104759725 -7.869967270719461e-5 2.6729300549302098e-5 -4.338042821812204e-5 -0.00014315193879495047 2.6661781481641573e-5 -0.0001733976871434768 -2.6336444756196498e-5 9.159599244983697e-5 -0.00015508627006391803 -4.3109380607051135e-5 1.6835862658147272e-5 1.792073158528541e-5 -7.001163361902599e-5 -4.674345222495249e-5 3.036351404834028e-5 -2.1701625825915297e-6 2.439655395962534e-5 1.1773439615602058e-5 9.929855401085215e-5 0.00011054835591155347 9.563256531943786e-6 -4.707818629296995e-5 9.590648801920449e-5 -4.885345082923397e-5 0.0001208191922671903 -0.0002225035034313972 -9.059373882676194e-5 -3.268187469414162e-5; 0.00012063635157848382 -6.609181646762502e-5 -8.302260603794181e-5 -1.9541194621509316e-5 -6.673466915070392e-5 -1.8391583189296476e-6 -2.9914954854560117e-5 0.0001361124807708773 -0.00010268483375237967 7.048662454782473e-5 -1.6720200353106822e-5 7.6481518856795e-5 8.879042369972584e-5 3.933901033906554e-5 -2.3042796517059236e-5 1.4161023766989025e-5 -3.0308782431336086e-5 -4.231336125211143e-5 -5.7871654195867056e-5 -1.2361219621183299e-5 0.00011966117680732812 -8.016287093322083e-5 3.7681201586580233e-5 -3.460617945800818e-5 -9.54333233557773e-5 6.842894007878977e-5 1.1556674774525573e-5 -0.00011588641128308456 -6.523252587338435e-5 5.238058228589095e-5 0.00024232536751971285 7.974424345790755e-6; 8.224337639854286e-5 -1.218803148938489e-5 2.7367696618746988e-5 4.015808068084576e-5 -4.848885813975649e-5 5.793578080352111e-5 7.150272418380917e-5 7.017520388922748e-5 -8.642651968535577e-5 1.970483450145065e-5 4.980705549109859e-5 9.72932799938929e-5 9.627208479078838e-5 -9.517727752818883e-5 -0.00010894272722202147 -6.569878618596989e-5 -0.00014163293539446397 0.00020533117336322745 6.451986397017856e-5 -0.00010703093297911067 2.3383893365242713e-5 4.1318886958613766e-5 -9.12378557970847e-5 2.819155693737988e-5 3.441893264403294e-5 8.88329711031072e-5 -9.182458901909231e-5 0.000141080946066501 5.8570116978216505e-5 8.331143604459376e-5 8.733199925496392e-5 2.7256763730982073e-5; 2.0639607059931383e-5 -6.556483098836061e-5 -9.26717530500157e-5 1.2781671971535676e-5 -2.19400917966395e-5 4.872884706412406e-5 -0.00020592711193319547 0.00019216477262955044 -0.0001082424915192628 -0.00010664191908733837 -0.00011950996672101251 4.0736262194625994e-5 3.815132181910655e-5 1.5810668587742027e-5 -5.011134821404839e-5 -1.1253566340090383e-5 -9.231913831615613e-5 0.0001426945472804816 -9.049288750691079e-5 0.00019282329044938032 -8.383033677535268e-5 -3.1071643160232267e-6 1.3963474843987855e-5 3.5590439484819587e-6 -0.00016782591960225285 -5.056347985817206e-5 5.644821608723057e-5 0.00014241667845919624 0.00012270893235392902 1.297179547248012e-5 6.824178304225082e-5 1.0279214580006563e-5; -1.3928808630423847e-5 -2.580913592300915e-5 -3.3537372519475696e-5 -0.00015090158835866352 -5.633276218779253e-5 2.3137669768206132e-5 -9.296012563046942e-5 -1.6965826080393775e-5 8.621489496172741e-6 -2.2875613513145147e-5 -9.810577744266379e-5 0.0001954865410423597 -1.4209998016843574e-5 4.283326866226591e-5 -0.00014059855589059123 0.00012722292462947907 7.49152067781239e-5 -0.00013340454011105866 -7.446950899084912e-5 6.688554319983476e-7 7.388097810490849e-6 7.2399238431729244e-6 0.0001258476231212468 2.1599497767766602e-5 -6.403067804987242e-5 9.297352239608012e-5 -0.0001389406416367069 -2.049448585437825e-6 9.337108799607962e-5 1.4604740497695822e-6 -4.2784514380599464e-5 -4.6444779445863475e-5; -9.859231075038756e-6 -8.92166064731826e-5 9.193804707326044e-5 4.697279636463018e-5 -7.961775925591352e-6 -0.00015716880462417928 1.1460652596785891e-5 4.223163322609779e-5 1.0276937966449545e-5 -0.00013143228735129704 -6.983421208855968e-5 3.282657761888761e-5 -2.204743960464444e-5 4.16758846695159e-5 8.68817804003514e-5 0.00012558122379420223 9.739221404019534e-5 5.784216390754152e-5 0.00013859734444125957 -5.485830862011259e-5 9.106923225050862e-5 0.00010087260288134235 0.00014309589441114764 -1.206883208208888e-5 -5.216583947857849e-6 4.1126604439399654e-5 0.00015406645233457655 -3.986819156847103e-5 5.189840866665336e-5 8.516869255986725e-5 6.19362906873599e-5 -9.757490643817917e-5; 0.00010022519278392329 3.801946592103462e-5 -0.00011406373518001133 0.00020252994441945248 0.000124030307108128 -7.962125107938336e-6 -7.458327594393708e-5 6.384426238766085e-5 0.00011404573602886837 0.00016995428892370348 8.429847808839796e-5 4.9968596487828436e-5 0.00011434576741704361 0.0001638659131113016 -6.057462009083322e-5 -9.412515931943289e-5 -0.00015873539372295175 -6.482401944411764e-5 3.0919208131320155e-5 -2.1594392035681557e-5 0.0001210999806626343 -7.771278900507687e-5 -1.8475592587357495e-6 -9.604004060733888e-6 -0.000106139860816243 -0.00013608629809640216 -3.699267030059231e-5 -9.477077686641228e-5 -0.00010653282801102781 -1.4092722392875731e-5 -0.00010621809191251024 -1.5991102498432955e-5; -6.85945405234782e-5 5.955887999559289e-5 0.00013538516313245464 -5.856513863138032e-5 -1.4058173587873332e-6 -9.187440346751602e-5 4.963713681034698e-5 -2.688422825954933e-6 0.00010794513317930726 3.837898211029819e-5 0.0002389099909964718 -0.00011218212503765081 3.951167408783001e-5 7.633592858151902e-5 -6.472568102937236e-5 2.6780157979814266e-5 1.481328499037518e-5 -8.19167534314287e-5 1.378444730332931e-5 -0.00013009375007415393 0.0001079858057823773 9.007736761530801e-5 -5.176919413089905e-5 -3.248019757621968e-5 1.4517148058514547e-5 7.808188192255654e-5 -4.971269564295335e-5 -4.636922941018939e-5 0.000126796724628076 7.444591127976329e-5 -0.00014616011222416605 -9.482908981910036e-5; -5.844121978835604e-5 -7.837145376351397e-5 -0.00015722129746674127 0.00013298625281212424 0.00020579882702922013 0.00011313802979320028 -6.057538548585531e-5 6.835987097737537e-6 8.129581992879848e-5 5.070884735805013e-5 0.0002587695883460528 -0.00015811274779362782 -2.55861693391912e-5 -0.00010511739374003677 -0.00012036865218643604 9.864171015696166e-5 -4.523079677439581e-6 -0.00011798592886290081 2.915397136620633e-6 0.0001434153630737371 -5.82827676214142e-5 4.964991176283433e-5 -6.646095125571264e-5 -0.0001173525713045014 2.099080602537575e-5 -1.2118468201797582e-5 -0.0001377560881325826 -5.26452283326858e-5 -8.246047126437611e-6 -6.679808275176214e-5 -0.00011273330592185735 2.2168856322686e-5; 1.807031263771555e-5 -8.41687352815843e-5 4.027746114181075e-5 -9.122214318020622e-5 1.6445567655606466e-5 8.070489332307503e-6 -4.988372814958256e-5 0.0001852610738412444 7.757903300690267e-5 -0.0001033573636389419 -5.785060342285634e-5 3.672856188378627e-5 0.00014628277937180577 0.00011685057767197896 0.00014759533302159713 0.00010566372735658319 0.00016843380659584684 7.314829496200851e-5 -0.0002992532673827499 -9.248457275797153e-5 -0.00010036986271851036 2.3871129872274463e-5 -8.735494988039545e-5 -0.00014760663829045643 8.600705119780681e-5 -0.00010550347282085964 -1.5242485585335465e-5 -2.606376723386703e-5 0.00014419519071699367 9.088176817632428e-5 4.8678929363374415e-5 4.9175829974698705e-5; -6.598751317803042e-5 -6.030990155271338e-5 -7.217605709890307e-5 -7.999448100896822e-5 -0.00020351446985593347 8.729900034517088e-5 3.311363431997184e-5 -7.431861561350837e-5 -6.288851661567153e-6 -4.8014057053713016e-5 0.0001289141673375904 4.1720859346913115e-5 -8.892866574317183e-5 0.00024919270251556733 4.605136379971683e-5 8.19395226906025e-5 -4.1751989581145076e-5 -4.782373255445095e-5 -7.793842635442057e-5 -4.7223255048506476e-5 -5.63738012963482e-5 -2.9043554629668638e-5 -4.210199952218848e-5 -4.4029447987912596e-5 -2.1515656069153655e-5 0.00010635621035611067 -0.00018420626113517394 7.879130314243693e-5 6.568936985126093e-5 -0.00014872338035935024 -0.00025311636093398925 0.000164341023239769; -3.258934873928717e-5 -6.829450274425075e-5 -8.627620252854646e-5 -2.488858615878919e-5 -0.000108696694842814 -1.8759535924339176e-5 -5.000216686357272e-5 0.00010093201137306672 -0.00017555949296410188 8.38374982045109e-5 -0.00019008694642297169 -2.05099385134345e-5 9.822381263758857e-5 -0.00014449343860221752 -4.487419562837569e-5 -0.00011297130538923156 5.281707864879642e-5 0.0001025940219910863 1.2051929042638165e-5 0.0001414263898545934 1.6517353396471644e-5 0.00011433706833441294 -9.066640621357982e-5 3.489854872851174e-5 2.456762205793661e-5 3.055989884115363e-5 0.0001093042593488524 -4.3657451788470566e-5 5.348321075826649e-5 5.2719075137710264e-5 -6.0060319702182694e-5 6.099252652302867e-5; 4.684605982778897e-5 0.00015922139527554951 3.730149137131014e-5 0.00010522137699458063 -1.7152006424891324e-5 9.699498286513317e-5 8.678879339289912e-5 -5.9812624844068647e-5 0.00010268462164837113 -0.00011774689528815062 -9.323463064342426e-5 -7.073884671976903e-5 9.475809156010668e-5 -2.0643809424456035e-5 -0.0002366002740962891 -0.00015139882498611479 -7.254450656895337e-5 0.00018710389803904177 -7.661572499622105e-5 -6.959221400743762e-5 -0.00023375447245039922 0.00011397922162008577 6.118481496391263e-5 -0.00014835137745090697 -2.0492704155778817e-5 2.8101463154730067e-5 2.545392813903837e-6 6.996500405525111e-5 -9.364928019190132e-5 -0.00014223980588562476 -0.00017213980072834412 -0.00018664228310068032; -0.00011808607541993425 4.8038628844860654e-5 1.6432158188171453e-5 -0.0001537796242725119 7.191083113625497e-5 -0.0001872625625789189 -5.120785590647832e-5 -0.00010370548716550833 0.0002961341187959484 3.4529158253777705e-5 -7.646508766143015e-5 5.085647262362489e-5 9.7541148531797e-5 6.58443632322203e-5 -4.7161783188722274e-5 -2.1120003114129957e-5 -0.00011250213447297528 8.57567456454027e-5 -1.6246659472613986e-5 -0.0002794219808517045 1.363162025290119e-6 8.212484226291877e-5 6.171654513068318e-5 6.960662267133479e-5 -1.3250552913316052e-5 0.0001021550660856051 0.000116517733656427 -2.596387548933425e-5 4.812307360893733e-5 4.640898532429348e-5 0.00014826165988538294 -7.550835744282355e-5; -6.17154759238133e-5 -2.249270200262119e-5 -0.00011052547360301221 0.00010846958012556371 -6.269449967654701e-5 -9.421800804554029e-5 0.0001429795125732551 3.136664291381576e-5 8.234448306032229e-5 7.665271012258093e-6 -0.00010900864016727386 -8.210261868921642e-6 -4.782299164798104e-5 -0.0001506850488949556 -7.13813455076535e-5 0.0001220554519095803 0.00010132866428469075 -2.9438787707441644e-5 8.61451813834961e-5 -3.230415622234766e-5 5.361084241307146e-5 9.664669469933737e-5 -1.5967776009313548e-5 -8.983938040906761e-5 5.9496953879748424e-5 1.088932604765186e-5 4.291803689560512e-5 -0.0001035132185206405 0.00012603782361842033 5.6519148172840214e-5 6.545989582085756e-5 0.00019466919603295402; -6.035587200219031e-5 -3.048070636485676e-5 1.9147372703658136e-5 4.1489395634045756e-5 -2.497117481740846e-5 -9.497523485239959e-5 -5.7456613895707984e-5 0.00010517491403246128 6.864511026744694e-5 -5.116598977995619e-6 9.274737470061827e-5 1.2732370968505513e-5 2.075340574175034e-5 -1.2390137336297228e-5 3.9403538861096767e-7 1.626717760612538e-5 8.14697204341642e-5 9.05253045212093e-5 0.0001821270206428582 7.847962927283407e-5 1.3242259164174353e-5 -6.920653601898014e-5 1.9616315447848574e-5 -0.00020374188086040435 3.897193248943069e-5 -0.00012801252075493572 -6.932340121209963e-6 0.0001030455395530517 0.00010479109999235196 4.828026639213363e-6 6.819267667109008e-5 -0.00012826038352701946; -2.8094946705911407e-5 -3.083253303418634e-5 2.28271104925644e-5 8.898699471965577e-5 1.207003371301013e-5 -3.803506896010253e-5 -1.0439388993198004e-5 5.42278607236792e-5 4.867334374931775e-5 -4.518214753706244e-5 7.921917441772954e-5 -0.00010169376454077221 0.00014349088761731043 -7.677398078600014e-5 0.00011103335729340141 -5.2174575634941974e-5 1.2260983704556828e-6 -2.8247074246699347e-5 9.942554189830378e-5 1.1441331857990563e-5 1.461823480987718e-6 -5.285105051816366e-5 -4.745934037136632e-5 -5.7792964603229816e-5 -2.5201509365519844e-5 2.534265824133726e-5 9.225348495884073e-5 5.786641433142276e-5 -4.426601716194696e-5 -0.00022661089732072592 -0.0002966056968806676 -9.220337650267332e-5; 6.735962678692488e-5 -4.1766666167990785e-5 -1.2230763882255954e-5 -0.00010595445483443115 5.75487764716636e-5 -6.401900018784194e-5 -0.0001401495784240279 0.00011234512648559416 3.367131002493477e-5 -0.00012897993574836596 -0.00012430046181646445 7.917755001005653e-5 0.00021940120941039688 -1.7720901543366787e-6 1.3755347840630702e-5 3.097948219408506e-5 1.2615215286920207e-5 3.2222539534702144e-5 1.179245272839102e-5 0.0001234499840346639 2.063462388366766e-5 -0.00019241103573688025 3.0956541099726786e-5 -4.40704671792656e-5 0.00013447786195217666 0.00015082376639372836 2.4749576273166298e-5 7.190370242358394e-5 5.500512351761415e-5 -2.351658263368668e-6 0.00010669987104381709 -0.00015557776633170152; 0.00015004433877961992 7.171248505085587e-5 7.186821237169626e-5 -4.4490909958627636e-5 0.00010725007375199672 -3.0032770909973542e-5 -6.131717312186528e-5 -7.843955090580263e-6 7.125566405206153e-5 -1.1301954900639649e-5 4.3303734100979687e-5 1.4667064688319504e-5 7.304025272798725e-5 0.0001203544299370227 3.312941696108749e-5 -1.6958564474267366e-5 9.650549979609595e-5 -0.00011026335489973004 1.5609365773470854e-5 -7.302637171037346e-5 2.9099054854809883e-5 -0.0001471923977349845 1.1051333940625235e-5 -2.716893762212287e-5 6.98817959074871e-5 -0.0001721674278358903 -5.542232323009273e-5 0.00015735103003876595 -2.357148589894786e-5 -9.349403471653878e-6 -4.482258084849063e-5 0.00015532227477720406; 9.200155002588563e-5 -7.894742336531445e-5 6.909560581959909e-5 6.397325162368042e-5 -7.337227905111243e-5 1.81069772469319e-5 -1.3611788720594416e-5 -0.00011573251865513483 6.840369863028096e-5 0.00016948325176315674 -3.591863239591659e-5 -0.00010666078921815155 5.9028980128354384e-5 -1.1826329658209398e-5 5.607247478735757e-5 4.8404866926718095e-5 0.00019965971844328957 -7.507221923617522e-5 3.281272622814832e-5 -0.00026297763722425135 0.00013116674929449456 -6.360763834265864e-5 1.3418215555135889e-5 -5.0055308723243565e-5 -6.63661867048385e-5 -5.73548986477575e-5 -0.00027880083687079906 5.80557334859591e-5 4.770976854049545e-6 0.00023913896259444115 -0.00027277934166079255 0.00013627896808145697; -3.847454513714503e-5 0.00013499990222799242 -0.00011172731115191943 -1.5791521720241938e-5 1.746134534943557e-5 8.991045092290287e-5 -0.00013876824755396457 0.00010837116337672065 2.5287605465845612e-5 4.091738287416977e-5 -8.683316787181636e-5 2.7275808177650394e-5 -0.0002117683668545441 -4.1985100098565496e-5 2.5652245538656208e-5 -4.386319574376392e-5 0.00010112778750072644 8.595907479349758e-5 -0.00011121149668878578 4.6184361279588816e-5 -2.018655118416289e-5 0.00010101308202894625 -6.718531017366678e-5 0.00016857952345919098 -6.646065389912549e-5 -8.186676655139642e-5 5.893228649103207e-5 6.0351993168584786e-5 0.00011074483540898065 2.197029266272924e-5 8.477605501713446e-5 2.8409362356176015e-5; 0.00015168579624614677 -0.00015583055870494473 9.538919617630628e-7 3.832680953664729e-5 3.7978788815051777e-6 -2.4169362640261138e-5 -5.114657620551897e-5 -1.847409772281022e-5 2.1159557864773197e-6 2.0241523762792873e-5 -0.00019447271848448938 7.808523581943265e-5 -1.986637222118892e-5 2.674661913358162e-5 2.334530358084579e-5 -7.56245056062654e-5 3.0029402060874113e-5 0.00020181267676989644 0.0001235838214971321 -4.559180457263342e-5 0.00010591180829678763 0.00026848768970112405 -3.4460484365718736e-5 2.3217581420886376e-5 8.163944652651308e-5 -1.3016809370038249e-5 0.00011559072367263004 -6.941009565607624e-5 -0.0001963581228291365 -8.809125507369087e-5 0.0001532562971452538 -0.00014338266124368565; 1.757703004370722e-5 -0.0001143078671567958 -2.315727859711487e-5 -1.965702457271244e-5 -0.00010862327061220267 8.574516952178291e-5 -6.518635550652207e-6 -4.5364523507647356e-5 -5.5962072188025316e-5 0.00012402023541183133 5.957240402024154e-5 0.00014366027145554176 -2.3027693792000462e-5 -2.1112743602468684e-5 1.875388162700245e-5 6.527077759058019e-5 -6.91717624114733e-5 0.00015894688377995764 8.440316279559432e-5 0.00015212992267590782 4.881772291117934e-7 -7.422075312677048e-5 4.410387820355415e-5 -5.293577222362373e-5 3.839174579736624e-5 -0.00026345439717731614 -0.0002799995755460038 -0.00010818517792829506 0.000100323169058197 0.0001395339012695561 -0.00015231897585652043 9.96289262864811e-5; 3.5567809000901426e-5 -7.706454964950166e-5 -0.00021089415570622416 0.00011735034019785079 -8.083291894757975e-6 0.00020298259875836352 -7.773799045844129e-5 0.00011110823067702369 0.00021550229536935868 -0.00011431477508174869 -5.7657664391610264e-5 -9.974130119097646e-5 -0.00012480812477309142 0.0001282768203701277 -7.215671251196544e-5 -9.859565073292652e-5 -5.0619583297677294e-5 6.369928412005192e-5 -1.0572646196095261e-5 -8.19449491509571e-5 -9.611637545185293e-5 4.977304667682311e-5 7.011605300341022e-5 -3.112052984669926e-5 -0.00011037792903970437 -0.0001152873232322168 -7.007802235329321e-5 -0.0001132779438457816 2.469945827307626e-5 -0.00013523812270157737 0.0001209863399444701 -0.00025738433799142086; 0.00026173845471244757 1.2837046954176047e-5 -1.7063366256061072e-5 -0.00010499090931947774 -1.4816968912365366e-5 0.00016174985866423866 5.1993764759212034e-6 0.00012913095487643582 1.9697625605151052e-5 0.00014575480089883838 6.317605074623253e-5 -5.233565486875458e-6 -5.676636065581032e-5 -0.000145716190315948 -0.000178552077711665 5.977641412069611e-5 -0.00018953242907405934 6.730883105088676e-5 0.00015681993171932752 7.13500070491646e-5 6.833150327334844e-5 -3.3560644238060075e-6 -5.0175547228302636e-5 2.9539240488211657e-5 1.9837260326726308e-5 4.025970555854449e-5 -0.00021762433389771325 9.82363948355292e-5 2.9598630492229406e-5 -5.052273409778598e-5 0.00011400749862556023 0.00014911272809737047; 3.034694161183524e-5 0.00011196417396575523 6.701272574869138e-5 -0.0001320288436821541 -4.679154592881643e-5 8.550622234496614e-7 9.169767069547579e-5 -2.3225470459100275e-5 -0.00010554614057612856 8.78538332196153e-5 5.0879133750331555e-5 -9.156671952072757e-5 -6.564313010158575e-5 3.4502088210235905e-5 -0.00016190548270092207 -6.214793468675314e-5 1.916295256675445e-5 -7.877992978169433e-5 -4.0810577802704876e-5 2.1361170338108406e-5 -7.858580723254803e-5 -8.121608362079694e-6 0.0001343505565907006 4.441404523245204e-5 -0.0001237098920958899 0.00010197222506544903 -4.80500405533108e-6 -0.00010022550925927883 1.2405247771001035e-5 -0.00015726659406058402 -0.00010572152025847508 -3.0795567754784686e-5; 8.026301135713982e-5 -2.2176003524236732e-5 -0.0001247968009306606 -4.023423782662333e-6 -0.00021152006023158516 2.9970157145265214e-5 -4.1155971969154164e-5 0.00014840522114126738 5.487954028918456e-6 0.00029170960103101106 0.0001364494985568802 -0.00017074520085998894 0.0001822852957661807 -0.00011459839673306628 3.7666128507673295e-5 -1.6101906778665383e-5 0.00018429903345122147 -0.00011690346376099852 -1.4439217677594005e-5 -6.349963017101542e-5 -9.64337575464998e-5 -0.00017683811686993795 5.719009847552347e-5 -3.922447443302018e-5 -3.1977191367793196e-5 -2.1097126348202013e-5 -3.903475528666147e-6 -0.00013517490672939649 0.00016719957263148342 -0.00013073334205956205 8.546639061704991e-5 -1.5269046100464052e-5; -4.285495149137722e-5 -0.00015808186933328908 -8.367387156383673e-6 2.5162637734446107e-6 -0.00013412299947280758 -9.557863583954096e-5 8.171517631804133e-5 -5.246708812823869e-5 -1.7444965328947305e-6 9.215804541550998e-5 -7.319868516109516e-5 -0.00027948244448684073 0.00015721960019390505 3.110966875056439e-5 -0.0002053108669065728 -5.454849221909746e-5 -6.301397636464194e-6 -0.00014283853440478876 3.911820979507831e-5 3.7215132149372895e-5 -6.609561157579041e-5 -1.8971257737095413e-6 -9.650786293836533e-5 2.1515796586256786e-5 5.0987890542689164e-5 3.5147839778305374e-5 0.00013020407143581424 0.00012702088363535395 -7.683541977943642e-5 0.00011238765958065533 2.683416848824828e-6 4.7270469192383486e-5; -4.81021370076556e-5 4.5161818924709434e-5 0.00011199531312668745 5.2031405532107285e-6 -6.459508855258904e-5 0.00010572515478926155 3.919753489032578e-6 6.903685552249666e-6 -0.00015121389420599422 1.7531731863723377e-5 -3.0011130512125973e-5 -3.682348957298774e-5 -0.0001382626168931522 -0.00013444802149092396 -0.00011880085636293478 -0.00011516978243961795 -2.658730314649844e-5 -1.0312354753107738e-5 -0.00013526252129011354 8.309089061376478e-6 1.3917499963287668e-5 4.067580179181019e-5 5.7374953975530736e-5 -4.978239208939439e-5 0.00015902992621632663 3.7446948785966035e-8 -1.5800279434907896e-5 5.3139531736822475e-5 -0.00014180776903302748 -0.00013860250597716513 0.00013365241924403315 0.00013656724062385112; 0.0001716806770309658 -2.2274967104751507e-5 -9.818170232392436e-5 -6.3892529706206e-5 -9.407238698258605e-5 -3.0187476022524926e-6 8.873553148078214e-5 6.795901996156597e-5 -0.00012123663631427866 7.99799860578801e-5 0.0001351110883297239 0.00013809933867378477 -2.510291900762605e-5 0.00012684979143159005 -0.00014433913323955196 -9.857484414169174e-5 0.00017308623197095634 4.2674845335571465e-5 -1.679097053056454e-5 -8.397930227147059e-5 1.064211361154162e-5 -5.046389500421066e-5 -0.00014105983003900941 -2.9611733534690534e-5 1.5866286648817994e-5 6.131247633675491e-5 -1.0022676488003671e-5 -2.2031875541867535e-5 1.003374260108281e-5 -0.0001272431503275569 -1.5284858218362233e-5 0.00014202934531568027; -1.4122403897882985e-5 -5.004591393831892e-5 7.915055148453373e-5 0.00010705464201430764 -4.692567044719153e-5 4.8652516581687163e-5 -6.473215789617122e-5 -7.822139041115574e-5 -1.070142779814479e-5 2.0027607225820363e-6 0.00012267875690861888 3.5943811747557506e-7 1.9521455853597164e-5 -0.00010675962430272944 -0.00013979616078165446 -4.262539032231118e-5 -6.636686184525886e-5 -2.5302475749296935e-5 0.0001797804577128186 8.46011095765146e-5 -9.998503837584602e-5 -0.0001354948329188775 -9.500132635249613e-6 -4.3619882760933495e-5 -0.00013717771825929085 4.5246957326643204e-5 0.00016297076918101878 7.414646797963396e-5 -0.00013415243329097305 0.00012045182638951863 -8.078019918491195e-5 1.8181799097218416e-6; 8.464045504599007e-5 -0.00019113839834670673 -0.00013323667689899772 -5.1797763714314024e-5 3.05359831175501e-5 -3.606974450729679e-5 5.702082171712219e-5 -6.241512482727741e-5 -3.756840623890042e-5 -9.84303728698095e-5 0.00024355072355621558 -0.00013379563051483503 -6.849695227782469e-5 1.1523379756114767e-6 -1.693282638069095e-5 6.92288854255075e-5 0.00015571431549989628 -8.287424452344741e-5 4.693981873362311e-5 0.00018321261859757007 7.156047561982724e-6 -0.00023000075770749568 -2.324903786468119e-6 9.687769898238739e-5 8.636373650307874e-5 8.07963410837309e-5 6.684051595886362e-5 4.869451049973391e-5 -7.006313350468768e-6 -5.831089962910616e-7 -4.4962754597401675e-5 -6.1184050506944375e-6], bias = [-2.042037274184826e-9, 5.450538238276147e-10, 2.5289934987238694e-9, -5.148148177753555e-10, -1.2833426980289933e-9, 3.2157841518188328e-9, 2.881843239953348e-10, 1.0123517173553542e-9, -1.2702900780703617e-9, 1.3104570071210386e-9, -2.326512247765953e-9, -7.056591985203844e-10, -2.954497048078123e-9, 6.357343856216914e-10, 1.4448305944960618e-9, 1.2741375330697258e-9, -1.5175691242038057e-9, 1.4450914722628342e-9, 1.8109587064243945e-9, -4.019722143932669e-10, 1.1298182630716548e-9, 1.2157589486701711e-9, -3.2476707575809744e-10, -3.245467265503101e-9, 2.5269332221808443e-9, -2.1689905677970475e-9, -5.654624220598772e-10, -2.0276934697509827e-9, -1.7125428650239895e-9, 3.680361289786686e-10, -9.315934878893783e-10, 2.0782255714890612e-10]), layer_4 = (weight = [-0.0007904217429736376 -0.0006134657177658567 -0.0005707143385862188 -0.0006837621003302308 -0.0008341402321897578 -0.0005106042139339528 -0.0007003734721873451 -0.0005920055483634651 -0.0007073437719725264 -0.0004130149265874651 -0.0006887888845526638 -0.0006701476440012 -0.0007681066177949802 -0.0005443204642705102 -0.0007025799817840762 -0.0007784759925414032 -0.0007973507560408622 -0.0006752262663322216 -0.0004625431346698359 -0.0006631387855498853 -0.0007774837715133924 -0.0007242005769272898 -0.0007606024799357885 -0.0009091325645501539 -0.0007827024388413978 -0.0010325599774881834 -0.0005925590173431629 -0.0006246384804114001 -0.0007831547945311147 -0.0007289307705100168 -0.0005963366240466802 -0.0007263039337167419; 0.00029680230859930646 0.00044045159335233126 0.00024182199209542147 0.00029528814104235834 0.0002953885231182095 0.00026806443487209774 6.535156190517071e-5 0.00020707335391228292 0.00015261088000378096 4.947066717177559e-5 0.00029179127530466097 0.00017615732457630542 0.0003917327014566102 0.00020560982886453667 0.0002494370336098561 0.00034963475290870455 0.0002559733568663764 0.00020444168978222695 0.0003026522687613105 1.740610020494017e-5 0.00033405789094706863 0.00019530848486119698 0.00013907814899625118 5.343438806550787e-5 0.00017462112606557147 0.0003617533259373307 0.00011445079170911656 0.0003683949033299282 0.00028043017727226373 0.00023786085032865976 0.0001228108338075376 0.00032624596727140253], bias = [-0.0006861029847291114, 0.00022974120193819142]))

    Visualizing the Results ​

    Let us now plot the loss over time

    julia
    begin
     fig = Figure()
     ax = CairoMakie.Axis(fig[1, 1]; xlabel="Iteration", ylabel="Loss")
 
@@ -255,7 +255,7 @@
     scatter!(ax, 1:length(losses), losses; marker=:circle, markersize=12, strokewidth=2)
 
     fig
-end

    Finally let us visualize the results

    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
+end

    Finally let us visualize the results

    julia
    prob_nn = ODEProblem(ODE_model, u0, tspan, res.u)
 soln_nn = Array(solve(prob_nn, RK4(); u0, p=res.u, saveat=tsteps, dt, adaptive=false))
 waveform_nn_trained = first(compute_waveform(
     dt_data, soln_nn, mass_ratio, ode_model_params))
@@ -281,7 +281,7 @@
         position=:lb)
 
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -313,7 +313,7 @@
   JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
   JULIA_PKG_PRECOMPILE_AUTO = 0
   JULIA_DEBUG = Literate

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/beginner/1_Basics.html b/previews/PR1000/tutorials/beginner/1_Basics.html index 2665758b66..76c22ce671 100644 --- a/previews/PR1000/tutorials/beginner/1_Basics.html +++ b/previews/PR1000/tutorials/beginner/1_Basics.html @@ -5,12 +5,12 @@ Julia & Lux for the Uninitiated | Lux.jl Docs - + - + - + @@ -244,7 +244,7 @@ 1 device: 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.627 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/beginner/2_PolynomialFitting.html b/previews/PR1000/tutorials/beginner/2_PolynomialFitting.html index 44df221e98..c5d3c773c0 100644 --- a/previews/PR1000/tutorials/beginner/2_PolynomialFitting.html +++ b/previews/PR1000/tutorials/beginner/2_PolynomialFitting.html @@ -5,12 +5,12 @@ Fitting a Polynomial using MLP | Lux.jl Docs - + - + - + @@ -151,7 +151,7 @@ 1 device: 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.609 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/beginner/3_SimpleRNN.html b/previews/PR1000/tutorials/beginner/3_SimpleRNN.html index 760a100e90..8643eb0619 100644 --- a/previews/PR1000/tutorials/beginner/3_SimpleRNN.html +++ b/previews/PR1000/tutorials/beginner/3_SimpleRNN.html @@ -5,14 +5,14 @@ Training a Simple LSTM | Lux.jl Docs - + - + - + - + @@ -126,455 +126,455 @@ return (train_state.parameters, train_state.states) |> cpu_device() end -ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.63922
-Epoch [  1]: Loss 0.59006
-Epoch [  1]: Loss 0.56046
-Epoch [  1]: Loss 0.53851
-Epoch [  1]: Loss 0.51878
-Epoch [  1]: Loss 0.50837
-Epoch [  1]: Loss 0.47988
-Validation: Loss 0.46056 Accuracy 1.00000
-Validation: Loss 0.46788 Accuracy 1.00000
-Epoch [  2]: Loss 0.47123
-Epoch [  2]: Loss 0.44911
-Epoch [  2]: Loss 0.44093
-Epoch [  2]: Loss 0.43190
-Epoch [  2]: Loss 0.41116
-Epoch [  2]: Loss 0.39899
-Epoch [  2]: Loss 0.42438
-Validation: Loss 0.36182 Accuracy 1.00000
-Validation: Loss 0.37078 Accuracy 1.00000
-Epoch [  3]: Loss 0.37969
-Epoch [  3]: Loss 0.36815
-Epoch [  3]: Loss 0.34446
-Epoch [  3]: Loss 0.34307
-Epoch [  3]: Loss 0.31119
-Epoch [  3]: Loss 0.30866
-Epoch [  3]: Loss 0.29978
-Validation: Loss 0.27641 Accuracy 1.00000
-Validation: Loss 0.28617 Accuracy 1.00000
-Epoch [  4]: Loss 0.29006
-Epoch [  4]: Loss 0.27774
-Epoch [  4]: Loss 0.26425
-Epoch [  4]: Loss 0.26136
-Epoch [  4]: Loss 0.25087
-Epoch [  4]: Loss 0.23835
-Epoch [  4]: Loss 0.21375
-Validation: Loss 0.20609 Accuracy 1.00000
-Validation: Loss 0.21591 Accuracy 1.00000
-Epoch [  5]: Loss 0.20882
-Epoch [  5]: Loss 0.22010
-Epoch [  5]: Loss 0.20382
-Epoch [  5]: Loss 0.19422
-Epoch [  5]: Loss 0.18539
-Epoch [  5]: Loss 0.17881
-Epoch [  5]: Loss 0.16316
-Validation: Loss 0.15119 Accuracy 1.00000
-Validation: Loss 0.16004 Accuracy 1.00000
-Epoch [  6]: Loss 0.16827
-Epoch [  6]: Loss 0.15449
-Epoch [  6]: Loss 0.15191
-Epoch [  6]: Loss 0.13655
-Epoch [  6]: Loss 0.13572
-Epoch [  6]: Loss 0.13289
-Epoch [  6]: Loss 0.13403
-Validation: Loss 0.10993 Accuracy 1.00000
-Validation: Loss 0.11702 Accuracy 1.00000
-Epoch [  7]: Loss 0.11842
-Epoch [  7]: Loss 0.12253
-Epoch [  7]: Loss 0.09597
-Epoch [  7]: Loss 0.11399
-Epoch [  7]: Loss 0.09454
-Epoch [  7]: Loss 0.09874
-Epoch [  7]: Loss 0.08202
-Validation: Loss 0.07828 Accuracy 1.00000
-Validation: Loss 0.08338 Accuracy 1.00000
-Epoch [  8]: Loss 0.08202
-Epoch [  8]: Loss 0.08031
-Epoch [  8]: Loss 0.08125
-Epoch [  8]: Loss 0.07767
-Epoch [  8]: Loss 0.07224
-Epoch [  8]: Loss 0.05991
-Epoch [  8]: Loss 0.06217
-Validation: Loss 0.05471 Accuracy 1.00000
-Validation: Loss 0.05807 Accuracy 1.00000
-Epoch [  9]: Loss 0.06353
-Epoch [  9]: Loss 0.05217
-Epoch [  9]: Loss 0.05646
-Epoch [  9]: Loss 0.05128
-Epoch [  9]: Loss 0.05138
-Epoch [  9]: Loss 0.04732
-Epoch [  9]: Loss 0.04093
-Validation: Loss 0.04107 Accuracy 1.00000
-Validation: Loss 0.04349 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04294
-Epoch [ 10]: Loss 0.04586
-Epoch [ 10]: Loss 0.04159
-Epoch [ 10]: Loss 0.04004
-Epoch [ 10]: Loss 0.03786
-Epoch [ 10]: Loss 0.03902
-Epoch [ 10]: Loss 0.03803
-Validation: Loss 0.03334 Accuracy 1.00000
-Validation: Loss 0.03532 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03516
-Epoch [ 11]: Loss 0.03884
-Epoch [ 11]: Loss 0.03437
-Epoch [ 11]: Loss 0.03213
-Epoch [ 11]: Loss 0.03224
-Epoch [ 11]: Loss 0.03263
-Epoch [ 11]: Loss 0.02999
-Validation: Loss 0.02831 Accuracy 1.00000
-Validation: Loss 0.03002 Accuracy 1.00000
-Epoch [ 12]: Loss 0.02805
-Epoch [ 12]: Loss 0.03113
-Epoch [ 12]: Loss 0.02716
-Epoch [ 12]: Loss 0.03118
-Epoch [ 12]: Loss 0.03009
-Epoch [ 12]: Loss 0.02828
-Epoch [ 12]: Loss 0.02706
-Validation: Loss 0.02463 Accuracy 1.00000
-Validation: Loss 0.02615 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02656
-Epoch [ 13]: Loss 0.02839
-Epoch [ 13]: Loss 0.02328
-Epoch [ 13]: Loss 0.02764
-Epoch [ 13]: Loss 0.02471
-Epoch [ 13]: Loss 0.02318
-Epoch [ 13]: Loss 0.02648
-Validation: Loss 0.02177 Accuracy 1.00000
-Validation: Loss 0.02313 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02412
-Epoch [ 14]: Loss 0.02352
-Epoch [ 14]: Loss 0.02210
-Epoch [ 14]: Loss 0.02238
-Epoch [ 14]: Loss 0.02261
-Epoch [ 14]: Loss 0.02210
-Epoch [ 14]: Loss 0.02290
-Validation: Loss 0.01947 Accuracy 1.00000
-Validation: Loss 0.02070 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02160
-Epoch [ 15]: Loss 0.02149
-Epoch [ 15]: Loss 0.02041
-Epoch [ 15]: Loss 0.02102
-Epoch [ 15]: Loss 0.01964
-Epoch [ 15]: Loss 0.01899
-Epoch [ 15]: Loss 0.01997
-Validation: Loss 0.01755 Accuracy 1.00000
-Validation: Loss 0.01868 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01773
-Epoch [ 16]: Loss 0.01748
-Epoch [ 16]: Loss 0.01898
-Epoch [ 16]: Loss 0.01945
-Epoch [ 16]: Loss 0.01935
-Epoch [ 16]: Loss 0.01908
-Epoch [ 16]: Loss 0.01547
-Validation: Loss 0.01594 Accuracy 1.00000
-Validation: Loss 0.01699 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01776
-Epoch [ 17]: Loss 0.01820
-Epoch [ 17]: Loss 0.01682
-Epoch [ 17]: Loss 0.01693
-Epoch [ 17]: Loss 0.01560
-Epoch [ 17]: Loss 0.01681
-Epoch [ 17]: Loss 0.01475
-Validation: Loss 0.01456 Accuracy 1.00000
-Validation: Loss 0.01553 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01625
-Epoch [ 18]: Loss 0.01666
-Epoch [ 18]: Loss 0.01450
-Epoch [ 18]: Loss 0.01508
-Epoch [ 18]: Loss 0.01564
-Epoch [ 18]: Loss 0.01561
-Epoch [ 18]: Loss 0.01295
-Validation: Loss 0.01337 Accuracy 1.00000
-Validation: Loss 0.01428 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01438
-Epoch [ 19]: Loss 0.01278
-Epoch [ 19]: Loss 0.01462
-Epoch [ 19]: Loss 0.01433
-Epoch [ 19]: Loss 0.01527
-Epoch [ 19]: Loss 0.01456
-Epoch [ 19]: Loss 0.01358
-Validation: Loss 0.01235 Accuracy 1.00000
-Validation: Loss 0.01320 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01343
-Epoch [ 20]: Loss 0.01354
-Epoch [ 20]: Loss 0.01340
-Epoch [ 20]: Loss 0.01300
-Epoch [ 20]: Loss 0.01298
-Epoch [ 20]: Loss 0.01360
-Epoch [ 20]: Loss 0.01105
-Validation: Loss 0.01143 Accuracy 1.00000
-Validation: Loss 0.01222 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01269
-Epoch [ 21]: Loss 0.01341
-Epoch [ 21]: Loss 0.01266
-Epoch [ 21]: Loss 0.01082
-Epoch [ 21]: Loss 0.01322
-Epoch [ 21]: Loss 0.01105
-Epoch [ 21]: Loss 0.01117
+ps_trained, st_trained = main(SpiralClassifier)
    Epoch [  1]: Loss 0.63034
+Epoch [  1]: Loss 0.60553
+Epoch [  1]: Loss 0.56538
+Epoch [  1]: Loss 0.54630
+Epoch [  1]: Loss 0.50381
+Epoch [  1]: Loss 0.50074
+Epoch [  1]: Loss 0.47526
+Validation: Loss 0.46276 Accuracy 1.00000
+Validation: Loss 0.47391 Accuracy 1.00000
+Epoch [  2]: Loss 0.47023
+Epoch [  2]: Loss 0.44691
+Epoch [  2]: Loss 0.43783
+Epoch [  2]: Loss 0.43551
+Epoch [  2]: Loss 0.41303
+Epoch [  2]: Loss 0.39960
+Epoch [  2]: Loss 0.38515
+Validation: Loss 0.36395 Accuracy 1.00000
+Validation: Loss 0.37779 Accuracy 1.00000
+Epoch [  3]: Loss 0.37114
+Epoch [  3]: Loss 0.34915
+Epoch [  3]: Loss 0.36132
+Epoch [  3]: Loss 0.33572
+Epoch [  3]: Loss 0.31298
+Epoch [  3]: Loss 0.31052
+Epoch [  3]: Loss 0.30406
+Validation: Loss 0.27789 Accuracy 1.00000
+Validation: Loss 0.29328 Accuracy 1.00000
+Epoch [  4]: Loss 0.28022
+Epoch [  4]: Loss 0.27531
+Epoch [  4]: Loss 0.26726
+Epoch [  4]: Loss 0.26451
+Epoch [  4]: Loss 0.23373
+Epoch [  4]: Loss 0.24550
+Epoch [  4]: Loss 0.21351
+Validation: Loss 0.20780 Accuracy 1.00000
+Validation: Loss 0.22336 Accuracy 1.00000
+Epoch [  5]: Loss 0.22156
+Epoch [  5]: Loss 0.20017
+Epoch [  5]: Loss 0.19203
+Epoch [  5]: Loss 0.20301
+Epoch [  5]: Loss 0.18432
+Epoch [  5]: Loss 0.16845
+Epoch [  5]: Loss 0.18846
+Validation: Loss 0.15293 Accuracy 1.00000
+Validation: Loss 0.16724 Accuracy 1.00000
+Epoch [  6]: Loss 0.16454
+Epoch [  6]: Loss 0.15258
+Epoch [  6]: Loss 0.14479
+Epoch [  6]: Loss 0.13681
+Epoch [  6]: Loss 0.13567
+Epoch [  6]: Loss 0.12978
+Epoch [  6]: Loss 0.14006
+Validation: Loss 0.11109 Accuracy 1.00000
+Validation: Loss 0.12287 Accuracy 1.00000
+Epoch [  7]: Loss 0.12029
+Epoch [  7]: Loss 0.10976
+Epoch [  7]: Loss 0.11662
+Epoch [  7]: Loss 0.09244
+Epoch [  7]: Loss 0.10328
+Epoch [  7]: Loss 0.08922
+Epoch [  7]: Loss 0.08470
+Validation: Loss 0.07890 Accuracy 1.00000
+Validation: Loss 0.08760 Accuracy 1.00000
+Epoch [  8]: Loss 0.07588
+Epoch [  8]: Loss 0.07787
+Epoch [  8]: Loss 0.07559
+Epoch [  8]: Loss 0.07404
+Epoch [  8]: Loss 0.07644
+Epoch [  8]: Loss 0.06436
+Epoch [  8]: Loss 0.05506
+Validation: Loss 0.05515 Accuracy 1.00000
+Validation: Loss 0.06097 Accuracy 1.00000
+Epoch [  9]: Loss 0.05574
+Epoch [  9]: Loss 0.05409
+Epoch [  9]: Loss 0.05341
+Epoch [  9]: Loss 0.05125
+Epoch [  9]: Loss 0.05115
+Epoch [  9]: Loss 0.04846
+Epoch [  9]: Loss 0.04370
+Validation: Loss 0.04143 Accuracy 1.00000
+Validation: Loss 0.04557 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04284
+Epoch [ 10]: Loss 0.04111
+Epoch [ 10]: Loss 0.03779
+Epoch [ 10]: Loss 0.03960
+Epoch [ 10]: Loss 0.04298
+Epoch [ 10]: Loss 0.03869
+Epoch [ 10]: Loss 0.03524
+Validation: Loss 0.03367 Accuracy 1.00000
+Validation: Loss 0.03705 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03339
+Epoch [ 11]: Loss 0.03459
+Epoch [ 11]: Loss 0.03342
+Epoch [ 11]: Loss 0.03407
+Epoch [ 11]: Loss 0.03388
+Epoch [ 11]: Loss 0.03202
+Epoch [ 11]: Loss 0.02971
+Validation: Loss 0.02860 Accuracy 1.00000
+Validation: Loss 0.03154 Accuracy 1.00000
+Epoch [ 12]: Loss 0.02859
+Epoch [ 12]: Loss 0.02878
+Epoch [ 12]: Loss 0.02993
+Epoch [ 12]: Loss 0.02963
+Epoch [ 12]: Loss 0.02878
+Epoch [ 12]: Loss 0.02608
+Epoch [ 12]: Loss 0.03050
+Validation: Loss 0.02489 Accuracy 1.00000
+Validation: Loss 0.02750 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02694
+Epoch [ 13]: Loss 0.02671
+Epoch [ 13]: Loss 0.02622
+Epoch [ 13]: Loss 0.02467
+Epoch [ 13]: Loss 0.02467
+Epoch [ 13]: Loss 0.02212
+Epoch [ 13]: Loss 0.02449
+Validation: Loss 0.02201 Accuracy 1.00000
+Validation: Loss 0.02435 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02324
+Epoch [ 14]: Loss 0.02266
+Epoch [ 14]: Loss 0.02391
+Epoch [ 14]: Loss 0.02036
+Epoch [ 14]: Loss 0.02121
+Epoch [ 14]: Loss 0.02264
+Epoch [ 14]: Loss 0.02342
+Validation: Loss 0.01970 Accuracy 1.00000
+Validation: Loss 0.02183 Accuracy 1.00000
+Epoch [ 15]: Loss 0.02103
+Epoch [ 15]: Loss 0.02069
+Epoch [ 15]: Loss 0.02057
+Epoch [ 15]: Loss 0.01919
+Epoch [ 15]: Loss 0.01962
+Epoch [ 15]: Loss 0.01935
+Epoch [ 15]: Loss 0.02118
+Validation: Loss 0.01777 Accuracy 1.00000
+Validation: Loss 0.01973 Accuracy 1.00000
+Epoch [ 16]: Loss 0.02003
+Epoch [ 16]: Loss 0.01873
+Epoch [ 16]: Loss 0.01931
+Epoch [ 16]: Loss 0.01905
+Epoch [ 16]: Loss 0.01646
+Epoch [ 16]: Loss 0.01608
+Epoch [ 16]: Loss 0.01732
+Validation: Loss 0.01614 Accuracy 1.00000
+Validation: Loss 0.01795 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01741
+Epoch [ 17]: Loss 0.01719
+Epoch [ 17]: Loss 0.01619
+Epoch [ 17]: Loss 0.01722
+Epoch [ 17]: Loss 0.01639
+Epoch [ 17]: Loss 0.01571
+Epoch [ 17]: Loss 0.01460
+Validation: Loss 0.01474 Accuracy 1.00000
+Validation: Loss 0.01642 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01682
+Epoch [ 18]: Loss 0.01484
+Epoch [ 18]: Loss 0.01554
+Epoch [ 18]: Loss 0.01390
+Epoch [ 18]: Loss 0.01602
+Epoch [ 18]: Loss 0.01499
+Epoch [ 18]: Loss 0.01172
+Validation: Loss 0.01352 Accuracy 1.00000
+Validation: Loss 0.01510 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01519
+Epoch [ 19]: Loss 0.01364
+Epoch [ 19]: Loss 0.01484
+Epoch [ 19]: Loss 0.01413
+Epoch [ 19]: Loss 0.01383
+Epoch [ 19]: Loss 0.01316
+Epoch [ 19]: Loss 0.01076
+Validation: Loss 0.01246 Accuracy 1.00000
+Validation: Loss 0.01394 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01428
+Epoch [ 20]: Loss 0.01335
+Epoch [ 20]: Loss 0.01268
+Epoch [ 20]: Loss 0.01281
+Epoch [ 20]: Loss 0.01328
+Epoch [ 20]: Loss 0.01177
+Epoch [ 20]: Loss 0.01040
+Validation: Loss 0.01150 Accuracy 1.00000
+Validation: Loss 0.01288 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01271
+Epoch [ 21]: Loss 0.01233
+Epoch [ 21]: Loss 0.01054
+Epoch [ 21]: Loss 0.01321
+Epoch [ 21]: Loss 0.01203
+Epoch [ 21]: Loss 0.01177
+Epoch [ 21]: Loss 0.00776
 Validation: Loss 0.01057 Accuracy 1.00000
-Validation: Loss 0.01130 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01172
-Epoch [ 22]: Loss 0.01148
-Epoch [ 22]: Loss 0.01044
-Epoch [ 22]: Loss 0.01167
-Epoch [ 22]: Loss 0.01094
-Epoch [ 22]: Loss 0.01203
-Epoch [ 22]: Loss 0.00936
-Validation: Loss 0.00968 Accuracy 1.00000
-Validation: Loss 0.01034 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01106
-Epoch [ 23]: Loss 0.01014
-Epoch [ 23]: Loss 0.01062
-Epoch [ 23]: Loss 0.00968
-Epoch [ 23]: Loss 0.00951
-Epoch [ 23]: Loss 0.01021
-Epoch [ 23]: Loss 0.01148
-Validation: Loss 0.00868 Accuracy 1.00000
-Validation: Loss 0.00925 Accuracy 1.00000
-Epoch [ 24]: Loss 0.01007
-Epoch [ 24]: Loss 0.00907
-Epoch [ 24]: Loss 0.00987
-Epoch [ 24]: Loss 0.00865
-Epoch [ 24]: Loss 0.00877
-Epoch [ 24]: Loss 0.00827
-Epoch [ 24]: Loss 0.00907
-Validation: Loss 0.00770 Accuracy 1.00000
-Validation: Loss 0.00819 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00904
-Epoch [ 25]: Loss 0.00849
-Epoch [ 25]: Loss 0.00762
-Epoch [ 25]: Loss 0.00839
-Epoch [ 25]: Loss 0.00697
-Epoch [ 25]: Loss 0.00816
-Epoch [ 25]: Loss 0.00820
-Validation: Loss 0.00699 Accuracy 1.00000
-Validation: Loss 0.00742 Accuracy 1.00000

    We can also train the compact model with the exact same code!

    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.61964
-Epoch [  1]: Loss 0.60676
-Epoch [  1]: Loss 0.57950
-Epoch [  1]: Loss 0.54075
-Epoch [  1]: Loss 0.52232
-Epoch [  1]: Loss 0.49256
-Epoch [  1]: Loss 0.48370
-Validation: Loss 0.45461 Accuracy 1.00000
-Validation: Loss 0.46667 Accuracy 1.00000
-Epoch [  2]: Loss 0.46845
-Epoch [  2]: Loss 0.45738
-Epoch [  2]: Loss 0.43860
-Epoch [  2]: Loss 0.43470
-Epoch [  2]: Loss 0.41848
-Epoch [  2]: Loss 0.39610
-Epoch [  2]: Loss 0.38415
-Validation: Loss 0.35426 Accuracy 1.00000
-Validation: Loss 0.36856 Accuracy 1.00000
-Epoch [  3]: Loss 0.35825
-Epoch [  3]: Loss 0.35951
-Epoch [  3]: Loss 0.36279
-Epoch [  3]: Loss 0.32950
-Epoch [  3]: Loss 0.31807
-Epoch [  3]: Loss 0.32480
-Epoch [  3]: Loss 0.30269
-Validation: Loss 0.26718 Accuracy 1.00000
-Validation: Loss 0.28308 Accuracy 1.00000
-Epoch [  4]: Loss 0.27659
-Epoch [  4]: Loss 0.28368
-Epoch [  4]: Loss 0.25963
-Epoch [  4]: Loss 0.26311
-Epoch [  4]: Loss 0.25752
-Epoch [  4]: Loss 0.23747
-Epoch [  4]: Loss 0.21984
-Validation: Loss 0.19750 Accuracy 1.00000
-Validation: Loss 0.21342 Accuracy 1.00000
-Epoch [  5]: Loss 0.21820
-Epoch [  5]: Loss 0.20098
-Epoch [  5]: Loss 0.20661
-Epoch [  5]: Loss 0.19340
-Epoch [  5]: Loss 0.18270
-Epoch [  5]: Loss 0.18410
-Epoch [  5]: Loss 0.18297
-Validation: Loss 0.14377 Accuracy 1.00000
-Validation: Loss 0.15836 Accuracy 1.00000
-Epoch [  6]: Loss 0.16764
-Epoch [  6]: Loss 0.14448
-Epoch [  6]: Loss 0.14964
-Epoch [  6]: Loss 0.13646
-Epoch [  6]: Loss 0.15055
-Epoch [  6]: Loss 0.13269
-Epoch [  6]: Loss 0.13151
-Validation: Loss 0.10387 Accuracy 1.00000
-Validation: Loss 0.11588 Accuracy 1.00000
-Epoch [  7]: Loss 0.12214
-Epoch [  7]: Loss 0.12336
-Epoch [  7]: Loss 0.10728
-Epoch [  7]: Loss 0.10316
-Epoch [  7]: Loss 0.09760
-Epoch [  7]: Loss 0.09230
-Epoch [  7]: Loss 0.08866
-Validation: Loss 0.07395 Accuracy 1.00000
-Validation: Loss 0.08278 Accuracy 1.00000
-Epoch [  8]: Loss 0.08347
-Epoch [  8]: Loss 0.08922
-Epoch [  8]: Loss 0.07872
-Epoch [  8]: Loss 0.07093
-Epoch [  8]: Loss 0.06854
-Epoch [  8]: Loss 0.06745
-Epoch [  8]: Loss 0.05531
-Validation: Loss 0.05181 Accuracy 1.00000
-Validation: Loss 0.05771 Accuracy 1.00000
-Epoch [  9]: Loss 0.06019
-Epoch [  9]: Loss 0.05877
-Epoch [  9]: Loss 0.05510
-Epoch [  9]: Loss 0.05308
-Epoch [  9]: Loss 0.04923
-Epoch [  9]: Loss 0.04534
-Epoch [  9]: Loss 0.04535
-Validation: Loss 0.03873 Accuracy 1.00000
-Validation: Loss 0.04297 Accuracy 1.00000
-Epoch [ 10]: Loss 0.04256
-Epoch [ 10]: Loss 0.04386
-Epoch [ 10]: Loss 0.04479
-Epoch [ 10]: Loss 0.03830
-Epoch [ 10]: Loss 0.03930
-Epoch [ 10]: Loss 0.03758
-Epoch [ 10]: Loss 0.03751
-Validation: Loss 0.03130 Accuracy 1.00000
-Validation: Loss 0.03479 Accuracy 1.00000
-Epoch [ 11]: Loss 0.03591
-Epoch [ 11]: Loss 0.03614
-Epoch [ 11]: Loss 0.03522
-Epoch [ 11]: Loss 0.03385
-Epoch [ 11]: Loss 0.03280
-Epoch [ 11]: Loss 0.03028
-Epoch [ 11]: Loss 0.02854
-Validation: Loss 0.02646 Accuracy 1.00000
-Validation: Loss 0.02951 Accuracy 1.00000
-Epoch [ 12]: Loss 0.03022
-Epoch [ 12]: Loss 0.03157
-Epoch [ 12]: Loss 0.02827
-Epoch [ 12]: Loss 0.02888
-Epoch [ 12]: Loss 0.02807
-Epoch [ 12]: Loss 0.02731
-Epoch [ 12]: Loss 0.02753
-Validation: Loss 0.02297 Accuracy 1.00000
-Validation: Loss 0.02568 Accuracy 1.00000
-Epoch [ 13]: Loss 0.02879
-Epoch [ 13]: Loss 0.02518
-Epoch [ 13]: Loss 0.02496
-Epoch [ 13]: Loss 0.02617
-Epoch [ 13]: Loss 0.02322
-Epoch [ 13]: Loss 0.02429
-Epoch [ 13]: Loss 0.02511
-Validation: Loss 0.02027 Accuracy 1.00000
-Validation: Loss 0.02272 Accuracy 1.00000
-Epoch [ 14]: Loss 0.02328
-Epoch [ 14]: Loss 0.02450
-Epoch [ 14]: Loss 0.02156
-Epoch [ 14]: Loss 0.02238
-Epoch [ 14]: Loss 0.02300
-Epoch [ 14]: Loss 0.02170
-Epoch [ 14]: Loss 0.01950
-Validation: Loss 0.01812 Accuracy 1.00000
-Validation: Loss 0.02035 Accuracy 1.00000
-Epoch [ 15]: Loss 0.02289
-Epoch [ 15]: Loss 0.02111
-Epoch [ 15]: Loss 0.01972
-Epoch [ 15]: Loss 0.02013
-Epoch [ 15]: Loss 0.02020
-Epoch [ 15]: Loss 0.01836
-Epoch [ 15]: Loss 0.01899
-Validation: Loss 0.01635 Accuracy 1.00000
-Validation: Loss 0.01839 Accuracy 1.00000
-Epoch [ 16]: Loss 0.01956
-Epoch [ 16]: Loss 0.01711
-Epoch [ 16]: Loss 0.01998
-Epoch [ 16]: Loss 0.01812
-Epoch [ 16]: Loss 0.01900
-Epoch [ 16]: Loss 0.01763
-Epoch [ 16]: Loss 0.01537
-Validation: Loss 0.01484 Accuracy 1.00000
-Validation: Loss 0.01674 Accuracy 1.00000
-Epoch [ 17]: Loss 0.01888
-Epoch [ 17]: Loss 0.01663
-Epoch [ 17]: Loss 0.01522
-Epoch [ 17]: Loss 0.01668
-Epoch [ 17]: Loss 0.01708
-Epoch [ 17]: Loss 0.01657
-Epoch [ 17]: Loss 0.01637
-Validation: Loss 0.01354 Accuracy 1.00000
-Validation: Loss 0.01531 Accuracy 1.00000
-Epoch [ 18]: Loss 0.01558
-Epoch [ 18]: Loss 0.01553
-Epoch [ 18]: Loss 0.01548
-Epoch [ 18]: Loss 0.01486
-Epoch [ 18]: Loss 0.01595
-Epoch [ 18]: Loss 0.01505
-Epoch [ 18]: Loss 0.01557
-Validation: Loss 0.01239 Accuracy 1.00000
-Validation: Loss 0.01405 Accuracy 1.00000
-Epoch [ 19]: Loss 0.01565
-Epoch [ 19]: Loss 0.01480
-Epoch [ 19]: Loss 0.01289
-Epoch [ 19]: Loss 0.01431
-Epoch [ 19]: Loss 0.01376
-Epoch [ 19]: Loss 0.01384
-Epoch [ 19]: Loss 0.01330
-Validation: Loss 0.01137 Accuracy 1.00000
-Validation: Loss 0.01291 Accuracy 1.00000
-Epoch [ 20]: Loss 0.01301
-Epoch [ 20]: Loss 0.01286
-Epoch [ 20]: Loss 0.01358
-Epoch [ 20]: Loss 0.01280
-Epoch [ 20]: Loss 0.01276
-Epoch [ 20]: Loss 0.01273
-Epoch [ 20]: Loss 0.01456
-Validation: Loss 0.01042 Accuracy 1.00000
 Validation: Loss 0.01185 Accuracy 1.00000
-Epoch [ 21]: Loss 0.01172
-Epoch [ 21]: Loss 0.01329
-Epoch [ 21]: Loss 0.01186
-Epoch [ 21]: Loss 0.01169
-Epoch [ 21]: Loss 0.01179
-Epoch [ 21]: Loss 0.01095
-Epoch [ 21]: Loss 0.01284
-Validation: Loss 0.00947 Accuracy 1.00000
-Validation: Loss 0.01075 Accuracy 1.00000
-Epoch [ 22]: Loss 0.01178
-Epoch [ 22]: Loss 0.01089
-Epoch [ 22]: Loss 0.01110
-Epoch [ 22]: Loss 0.01071
-Epoch [ 22]: Loss 0.01036
-Epoch [ 22]: Loss 0.00971
-Epoch [ 22]: Loss 0.01027
-Validation: Loss 0.00845 Accuracy 1.00000
-Validation: Loss 0.00955 Accuracy 1.00000
-Epoch [ 23]: Loss 0.01146
-Epoch [ 23]: Loss 0.00902
-Epoch [ 23]: Loss 0.00849
-Epoch [ 23]: Loss 0.00903
-Epoch [ 23]: Loss 0.00886
-Epoch [ 23]: Loss 0.00972
-Epoch [ 23]: Loss 0.01066
-Validation: Loss 0.00751 Accuracy 1.00000
-Validation: Loss 0.00846 Accuracy 1.00000
-Epoch [ 24]: Loss 0.00976
-Epoch [ 24]: Loss 0.00777
+Epoch [ 22]: Loss 0.01148
+Epoch [ 22]: Loss 0.01199
+Epoch [ 22]: Loss 0.01032
+Epoch [ 22]: Loss 0.01079
+Epoch [ 22]: Loss 0.01012
+Epoch [ 22]: Loss 0.01070
+Epoch [ 22]: Loss 0.01114
+Validation: Loss 0.00957 Accuracy 1.00000
+Validation: Loss 0.01072 Accuracy 1.00000
+Epoch [ 23]: Loss 0.01039
+Epoch [ 23]: Loss 0.00953
+Epoch [ 23]: Loss 0.01061
+Epoch [ 23]: Loss 0.00931
+Epoch [ 23]: Loss 0.00997
+Epoch [ 23]: Loss 0.00961
+Epoch [ 23]: Loss 0.00712
+Validation: Loss 0.00850 Accuracy 1.00000
+Validation: Loss 0.00949 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00893
+Epoch [ 24]: Loss 0.00878
+Epoch [ 24]: Loss 0.00866
+Epoch [ 24]: Loss 0.00924
+Epoch [ 24]: Loss 0.00824
 Epoch [ 24]: Loss 0.00825
-Epoch [ 24]: Loss 0.00881
-Epoch [ 24]: Loss 0.00796
-Epoch [ 24]: Loss 0.00826
-Epoch [ 24]: Loss 0.00787
-Validation: Loss 0.00682 Accuracy 1.00000
-Validation: Loss 0.00765 Accuracy 1.00000
-Epoch [ 25]: Loss 0.00840
-Epoch [ 25]: Loss 0.00781
-Epoch [ 25]: Loss 0.00786
-Epoch [ 25]: Loss 0.00724
-Epoch [ 25]: Loss 0.00762
-Epoch [ 25]: Loss 0.00726
+Epoch [ 24]: Loss 0.00878
+Validation: Loss 0.00763 Accuracy 1.00000
+Validation: Loss 0.00849 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00802
+Epoch [ 25]: Loss 0.00754
+Epoch [ 25]: Loss 0.00771
+Epoch [ 25]: Loss 0.00777
+Epoch [ 25]: Loss 0.00775
+Epoch [ 25]: Loss 0.00812
+Epoch [ 25]: Loss 0.00833
+Validation: Loss 0.00700 Accuracy 1.00000
+Validation: Loss 0.00776 Accuracy 1.00000

    We can also train the compact model with the exact same code!

    julia
    ps_trained2, st_trained2 = main(SpiralClassifierCompact)
    Epoch [  1]: Loss 0.62243
+Epoch [  1]: Loss 0.61271
+Epoch [  1]: Loss 0.55580
+Epoch [  1]: Loss 0.53213
+Epoch [  1]: Loss 0.52075
+Epoch [  1]: Loss 0.49787
+Epoch [  1]: Loss 0.47505
+Validation: Loss 0.47777 Accuracy 1.00000
+Validation: Loss 0.46300 Accuracy 1.00000
+Epoch [  2]: Loss 0.46806
+Epoch [  2]: Loss 0.44811
+Epoch [  2]: Loss 0.44868
+Epoch [  2]: Loss 0.42514
+Epoch [  2]: Loss 0.41207
+Epoch [  2]: Loss 0.38999
+Epoch [  2]: Loss 0.40689
+Validation: Loss 0.38313 Accuracy 1.00000
+Validation: Loss 0.36497 Accuracy 1.00000
+Epoch [  3]: Loss 0.37506
+Epoch [  3]: Loss 0.35458
+Epoch [  3]: Loss 0.34745
+Epoch [  3]: Loss 0.32494
+Epoch [  3]: Loss 0.31280
+Epoch [  3]: Loss 0.32159
+Epoch [  3]: Loss 0.31559
+Validation: Loss 0.29940 Accuracy 1.00000
+Validation: Loss 0.27966 Accuracy 1.00000
+Epoch [  4]: Loss 0.27458
+Epoch [  4]: Loss 0.28234
+Epoch [  4]: Loss 0.26623
+Epoch [  4]: Loss 0.24585
+Epoch [  4]: Loss 0.24220
+Epoch [  4]: Loss 0.25250
+Epoch [  4]: Loss 0.22705
+Validation: Loss 0.22949 Accuracy 1.00000
+Validation: Loss 0.20991 Accuracy 1.00000
+Epoch [  5]: Loss 0.20868
+Epoch [  5]: Loss 0.20777
+Epoch [  5]: Loss 0.21327
+Epoch [  5]: Loss 0.18019
+Epoch [  5]: Loss 0.18150
+Epoch [  5]: Loss 0.17559
+Epoch [  5]: Loss 0.20257
+Validation: Loss 0.17268 Accuracy 1.00000
+Validation: Loss 0.15497 Accuracy 1.00000
+Epoch [  6]: Loss 0.15827
+Epoch [  6]: Loss 0.15591
+Epoch [  6]: Loss 0.14491
+Epoch [  6]: Loss 0.13292
+Epoch [  6]: Loss 0.14914
+Epoch [  6]: Loss 0.12443
+Epoch [  6]: Loss 0.13808
+Validation: Loss 0.12726 Accuracy 1.00000
+Validation: Loss 0.11293 Accuracy 1.00000
+Epoch [  7]: Loss 0.12103
+Epoch [  7]: Loss 0.11242
+Epoch [  7]: Loss 0.10596
+Epoch [  7]: Loss 0.09875
+Epoch [  7]: Loss 0.09495
+Epoch [  7]: Loss 0.09276
+Epoch [  7]: Loss 0.10884
+Validation: Loss 0.09070 Accuracy 1.00000
+Validation: Loss 0.08028 Accuracy 1.00000
+Epoch [  8]: Loss 0.09406
+Epoch [  8]: Loss 0.08342
+Epoch [  8]: Loss 0.07543
+Epoch [  8]: Loss 0.06123
+Epoch [  8]: Loss 0.06837
+Epoch [  8]: Loss 0.06419
+Epoch [  8]: Loss 0.05732
+Validation: Loss 0.06270 Accuracy 1.00000
+Validation: Loss 0.05587 Accuracy 1.00000
+Epoch [  9]: Loss 0.05821
+Epoch [  9]: Loss 0.05601
+Epoch [  9]: Loss 0.05212
+Epoch [  9]: Loss 0.05552
+Epoch [  9]: Loss 0.04673
+Epoch [  9]: Loss 0.04686
+Epoch [  9]: Loss 0.03952
+Validation: Loss 0.04691 Accuracy 1.00000
+Validation: Loss 0.04198 Accuracy 1.00000
+Epoch [ 10]: Loss 0.04253
+Epoch [ 10]: Loss 0.04115
+Epoch [ 10]: Loss 0.04191
+Epoch [ 10]: Loss 0.04000
+Epoch [ 10]: Loss 0.03686
+Epoch [ 10]: Loss 0.03965
+Epoch [ 10]: Loss 0.03798
+Validation: Loss 0.03820 Accuracy 1.00000
+Validation: Loss 0.03413 Accuracy 1.00000
+Epoch [ 11]: Loss 0.03729
+Epoch [ 11]: Loss 0.03196
+Epoch [ 11]: Loss 0.03405
+Epoch [ 11]: Loss 0.03641
+Epoch [ 11]: Loss 0.03278
+Epoch [ 11]: Loss 0.02875
+Epoch [ 11]: Loss 0.03003
+Validation: Loss 0.03254 Accuracy 1.00000
+Validation: Loss 0.02899 Accuracy 1.00000
+Epoch [ 12]: Loss 0.03042
+Epoch [ 12]: Loss 0.02825
+Epoch [ 12]: Loss 0.02816
+Epoch [ 12]: Loss 0.03091
+Epoch [ 12]: Loss 0.02586
+Epoch [ 12]: Loss 0.02826
+Epoch [ 12]: Loss 0.02898
+Validation: Loss 0.02840 Accuracy 1.00000
+Validation: Loss 0.02524 Accuracy 1.00000
+Epoch [ 13]: Loss 0.02510
+Epoch [ 13]: Loss 0.02628
+Epoch [ 13]: Loss 0.02558
+Epoch [ 13]: Loss 0.02468
+Epoch [ 13]: Loss 0.02447
+Epoch [ 13]: Loss 0.02516
+Epoch [ 13]: Loss 0.02303
+Validation: Loss 0.02517 Accuracy 1.00000
+Validation: Loss 0.02233 Accuracy 1.00000
+Epoch [ 14]: Loss 0.02156
+Epoch [ 14]: Loss 0.02310
+Epoch [ 14]: Loss 0.02294
+Epoch [ 14]: Loss 0.02135
+Epoch [ 14]: Loss 0.02244
+Epoch [ 14]: Loss 0.02250
+Epoch [ 14]: Loss 0.02294
+Validation: Loss 0.02259 Accuracy 1.00000
+Validation: Loss 0.01999 Accuracy 1.00000
+Epoch [ 15]: Loss 0.01831
+Epoch [ 15]: Loss 0.01957
+Epoch [ 15]: Loss 0.01967
+Epoch [ 15]: Loss 0.02107
+Epoch [ 15]: Loss 0.01948
+Epoch [ 15]: Loss 0.02159
+Epoch [ 15]: Loss 0.02296
+Validation: Loss 0.02044 Accuracy 1.00000
+Validation: Loss 0.01805 Accuracy 1.00000
+Epoch [ 16]: Loss 0.01883
+Epoch [ 16]: Loss 0.01874
+Epoch [ 16]: Loss 0.01780
+Epoch [ 16]: Loss 0.01667
+Epoch [ 16]: Loss 0.01866
+Epoch [ 16]: Loss 0.01859
+Epoch [ 16]: Loss 0.01752
+Validation: Loss 0.01859 Accuracy 1.00000
+Validation: Loss 0.01638 Accuracy 1.00000
+Epoch [ 17]: Loss 0.01578
+Epoch [ 17]: Loss 0.01836
+Epoch [ 17]: Loss 0.01503
+Epoch [ 17]: Loss 0.01757
+Epoch [ 17]: Loss 0.01687
+Epoch [ 17]: Loss 0.01556
+Epoch [ 17]: Loss 0.01723
+Validation: Loss 0.01700 Accuracy 1.00000
+Validation: Loss 0.01495 Accuracy 1.00000
+Epoch [ 18]: Loss 0.01577
+Epoch [ 18]: Loss 0.01588
+Epoch [ 18]: Loss 0.01479
+Epoch [ 18]: Loss 0.01476
+Epoch [ 18]: Loss 0.01442
+Epoch [ 18]: Loss 0.01556
+Epoch [ 18]: Loss 0.01408
+Validation: Loss 0.01562 Accuracy 1.00000
+Validation: Loss 0.01370 Accuracy 1.00000
+Epoch [ 19]: Loss 0.01447
+Epoch [ 19]: Loss 0.01459
+Epoch [ 19]: Loss 0.01301
+Epoch [ 19]: Loss 0.01425
+Epoch [ 19]: Loss 0.01311
+Epoch [ 19]: Loss 0.01418
+Epoch [ 19]: Loss 0.01348
+Validation: Loss 0.01438 Accuracy 1.00000
+Validation: Loss 0.01260 Accuracy 1.00000
+Epoch [ 20]: Loss 0.01398
+Epoch [ 20]: Loss 0.01408
+Epoch [ 20]: Loss 0.01339
+Epoch [ 20]: Loss 0.01272
+Epoch [ 20]: Loss 0.01167
+Epoch [ 20]: Loss 0.01151
+Epoch [ 20]: Loss 0.01086
+Validation: Loss 0.01319 Accuracy 1.00000
+Validation: Loss 0.01155 Accuracy 1.00000
+Epoch [ 21]: Loss 0.01245
+Epoch [ 21]: Loss 0.01062
+Epoch [ 21]: Loss 0.01189
+Epoch [ 21]: Loss 0.01221
+Epoch [ 21]: Loss 0.01134
+Epoch [ 21]: Loss 0.01185
+Epoch [ 21]: Loss 0.01080
+Validation: Loss 0.01194 Accuracy 1.00000
+Validation: Loss 0.01047 Accuracy 1.00000
+Epoch [ 22]: Loss 0.01159
+Epoch [ 22]: Loss 0.01187
+Epoch [ 22]: Loss 0.01073
+Epoch [ 22]: Loss 0.00970
+Epoch [ 22]: Loss 0.01003
+Epoch [ 22]: Loss 0.00966
+Epoch [ 22]: Loss 0.00883
+Validation: Loss 0.01057 Accuracy 1.00000
+Validation: Loss 0.00929 Accuracy 1.00000
+Epoch [ 23]: Loss 0.00981
+Epoch [ 23]: Loss 0.01000
+Epoch [ 23]: Loss 0.00948
+Epoch [ 23]: Loss 0.00895
+Epoch [ 23]: Loss 0.00924
+Epoch [ 23]: Loss 0.00850
+Epoch [ 23]: Loss 0.00887
+Validation: Loss 0.00937 Accuracy 1.00000
+Validation: Loss 0.00828 Accuracy 1.00000
+Epoch [ 24]: Loss 0.00856
+Epoch [ 24]: Loss 0.00821
+Epoch [ 24]: Loss 0.00878
+Epoch [ 24]: Loss 0.00829
+Epoch [ 24]: Loss 0.00788
+Epoch [ 24]: Loss 0.00851
+Epoch [ 24]: Loss 0.00775
+Validation: Loss 0.00852 Accuracy 1.00000
+Validation: Loss 0.00755 Accuracy 1.00000
+Epoch [ 25]: Loss 0.00766
+Epoch [ 25]: Loss 0.00806
+Epoch [ 25]: Loss 0.00790
 Epoch [ 25]: Loss 0.00775
-Validation: Loss 0.00631 Accuracy 1.00000
-Validation: Loss 0.00707 Accuracy 1.00000

    Saving the Model ​

    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.

    julia
    @save "trained_model.jld2" ps_trained st_trained

    Let's try loading the model

    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
+Epoch [ 25]: Loss 0.00713
+Epoch [ 25]: Loss 0.00745
+Epoch [ 25]: Loss 0.00773
+Validation: Loss 0.00790 Accuracy 1.00000
+Validation: Loss 0.00701 Accuracy 1.00000

    Saving the Model ​

    We can save the model using JLD2 (and any other serialization library of your choice) Note that we transfer the model to CPU before saving. Additionally, we recommend that you don't save the model struct and only save the parameters and states.

    julia
    @save "trained_model.jld2" ps_trained st_trained

    Let's try loading the model

    julia
    @load "trained_model.jld2" ps_trained st_trained
    2-element Vector{Symbol}:
  :ps_trained
  :st_trained

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
@@ -637,7 +637,7 @@
 
 1 device:
   0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.422 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/beginner/4_SimpleChains.html b/previews/PR1000/tutorials/beginner/4_SimpleChains.html index c4845c8d33..df601cf95d 100644 --- a/previews/PR1000/tutorials/beginner/4_SimpleChains.html +++ b/previews/PR1000/tutorials/beginner/4_SimpleChains.html @@ -5,14 +5,14 @@ MNIST Classification with SimpleChains | Lux.jl Docs - + - + - + - + @@ -119,25 +119,25 @@ end return tr_acc, te_acc -end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 105.39s 	 Training Accuracy: 22.72% 	 Test Accuracy: 19.00%
-[ 2/10] 	 Time 97.47s 	 Training Accuracy: 47.39% 	 Test Accuracy: 46.00%
-[ 3/10] 	 Time 99.44s 	 Training Accuracy: 60.33% 	 Test Accuracy: 61.00%
-[ 4/10] 	 Time 99.45s 	 Training Accuracy: 69.56% 	 Test Accuracy: 65.50%
-[ 5/10] 	 Time 107.39s 	 Training Accuracy: 74.89% 	 Test Accuracy: 71.50%
-[ 6/10] 	 Time 104.03s 	 Training Accuracy: 78.22% 	 Test Accuracy: 73.50%
-[ 7/10] 	 Time 106.84s 	 Training Accuracy: 80.00% 	 Test Accuracy: 76.50%
-[ 8/10] 	 Time 104.29s 	 Training Accuracy: 83.00% 	 Test Accuracy: 80.00%
-[ 9/10] 	 Time 102.81s 	 Training Accuracy: 84.78% 	 Test Accuracy: 81.00%
-[10/10] 	 Time 108.25s 	 Training Accuracy: 86.22% 	 Test Accuracy: 83.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.76s 	 Training Accuracy: 32.89% 	 Test Accuracy: 29.00%
-[ 2/10] 	 Time 17.69s 	 Training Accuracy: 44.83% 	 Test Accuracy: 42.00%
-[ 3/10] 	 Time 17.70s 	 Training Accuracy: 58.11% 	 Test Accuracy: 53.00%
-[ 4/10] 	 Time 17.70s 	 Training Accuracy: 65.56% 	 Test Accuracy: 62.50%
-[ 5/10] 	 Time 17.70s 	 Training Accuracy: 73.06% 	 Test Accuracy: 67.00%
-[ 6/10] 	 Time 17.70s 	 Training Accuracy: 78.50% 	 Test Accuracy: 76.50%
-[ 7/10] 	 Time 17.71s 	 Training Accuracy: 82.17% 	 Test Accuracy: 79.00%
-[ 8/10] 	 Time 17.70s 	 Training Accuracy: 83.44% 	 Test Accuracy: 79.00%
-[ 9/10] 	 Time 17.70s 	 Training Accuracy: 85.89% 	 Test Accuracy: 85.00%
-[10/10] 	 Time 17.70s 	 Training Accuracy: 86.94% 	 Test Accuracy: 83.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
+end
    train (generic function with 1 method)

    Finally Training the Model ​

    First we will train the Lux model

    julia
    tr_acc, te_acc = train(lux_model)
    [ 1/10] 	 Time 49.05s 	 Training Accuracy: 21.61% 	 Test Accuracy: 16.00%
+[ 2/10] 	 Time 37.82s 	 Training Accuracy: 46.17% 	 Test Accuracy: 43.50%
+[ 3/10] 	 Time 45.59s 	 Training Accuracy: 59.50% 	 Test Accuracy: 60.00%
+[ 4/10] 	 Time 60.41s 	 Training Accuracy: 69.22% 	 Test Accuracy: 65.50%
+[ 5/10] 	 Time 77.59s 	 Training Accuracy: 74.11% 	 Test Accuracy: 71.00%
+[ 6/10] 	 Time 78.28s 	 Training Accuracy: 78.11% 	 Test Accuracy: 75.00%
+[ 7/10] 	 Time 73.89s 	 Training Accuracy: 80.22% 	 Test Accuracy: 79.50%
+[ 8/10] 	 Time 82.82s 	 Training Accuracy: 82.39% 	 Test Accuracy: 79.50%
+[ 9/10] 	 Time 64.69s 	 Training Accuracy: 85.06% 	 Test Accuracy: 83.50%
+[10/10] 	 Time 72.24s 	 Training Accuracy: 86.56% 	 Test Accuracy: 83.50%

    Now we will train the SimpleChains model

    julia
    train(simple_chains_model)
    [ 1/10] 	 Time 18.81s 	 Training Accuracy: 30.56% 	 Test Accuracy: 26.00%
+[ 2/10] 	 Time 17.57s 	 Training Accuracy: 47.06% 	 Test Accuracy: 45.00%
+[ 3/10] 	 Time 17.58s 	 Training Accuracy: 59.72% 	 Test Accuracy: 51.50%
+[ 4/10] 	 Time 17.54s 	 Training Accuracy: 67.61% 	 Test Accuracy: 64.00%
+[ 5/10] 	 Time 17.53s 	 Training Accuracy: 75.22% 	 Test Accuracy: 72.50%
+[ 6/10] 	 Time 17.54s 	 Training Accuracy: 80.22% 	 Test Accuracy: 73.50%
+[ 7/10] 	 Time 17.56s 	 Training Accuracy: 81.17% 	 Test Accuracy: 81.00%
+[ 8/10] 	 Time 17.51s 	 Training Accuracy: 85.61% 	 Test Accuracy: 83.50%
+[ 9/10] 	 Time 17.51s 	 Training Accuracy: 86.33% 	 Test Accuracy: 83.50%
+[10/10] 	 Time 17.51s 	 Training Accuracy: 87.33% 	 Test Accuracy: 85.50%

    On my local machine we see a 3-4x speedup when using SimpleChains.jl. The conditions of the server this documentation is being built on is not ideal for CPU benchmarking hence, the speedup may not be as significant and even there might be regressions.

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -170,7 +170,7 @@
   JULIA_CUDA_HARD_MEMORY_LIMIT = 100%
   JULIA_PKG_PRECOMPILE_AUTO = 0
   JULIA_DEBUG = Literate

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/beginner/5_OptimizationIntegration.html b/previews/PR1000/tutorials/beginner/5_OptimizationIntegration.html index 00d2718935..f5d607c8f9 100644 --- a/previews/PR1000/tutorials/beginner/5_OptimizationIntegration.html +++ b/previews/PR1000/tutorials/beginner/5_OptimizationIntegration.html @@ -5,14 +5,14 @@ Training Lux Models using Optimization.jl | Lux.jl Docs - + - + - + - + @@ -109,31 +109,31 @@ return StatefulLuxLayer{true}(model, res.u, smodel.st) end -trained_model = train_model(dataloader)
    Iteration:     1, Loss: 1.133634e-01
-Iteration:    26, Loss: 1.063336e-01
-Iteration:    51, Loss: 2.078427e-01
-Iteration:    76, Loss: 2.046876e-01
-Iteration:     1, Loss: 1.987609e-01
-Iteration:     1, Loss: 2.268675e-02
-Iteration:    26, Loss: 3.252327e-02
-Iteration:    51, Loss: 2.324151e-02
-Iteration:    76, Loss: 2.137157e-02
-Iteration:   101, Loss: 1.924721e-02
-Iteration:   126, Loss: 1.698606e-02
-Iteration:   151, Loss: 1.479297e-02
-Iteration:   176, Loss: 1.256829e-02
-Iteration:   201, Loss: 1.779426e-02
-Iteration:   226, Loss: 1.126407e-02
-Iteration:   251, Loss: 9.760977e-03
-Iteration:   276, Loss: 8.756177e-03
-Iteration:   301, Loss: 1.514340e-02
-Iteration:   326, Loss: 9.525211e-03
-Iteration:   351, Loss: 8.062281e-03
-Iteration:   376, Loss: 7.160148e-03
-Iteration:   401, Loss: 6.379412e-03
-Iteration:   426, Loss: 1.824650e-02
-Iteration:   451, Loss: 8.544287e-03
-Iteration:   476, Loss: 6.913106e-03

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
+trained_model = train_model(dataloader)
    Iteration:     1, Loss: 1.894389e-01
+Iteration:    26, Loss: 2.794352e-02
+Iteration:    51, Loss: 1.110121e-01
+Iteration:    76, Loss: 2.543880e-01
+Iteration:     1, Loss: 3.234872e-01
+Iteration:     1, Loss: 2.432619e-02
+Iteration:    26, Loss: 3.241556e-02
+Iteration:    51, Loss: 2.396435e-02
+Iteration:    76, Loss: 2.210484e-02
+Iteration:   101, Loss: 2.054718e-02
+Iteration:   126, Loss: 1.922303e-02
+Iteration:   151, Loss: 1.797500e-02
+Iteration:   176, Loss: 1.671544e-02
+Iteration:   201, Loss: 1.529878e-02
+Iteration:   226, Loss: 1.391988e-02
+Iteration:   251, Loss: 1.273907e-02
+Iteration:   276, Loss: 1.162500e-02
+Iteration:   301, Loss: 2.115832e-02
+Iteration:   326, Loss: 1.171498e-02
+Iteration:   351, Loss: 9.864081e-03
+Iteration:   376, Loss: 8.805853e-03
+Iteration:   401, Loss: 7.982390e-03
+Iteration:   426, Loss: 1.730291e-02
+Iteration:   451, Loss: 1.051912e-02
+Iteration:   476, Loss: 8.427156e-03

    Plotting the results ​

    julia
    dudt(u, p, t) = trained_model(u, p)
 prob = ODEProblem(dudt, gdev(u0), (tspan[1], tspan[2]), trained_model.ps)
 sol = solve(prob, Tsit5(); saveat=t)
 pred = convert(AbstractArray, sol) |> cdev
@@ -147,7 +147,7 @@
     lines!(ax, t, pred[2, :]; label=L"\hat{u}_2(t)", color=:red, linewidth=4)
     axislegend(ax; position=:lt)
     fig
-end

    Appendix ​

    julia
    using InteractiveUtils
+end

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -208,7 +208,7 @@
 
 1 device:
   0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/index.html b/previews/PR1000/tutorials/index.html index 3d2f6e35bb..d57d6df056 100644 --- a/previews/PR1000/tutorials/index.html +++ b/previews/PR1000/tutorials/index.html @@ -5,12 +5,12 @@ Tutorials | Lux.jl Docs - + - + - + @@ -28,7 +28,7 @@
    Lux.jl Docs

    Appearance


    Trusted by

    Scientific Computing

    SciML.ai

    Machine Learning

    Tutorials ​

    Beginner Tutorials ​

    Julia & Lux for the Uninitiated

    How to get started with Julia and Lux for those who have never used Julia before.

    Fitting a Polynomial using MLP

    Learn the Basics of Lux by fitting a Multi-Layer Perceptron to a Polynomial.

    Training a Simple LSTM

    Learn how to define custom layers and train an RNN on time-series data.

    Use SimpleChains.jl as a Backend

    Learn how to train small neural networks really fast on CPU.

    Fitting with Optimization.jl

    Learn how to use Optimization.jl with Lux (on GPUs).

    Pre-Built Deep Learning Models

    Use Boltz.jl to load pre-built DL and SciML models.

    Intermediate Tutorials ​

    MNIST Classification using Neural ODE

    Train a Neural Ordinary Differential Equations to classify MNIST Images.

    Bayesian Neural Networks

    Figure out how to use Probabilistic Programming Frameworks like Turing with Lux.

    Training a HyperNetwork

    Train a hypernetwork to work on multiple datasets by predicting NN parameters.

    Training a PINN

    Train a PINN to solve 2D PDEs (using Nested AD).

    Advanced Tutorials ​

    Neural ODE to Model Gravitational Waveforms

    Training a Neural ODE to fit simulated data of gravitational waveforms.

    Optimal Control with Symbolic UDE

    Train a UDE and replace a part of it with Symbolic Regression.

    Larger Models ​

    WARNING

    These models are part of the Lux examples, however, these are larger model that cannot be run on CI and aren't frequently tested. If you find a bug in one of these models, please open an issue or PR to fix it.

    ImageNet Classification

    Train Large Image Classifiers using Lux (on Distributed GPUs).

    Denoising Diffusion Implicit Model (DDIM)

    Train a Diffusion Model to generate images from Gaussian noises.

    ConvMixer on CIFAR-10

    Train ConvMixer on CIFAR-10 to 90% accuracy within 10 minutes.

    Selected 3rd Party Tutorials ​

    WARNING

    These tutorials are developed by the community and may not be up-to-date with the latest version of Lux.jl. Please refer to the official documentation for the most up-to-date information.

    Please open an issue (ideally both at Lux.jl and at the downstream linked package) if any of them are non-functional and we will try to get them updated.

    GPU-Accelerated Physics-Informed Neural Networks

    Use Machine Learning (PINNs) to solve the Heat Equation PDE on a GPU.

    Weather Forecasting with Neural ODEs

    Train a neural ODEs to a multidimensional weather dataset and use it for weather forecasting.

    Controlling Stochastic Differential Equations

    Control the time evolution of a continuously monitored qubit described by an SDE with multiplicative scalar noise.

    Medical Image Segmentation

    Explore various aspects of deep learning for medical imaging and a comprehensive overview of Julia packages.

    Neural PDE closures

    Learn an unknown term in a PDE using convolutional neural networks and Fourier neural operators.

    TIP

    If you found an amazing tutorial showcasing Lux.jl online, or wrote one yourself, please open an issue or PR to add it to the list!

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/intermediate/1_NeuralODE.html b/previews/PR1000/tutorials/intermediate/1_NeuralODE.html index 5de139e183..f158c2f515 100644 --- a/previews/PR1000/tutorials/intermediate/1_NeuralODE.html +++ b/previews/PR1000/tutorials/intermediate/1_NeuralODE.html @@ -5,14 +5,14 @@ MNIST Classification using Neural ODEs | Lux.jl Docs - + - + - + - + @@ -136,47 +136,47 @@ end end -train(NeuralODECompact)
    [1/9]	Time 118.1406s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.7178s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5016s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
-[4/9]	Time 0.6758s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
-[5/9]	Time 0.4877s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
-[6/9]	Time 0.6598s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
-[7/9]	Time 0.4953s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
-[8/9]	Time 0.4947s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
-[9/9]	Time 0.7741s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 35.6246s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.5096s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
-[3/9]	Time 0.5355s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
-[4/9]	Time 0.8174s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
-[5/9]	Time 0.5219s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
-[6/9]	Time 0.5240s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
-[7/9]	Time 0.8928s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
-[8/9]	Time 0.5157s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
-[9/9]	Time 0.5063s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 44.6107s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4743s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
-[3/9]	Time 0.5032s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
-[4/9]	Time 0.7259s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
-[5/9]	Time 0.5070s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
-[6/9]	Time 0.5044s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
-[7/9]	Time 0.7081s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
-[8/9]	Time 0.4944s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
-[9/9]	Time 0.5164s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 102.5954s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 21.1668s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
-[3/9]	Time 19.6983s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
-[4/9]	Time 22.0427s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
-[5/9]	Time 16.8381s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
-[6/9]	Time 20.4067s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 13.9161s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
-[8/9]	Time 14.8513s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
-[9/9]	Time 14.7005s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 51.9802s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 27.0563s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
-[3/9]	Time 30.0257s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
-[4/9]	Time 32.3402s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
-[5/9]	Time 28.1834s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
-[6/9]	Time 36.2859s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
-[7/9]	Time 32.9532s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
-[8/9]	Time 27.0422s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
-[9/9]	Time 27.9113s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
+train(NeuralODECompact)
    [1/9]	Time 136.7023s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5255s	Training Accuracy: 58.22222%	Test Accuracy: 57.33333%
+[3/9]	Time 0.7318s	Training Accuracy: 67.85185%	Test Accuracy: 70.66667%
+[4/9]	Time 0.5021s	Training Accuracy: 74.29630%	Test Accuracy: 74.66667%
+[5/9]	Time 0.5467s	Training Accuracy: 76.29630%	Test Accuracy: 76.00000%
+[6/9]	Time 0.8045s	Training Accuracy: 78.74074%	Test Accuracy: 80.00000%
+[7/9]	Time 0.5118s	Training Accuracy: 82.22222%	Test Accuracy: 81.33333%
+[8/9]	Time 0.5154s	Training Accuracy: 83.62963%	Test Accuracy: 83.33333%
+[9/9]	Time 0.9002s	Training Accuracy: 85.18519%	Test Accuracy: 82.66667%
    julia
    train(NeuralODE)
    [1/9]	Time 38.9844s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5079s	Training Accuracy: 57.18519%	Test Accuracy: 57.33333%
+[3/9]	Time 0.5245s	Training Accuracy: 68.37037%	Test Accuracy: 68.00000%
+[4/9]	Time 0.7894s	Training Accuracy: 73.77778%	Test Accuracy: 75.33333%
+[5/9]	Time 0.5193s	Training Accuracy: 76.14815%	Test Accuracy: 77.33333%
+[6/9]	Time 0.5215s	Training Accuracy: 79.48148%	Test Accuracy: 80.66667%
+[7/9]	Time 0.7914s	Training Accuracy: 81.25926%	Test Accuracy: 80.66667%
+[8/9]	Time 0.5144s	Training Accuracy: 83.40741%	Test Accuracy: 82.66667%
+[9/9]	Time 0.5315s	Training Accuracy: 84.81481%	Test Accuracy: 82.00000%

    We can also change the sensealg and train the model! GaussAdjoint allows you to use any arbitrary parameter structure and not just a flat vector (ComponentArray).

    julia
    train(NeuralODE; sensealg=GaussAdjoint(; autojacvec=ZygoteVJP()), use_named_tuple=true)
    [1/9]	Time 44.4626s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.5993s	Training Accuracy: 57.55556%	Test Accuracy: 54.00000%
+[3/9]	Time 0.5022s	Training Accuracy: 69.85185%	Test Accuracy: 69.33333%
+[4/9]	Time 0.5087s	Training Accuracy: 72.51852%	Test Accuracy: 74.00000%
+[5/9]	Time 0.7353s	Training Accuracy: 75.33333%	Test Accuracy: 76.00000%
+[6/9]	Time 0.4898s	Training Accuracy: 78.88889%	Test Accuracy: 79.33333%
+[7/9]	Time 0.4922s	Training Accuracy: 81.03704%	Test Accuracy: 80.00000%
+[8/9]	Time 0.4996s	Training Accuracy: 83.77778%	Test Accuracy: 81.33333%
+[9/9]	Time 0.4869s	Training Accuracy: 85.25926%	Test Accuracy: 82.66667%

    But remember some AD backends like ReverseDiff is not GPU compatible. For a model this size, you will notice that training time is significantly lower for training on CPU than on GPU.

    julia
    train(NeuralODE; sensealg=InterpolatingAdjoint(; autojacvec=ReverseDiffVJP()), cpu=true)
    [1/9]	Time 101.8574s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 13.8755s	Training Accuracy: 58.74074%	Test Accuracy: 56.66667%
+[3/9]	Time 10.5309s	Training Accuracy: 69.92593%	Test Accuracy: 71.33333%
+[4/9]	Time 10.5302s	Training Accuracy: 72.81481%	Test Accuracy: 74.00000%
+[5/9]	Time 12.7074s	Training Accuracy: 76.37037%	Test Accuracy: 78.66667%
+[6/9]	Time 14.3102s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 9.5009s	Training Accuracy: 81.62963%	Test Accuracy: 80.66667%
+[8/9]	Time 14.1295s	Training Accuracy: 83.33333%	Test Accuracy: 80.00000%
+[9/9]	Time 7.7721s	Training Accuracy: 85.40741%	Test Accuracy: 82.00000%

    For completeness, let's also test out discrete sensitivities!

    julia
    train(NeuralODE; sensealg=ReverseDiffAdjoint(), cpu=true)
    [1/9]	Time 43.2785s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 24.6150s	Training Accuracy: 58.66667%	Test Accuracy: 57.33333%
+[3/9]	Time 27.3617s	Training Accuracy: 69.70370%	Test Accuracy: 71.33333%
+[4/9]	Time 20.8862s	Training Accuracy: 72.74074%	Test Accuracy: 74.00000%
+[5/9]	Time 23.0043s	Training Accuracy: 76.14815%	Test Accuracy: 78.66667%
+[6/9]	Time 20.1717s	Training Accuracy: 79.03704%	Test Accuracy: 80.66667%
+[7/9]	Time 22.9598s	Training Accuracy: 81.55556%	Test Accuracy: 80.66667%
+[8/9]	Time 19.8321s	Training Accuracy: 83.40741%	Test Accuracy: 80.00000%
+[9/9]	Time 18.7652s	Training Accuracy: 85.25926%	Test Accuracy: 81.33333%

    Alternate Implementation using Stateful Layer ​

    Starting v0.5.5, Lux provides a StatefulLuxLayer which can be used to avoid the Boxing of st. Using the @compact API avoids this problem entirely.

    julia
    struct StatefulNeuralODE{M <: Lux.AbstractLuxLayer, So, T, K} <:
        Lux.AbstractLuxWrapperLayer{:model}
     model::M
     solver::So
@@ -194,20 +194,20 @@
     dudt(u, p, t) = st_model(u, p)
     prob = ODEProblem{false}(ODEFunction{false}(dudt), x, n.tspan, ps)
     return solve(prob, n.solver; n.kwargs...), st_model.st
-end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 40.6684s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
-[2/9]	Time 0.4853s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
-[3/9]	Time 0.5115s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
-[4/9]	Time 0.5330s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
-[5/9]	Time 0.4983s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
-[6/9]	Time 0.4850s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
-[7/9]	Time 0.5257s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
-[8/9]	Time 0.6022s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
-[9/9]	Time 0.8892s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
+end

    Train the new Stateful Neural ODE ​

    julia
    train(StatefulNeuralODE)
    [1/9]	Time 40.9347s	Training Accuracy: 37.48148%	Test Accuracy: 40.00000%
+[2/9]	Time 0.4784s	Training Accuracy: 58.22222%	Test Accuracy: 55.33333%
+[3/9]	Time 0.7823s	Training Accuracy: 68.29630%	Test Accuracy: 68.66667%
+[4/9]	Time 0.4966s	Training Accuracy: 73.11111%	Test Accuracy: 76.00000%
+[5/9]	Time 0.4866s	Training Accuracy: 75.92593%	Test Accuracy: 76.66667%
+[6/9]	Time 0.5181s	Training Accuracy: 78.96296%	Test Accuracy: 80.66667%
+[7/9]	Time 0.8805s	Training Accuracy: 80.81481%	Test Accuracy: 81.33333%
+[8/9]	Time 0.4999s	Training Accuracy: 83.25926%	Test Accuracy: 82.66667%
+[9/9]	Time 0.4981s	Training Accuracy: 84.59259%	Test Accuracy: 82.00000%

    We might not see a significant difference in the training time, but let us investigate the type stabilities of the layers.

    Type Stability ​

    julia
    model, ps, st = create_model(NeuralODE)
 
 model_stateful, ps_stateful, st_stateful = create_model(StatefulNeuralODE)
 
 x = gpu_device()(ones(Float32, 28, 28, 1, 3));

    NeuralODE is not type stable due to the boxing of st

    julia
    @code_warntype model(x, ps, st)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -217,7 +217,7 @@
 1 ─ %1 = Base.getproperty(c, :layers)::@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".NeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}
 │   %2 = Lux.applychain(%1, x, ps, st)::TUPLE{CUDA.CUARRAY{FLOAT32, 2, CUDA.DEVICEMEMORY}, NAMEDTUPLE{(:LAYER_1, :LAYER_2, :LAYER_3, :LAYER_4, :LAYER_5), <:TUPLE{@NAMEDTUPLE{}, @NAMEDTUPLE{}, ANY, @NAMEDTUPLE{}, @NAMEDTUPLE{}}}}
 └──      return %2

    We avoid the problem entirely by using StatefulNeuralODE

    julia
    @code_warntype model_stateful(x, ps_stateful, st_stateful)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".StatefulNeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Main.var"##225".StatefulNeuralODE{Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}, OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, Tuple{Float32, Float32}, Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -229,7 +229,7 @@
 └──      return %2

    Note, that we still recommend using this layer internally and not exposing this as the default API to the users.

    Finally checking the compact model

    julia
    model_compact, ps_compact, st_compact = create_model(NeuralODECompact)
 
 @code_warntype model_compact(x, ps_compact, st_compact)
    MethodInstance for (::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.CompactLuxLayer{:₋₋₋no_special_dispatch₋₋₋, Main.var"##225".var"#2#3", Nothing, @NamedTuple{model::Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}}, Lux.CompactMacroImpl.ValueStorage{@NamedTuple{}, @NamedTuple{solver::Returns{OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}}, tspan::Returns{Tuple{Float32, Float32}}}}, Tuple{Tuple{Symbol}, Tuple{Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing})(::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}, ::ComponentArrays.ComponentVector{Float32, CUDA.CuArray{Float32, 1, CUDA.DeviceMemory}, Tuple{ComponentArrays.Axis{(layer_1 = 1:0, layer_2 = ViewAxis(1:15700, Axis(weight = ViewAxis(1:15680, ShapedAxis((20, 784))), bias = 15681:15700)), layer_3 = ViewAxis(15701:16240, Axis(model = ViewAxis(1:540, Axis(layer_1 = ViewAxis(1:210, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)), layer_2 = ViewAxis(211:320, Axis(weight = ViewAxis(1:100, ShapedAxis((10, 10))), bias = 101:110)), layer_3 = ViewAxis(321:540, Axis(weight = ViewAxis(1:200, ShapedAxis((20, 10))), bias = 201:220)))),)), layer_4 = 16241:16240, layer_5 = ViewAxis(16241:16450, Axis(weight = ViewAxis(1:200, ShapedAxis((10, 20))), bias = 201:210)))}}}, ::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{model::@NamedTuple{layer_1::@NamedTuple{}, layer_2::@NamedTuple{}, layer_3::@NamedTuple{}}, solver::OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}, tspan::Tuple{Float32, Float32}, ₋₋₋kwargs₋₋₋::@NamedTuple{kwargs::Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}, layer_4::@NamedTuple{}, layer_5::@NamedTuple{}})
-  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/src/layers/containers.jl:480
+  from (c::Lux.Chain)(x, ps, st::NamedTuple) @ Lux /var/lib/buildkite-agent/builds/gpuci-4/julialang/lux-dot-jl/src/layers/containers.jl:480
 Arguments
   c::Lux.Chain{@NamedTuple{layer_1::Lux.FlattenLayer{Nothing}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.CompactLuxLayer{:₋₋₋no_special_dispatch₋₋₋, Main.var"##225".var"#2#3", Nothing, @NamedTuple{model::Lux.Chain{@NamedTuple{layer_1::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_2::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}, layer_3::Lux.Dense{typeof(tanh), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}}, Lux.CompactMacroImpl.ValueStorage{@NamedTuple{}, @NamedTuple{solver::Returns{OrdinaryDiffEqTsit5.Tsit5{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False}}, tspan::Returns{Tuple{Float32, Float32}}}}, Tuple{Tuple{Symbol}, Tuple{Base.Pairs{Symbol, Any, NTuple{5, Symbol}, @NamedTuple{save_everystep::Bool, reltol::Float32, abstol::Float32, save_start::Bool, sensealg::SciMLSensitivity.InterpolatingAdjoint{0, true, Val{:central}, SciMLSensitivity.ZygoteVJP}}}}}}, layer_4::Lux.WrappedFunction{Base.Fix1{typeof(Main.var"##225".diffeqsol_to_array), Int64}}, layer_5::Lux.Dense{typeof(identity), Int64, Int64, Nothing, Nothing, Static.True}}, Nothing}
   x::CUDA.CuArray{Float32, 4, CUDA.DeviceMemory}
@@ -298,8 +298,8 @@
 - JULIA_CUDA_HARD_MEMORY_LIMIT: 100%
 
 1 device:
-  0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.889 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + 0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 3.920 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    + \ No newline at end of file diff --git a/previews/PR1000/tutorials/intermediate/2_BayesianNN.html b/previews/PR1000/tutorials/intermediate/2_BayesianNN.html index 2fced190c8..d2e0a211ca 100644 --- a/previews/PR1000/tutorials/intermediate/2_BayesianNN.html +++ b/previews/PR1000/tutorials/intermediate/2_BayesianNN.html @@ -5,14 +5,14 @@ Bayesian Neural Network | Lux.jl Docs - + - + - + - + @@ -114,8 +114,8 @@ Iterations = 1:1:5000 Number of chains = 1 Samples per chain = 5000 -Wall duration = 26.66 seconds -Compute duration = 26.66 seconds +Wall duration = 20.32 seconds +Compute duration = 20.32 seconds parameters = parameters[1], parameters[2], parameters[3], parameters[4], parameters[5], parameters[6], parameters[7], parameters[8], parameters[9], parameters[10], parameters[11], parameters[12], parameters[13], parameters[14], parameters[15], parameters[16], parameters[17], parameters[18], parameters[19], parameters[20] internals = lp, n_steps, is_accept, acceptance_rate, log_density, hamiltonian_energy, hamiltonian_energy_error, numerical_error, step_size, nom_step_size @@ -123,26 +123,26 @@ parameters mean std mcse ess_bulk ess_tail rhat ess_per_sec Symbol Float64 Float64 Float64 Float64 Float64 Float64 Float64 - parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.4534 - parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 2.9983 - parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.7274 - parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.5388 - parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.5651 - parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 0.8146 - parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.4667 - parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 0.9823 - parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.5304 - parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.5283 - parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.6875 - parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 0.8656 - parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.7294 - parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.5382 - parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.6115 - parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.1765 - parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.5879 - parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 1.6856 - parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.6631 - parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.1435 + parameters[1] 2.0965 4.4644 1.3270 12.0887 23.0787 1.3335 0.5950 + parameters[2] 0.0843 0.3952 0.0510 79.9413 35.2287 1.0240 3.9345 + parameters[3] 4.9916 1.9230 0.4456 19.3935 81.5852 1.2170 0.9545 + parameters[4] 0.3356 2.3096 0.6117 14.3653 24.1405 1.2067 0.7070 + parameters[5] 5.0569 2.3192 0.6141 15.0672 34.9402 1.1058 0.7416 + parameters[6] 0.7127 1.2191 0.2991 21.7200 21.7891 1.1550 1.0690 + parameters[7] 1.7149 3.8638 1.1143 12.4420 21.7560 1.1439 0.6124 + parameters[8] 0.3690 1.2835 0.2555 26.1914 29.6846 1.0246 1.2891 + parameters[9] -0.4968 2.2271 0.6133 14.1406 24.5983 1.2648 0.6960 + parameters[10] 0.0842 2.1828 0.5840 14.0865 21.8689 1.1832 0.6933 + parameters[11] -1.0288 1.5663 0.3628 18.3294 27.7789 1.0536 0.9021 + parameters[12] -4.1763 1.7426 0.3705 23.0782 28.1565 1.0633 1.1359 + parameters[13] 3.1846 1.4791 0.3401 19.4472 53.5564 1.0444 0.9571 + parameters[14] 2.7199 1.9547 0.5178 14.3490 46.8984 1.3048 0.7062 + parameters[15] -2.0613 1.4937 0.3727 16.3035 39.2852 1.0885 0.8024 + parameters[16] -2.9853 1.4059 0.2557 31.3669 31.3956 1.0012 1.5438 + parameters[17] -2.4061 2.6897 0.7370 15.6752 20.0204 1.0733 0.7715 + parameters[18] -5.3040 1.1943 0.1791 44.9414 68.7121 1.0867 2.2119 + parameters[19] -5.1706 2.2709 0.5991 17.6788 18.3886 1.0540 0.8701 + parameters[20] -5.1303 1.3008 0.2366 30.4880 60.9517 1.0163 1.5005 Quantiles parameters 2.5% 25.0% 50.0% 75.0% 97.5% @@ -224,16 +224,16 @@ WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-15.0.7 (ORCJIT, znver2) -Threads: 128 default, 0 interactive, 64 GC (on 128 virtual cores) +Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores) Environment: - JULIA_CPU_THREADS = 128 + JULIA_CPU_THREADS = 16 JULIA_DEPOT_PATH = /cache/julia-buildkite-plugin/depots/01872db4-8c79-43af-ab7d-12abac4f24f6 JULIA_PKG_SERVER = - JULIA_NUM_THREADS = 128 + JULIA_NUM_THREADS = 16 JULIA_CUDA_HARD_MEMORY_LIMIT = 100% JULIA_PKG_PRECOMPILE_AUTO = 0 JULIA_DEBUG = Literate

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/intermediate/3_HyperNet.html b/previews/PR1000/tutorials/intermediate/3_HyperNet.html index 58ebf42726..c6fd4e90e8 100644 --- a/previews/PR1000/tutorials/intermediate/3_HyperNet.html +++ b/previews/PR1000/tutorials/intermediate/3_HyperNet.html @@ -5,14 +5,14 @@ Training a HyperNetwork on MNIST and FashionMNIST | Lux.jl Docs - + - + - + - + @@ -136,109 +136,109 @@ return test_acc_list end -test_acc_list = train()
    [  1/ 50]	       MNIST	Time 67.59538s	Training Accuracy: 60.45%	Test Accuracy: 56.25%
-[  1/ 50]	FashionMNIST	Time 0.02886s	Training Accuracy: 53.81%	Test Accuracy: 46.88%
-[  2/ 50]	       MNIST	Time 0.02877s	Training Accuracy: 69.43%	Test Accuracy: 62.50%
-[  2/ 50]	FashionMNIST	Time 0.02784s	Training Accuracy: 62.50%	Test Accuracy: 50.00%
-[  3/ 50]	       MNIST	Time 0.02909s	Training Accuracy: 78.22%	Test Accuracy: 65.62%
-[  3/ 50]	FashionMNIST	Time 0.02795s	Training Accuracy: 62.21%	Test Accuracy: 50.00%
-[  4/ 50]	       MNIST	Time 0.02895s	Training Accuracy: 76.76%	Test Accuracy: 68.75%
-[  4/ 50]	FashionMNIST	Time 0.02354s	Training Accuracy: 62.21%	Test Accuracy: 46.88%
-[  5/ 50]	       MNIST	Time 0.02153s	Training Accuracy: 83.40%	Test Accuracy: 78.12%
-[  5/ 50]	FashionMNIST	Time 0.02135s	Training Accuracy: 69.73%	Test Accuracy: 59.38%
-[  6/ 50]	       MNIST	Time 0.02083s	Training Accuracy: 85.74%	Test Accuracy: 75.00%
-[  6/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 73.93%	Test Accuracy: 62.50%
-[  7/ 50]	       MNIST	Time 0.02118s	Training Accuracy: 91.02%	Test Accuracy: 75.00%
-[  7/ 50]	FashionMNIST	Time 0.02260s	Training Accuracy: 68.65%	Test Accuracy: 53.12%
-[  8/ 50]	       MNIST	Time 0.02231s	Training Accuracy: 91.99%	Test Accuracy: 78.12%
-[  8/ 50]	FashionMNIST	Time 0.03982s	Training Accuracy: 73.14%	Test Accuracy: 62.50%
-[  9/ 50]	       MNIST	Time 0.02182s	Training Accuracy: 94.14%	Test Accuracy: 84.38%
-[  9/ 50]	FashionMNIST	Time 0.02077s	Training Accuracy: 77.93%	Test Accuracy: 56.25%
-[ 10/ 50]	       MNIST	Time 0.02068s	Training Accuracy: 95.70%	Test Accuracy: 84.38%
-[ 10/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 79.20%	Test Accuracy: 65.62%
-[ 11/ 50]	       MNIST	Time 0.02058s	Training Accuracy: 97.17%	Test Accuracy: 78.12%
-[ 11/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 82.42%	Test Accuracy: 68.75%
-[ 12/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 97.85%	Test Accuracy: 78.12%
-[ 12/ 50]	FashionMNIST	Time 0.02021s	Training Accuracy: 84.08%	Test Accuracy: 71.88%
-[ 13/ 50]	       MNIST	Time 0.02726s	Training Accuracy: 97.95%	Test Accuracy: 78.12%
-[ 13/ 50]	FashionMNIST	Time 0.02063s	Training Accuracy: 82.23%	Test Accuracy: 71.88%
-[ 14/ 50]	       MNIST	Time 0.02042s	Training Accuracy: 98.63%	Test Accuracy: 81.25%
-[ 14/ 50]	FashionMNIST	Time 0.02058s	Training Accuracy: 83.40%	Test Accuracy: 65.62%
-[ 15/ 50]	       MNIST	Time 0.02022s	Training Accuracy: 99.22%	Test Accuracy: 78.12%
-[ 15/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 84.08%	Test Accuracy: 68.75%
-[ 16/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 99.61%	Test Accuracy: 78.12%
-[ 16/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 85.35%	Test Accuracy: 68.75%
-[ 17/ 50]	       MNIST	Time 0.02028s	Training Accuracy: 99.80%	Test Accuracy: 78.12%
-[ 17/ 50]	FashionMNIST	Time 0.02797s	Training Accuracy: 86.13%	Test Accuracy: 68.75%
-[ 18/ 50]	       MNIST	Time 0.02152s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 18/ 50]	FashionMNIST	Time 0.02059s	Training Accuracy: 86.23%	Test Accuracy: 65.62%
-[ 19/ 50]	       MNIST	Time 0.02069s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 19/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 86.62%	Test Accuracy: 68.75%
-[ 20/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 20/ 50]	FashionMNIST	Time 0.02016s	Training Accuracy: 87.79%	Test Accuracy: 68.75%
-[ 21/ 50]	       MNIST	Time 0.02172s	Training Accuracy: 99.90%	Test Accuracy: 78.12%
-[ 21/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 88.57%	Test Accuracy: 68.75%
-[ 22/ 50]	       MNIST	Time 0.02040s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 22/ 50]	FashionMNIST	Time 0.02024s	Training Accuracy: 88.38%	Test Accuracy: 65.62%
-[ 23/ 50]	       MNIST	Time 0.02088s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 23/ 50]	FashionMNIST	Time 0.02052s	Training Accuracy: 87.89%	Test Accuracy: 68.75%
-[ 24/ 50]	       MNIST	Time 0.02050s	Training Accuracy: 100.00%	Test Accuracy: 78.12%
-[ 24/ 50]	FashionMNIST	Time 0.02080s	Training Accuracy: 88.96%	Test Accuracy: 62.50%
-[ 25/ 50]	       MNIST	Time 0.02047s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 25/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 90.53%	Test Accuracy: 65.62%
-[ 26/ 50]	       MNIST	Time 0.02916s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 26/ 50]	FashionMNIST	Time 0.02016s	Training Accuracy: 91.31%	Test Accuracy: 65.62%
-[ 27/ 50]	       MNIST	Time 0.02073s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 27/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 91.50%	Test Accuracy: 62.50%
-[ 28/ 50]	       MNIST	Time 0.02008s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 28/ 50]	FashionMNIST	Time 0.02012s	Training Accuracy: 92.48%	Test Accuracy: 68.75%
-[ 29/ 50]	       MNIST	Time 0.02081s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 29/ 50]	FashionMNIST	Time 0.02061s	Training Accuracy: 93.07%	Test Accuracy: 65.62%
-[ 30/ 50]	       MNIST	Time 0.02097s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 30/ 50]	FashionMNIST	Time 0.02821s	Training Accuracy: 92.97%	Test Accuracy: 68.75%
-[ 31/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 31/ 50]	FashionMNIST	Time 0.02067s	Training Accuracy: 93.65%	Test Accuracy: 65.62%
-[ 32/ 50]	       MNIST	Time 0.02055s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 32/ 50]	FashionMNIST	Time 0.02055s	Training Accuracy: 93.85%	Test Accuracy: 68.75%
-[ 33/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 33/ 50]	FashionMNIST	Time 0.02084s	Training Accuracy: 94.53%	Test Accuracy: 68.75%
-[ 34/ 50]	       MNIST	Time 0.02110s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 34/ 50]	FashionMNIST	Time 0.02081s	Training Accuracy: 94.34%	Test Accuracy: 68.75%
-[ 35/ 50]	       MNIST	Time 0.02813s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 35/ 50]	FashionMNIST	Time 0.02068s	Training Accuracy: 94.43%	Test Accuracy: 71.88%
-[ 36/ 50]	       MNIST	Time 0.02078s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 36/ 50]	FashionMNIST	Time 0.02239s	Training Accuracy: 95.02%	Test Accuracy: 75.00%
-[ 37/ 50]	       MNIST	Time 0.02039s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 37/ 50]	FashionMNIST	Time 0.02071s	Training Accuracy: 95.21%	Test Accuracy: 75.00%
-[ 38/ 50]	       MNIST	Time 0.02081s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 38/ 50]	FashionMNIST	Time 0.02029s	Training Accuracy: 95.70%	Test Accuracy: 75.00%
-[ 39/ 50]	       MNIST	Time 0.02027s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 39/ 50]	FashionMNIST	Time 0.02033s	Training Accuracy: 95.61%	Test Accuracy: 71.88%
-[ 40/ 50]	       MNIST	Time 0.02076s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 40/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 95.90%	Test Accuracy: 71.88%
-[ 41/ 50]	       MNIST	Time 0.02056s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 41/ 50]	FashionMNIST	Time 0.02013s	Training Accuracy: 96.39%	Test Accuracy: 75.00%
-[ 42/ 50]	       MNIST	Time 0.02089s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 42/ 50]	FashionMNIST	Time 0.02078s	Training Accuracy: 96.88%	Test Accuracy: 75.00%
-[ 43/ 50]	       MNIST	Time 0.02095s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 43/ 50]	FashionMNIST	Time 0.02954s	Training Accuracy: 96.48%	Test Accuracy: 65.62%
-[ 44/ 50]	       MNIST	Time 0.02079s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 44/ 50]	FashionMNIST	Time 0.02065s	Training Accuracy: 95.90%	Test Accuracy: 68.75%
-[ 45/ 50]	       MNIST	Time 0.02045s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
-[ 45/ 50]	FashionMNIST	Time 0.02060s	Training Accuracy: 95.90%	Test Accuracy: 71.88%
-[ 46/ 50]	       MNIST	Time 0.02054s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
-[ 46/ 50]	FashionMNIST	Time 0.02057s	Training Accuracy: 95.41%	Test Accuracy: 71.88%
-[ 47/ 50]	       MNIST	Time 0.02223s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 47/ 50]	FashionMNIST	Time 0.02075s	Training Accuracy: 96.48%	Test Accuracy: 71.88%
-[ 48/ 50]	       MNIST	Time 0.02729s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 48/ 50]	FashionMNIST	Time 0.02069s	Training Accuracy: 96.68%	Test Accuracy: 71.88%
-[ 49/ 50]	       MNIST	Time 0.02059s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 49/ 50]	FashionMNIST	Time 0.02063s	Training Accuracy: 96.58%	Test Accuracy: 75.00%
-[ 50/ 50]	       MNIST	Time 0.02060s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[ 50/ 50]	FashionMNIST	Time 0.02066s	Training Accuracy: 96.00%	Test Accuracy: 71.88%
+test_acc_list = train()
    [  1/ 50]	       MNIST	Time 80.69589s	Training Accuracy: 55.27%	Test Accuracy: 56.25%
+[  1/ 50]	FashionMNIST	Time 0.03641s	Training Accuracy: 32.81%	Test Accuracy: 21.88%
+[  2/ 50]	       MNIST	Time 0.03428s	Training Accuracy: 67.09%	Test Accuracy: 71.88%
+[  2/ 50]	FashionMNIST	Time 0.05600s	Training Accuracy: 52.25%	Test Accuracy: 46.88%
+[  3/ 50]	       MNIST	Time 0.03427s	Training Accuracy: 77.34%	Test Accuracy: 59.38%
+[  3/ 50]	FashionMNIST	Time 0.06616s	Training Accuracy: 63.48%	Test Accuracy: 59.38%
+[  4/ 50]	       MNIST	Time 0.02669s	Training Accuracy: 78.81%	Test Accuracy: 65.62%
+[  4/ 50]	FashionMNIST	Time 0.02622s	Training Accuracy: 65.92%	Test Accuracy: 59.38%
+[  5/ 50]	       MNIST	Time 0.02200s	Training Accuracy: 82.91%	Test Accuracy: 71.88%
+[  5/ 50]	FashionMNIST	Time 0.02199s	Training Accuracy: 67.97%	Test Accuracy: 59.38%
+[  6/ 50]	       MNIST	Time 0.05453s	Training Accuracy: 84.47%	Test Accuracy: 59.38%
+[  6/ 50]	FashionMNIST	Time 0.04688s	Training Accuracy: 71.19%	Test Accuracy: 59.38%
+[  7/ 50]	       MNIST	Time 0.04077s	Training Accuracy: 88.87%	Test Accuracy: 71.88%
+[  7/ 50]	FashionMNIST	Time 0.04811s	Training Accuracy: 71.48%	Test Accuracy: 75.00%
+[  8/ 50]	       MNIST	Time 0.05305s	Training Accuracy: 90.23%	Test Accuracy: 71.88%
+[  8/ 50]	FashionMNIST	Time 0.02038s	Training Accuracy: 74.71%	Test Accuracy: 65.62%
+[  9/ 50]	       MNIST	Time 0.03395s	Training Accuracy: 93.65%	Test Accuracy: 78.12%
+[  9/ 50]	FashionMNIST	Time 0.02593s	Training Accuracy: 75.98%	Test Accuracy: 68.75%
+[ 10/ 50]	       MNIST	Time 0.02077s	Training Accuracy: 95.21%	Test Accuracy: 78.12%
+[ 10/ 50]	FashionMNIST	Time 0.02246s	Training Accuracy: 81.25%	Test Accuracy: 71.88%
+[ 11/ 50]	       MNIST	Time 0.02136s	Training Accuracy: 97.07%	Test Accuracy: 78.12%
+[ 11/ 50]	FashionMNIST	Time 0.02048s	Training Accuracy: 81.54%	Test Accuracy: 65.62%
+[ 12/ 50]	       MNIST	Time 0.02282s	Training Accuracy: 97.07%	Test Accuracy: 81.25%
+[ 12/ 50]	FashionMNIST	Time 0.02859s	Training Accuracy: 78.12%	Test Accuracy: 62.50%
+[ 13/ 50]	       MNIST	Time 0.02120s	Training Accuracy: 98.05%	Test Accuracy: 81.25%
+[ 13/ 50]	FashionMNIST	Time 0.02173s	Training Accuracy: 82.13%	Test Accuracy: 71.88%
+[ 14/ 50]	       MNIST	Time 0.02161s	Training Accuracy: 99.02%	Test Accuracy: 81.25%
+[ 14/ 50]	FashionMNIST	Time 0.02257s	Training Accuracy: 84.38%	Test Accuracy: 62.50%
+[ 15/ 50]	       MNIST	Time 0.02224s	Training Accuracy: 99.32%	Test Accuracy: 81.25%
+[ 15/ 50]	FashionMNIST	Time 0.02504s	Training Accuracy: 84.47%	Test Accuracy: 62.50%
+[ 16/ 50]	       MNIST	Time 0.02212s	Training Accuracy: 99.41%	Test Accuracy: 81.25%
+[ 16/ 50]	FashionMNIST	Time 0.04436s	Training Accuracy: 85.64%	Test Accuracy: 62.50%
+[ 17/ 50]	       MNIST	Time 0.02359s	Training Accuracy: 99.80%	Test Accuracy: 81.25%
+[ 17/ 50]	FashionMNIST	Time 0.02177s	Training Accuracy: 86.52%	Test Accuracy: 59.38%
+[ 18/ 50]	       MNIST	Time 0.02864s	Training Accuracy: 99.90%	Test Accuracy: 81.25%
+[ 18/ 50]	FashionMNIST	Time 0.02119s	Training Accuracy: 87.99%	Test Accuracy: 62.50%
+[ 19/ 50]	       MNIST	Time 0.02161s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 19/ 50]	FashionMNIST	Time 0.02909s	Training Accuracy: 87.70%	Test Accuracy: 62.50%
+[ 20/ 50]	       MNIST	Time 0.02218s	Training Accuracy: 99.90%	Test Accuracy: 84.38%
+[ 20/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 88.96%	Test Accuracy: 62.50%
+[ 21/ 50]	       MNIST	Time 0.02941s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 21/ 50]	FashionMNIST	Time 0.03479s	Training Accuracy: 88.87%	Test Accuracy: 62.50%
+[ 22/ 50]	       MNIST	Time 0.02223s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 22/ 50]	FashionMNIST	Time 0.02101s	Training Accuracy: 89.06%	Test Accuracy: 62.50%
+[ 23/ 50]	       MNIST	Time 0.02614s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 23/ 50]	FashionMNIST	Time 0.02191s	Training Accuracy: 89.45%	Test Accuracy: 62.50%
+[ 24/ 50]	       MNIST	Time 0.02288s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 24/ 50]	FashionMNIST	Time 0.02745s	Training Accuracy: 89.94%	Test Accuracy: 59.38%
+[ 25/ 50]	       MNIST	Time 0.02736s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 25/ 50]	FashionMNIST	Time 0.02957s	Training Accuracy: 90.43%	Test Accuracy: 65.62%
+[ 26/ 50]	       MNIST	Time 0.02499s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 26/ 50]	FashionMNIST	Time 0.02296s	Training Accuracy: 91.02%	Test Accuracy: 62.50%
+[ 27/ 50]	       MNIST	Time 0.03058s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 27/ 50]	FashionMNIST	Time 0.02179s	Training Accuracy: 90.92%	Test Accuracy: 65.62%
+[ 28/ 50]	       MNIST	Time 0.03407s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 28/ 50]	FashionMNIST	Time 0.01977s	Training Accuracy: 91.60%	Test Accuracy: 65.62%
+[ 29/ 50]	       MNIST	Time 0.02877s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 29/ 50]	FashionMNIST	Time 0.02133s	Training Accuracy: 91.31%	Test Accuracy: 62.50%
+[ 30/ 50]	       MNIST	Time 0.02732s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 30/ 50]	FashionMNIST	Time 0.02149s	Training Accuracy: 91.41%	Test Accuracy: 68.75%
+[ 31/ 50]	       MNIST	Time 0.02000s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 31/ 50]	FashionMNIST	Time 0.02580s	Training Accuracy: 91.60%	Test Accuracy: 65.62%
+[ 32/ 50]	       MNIST	Time 0.03682s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 32/ 50]	FashionMNIST	Time 0.02041s	Training Accuracy: 92.29%	Test Accuracy: 65.62%
+[ 33/ 50]	       MNIST	Time 0.02557s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 33/ 50]	FashionMNIST	Time 0.02514s	Training Accuracy: 92.09%	Test Accuracy: 65.62%
+[ 34/ 50]	       MNIST	Time 0.02063s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 34/ 50]	FashionMNIST	Time 0.02045s	Training Accuracy: 92.68%	Test Accuracy: 65.62%
+[ 35/ 50]	       MNIST	Time 0.02030s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 35/ 50]	FashionMNIST	Time 0.02601s	Training Accuracy: 92.87%	Test Accuracy: 65.62%
+[ 36/ 50]	       MNIST	Time 0.02066s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 36/ 50]	FashionMNIST	Time 0.02098s	Training Accuracy: 93.16%	Test Accuracy: 68.75%
+[ 37/ 50]	       MNIST	Time 0.02270s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 37/ 50]	FashionMNIST	Time 0.02096s	Training Accuracy: 93.46%	Test Accuracy: 68.75%
+[ 38/ 50]	       MNIST	Time 0.02106s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 38/ 50]	FashionMNIST	Time 0.03050s	Training Accuracy: 93.75%	Test Accuracy: 68.75%
+[ 39/ 50]	       MNIST	Time 0.02782s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 39/ 50]	FashionMNIST	Time 0.02598s	Training Accuracy: 94.04%	Test Accuracy: 68.75%
+[ 40/ 50]	       MNIST	Time 0.02427s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 40/ 50]	FashionMNIST	Time 0.03184s	Training Accuracy: 94.24%	Test Accuracy: 68.75%
+[ 41/ 50]	       MNIST	Time 0.02034s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 41/ 50]	FashionMNIST	Time 0.02142s	Training Accuracy: 94.53%	Test Accuracy: 71.88%
+[ 42/ 50]	       MNIST	Time 0.02225s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 42/ 50]	FashionMNIST	Time 0.02270s	Training Accuracy: 94.43%	Test Accuracy: 71.88%
+[ 43/ 50]	       MNIST	Time 0.03666s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 43/ 50]	FashionMNIST	Time 0.02332s	Training Accuracy: 94.82%	Test Accuracy: 71.88%
+[ 44/ 50]	       MNIST	Time 0.03407s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 44/ 50]	FashionMNIST	Time 0.02114s	Training Accuracy: 95.31%	Test Accuracy: 75.00%
+[ 45/ 50]	       MNIST	Time 0.02362s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 45/ 50]	FashionMNIST	Time 0.02191s	Training Accuracy: 95.21%	Test Accuracy: 75.00%
+[ 46/ 50]	       MNIST	Time 0.02321s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 46/ 50]	FashionMNIST	Time 0.02938s	Training Accuracy: 95.31%	Test Accuracy: 75.00%
+[ 47/ 50]	       MNIST	Time 0.03179s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 47/ 50]	FashionMNIST	Time 0.04171s	Training Accuracy: 95.61%	Test Accuracy: 75.00%
+[ 48/ 50]	       MNIST	Time 0.02858s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 48/ 50]	FashionMNIST	Time 0.02325s	Training Accuracy: 95.70%	Test Accuracy: 75.00%
+[ 49/ 50]	       MNIST	Time 0.02226s	Training Accuracy: 100.00%	Test Accuracy: 81.25%
+[ 49/ 50]	FashionMNIST	Time 0.02216s	Training Accuracy: 96.09%	Test Accuracy: 75.00%
+[ 50/ 50]	       MNIST	Time 0.02234s	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[ 50/ 50]	FashionMNIST	Time 0.02797s	Training Accuracy: 96.00%	Test Accuracy: 75.00%
 
-[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 81.25%
-[FINAL]	FashionMNIST	Training Accuracy: 96.00%	Test Accuracy: 71.88%

    Appendix ​

    julia
    using InteractiveUtils
+[FINAL]	       MNIST	Training Accuracy: 100.00%	Test Accuracy: 84.38%
+[FINAL]	FashionMNIST	Training Accuracy: 96.00%	Test Accuracy: 75.00%

    Appendix ​

    julia
    using InteractiveUtils
 InteractiveUtils.versioninfo()
 
 if @isdefined(MLDataDevices)
@@ -299,7 +299,7 @@
 
 1 device:
   0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 2.170 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file diff --git a/previews/PR1000/tutorials/intermediate/4_PINN2DPDE.html b/previews/PR1000/tutorials/intermediate/4_PINN2DPDE.html index ae86991b83..9be1120010 100644 --- a/previews/PR1000/tutorials/intermediate/4_PINN2DPDE.html +++ b/previews/PR1000/tutorials/intermediate/4_PINN2DPDE.html @@ -5,14 +5,14 @@ Training a PINN on 2D PDE | Lux.jl Docs - + - + - + - + @@ -176,16 +176,16 @@ trained_u = Lux.testmode(StatefulLuxLayer{true}( trained_model.model.u, trained_model.ps.u, trained_model.st.u))
    ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 ┌ Warning: `Lag(T, b)` is deprecated.  Use `CircBuff(T,b,rev=true)` instead.
 │   caller = #6 at 4_PINN2DPDE.md:16 [inlined]
-└ @ Core /var/lib/buildkite-agent/builds/gpuci-13/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
+└ @ Core /var/lib/buildkite-agent/builds/gpuci-7/julialang/lux-dot-jl/docs/src/tutorials/intermediate/4_PINN2DPDE.md:16
 Iteration: [    1 / 50000] 	 Loss: 3.159042358 (3.159042358) 	 Physics Loss: 1.982162476 (1.982162476) 	 Data Loss: 0.578374863 (0.578374863) 	 BC Loss: 0.598505080 (0.598505080)
 Iteration: [  501 / 50000] 	 Loss: 0.040918160 (0.025583776) 	 Physics Loss: 0.000391877 (0.000269295) 	 Data Loss: 0.014243508 (0.009196416) 	 BC Loss: 0.026282774 (0.016118063)
 Iteration: [ 1001 / 50000] 	 Loss: 0.015340659 (0.025281426) 	 Physics Loss: 0.000071670 (0.000163182) 	 Data Loss: 0.007905648 (0.010876314) 	 BC Loss: 0.007363341 (0.014241929)
@@ -370,7 +370,7 @@
 
 1 device:
   0: NVIDIA A100-PCIE-40GB MIG 1g.5gb (sm_80, 4.484 GiB / 4.750 GiB available)

    This page was generated using Literate.jl.

    - + \ No newline at end of file