UncertaintyQuantification.jl

Appearance

0.1k

Gaussian Process Regression ​

Methods for Gaussian process regression.

Index ​

Types ​

UncertaintyQuantification.GaussianProcess Type
julia
GaussianProcess(data::DataFrame, output::Symbol; kwargs...)

Constructs a GaussianProcess model with the specified data and output variable.

Arguments

  • data: A DataFrame containing the input and output data.

  • output: The output variable for the Gaussian process.

Keyword Arguments

  • mean_fct: The mean function for the Gaussian process. Defaults to ZeroMean.

  • kernel: The kernel for the Gaussian process. Defaults to SqExponentialKernel.

  • input_transform: The transformation to apply to the input variables. Defaults to IdentityTransformChoice.

  • output_transform: The transformation to apply to the output variables. Defaults to IdentityTransformChoice.

  • σ²: The noise variance. Defaults to 1.0e-10.

  • learn_noise: Whether to learn the noise variance. Defaults to false.

  • learn_hyperparameters: Whether to learn the hyperparameters. Defaults to true.

  • optimizer: The optimization algorithm used to learn the hyperparameters. Defaults to MaximumLikelihoodEstimation(Optim.LBFGS(), Optim.Options(; iterations=100, show_trace=false)).

Examples

julia
julia> mean_fct = ConstMean(0.0);

julia> kernel = SqExponentialKernel();

julia> data = DataFrame(x = 1:10, y = [1, 4, 10, 15, 24, 37, 50, 62, 80, 101]);

julia> gp_model = GaussianProcess(data, :y; mean_fct = mean_fct, kernel = kernel, σ² = 1.0e-3);
source
julia
GaussianProcess(
    gp::GP,
    data::DataFrame,
    output::Symbol;
    kwargs...
)

Constructs a Gaussian process model for the given data and output variable using a pre-defined Gaussian process.

Arguments

  • gp: A Gaussian process object, typically from AbstractGPs, defining the kernel and mean.

  • data: A DataFrame containing the input and output data.

  • output: The name of the output (as a Symbol) to be modeled as the response variable.

Keyword Arguments

  • input_transform: Choice of transformation that is applied to input features before fitting. Defaults to IdentityTransformChoice().

  • output_transform: Choice of transformation that is applied to output data before fitting. Defaults to IdentityTransformChoice().

  • σ²: The noise variance. Defaults to 0.0.

  • learn_noise: Whether to learn the noise variance. Defaults to false.

  • learn_hyperparameters: Whether to learn the hyperparameters. Defaults to true.

  • optimizer: The optimizer for hyperparameter optimization. Defaults to MaximumLikelihoodEstimation with Optim.LBFGS() and Optim.Options(; iterations=100, show_trace=false).

Examples

julia
julia> gp = GP(0.0, SqExponentialKernel());

julia> data = DataFrame(x = 1:10, y = [1, 4, 10, 15, 24, 37, 50, 62, 80, 101]);

julia> gp_model = GaussianProcess(gp, data, :y);
source
julia
GaussianProcess(input::Union{UQInput, Vector{<:UQInput}}, model::Union{UQModel, Vector{<:UQModel}}, output::Symbol; kwargs...)

Constructs a GaussianProcess model with the specified input, model, and output.

Arguments

  • input: The input variable(s) for the Gaussian process.

  • model: The model(s) to be used for the Gaussian process.

  • output: The output variable for the Gaussian process.

Keyword Arguments

  • n_design_points: Number of design points to sample from the input space. Defaults to 10.

  • experimental_design: The strategy utilized for sampling the input variables. Defaults to LatinHypercubeSampling.

  • mean_fct: The mean function for the Gaussian process. Defaults to ZeroMean.

  • kernel: The kernel for the Gaussian process. Defaults to SqExponentialKernel.

  • input_transform: The transformation to apply to the input variables. Defaults to IdentityTransformChoice.

  • output_transform: The transformation to apply to the output variables. Defaults to IdentityTransformChoice.

  • σ²: The noise variance. Defaults to 0.0.

  • learn_noise: Whether to learn the noise variance. Defaults to false.

  • learn_hyperparameters: Whether to learn the hyperparameters. Defaults to true.

  • optimizer: The optimization algorithm used to learn the hyperparameters. Defaults to MaximumLikelihoodEstimation(Optim.LBFGS(), Optim.Options(; iterations=100, show_trace=false)).

Examples

julia
julia> begin # hide
           mean_fct = ConstMean(0.0)
           kernel = SqExponentialKernel()
           x = RandomVariable(Uniform(0, 5), :x)
           model = Model(df -> sin.(df.x), :y)
           design = LatinHypercubeSampling(10)
           gp_model = GaussianProcess(x, model, :y; experimental_design = design, mean_fct = mean_fct, kernel = kernel)
           nothing # hide
       end # hide
source
julia
GaussianProcess(
    gp::GP,
    input::Vector{<:UQInput},
    model::Union{UQModel, Vector{<:UQModel}},
    output::Symbol;
    kwargs...
)

Constructs a Gaussian process model for the given input and model. Evaluates the model using specified experimental design.

Arguments

  • gp: A Gaussian process object, typically from AbstractGPs, defining the kernel and mean.

  • input: Single input or vector of inputs. The Gaussian process will only consider inputs of type RandomVariable as input features.

  • model: Single model or vector of models of supertype UQModel that the Gaussian process is supposed to model.

  • output: The name of the output (as a Symbol) to be modeled as the response variable.

Keyword Arguments

  • n_design_points: Number of design points to sample from the input space. Defaults to 10.

  • experimental_design: The strategy utilized for sampling the input variables.

  • input_transform: Choice of transformation that is applied to input features before fitting. Defaults to IdentityTransformChoice().

  • output_transform: Choice of transformation that is applied to output data before fitting. Defaults to IdentityTransformChoice().

  • σ²: The noise variance. Defaults to 0.0.

  • learn_noise: Whether to learn the noise variance. Defaults to false.

  • learn_hyperparameters: Whether to learn the hyperparameters. Defaults to true.

  • optimizer: The optimization algorithm used to learn the hyperparameters. Defaults to MaximumLikelihoodEstimation(Optim.LBFGS(), Optim.Options(; iterations=100, show_trace=false)).

Examples

julia
julia> begin # hide
           gp = GP(0.0, SqExponentialKernel())
           x = RandomVariable(Uniform(0, 5), :x)
           model = Model(df -> sin.(df.x), :y)
           design = LatinHypercubeSampling(10)
           gp_model = GaussianProcess(gp, x, model, :y; experimental_design = design)
           nothing # hide
       end # hide
source
UncertaintyQuantification.MaximumLikelihoodEstimation Type
julia
MaximumLikelihoodEstimation(
    optimizer = LBFGS(),
    options = Optim.Options(; iterations = 100, show_trace = false),
    backend = AutoMooncake();
    restarts = 5,
)

Represents a hyperparameter optimization strategy that maximizes the log marginal likelihood of a Gaussian process model with random restarts of the optimization.

Arguments

  • optimizer::Optim.AbstractOptimizer: chosen optimizer (default: LBFGS())

  • options::Optim.Options: options for the optimizer (default: Optim.Options(; iterations = 100, show_trace = false))

  • backend::AbstractADType: automatic differentiation backend (default: AutoMooncake(); can be nothing when using gradient-free optimization)

Keyword Arguments

  • restarts: Number of additional randomized optimization runs. Defaults to 5.

Note

You can choose from any optimizer and set of options provided by Optim.jl, such as LBFGS(), Adam(), or ConjugateGradient().

Examples

julia
julia> using Optim

julia> MaximumLikelihoodEstimation(Optim.Adam(alpha = 0.01), Optim.Options(; iterations = 1000, show_trace = false))
MaximumLikelihoodEstimation(Adam{Float64, Float64, Flat}(0.01, 0.9, 0.999, 1.0e-8, Flat()), Optim.Options(x_abstol = 0.0, x_reltol = 0.0, f_abstol = 0.0, f_reltol = 0.0, g_abstol = 1.0e-8, outer_x_abstol = 0.0, outer_x_reltol = 0.0, outer_f_abstol = 0.0, outer_f_reltol = 0.0, outer_g_abstol = 1.0e-8, f_calls_limit = 0, g_calls_limit = 0, h_calls_limit = 0, allow_f_increases = true, allow_outer_f_increases = true, successive_f_tol = 1, iterations = 1000, outer_iterations = 1000, store_trace = false, trace_simplex = false, show_trace = false, extended_trace = false, show_warnings = true, show_every = 1, time_limit = NaN, )
, AutoMooncake(), 5)
source
UncertaintyQuantification.IdentityTransformChoice Type
julia
IdentityTransformChoice()

A standardization choice that specifies the application of an identity transform to data.

Used as an input or output transformation in a GaussianProcess. Internally, the DataStandardizer constructs the functions required for evaluation.

Examples

julia
julia> id = IdentityTransformChoice()
IdentityTransformChoice()
source
UncertaintyQuantification.ZScoreTransformChoice Type
julia
ZScoreTransformChoice()

A standardization choice that specifies the application of a Z-score-transformation to data.

Used as an input or output transformation in a GaussianProcess. Internally, the DataStandardizer constructs the functions required for evaluation.

Examples

julia
julia> zscore = ZScoreTransformChoice()
ZScoreTransformChoice()
source
UncertaintyQuantification.UnitRangeTransformChoice Type
julia
UnitRangeTransformChoice()

A standardization choice that specifies the application of a unit range transform to data.

Used as an input or output transformation in a GaussianProcess. Internally, the DataStandardizer constructs the functions required for evaluation.

Examples

julia
julia> unitrange = UnitRangeTransformChoice()
UnitRangeTransformChoice()
source
UncertaintyQuantification.StandardNormalTransformChoice Type
julia
StandardNormalTransformChoice()

A standardization choice that specifies the application of a standard normal transformation to data.

Can only be used as an input transformation in a GaussianProcess for inputs of type RandomVariable. Internally, the DataStandardizer constructs the function required for evaluation.

Examples

julia
julia> sns = StandardNormalTransformChoice()
StandardNormalTransformChoice()
source
UncertaintyQuantification.MaximumVariance Type
julia
MaximumVariance()

Selects the candidate with the largest posterior variance.

Reference

Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989). Design and Analysis of Computer Experiments. Statistical Science, 4(4), 409–423. https://doi.org/10.1214/ss/1177012413

source
UncertaintyQuantification.ExpectedImprovement Type
julia
ExpectedImprovement(ξ = 0.0)

Selects the candidate maximizing expected improvement over the current best (minimum) observed training output. ξ controls the exploration/exploitation trade-off (higher ξ favors exploration).

Reference

Jones, D. R., Schonlau, M., & Welch, W. J. (1998). Efficient Global Optimization of Expensive Black-Box Functions. Journal of Global Optimization, 13(4), 455–492. https://doi.org/10.1023/A:1008306431147

source
UncertaintyQuantification.ProbabilityOfImprovement Type
julia
ProbabilityOfImprovement(ξ = 0.0)

Selects the candidate maximizing the probability of improving over the current best (minimum) observed training output.

Reference

Kushner, H. J. (1964). A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise. Journal of Basic Engineering, 86(1), 97–106. https://doi.org/10.1115/1.3653121

source
UncertaintyQuantification.UpperConfidenceBound Type
julia
UpperConfidenceBound(κ = 2.0)

Selects the candidate minimizing μ(x) - κ·σ(x) (lower confidence bound for a minimization objective). κ controls the exploration weight.

Reference

Cox, D. D., & John, S. (1992). A Statistical Method for Global Optimization. Proceedings of the 1992 IEEE International Conference on Systems, Man, and Cybernetics, 1241–1246. https://doi.org/10.1109/ICSMC.1992.271617

source
UncertaintyQuantification.DeviationNumber Type
julia
DeviationNumber(threshold = 0.0)

U-function: selects the candidate minimizing |μ(x) - threshold| / σ(x), i.e. the point closest to the limit state and with the highest uncertainty.

Reference

Echard, B., Gayton, N., & Lemaire, M. (2011). AK-MCS: An active learning reliability method combining Kriging and Monte Carlo Simulation. Structural Safety, 33(2), 145–154. https://doi.org/10.1016/j.strusafe.2011.01.002

source
UncertaintyQuantification.ExpectedFeasibility Type
julia
ExpectedFeasibility(threshold = 0.0, epsilon_factor = 2.0)

Selects the candidate with the largest expected feasibility function (EFF) near the limit state G(x) = threshold. The feasibility half-width is set per candidate as epsilon_factor * σ(x).

Reference

Bichon, B. J., Eldred, M. S., Swiler, L. P., Mahadevan, S., & McFarland, J. M. (2008). Efficient Global Reliability Analysis for Nonlinear Implicit Performance Functions. AIAA Journal, 46(10), 2459-2468. https://doi.org/10.2514/1.34321

source
UncertaintyQuantification.MaximinDistance Type
julia
MaximinDistance()

Space-filling score that selects the candidate farthest from every existing training point (maximizes the nearest-neighbor distance).

Reference

Johnson, M. E., Moore, L. M., & Ylvisaker, D. (1990). Minimax and Maximin Distance Designs. Journal of Statistical Planning and Inference, 26(2), 131–148. https://doi.org/10.1016/0378-3758(90)90122-B

source
UncertaintyQuantification.ExpectedImprovementForGlobalFit Type
julia
ExpectedImprovementForGlobalFit()

EIGF score rewards candidates far (in output) from their nearest training observation, adjusted by local posterior variance.

Reference

Lam, C. Q. (2008). Sequential Adaptive Designs in Computer Experiments for Response Surface Model Fit. PhD dissertation, The Ohio State University.

source

Functions ​

UncertaintyQuantification.AdaptiveGaussianProcess Function
julia
AdaptiveGaussianProcess(
    gp::GP,
    input,
    model,
    output,
    acquisition_function,
    n_added_points,
    n_design_points = 10,
    experimental_design = LatinHypercubeSampling(n_design_points);
    kwargs...
)

Fit a Gaussian-process surrogate from an initial experimental design, then adaptively enrich its training data with points selected by acquisition_function.

At each adaptive iteration, candidate points are sampled from input, the acquisition function selects one candidate, model is evaluated at that point, and the Gaussian process is refitted using the enlarged training set.

Arguments

  • gp: Prior Gaussian process specifying the mean function and kernel.

  • input: Input variable or variables used for the initial design and candidate sampling.

  • model: Expensive model evaluated at selected adaptive points.

  • output: Name of the model output approximated by the surrogate.

  • acquisition_function: Learning function used to rank candidate points.

  • n_added_points: Number of adaptively selected training points.

  • n_design_points: Number of points in the initial experimental design.

  • experimental_design: Sampling/design method for the initial training data.

Keyword Arguments

  • input_transform: Transformation applied to GP input features.

  • output_transform: Transformation applied to the GP response.

  • σ²: Observation-noise variance.

  • learn_noise: Whether to infer the observation-noise variance.

  • learn_hyperparameters: Whether to optimize GP hyperparameters on each fit.

  • candidate_sampling: Monte Carlo sampling method used for acquisition candidates.

  • optimizer: Hyperparameter-optimization strategy.

Examples

julia
julia> x = RandomVariable(Uniform(-2, 2), :x);

julia> model = Model(df -> sin.(df.x), :y);

julia> prior = GP(ZeroMean(), SqExponentialKernel());

julia> surrogate = AdaptiveGaussianProcess(
           prior,
           x,
           model,
           :y,
           MaximumVariance(),
           5,
       );
source
julia
AdaptiveGaussianProcess(
    input,
    model,
    output,
    acquisition_function,
    n_added_points,
    n_design_points = 10,
    experimental_design = LatinHypercubeSampling(n_design_points);
    mean_fct = ZeroMean(),
    kernel = SqExponentialKernel(),
    kwargs...
)

Construct and adapt a Gaussian-process surrogate using a zero-mean, squared-exponential prior by default. Use mean_fct and kernel to choose a different prior.

source
julia
AdaptiveGaussianProcess(
    gp_model,
    input,
    model,
    acquisition_function,
    n_added_points;
    kwargs...
)

Adapt an already-fitted GaussianProcess by evaluating model at n_added_points selected from candidates sampled from input.

source
julia
AdaptiveGaussianProcess(
    gp,
    data,
    input,
    model,
    output,
    acquisition_function,
    n_added_points;
    kwargs...
)

Fit the initial surrogate from data with the specified prior gp, then adaptively add points selected from candidates sampled from input. The model supplies the expensive response at each selected point.

source
julia
AdaptiveGaussianProcess(
    data,
    input,
    model,
    output,
    acquisition_function,
    n_added_points;
    mean_fct = ZeroMean(),
    kernel = SqExponentialKernel(),
    kwargs...
)

Fit the initial surrogate from data, then adaptively add points selected from candidates sampled from input. Use mean_fct and kernel to specify the initial GP prior; model is required to evaluate newly selected points.

source
UncertaintyQuantification.evaluate! Method
julia
evaluate!(gp::GaussianProcess, data::DataFrame; mode::Symbol = :mean, n_samples::Int = 1)

Evaluates a fitted GaussianProcess model at the specified input locations.

Arguments

  • gp: Trained Gaussian process model to be evaluated.

  • data: A DataFrame containing the input locations at which predictions are computed.

Keyword Arguments

  • mode: A Symbol specifying the type of output to return. Supported options are:

    • :mean - predictive mean (default)

    • :var - predictive variance

    • :mean_and_var - both mean and variance

    • :sample - random samples from the predictive distribution

  • n_samples: Number of samples to draw when mode = :sample. Ignored otherwise. (Note: Sampling can be unstable when input locations are very close together, leading to numerical issues in the covariance matrix.)

Examples

julia
julia> gp = GP(0.0, SqExponentialKernel());

julia> data = DataFrame(x = 1:10, y = [1, 4, 10, 15, 24, 37, 50, 62, 80, 101]);

julia> gp_model = GaussianProcess(gp, data, :y; σ² = 1.0e-3);

julia> df = DataFrame(x = [0.5, 1.5, 2.5, 5.5, 8.5]);

julia> evaluate!(gp_model, df; mode = :mean_and_var);
source