,
Anders Kvellestad [aut] ,
Riccardo De Bin [ctb] | Title: | Dividing Local Gaussian Processes for Online Learning Regression |
|---|---|
| Description: | We implement and extend the Dividing Local Gaussian Process algorithm by Lederer et al. (2020) <doi:10.48550/arXiv.2006.09446>. Its main use case is in online learning where it is used to train a network of local GPs (referred to as tree) by cleverly partitioning the input space. In contrast to a single GP, 'GPTreeO' is able to deal with larger amounts of data. The package includes methods to create the tree and set its parameter, incorporating data points from a data stream as well as making joint predictions based on all relevant local GPs. |
| Authors: | Timo Braun [aut, cre] ,
Anders Kvellestad [aut] ,
Riccardo De Bin [ctb] |
| Maintainer: | Timo Braun <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.0.1 |
| Built: | 2025-02-13 03:57:09 UTC |
| Source: | https://github.com/timo-braun/gptreeo |
Factory function called by GPNode to create the wrapper for a specified GP package
CreateWrappedGP( wrapper, X, y, y_var, gp_control, init_covpars, retrain_buffer_length, add_buffer_in_prediction )CreateWrappedGP( wrapper, X, y, y_var, gp_control, init_covpars, retrain_buffer_length, add_buffer_in_prediction )
wrapper |
A string specifying what GP implementation is used |
X |
Input data matrix with x_dim columns and at maximum Nbar rows. Is used to create the first iteration of the local GP. |
y |
Value of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored |
y_var |
Variance of the target variable; has to be a one-dimensional matrix or vector |
gp_control |
A list of GP implementation-specific options, passed directly to the wrapped GP implementation |
init_covpars |
Initial covariance parameters of the local GP |
retrain_buffer_length |
Only retrain when the number of buffer points or collected points exceeds this value |
add_buffer_in_prediction |
If TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated. |
A detailed list of expected functions from GPTree and GPNode can be found in the comments of this file. Currently, GPs from the DiceKriging package (WrappedDiceKrigingGP) and mlegp package (WrappedmlegpGP) are implemented. The user can create their own wrapper using WrappedGP.
The wrapper of the chosen GP package, containing the respective GP and information on the shared points and those stored in the buffer.
The nodes contain the local GP if they are leaves (at the end of a branch). Nodes that are just nodes contain information on how the input space was split. They are responsible for computing and updating the splitting probabilities. Also, the tree interacts with the local GPs through the nodes.
Currently, GPs from the DiceKriging package (WrappedDiceKrigingGP) and mlegp package (WrappedmlegpGP) are implemented. The user can create their own wrapper using WrappedGP.
keyA string like "0110100" to identify the node in the binary tree
x_dimDimensionality of input points. It is set once the first point is received through the GPTree method update. It needs to be specified if min_ranges should be different from default.
thetaOverlap ratio between two leafs in the split direction. The default value is 0.
split_direction_criterionA string that indicates which spitting criterion to use. The options are:
"max_spread": Split along the direction which has the largest data spread.
"min_lengthscale": split along the direction with the smallest length-scale hyperparameter from the local GP.
"max_spread_per_lengthscale": Split along the direction with the largest data spread relative to the corresponding GP length-scale hyperparameter.
"max_corr": Split along the direction where the input data is most strongly correlated with the target variable.
"principal_component": Split along the first principal component.
The default value is "max_spread_per_lengthscale".
split_position_criterionA string indicating how the split position along the split direction should be set. Possible values are ("mean" and "median"). The default is "mean".
shape_decayA string specifying how the probability function for a point to be assigned to the left leaf should fall off in the overlap region. The available options are a linear shape ("linear"), an exponential shape ("exponential") or a Gaussian shape ("gaussian"). Another option is to select no overlap region. This can be achieved by selecting "deterministic" or to set theta to 0. The default is "linear".
prob_min_thetaMinimum probability after which the overlap shape gets truncated (either towards 0 or 1). The default value is 0.01.
NbarMaximum number of data points for each GP in a leaf before it is split. The default value is 1000.
min_rangesSmallest allowed input data spread (per dimension) before node splitting stops. It is set to its default min_ranges = rep(0.0, x_dim) once the first point is received through the update method. x_dim needs to be specified by the user if it should be different from the default.
is_leafIf TRUE, this node a leaf, i.e the last node on its branch
wrapped_gpAn instance of the WrappedGP type
can_splitIf TRUE for a given dimension, the leaf can be split along that dimension
rotation_matrixA rotation matrix, used for transforming the data
shiftA shift, used for transforming the data
use_pc_transformTRUE if principal components transformation is used for node splitting
x_spreadVector of data spread for each dimension
split_indexIndex for the split dimension
position_splitPosition of the split along dimension split_index
width_overlapWidth of overlap region along dimension split_index
point_idsIDs of the points assigned to this node
residualsVector of residuals
pred_errsVector of prediction uncertainties
error_scalerScaling factor for the prediction error to ensure desired coverage
use_n_residualsNumber of past residuals to use in calibrating the error_scaler
new()
Create a new node object
GPNode$new( key, x_dim, theta, split_direction_criterion, split_position_criterion, shape_decay, prob_min_theta, Nbar, wrapper, gp_control, retrain_buffer_length, add_buffer_in_prediction, min_ranges = NULL, is_leaf = TRUE )
keyA string like "0110100" to identify the node in the binary tree
x_dimDimensionality of input points. It is set once the first point is received through the GPTree method update. It needs to be specified if min_ranges should be different from default.
thetaOverlap ratio between two leafs in the split direction. The default value is 0.
split_direction_criterionA string that indicates which spitting criterion to use. The options are:
"max_spread": Split along the direction which has the largest data spread.
"min_lengthscale": split along the direction with the smallest length-scale hyperparameter from the local GP.
"max_spread_per_lengthscale": Split along the direction with the largest data spread relative to the corresponding GP length-scale hyperparameter.
"max_corr": Split along the direction where the input data is most strongly correlated with the target variable.
"principal_component": Split along the first principal component.
The default value is "max_spread_per_lengthscale".
split_position_criterionA string indicating how the split position along the split direction should be set. Possible values are ("mean" and "median"). The default is "mean".
shape_decayA string specifying how the probability function for a point to be assigned to the left leaf should fall off in the overlap region. The available options are a linear shape ("linear"), an exponential shape ("exponential") or a Gaussian shape ("gaussian"). Another option is to select no overlap region. This can be achieved by selecting "deterministic" or to set theta to 0. The default is "linear".
prob_min_thetaMinimum probability after which the overlap shape gets truncated (either towards 0 or 1). The default value is 0.01.
NbarMaximum number of data points for each GP in a leaf before it is split. The default value is 1000.
wrapperA string that indicates which GP implementation should be used. The current version includes wrappers for the packages "DiceKriging" and "mlegp". The default setting is "DiceKriging".
gp_controlA list of control parameter that is forwarded to the wrapper. Here, the covariance function is specified. DiceKriging allows for the following kernels, passed as string: "gauss", "matern5_2", "matern3_2", "exp", "powexp" where "matern3_2" is set as default.
retrain_buffer_lengthSize of the retrain buffer. The buffer for a each node collects data points and holds them until the buffer length is reached. Then the GP in the node is updated with the data in the buffer. For a fixed Nbar, higher values for retrain_buffer_length lead to faster run time (less frequent retraining), but the trade-off is a temporary reduced prediction accuracy. We advise that the choice for retrain_buffer_length should depend on the chosen Nbar. By default retrain_buffer_length is set equal to Nbar.
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated. The default is FALSE.
min_rangesSmallest allowed input data spread (per dimension) before node splitting stops. It is set to its default min_ranges = rep(0.0, x_dim) once the first point is received through the GPTree method update. x_dim needs to be specified by the user if it should be different from the default.
is_leafIf TRUE, this node a leaf, i.e the last node on its branch.
n_points_train_limitNumber of points at which a GP is created in the leaf
A new GPNode object. Contains the local GP in the field wrapped_gp, and information used for and related to splitting the node. If the node has been split, the local GP is removed.
transform()
Method to transform input data through a shift and a rotation. IS EXPECTED TO NOT BE CALLED BY THE USER
GPNode$transform(X)
XMatrix with x points
The transformed X matrix
update_prob_pars()
Method to update the probability parameters (x_spread, can_split, split_index, position_split, width_overlap). IS EXPECTED TO NOT BE CALLED BY THE USER
GPNode$update_prob_pars()
get_prob_child_1()
Method to compute the probability that a point x should go to child 1. IS EXPECTED TO NOT BE CALLED BY THE USER
GPNode$get_prob_child_1(x)
xSingle data point for which probability is computed; has to be a vector with length equal to x_dim
The probability that a point x should go to child 1
register_residual()
Method to register prediction performance
GPNode$register_residual(x, y)
xMost recent single input data point from the data stream; has to be a vector with length equal to x_dim
yTarget variable which has to be a one-dimensional matrix or a vector; any further columns will be ignored
update_empirical_error_pars()
Method for updating the empirical error parameters
GPNode$update_empirical_error_pars()
delete_gp()
Method to delete the GP. IS EXPECTED TO NOT BE CALLED BY THE USER
GPNode$delete_gp()
clone()
The objects of this class are cloneable with this method.
GPNode$clone(deep = FALSE)
deepWhether to make a deep clone.
GPTree() for the main methods
The base class which contains and where all parameters are set. Here, all information on how and when the splitting is carried out is stored.
wrapper and gp_control specify the Gaussian process (GP) implementation and its parameters. Moreover, minimum errors and calibration of the predictions are specified here, too.
Essential methods
The following three methods are essential for the package. The remaining ones are mostly not expected to be called by the user.
GPTree$new(): Creates a new tree with specified parameters
GPTree$update(): Adds the information from the input point to the tree and updates local GPs
GPTree$joint_prediction(): Computes the joint prediction for a given input point
The tree collects the information from all GPNodes which in turn contain the local GP. Currently, GPs from the DiceKriging package (WrappedDiceKrigingGP) and mlegp package (WrappedmlegpGP) are implemented. The user can create their own wrapper using WrappedGP.
NbarMaximum number of data points for each GP in a leaf before it is split. The default value is 1000.
retrain_buffer_lengthSize of the retrain buffer. The buffer for a each node collects data points and holds them until the buffer length is reached. Then the GP in the node is updated with the data in the buffer. For a fixed Nbar, higher values for retrain_buffer_length lead to faster run time (less frequent retraining), but the trade-off is a temporary reduced prediction accuracy. We advise that the choice for retrain_buffer_length should depend on the chosen Nbar. By default retrain_buffer_length is set equal to Nbar.
gradual_splitIf TRUE, gradual splitting is used for splitting. The default value is TRUE.
thetaOverlap ratio between two leafs in the split direction. The default value is 0.
wrapperA string that indicates which GP implementation should be used. The current version includes wrappers for the packages "DiceKriging" and "mlegp". The default setting is "DiceKriging".
gp_controlA list of control parameter that is forwarded to the wrapper. Here, the covariance function is specified. DiceKriging allows for the following kernels, passed as string: "gauss", "matern5_2", "matern3_2", "exp", "powexp" where "matern3_2" is set as default.
split_direction_criterionA string that indicates which spitting criterion to use. The options are:
"max_spread": Split along the direction which has the largest data spread.
"min_lengthscale": split along the direction with the smallest length-scale hyperparameter from the local GP.
"max_spread_per_lengthscale": Split along the direction with the largest data spread relative to the corresponding GP length-scale hyperparameter.
"max_corr": Split along the direction where the input data is most strongly correlated with the target variable.
"principal_component": Split along the first principal component.
The default value is "max_spread_per_lengthscale".
split_position_criterionA string indicating how the split position along the split direction should be set. Possible values are ("median" and "mean"). The default is "median".
shape_decayA string specifying how the probability function for a point to be assigned to the left leaf should fall off in the overlap region. The available options are a linear shape ("linear"), an exponential shape ("exponential") or a Gaussian shape ("gaussian"). Another option is to select no overlap region. This can be achieved by selecting "deterministic" or to set theta to 0. The default is "linear".
use_empirical_errorIf TRUE, the uncertainty is calibrated using recent data points. The default value is TRUE.
The most recent 25 observations are used to ensure that the prediction uncertainty yields approximately 68 % coverage. This coverage is only achieved if theta = 0 (also together with gradual_split = TRUE) is used. Nevertheless, the coverage will be closer to 68 % than it would be without calibration. The prediction uncertainties at the beginning are conservative and become less conservative with increasing number of input points.
use_reference_gpIf TRUE, the covariance parameters determined for the GP in node 0 will be used for all subsequent GPs. The default is FALSE.
min_abs_y_errMinimum absolute error assumed for y data. The default value is 0.
min_rel_y_errMinimum relative error assumed for y data. The default value is 100 * .Machine$double.eps.
min_abs_node_pred_errMinimum absolute error on the prediction from a single node. The default value is 0.
min_rel_node_pred_errMinimum relative error on the prediction from a single node. The default value is 100 * .Machine$double.eps.
prob_min_thetaMinimum probability after which the overlap shape gets truncated (either towards 0 or 1). The default value is 0.01.
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated. The default is FALSE.
x_dimDimensionality of input points. It is set once the first point is received through the update() or joint_prediction() method. It needs to be specified if min_ranges should be different from default.
min_rangesSmallest allowed input data spread (per dimension) before node splitting stops. It is set to its default min_ranges = rep(0.0, x_dim) once the first point is received through the update() method. x_dim needs to be specified by the user if it should be different from the default.
max_cond_numAdd additional noise if the covariance matrix condition number exceeds this value. The default is NULL.
max_pointsThe maximum number of points the tree is allowed to store. The default value is Inf.
End of the user-defined input fields.
nodesA hash to hold the GP tree, using string keys to identify nodes and their position in the tree ("0", "00", "01", "000", "001", "010", "011", etc.)
leaf_keysStores the keys ("0", "00", "01", "000", "001", "010", "011", etc.) for the leaves
n_pointsNumber of points in the tree
n_fedNumber of points fed to the tree
new()
GPTree$new( Nbar = 1000, retrain_buffer_length = Nbar, gradual_split = TRUE, theta = 0, wrapper = "DiceKriging", gp_control = list(covtype = "matern3_2"), split_direction_criterion = "max_spread_per_lengthscale", split_position_criterion = "median", shape_decay = "linear", use_empirical_error = TRUE, use_reference_gp = FALSE, min_abs_y_err = 0, min_rel_y_err = 100 * .Machine$double.eps, min_abs_node_pred_err = 0, min_rel_node_pred_err = 100 * .Machine$double.eps, prob_min_theta = 0.01, add_buffer_in_prediction = FALSE, x_dim = 0, min_ranges = NULL, max_cond_num = NULL, max_points = Inf )
NbarMaximum number of data points for each GP in a leaf before it is split. The default value is 1000.
retrain_buffer_lengthSize of the retrain buffer. The buffer for a each node collects data points and holds them until the buffer length is reached. Then the GP in the node is updated with the data in the buffer. For a fixed Nbar, higher values for retrain_buffer_length lead to faster run time (less frequent retraining), but the trade-off is a temporary reduced prediction accuracy. We advise that the choice for retrain_buffer_length should depend on the chosen Nbar. By default retrain_buffer_length is set equal to Nbar.
gradual_splitIf TRUE, gradual splitting is used for splitting. The default value is TRUE.
thetaOverlap ratio between two leafs in the split direction. The default value is 0.
wrapperA string that indicates which GP implementation should be used. The current version includes wrappers for the packages "DiceKriging" and "mlegp". The default setting is "DiceKriging".
gp_controlA list of control parameter that is forwarded to the wrapper. Here, the covariance function is specified. DiceKriging allows for the following kernels, passed as string: "gauss", "matern5_2", "matern3_2", "exp", "powexp" where "matern3_2" is set as default.
split_direction_criterionA string that indicates which spitting criterion to use. The options are:
"max_spread": Split along the direction which has the largest data spread.
"min_lengthscale": split along the direction with the smallest length-scale hyperparameter from the local GP.
"max_spread_per_lengthscale": Split along the direction with the largest data spread relative to the corresponding GP length-scale hyperparameter.
"max_corr": Split along the direction where the input data is most strongly correlated with the target variable.
"principal_component": Split along the first principal component.
The default value is "max_spread_per_lengthscale".
split_position_criterionA string indicating how the split position along the split direction should be set. Possible values are ("median" and "mean"). The default is "median".
shape_decayA string specifying how the probability function for a point to be assigned to the left leaf should fall off in the overlap region. The available options are a linear shape ("linear"), an exponential shape ("exponential") or a Gaussian shape ("gaussian"). Another option is to select no overlap region. This can be achieved by selecting "deterministic" or to set theta to 0. The default is "linear".
use_empirical_errorIf TRUE, the uncertainty is calibrated using recent data points. The default value is TRUE.
The most recent 25 observations are used to ensure that the prediction uncertainty yields approximately 68 % coverage. This coverage is only achieved if theta = 0 (also together with gradual_split = TRUE) is used. Nevertheless, the coverage will be closer to 68 % than it would be without calibration. The prediction uncertainties at the beginning are conservative and become less conservative with increasing number of input points.
use_reference_gpIf TRUE, the covariance parameters determined for the GP in node 0 will be used for all subsequent GPs. The default is FALSE.
min_abs_y_errMinimum absolute error assumed for y data. The default value is 0.
min_rel_y_errMinimum relative error assumed for y data. The default value is 100 * .Machine$double.eps.
min_abs_node_pred_errMinimum absolute error on the prediction from a single node. The default value is 0.
min_rel_node_pred_errMinimum relative error on the prediction from a single node. The default value is 100 * .Machine$double.eps.
prob_min_thetaMinimum probability after which the overlap shape gets truncated (either towards 0 or 1). The default value is 0.01.
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated. The default is FALSE.
x_dimDimensionality of input points. It is set once the first point is received through the update method. It needs to be specified if min_ranges should be different from default.
min_rangesSmallest allowed input data spread (per dimension) before node splitting stops. It is set to its default min_ranges = rep(0.0, x_dim) once the first point is received through the update method. x_dim needs to be specified by the user if it should be different from the default.
max_cond_numAdd additional noise if the covariance matrix condition number exceeds this value. The default is NULL.
max_pointsThe maximum number of points the tree is allowed to store. The default value is Inf.
A new GPTree object. Tree-specific parameters are listed in this object. The field nodes contains a hash with all GPNodes and information related to nodes. The nodes in turn contain the local GPs. Nodes that have been split no longer contain a GP.
set.seed(42)
## Use the 1d toy data set from Higdon (2002)
X <- as.matrix(sample(seq(0, 10, length.out = 31)))
y <- sin(2 * pi * X / 10) + 0.2 * sin(2 * pi * X / 2.5)
y_variance <- rep(0.1**2, 31)
## Initialize a tree with Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE,
## and default parameters otherwise
gptree <- GPTree$new(Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE)
## For the purpose of this example, we simulate the data stream through a simple for loop.
## In actual applications, the input stream comes from e.g. a differential evolutionary scanner.
## We follow the procedure in the associated paper, thus letting the tree make a prediction
## first before we update the tree with the point.
for (i in 1:nrow(X)) {
y_pred_with_err = gptree$joint_prediction(X[i,], return_std = TRUE)
## Update the tree with the true (X,y) pair
gptree$update(X[i,], y[i], y_variance[i])
}
## In the following, we go over different initializations of the tree
## 1. The same tree as before, but using the package mlegp:
## Note: since the default for gp_control is gp_control = list(covtype = "matern3_2"),
## we set gp_control to an empty list when using mlegp.
gptree <- GPTree$new(Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE,
wrapper = "mlegp", gp_control = list())
## 2. Minimum working example:
gptree <- GPTree$new()
## 3. Fully specified example corresponding to the default settings
## Here, we choose to specify x_dim and min_ranges so that they correspond to the default values.
## If we do not specifiy them here, they will be automatically specified once
## the update or predict method is called.
gptree <- GPTree$new(Nbar = 1000, retrain_buffer_length = 1000,
gradual_split = TRUE, theta = 0, wrapper = "DiceKriging",
gp_control = list(covtype = "matern3_2"),
split_direction_criterion = "max_spread_per_lengthscale", split_position_criterion = "mean",
shape_decay = "linear", use_empirical_error = TRUE,
use_reference_gp = FALSE, min_abs_y_err = 0, min_rel_y_err = 100 * .Machine$double.eps,
min_abs_node_pred_err = 0, min_rel_node_pred_err = 100 * .Machine$double.eps,
prob_min_theta = 0.01, add_buffer_in_prediction = FALSE, x_dim = ncol(X),
min_ranges = rep(0.0, ncol(X)), max_cond_num = NULL, max_points = Inf)
add_node()
Add a new GPNode to the tree. IS EXPECTED TO NOT BE CALLED BY THE USER
GPTree$add_node(key)
keyKey of the new leaf
get_marginal_point_prob()
Marginal probability for point x to belong to node with given key. IS EXPECTED TO NOT BE CALLED BY THE USER
GPTree$get_marginal_point_prob(x, key)
xSingle input data point from the data stream; has to be a vector with length equal to x_dim
keyKey of the node
Returns the marginal probability for point x to belong to node with given key
update()
Assigns the given input point x with target variable y and associated variance y_var to a node and updates the tree accordingly
GPTree$update(x, y, y_var = 0, retrain_node = TRUE)
xMost recent single input data point from the data stream; has to be a vector with length equal to x_dim
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
retrain_nodeIf TRUE, the GP node will be retrained after the point is added.
The methods takes care of both updating an existing node and splitting the parent node into two child nodes. It ensures that the each child node has at least n_points_train_limit in each GP. Further handling of duplicate points is also done here.
get_data_split_table()
Generates a table used to distribute data points from a node to two child nodes
GPTree$get_data_split_table(current_node)
current_nodeThe GPNode whose data should be distributed
A matrix object
joint_prediction()
Compute the joint prediction from all relevant leaves for an input point x
GPTree$joint_prediction(x, return_std = TRUE)
xSingle data point for which the predicted joint mean (and standard deviation) is computed; has to be a vector with length equal to x_dim
return_stdIf TRUE, the standard error of the prediction is returned
We follow Eqs. (5) and (6) in this paper
The prediction (and its standard error) for input point x from this tree
clone()
The objects of this class are cloneable with this method.
GPTree$clone(deep = FALSE)
deepWhether to make a deep clone.
## ------------------------------------------------ ## Method `GPTree$new` ## ------------------------------------------------ set.seed(42) ## Use the 1d toy data set from Higdon (2002) X <- as.matrix(sample(seq(0, 10, length.out = 31))) y <- sin(2 * pi * X / 10) + 0.2 * sin(2 * pi * X / 2.5) y_variance <- rep(0.1**2, 31) ## Initialize a tree with Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE, ## and default parameters otherwise gptree <- GPTree$new(Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE) ## For the purpose of this example, we simulate the data stream through a simple for loop. ## In actual applications, the input stream comes from e.g. a differential evolutionary scanner. ## We follow the procedure in the associated paper, thus letting the tree make a prediction ## first before we update the tree with the point. for (i in 1:nrow(X)) { y_pred_with_err = gptree$joint_prediction(X[i,], return_std = TRUE) ## Update the tree with the true (X,y) pair gptree$update(X[i,], y[i], y_variance[i]) } ## In the following, we go over different initializations of the tree ## 1. The same tree as before, but using the package mlegp: ## Note: since the default for gp_control is gp_control = list(covtype = "matern3_2"), ## we set gp_control to an empty list when using mlegp. gptree <- GPTree$new(Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE, wrapper = "mlegp", gp_control = list()) ## 2. Minimum working example: gptree <- GPTree$new() ## 3. Fully specified example corresponding to the default settings ## Here, we choose to specify x_dim and min_ranges so that they correspond to the default values. ## If we do not specifiy them here, they will be automatically specified once ## the update or predict method is called. gptree <- GPTree$new(Nbar = 1000, retrain_buffer_length = 1000, gradual_split = TRUE, theta = 0, wrapper = "DiceKriging", gp_control = list(covtype = "matern3_2"), split_direction_criterion = "max_spread_per_lengthscale", split_position_criterion = "mean", shape_decay = "linear", use_empirical_error = TRUE, use_reference_gp = FALSE, min_abs_y_err = 0, min_rel_y_err = 100 * .Machine$double.eps, min_abs_node_pred_err = 0, min_rel_node_pred_err = 100 * .Machine$double.eps, prob_min_theta = 0.01, add_buffer_in_prediction = FALSE, x_dim = ncol(X), min_ranges = rep(0.0, ncol(X)), max_cond_num = NULL, max_points = Inf)## ------------------------------------------------ ## Method `GPTree$new` ## ------------------------------------------------ set.seed(42) ## Use the 1d toy data set from Higdon (2002) X <- as.matrix(sample(seq(0, 10, length.out = 31))) y <- sin(2 * pi * X / 10) + 0.2 * sin(2 * pi * X / 2.5) y_variance <- rep(0.1**2, 31) ## Initialize a tree with Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE, ## and default parameters otherwise gptree <- GPTree$new(Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE) ## For the purpose of this example, we simulate the data stream through a simple for loop. ## In actual applications, the input stream comes from e.g. a differential evolutionary scanner. ## We follow the procedure in the associated paper, thus letting the tree make a prediction ## first before we update the tree with the point. for (i in 1:nrow(X)) { y_pred_with_err = gptree$joint_prediction(X[i,], return_std = TRUE) ## Update the tree with the true (X,y) pair gptree$update(X[i,], y[i], y_variance[i]) } ## In the following, we go over different initializations of the tree ## 1. The same tree as before, but using the package mlegp: ## Note: since the default for gp_control is gp_control = list(covtype = "matern3_2"), ## we set gp_control to an empty list when using mlegp. gptree <- GPTree$new(Nbar = 15, retrain_buffer_length = 15, use_empirical_error = FALSE, wrapper = "mlegp", gp_control = list()) ## 2. Minimum working example: gptree <- GPTree$new() ## 3. Fully specified example corresponding to the default settings ## Here, we choose to specify x_dim and min_ranges so that they correspond to the default values. ## If we do not specifiy them here, they will be automatically specified once ## the update or predict method is called. gptree <- GPTree$new(Nbar = 1000, retrain_buffer_length = 1000, gradual_split = TRUE, theta = 0, wrapper = "DiceKriging", gp_control = list(covtype = "matern3_2"), split_direction_criterion = "max_spread_per_lengthscale", split_position_criterion = "mean", shape_decay = "linear", use_empirical_error = TRUE, use_reference_gp = FALSE, min_abs_y_err = 0, min_rel_y_err = 100 * .Machine$double.eps, min_abs_node_pred_err = 0, min_rel_node_pred_err = 100 * .Machine$double.eps, prob_min_theta = 0.01, add_buffer_in_prediction = FALSE, x_dim = ncol(X), min_ranges = rep(0.0, ncol(X)), max_cond_num = NULL, max_points = Inf)
Contains the GP created by DiceKriging::km from the DiceKriging package
gpThe DiceKriging GP object (DiceKriging::km in the DiceKriging manual)
X_bufferBuffer matrix to collect x points until first GP can be trained
y_bufferBuffer vector to collect y points until first GP can be trained
y_var_bufferBuffer vector to collect variance of y points until first GP can be trained
add_y_varSmall additional variance used to keep the covariance matrix condition number under control
n_points_train_limitNumber of points needed before we can create the GP
n_pointsThe number of collected points belonging to this GP
x_dimDimensionality of input points
gp_controlA list of GP implementation-specific options, passed directly to the wrapped GP implementation
init_covparsThe initial covariance parameters when training the DiceKriging GP object in self@gp
estimate_covparsIf TRUE, the parameters are estimated by the package. Otherwise, the parameters from init_covpars are taken
retrain_buffer_lengthOnly retrain after this many new points have been added to the buffer
retrain_buffer_counterCounter for the number of new points added since last retraining
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated.
X_sharedMatrix with x points that this GP shares with the GP in the sibling node
y_sharedVector of y points that this GP shares with the GP in the sibling node
y_var_sharedVector of y_var points that this GP shares with the GP in the sibling node
n_shared_pointsThe number of own points shared with the GP in the sibling node
new()
Create a new WrappedDiceKrigingGP object
WrappedDiceKrigingGP$new( X, y, y_var, gp_control, init_covpars, retrain_buffer_length, add_buffer_in_prediction, estimate_covpars = TRUE, X_shared = NULL, y_shared = NULL, y_var_shared = NULL )
XInput data matrix with x_dim columns and at maximum Nbar rows. Is used to create the first iteration of the local GP.
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
gp_controlA list of GP implementation-specific options, passed directly to the wrapped GP implementation
init_covparsInitial covariance parameters of the local GP
retrain_buffer_lengthOnly retrain when the number of buffer points or collected points exceeds this value
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated.
estimate_covparsIf TRUE, the parameters are estimated by the package. Otherwise, the parameters from init_covpars are taken
X_sharedMatrix with x points that this GP shares with the GP in the sibling node
y_sharedVector of y points that this GP shares with the GP in the sibling node
y_var_sharedVector of y_var points that this GP shares with the GP in the sibling node
A new WrappedDiceKrigingGP object. Besides the local GP, information on the shared points and those stored in the buffer are collected. For more information on the GP, consult the method DiceKriging::km in the DiceKriging package.
update_init_covpars()
Stores the initial covariance parameters (length-scales, standard deviation and trend coefficients) of the GP in the field init_covpars
WrappedDiceKrigingGP$update_init_covpars()
get_lengthscales()
Retrieves the length-scales of the kernel of the local GP
WrappedDiceKrigingGP$get_lengthscales()
get_X_data()
Retrieves the design matrix X
WrappedDiceKrigingGP$get_X_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_y_data()
Retrieves the response
WrappedDiceKrigingGP$get_y_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_y_var_data()
Retrieves the individual variances from the response
WrappedDiceKrigingGP$get_y_var_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_cov_mat()
Retrieves the covariance matrix
WrappedDiceKrigingGP$get_cov_mat()
the covariance matrix
update_add_y_var()
Method for updating add_y_var based on a bound for the covariance matrix condition number, based on this paper, Section 5.4
WrappedDiceKrigingGP$update_add_y_var(max_cond_num)
max_cond_numMax allowed condition number
store_point()
Stores a new point into the respective buffer method
WrappedDiceKrigingGP$store_point( x, y, y_var, shared = FALSE, remove_shared = TRUE )
xSingle input data point from the data stream; has to be a vector or row matrix with length equal to x_dim
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
sharedIf TRUE, this point is shared between this GP and its sibling GP
remove_sharedIf TRUE, the last of the shared points is removed
delete_buffers()
Method for clearing the buffers
WrappedDiceKrigingGP$delete_buffers()
train()
Method for (re)creating / (re)training the GP
WrappedDiceKrigingGP$train(do_buffer_check = TRUE)
do_buffer_checkIf TRUE, only train the GP if the number of stored points is larger than retrain_buffer_length
TRUE if training was performed, otherwise FALSE
predict()
Method for prediction
WrappedDiceKrigingGP$predict(x, return_std = TRUE)
xSingle data point for which the predicted mean (and standard deviation) is computed; has to be a vector or row matrix with length equal to x_dim
return_stdIf TRUE, the standard error is returned in addition to the prediction
Prediction for input point x
delete_gp()
Method to delete the GP object in self$gp
WrappedDiceKrigingGP$delete_gp()
create_DiceKriging_gp()
Method for calling the 'km' function in DiceKriging to create a GP object, stored in self$gp
WrappedDiceKrigingGP$create_DiceKriging_gp(X, y, y_var)
XInput data matrix with x_dim columns and at maximum Nbar rows for the local GP.
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
TRUE
call_DiceKriging_predict()
Method for calling the 'predict' function in DiceKriging
WrappedDiceKrigingGP$call_DiceKriging_predict(x, use_gp = NULL)
xSingle data point for which the predicted mean (and standard deviation) is computed; has to be a vector with length equal to x_dim
use_gpoptional user-defined GP which is evaluated instead of the local GP
The predictions for x from the specified GP, by default the local GP
clone()
The objects of this class are cloneable with this method.
WrappedDiceKrigingGP$clone(deep = FALSE)
deepWhether to make a deep clone.
Contains the GP created by a user-defined GP package
This is effectively a dummy wrapper based on the wrapper for the mlegp package (see WrappedmlegpGP). It contains a basic implementation of the wrapper. The vignette offers a tutorial on how to change this wrapper for the new GP package.
gpThe mlegp GP object (mlegp::mlegp in the mlegp manual)
X_bufferBuffer matrix to collect x points until first GP can be trained
y_bufferBuffer vector to collect y points until first GP can be trained
y_var_bufferBuffer vector to collect variance of y points until first GP can be trained
add_y_varSmall additional variance used to keep the covariance matrix condition number under control
n_points_train_limitNumber of points needed before we can create the GP
n_pointsThe number of collected points belonging to this GP
x_dimDimensionality of input points
gp_controlA list of GP implementation-specific options, passed directly to the wrapped GP implementation
init_covparsThe initial covariance parameters when training the mlegp GP object in self@gp
estimate_covparsIf TRUE, the parameters are estimated by the package. Otherwise, the parameters from init_covpars are taken
retrain_buffer_lengthOnly retrain after this many new points have been added to the buffer
retrain_buffer_counterCounter for the number of new points added since last retraining
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated.
X_sharedMatrix with x points that this GP shares with the GP in the sibling node
y_sharedVector of y points that this GP shares with the GP in the sibling node
y_var_sharedVector of y_var points that this GP shares with the GP in the sibling node
n_shared_pointsThe number of own points shared with the GP in the sibling node
new()
Create a new WrappedmlegpGP object
WrappedGP$new( X, y, y_var, gp_control, init_covpars, retrain_buffer_length, add_buffer_in_prediction, estimate_covpars = TRUE, X_shared = NULL, y_shared = NULL, y_var_shared = NULL )
XInput data matrix with x_dim columns and at maximum Nbar rows. Is used to create the first iteration of the local GP.
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
gp_controlA list of GP implementation-specific options, passed directly to the wrapped GP implementation
init_covparsInitial covariance parameters of the local GP
retrain_buffer_lengthOnly retrain when the number of buffer points or collected points exceeds this value
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated.
estimate_covparsIf TRUE, the parameters are estimated by the package. Otherwise, the parameters from init_covpars are taken
X_sharedMatrix with x points that this GP shares with the GP in the sibling node
y_sharedVector of y points that this GP shares with the GP in the sibling node
y_var_sharedVector of y_var points that this GP shares with the GP in the sibling node
A new WrappedGP object. Besides the local GP, information on the shared points and those stored in the buffer are collected. For more information on the GP, consult the respective met in the GP package.
update_init_covpars()
Stores the initial covariance parameters (length-scales, standard deviation and trend coefficients) of the GP in the field init_covpars
WrappedGP$update_init_covpars()
get_lengthscales()
Retrieves the length-scales of the kernel of the local GP
WrappedGP$get_lengthscales()
get_X_data()
Retrieves the design matrix X
WrappedGP$get_X_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_y_data()
Retrieves the response
WrappedGP$get_y_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_y_var_data()
Retrieves the individual variances from the response
WrappedGP$get_y_var_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_cov_mat()
Retrieves the covariance matrix
WrappedGP$get_cov_mat()
the covariance matrix
update_add_y_var()
Method for updating add_y_var based on a bound for the covariance matrix condition number, based on this paper, Section 5.4
WrappedGP$update_add_y_var(max_cond_num)
max_cond_numMax allowed condition number
store_point()
Stores a new point into the respective buffer method
WrappedGP$store_point(x, y, y_var, shared = FALSE, remove_shared = TRUE)
xSingle input data point from the data stream; has to be a vector or row matrix with length equal to x_dim
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
sharedIf TRUE, this point is shared between this GP and its sibling GP
remove_sharedIf TRUE, the last of the shared points is removed
delete_buffers()
Method for clearing the buffers
WrappedGP$delete_buffers()
delete_gp()
Method to delete the GP object in self$gp
WrappedGP$delete_gp()
call_create_gp()
Method for calling the 'mlegp' function in mlegp to create a GP object, stored in self$gp
WrappedGP$call_create_gp(X, y, y_var)
XInput data matrix with x_dim columns and at maximum Nbar rows for the local GP.
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
TRUE
call_predict()
Method for calling the 'predict' function in mlegp
WrappedGP$call_predict(x, use_gp = NULL)
xSingle data point for which the predicted mean (and standard deviation) is computed; has to be a vector with length equal to x_dim
use_gpOptional user-defined GP which is evaluated instead of the local GP
The predictions for x from the specified GP, by default the local GP. The output needs to be a list with fields mean and sd for the prediction and prediction error, respectively.
train()
Method for (re)creating / (re)training the GP
WrappedGP$train(do_buffer_check = TRUE)
do_buffer_checkIf TRUE, only train the GP if the number of stored points is larger than retrain_buffer_length
TRUE if training was performed, otherwise FALSE
predict()
Method for prediction
WrappedGP$predict(x, return_std = TRUE)
xSingle data point for which the predicted mean (and standard deviation) is computed; has to be a vector or row matrix with length equal to x_dim
return_stdIf TRUE, the standard error is returned in addition to the prediction
Prediction for input point x
clone()
The objects of this class are cloneable with this method.
WrappedGP$clone(deep = FALSE)
deepWhether to make a deep clone.
Contains the GP created by mlegp::mlegp from the mlegp package
This package is by default not able to include individual uncertainties for input points. For this reason, all fields related to y_var are not used when updating the GP. No covariance kernel can be specified either. This implementation also assumes a vector for y (and not a matrix with multiple columns). Moreover, since no parameters can be specified for the GP, we will only update the GP parameters due to internal dependencies, but not use init_covpars.
gpThe mlegp GP object (mlegp::mlegp in the mlegp manual)
X_bufferBuffer matrix to collect x points until first GP can be trained
y_bufferBuffer vector to collect y points until first GP can be trained
y_var_bufferBuffer vector to collect variance of y points until first GP can be trained
add_y_varSmall additional variance used to keep the covariance matrix condition number under control
n_points_train_limitNumber of points needed before we can create the GP
n_pointsThe number of collected points belonging to this GP
x_dimDimensionality of input points
gp_controlA list of GP implementation-specific options, passed directly to the wrapped GP implementation
init_covparsThe initial covariance parameters when training the mlegp GP object in self@gp
estimate_covparsIf TRUE, the parameters are estimated by the package. Otherwise, the parameters from init_covpars are taken
retrain_buffer_lengthOnly retrain after this many new points have been added to the buffer
retrain_buffer_counterCounter for the number of new points added since last retraining
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated.
X_sharedMatrix with x points that this GP shares with the GP in the sibling node
y_sharedVector of y points that this GP shares with the GP in the sibling node
y_var_sharedVector of y_var points that this GP shares with the GP in the sibling node
n_shared_pointsThe number of own points shared with the GP in the sibling node
new()
Create a new WrappedmlegpGP object
WrappedmlegpGP$new( X, y, y_var, gp_control, init_covpars, retrain_buffer_length, add_buffer_in_prediction, estimate_covpars = TRUE, X_shared = NULL, y_shared = NULL, y_var_shared = NULL )
XInput data matrix with x_dim columns and at maximum Nbar rows. Is used to create the first iteration of the local GP.
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
gp_controlA list of GP implementation-specific options, passed directly to the wrapped GP implementation
init_covparsInitial covariance parameters of the local GP
retrain_buffer_lengthOnly retrain when the number of buffer points or collected points exceeds this value
add_buffer_in_predictionIf TRUE, points in the data buffers are added to the GP before prediction. They are added into a temporarily created GP which contains the not yet included points. The GP in the node is not yet updated.
estimate_covparsIf TRUE, the parameters are estimated by the package. Otherwise, the parameters from init_covpars are taken
X_sharedMatrix with x points that this GP shares with the GP in the sibling node
y_sharedVector of y points that this GP shares with the GP in the sibling node
y_var_sharedVector of y_var points that this GP shares with the GP in the sibling node
A new WrappedmlegpGP object. Besides the local GP, information on the shared points and those stored in the buffer are collected. For more information on the GP, consult the method mlegp::mlegp in the mlegp package.
update_init_covpars()
Stores the initial covariance parameters (length-scales, standard deviation and trend coefficients) of the GP in the field init_covpars
WrappedmlegpGP$update_init_covpars()
get_lengthscales()
Retrieves the length-scales of the kernel of the local GP
WrappedmlegpGP$get_lengthscales()
get_X_data()
Retrieves the design matrix X
WrappedmlegpGP$get_X_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_y_data()
Retrieves the response
WrappedmlegpGP$get_y_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_y_var_data()
Retrieves the individual variances from the response
WrappedmlegpGP$get_y_var_data(include_shared = FALSE)
include_sharedIf TRUE, shared points between this GP and its sibling GP are included
get_cov_mat()
Retrieves the covariance matrix
WrappedmlegpGP$get_cov_mat()
the covariance matrix
update_add_y_var()
Method for updating add_y_var based on a bound for the covariance matrix condition number, based on this paper, Section 5.4
WrappedmlegpGP$update_add_y_var(max_cond_num)
max_cond_numMax allowed condition number
store_point()
Stores a new point into the respective buffer method
WrappedmlegpGP$store_point(x, y, y_var, shared = FALSE, remove_shared = TRUE)
xSingle input data point from the data stream; has to be a vector or row matrix with length equal to x_dim
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
sharedIf TRUE, this point is shared between this GP and its sibling GP
remove_sharedIf TRUE, the last of the shared points is removed
delete_buffers()
Method for clearing the buffers
WrappedmlegpGP$delete_buffers()
train()
Method for (re)creating / (re)training the GP
WrappedmlegpGP$train(do_buffer_check = TRUE)
do_buffer_checkIf TRUE, only train the GP if the number of stored points is larger than retrain_buffer_length
TRUE if training was performed, otherwise FALSE
predict()
Method for prediction
WrappedmlegpGP$predict(x, return_std = TRUE)
xSingle data point for which the predicted mean (and standard deviation) is computed; has to be a vector or row matrix with length equal to x_dim
return_stdIf TRUE, the standard error is returned in addition to the prediction
Prediction for input point x
delete_gp()
Method to delete the GP object in self$gp
WrappedmlegpGP$delete_gp()
create_mlegp_gp()
Method for calling the 'mlegp' function in mlegp to create a GP object, stored in self$gp
WrappedmlegpGP$create_mlegp_gp(X, y, y_var)
XInput data matrix with x_dim columns and at maximum Nbar rows for the local GP.
yValue of target variable at input point x; has to be a one-dimensional matrix or a vector; any further columns will be ignored
y_varVariance of the target variable; has to be a one-dimensional matrix or vector
TRUE
call_mlegp_predict()
Method for calling the 'predict' function in mlegp
WrappedmlegpGP$call_mlegp_predict(x, use_gp = NULL)
xSingle data point for which the predicted mean (and standard deviation) is computed; has to be a vector with length equal to x_dim
use_gpOptional user-defined GP which is evaluated instead of the local GP
The predictions for x from the specified GP, by default the local GP. The output needs to be a list with fields mean and sd for the prediction and prediction error, respectively.
clone()
The objects of this class are cloneable with this method.
WrappedmlegpGP$clone(deep = FALSE)
deepWhether to make a deep clone.