ikpls.jax_ikpls_base

This file contains an abstract class for partial least-squares regression using Improved Kernel PLS by Dayal and MacGregor.

Implementations of concrete classes exist for both Improved Kernel PLS Algorithm #1 and Improved Kernel PLS Algorithm #2.

For more details, refer to the paper: “Improved Kernel Partial Least Squares Regression” by Dayal and MacGregor.

This file also contains the _R_Y_Mapping class which is a concrete Mapping to store the PLS weights matrix to compute scores U directly from original Y for different numbers of components.

Author: Ole-Christian Galbo Engstrøm E-mail: ocge@foss.dk

Classes

PLSBase(center_X, center_Y, scale_X, ...)

Implements an abstract class for partial least-squares regression using Improved Kernel PLS by Dayal and MacGregor: https://doi.org/10.1002/(SICI)1099-128X(199701)11:1%3C73::AID-CEM435%3E3.0.CO;2-%23

class ikpls.jax_ikpls_base.PLSBase(center_X: bool = True, center_Y: bool = True, scale_X: bool = True, scale_Y: bool = True, ddof: int = 1, copy: bool = True, dtype: str | type[Any] | dtype | SupportsDType = <class 'jax.numpy.float64'>, differentiable: bool = False, verbose: bool = False)

Bases: ABC

Implements an abstract class for partial least-squares regression using Improved Kernel PLS by Dayal and MacGregor: https://doi.org/10.1002/(SICI)1099-128X(199701)11:1%3C73::AID-CEM435%3E3.0.CO;2-%23

Implementations of concrete classes exist for both Improved Kernel PLS Algorithm #1 and Improved Kernel PLS Algorithm #2.

Parameters:

center_X (bool, default=True) – Whether to center the predictor variables (X) before fitting. If True, then the mean of the training data is subtracted from the predictor variables.
center_Y (bool, default=True) – Whether to center the response variables (Y) before fitting. If True, then the mean of the training data is subtracted from the response variables.
scale_X (bool, default=True) – Whether to scale the predictor variables (X) before fitting by dividing each row with their row of column-wise standard deviatons.
scale_Y (bool, default=True) – Whether to scale the predictor variables (Y) before fitting by dividing each row with their row of column-wise standard deviatons.
ddof (int, default=1) – The delta degrees of freedom to use when computing the sample standard deviation. A value of 0 corresponds to the biased estimate of the sample standard deviation, while a value of 1 corresponds to Bessel’s correction for the sample standard deviation.
copy (bool, default=True) – Whether to copy X and Y in fit before potentially applying centering and scaling. If True, then the data is copied before fitting. If False, and dtype matches the type of X and Y, then centering and scaling is done inplace, modifying both arrays.
dtype (DTypeLike, default=jnp.float64) – The float datatype to use in computation of the PLS algorithm. This should be jax.numpy.float32 or jax.numpy.float64. Using a lower precision than float64 will yield significantly worse results when using an increasing number of components due to propagation of numerical errors.
differentiable (bool, default=False) – Whether to make the implementation end-to-end differentiable. The differentiable version is slightly slower. Results among the two versions are identical. If this is True, fit and stateless_fit will not issue a warning if the residual goes below machine epsilon, and max_stable_components will not be set.
verbose (bool, default=False) – If True, each sub-function will print when it will be JIT compiled. This can be useful to track if recompilation is triggered due to passing inputs with different shapes.

Notes

Any centering and scaling is undone before returning predictions with fit to ensure that predictions are on the original scale. If both centering and scaling are True, then the data is first centered and then scaled.

Fits Improved Kernel PLS Algorithm #1 on X and Y using A components. Returns the internal matrices instead of storing them in the class instance.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.

Returns:

B (Array of shape (A, K, M)) – PLS regression coefficients tensor.
W (Array of shape (A, K)) – PLS weights matrix for X.
P (Array of shape (A, K)) – PLS loadings matrix for X.
Q (Array of shape (A, M)) – PLS Loadings matrix for Y.
R (Array of shape (A, K)) – PLS weights matrix to compute scores T directly from original X.
R_Y (Mapping[int, Array]) – Mapping from number of components to PLS weights matrix to compute scores U directly from original Y. Keys range from 1 to A. Values are arrays of shape (M, n_components) where n_components is the key. Values are computed lazily and cached upon first access. See Notes for more information.
T (Array of shape (A, N)) – PLS scores matrix of X. Only Returned for Improved Kernel PLS Algorithm #1.
X_mean (Array of shape (1, K) or None) – Mean of the predictor variables center_X is True, otherwise None.
Y_mean (Array of shape (1, M) or None) – Mean of the response variables center_Y is True, otherwise None.
X_std (Array of shape (1, K) or None) – Sample standard deviation of the predictor variables scale_X is True, otherwise None.
Y_std (Array of shape (1, M) or None) – Sample standard deviation of the response variables scale_Y is True, otherwise None.

Warns:

UserWarning. – If at any point during iteration over the number of components A, the residual goes below machine epsilon.

See also

fit: Performs the same operation but stores the output matrices in the class

instance

Notes

For optimization purposes, the internal representation of all matrices (except B) is transposed from the usual representation.

Fits Improved Kernel PLS Algorithm #1 on X and Y using A components.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.

A

Number of components in the PLS model.

Type:: int

max_stable_components

Maximum number of components that can be used without the residual going below machine epsilon.

Type:: int

B

PLS regression coefficients tensor.

Type:: Array of shape (A, K, M)

W

PLS weights matrix for X.

Type:: Array of shape (K, A)

P

PLS loadings matrix for X.

Type:: Array of shape (K, A)

Q

PLS Loadings matrix for Y.

Type:: Array of shape (M, A)

R

PLS weights matrix to compute scores T directly from original X.

Type:: Array of shape (K, A)

R_Y

Mapping from number of components to PLS weights matrix to compute scores U directly from original Y. Keys range from 1 to A. Values are arrays of shape (M, n_components) where n_components is the key. Values are computed lazily and cached upon first access. See Notes for more information.

Type:: Mapping[int, Array]

T

PLS scores matrix of X. Only assigned for Improved Kernel PLS Algorithm #1. IMPORTANT: If weights are provided, these are NOT the scores of X but instead weighted scores. In this case, scores can be computerd using transform.

Type:: Array of shape (N, A)

Returns:: self – Fitted model.
Return type:: PLS
Raises:: ValueError – If weights are provided and not all weights are non-negative.
Warns:: UserWarning. – If at any point during iteration over the number of components A, the residual goes below machine epsilon.

See also

stateless_fit: Performs the same operation but returns the output matrices

instead

Predicts with Improved Kernel PLS Algorithm #1 on X with B using n_components components. If n_components is None, then predictions are returned for all number of components.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
B (Array of shape (A, K, M)) – PLS regression coefficients tensor.
n_components (int or None, optional) – Number of components in the PLS model. If None, then all number of components are used.
X_mean (Array of shape (1, K) or None, optional, default=None) – Mean of the predictor variables. If None, then no mean is subtracted from X.
X_std (Array of shape (1, K) or None, optional, default=None) – Sample standard deviation of the predictor variables. If None, then no scaling is applied to X.
Y_mean (Array of shape (1, M) or None, optional, default=None) – Mean of the response variables. If None, then no mean is subtracted from Y.
Y_std (Array of shape (1, M) or None, optional, default=None) – Sample standard deviation of the response variables. If None, then no scaling is applied to Y.

Returns:

Y_pred – If n_components is an int, then an array of shape (N, M) with the predictions for that specific number of components is used. If n_components is None, returns a prediction for each number of components up to A.

Return type:

Array of shape (N, M) or (A, N, M)

See also

predict: Performs the same operation but uses the class instances of B,

None, None, None, and, arguments.

Predicts with Improved Kernel PLS Algorithm #1 on X with B using n_components components. If n_components is None, then predictions are returned for all number of components.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
n_components (int or None, optional) – Number of components in the PLS model. If None, then all number of components are used.

Returns:

Y_pred – If n_components is an int, then an array of shape (N, M) with the predictions for that specific number of components is used. If n_components is None, returns a prediction for each number of components up to A.

Return type:

Array of shape (N, M) or (A, N, M)

See also

stateless_predict: Performs the same operation but uses inputs B, X_mean,

None, None, and, instance.

Transforms X and Y to their respective scores using n_components components. If n_components is None, then scores for all components up to A are returned.

Parameters:

X (Array of shape (N, K) or None, optional, default=None) – Predictor variables.
Y (Array of shape (N, M) or None, optional, default=None) – Response variables.
n_components (int or None, optional, default=None.) – Number of components in the PLS model. If None, then scores for all components up to A are returned.
copy (bool, default=True) – Whether to copy X and Y before potentially applying centering and scaling. If True, then the data is copied beforehand. If False, and dtype matches the type of X and Y, then centering and scaling is done inplace, modifying both arrays.

Returns:

T (Array of shape (N, n_components) or (N, A) or None) – X scores of X. If n_components is an int, then the scores up to n_components is used. If n_components is None, returns scores for all components up to A.
U (Array of shape (N, n_components) or (N, A) or None) – Y scores of Y. If n_components is an int, then the scores up to n_components is used. If n_components is None, returns scores for all components up to A.

Raises:

NotFittedError – If the model has not been fitted before calling transform().

See also

fit_transform: Fits the model and returns the scores of X and Y.
inverse_transform: Reconstructs X and Y from their respective scores.

Notes

If multiple calls to transform are made with Y provided and a previously seen value of n_components, then self.R_Y[n_components] is only computed once and stored for future use.

Any centering and scaling of X`and `Y is carried out before computation of the scores and is undone afterwards.

Fits Improved Kernel PLS Algorithm #1 on X and Y using A components and returns T and U which are the scores of X and Y, respectively.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.

Returns:

T (Array of shape (N, A)) – PLS scores matrix of X.
U (Array of shape (N, A)) – PLS scores matrix of Y.

See also

transform: Transforms X and Y to their respective scores.
inverse_transform: Reconstructs X and Y from their respective scores.

Reconstructs X and Y from their respective scores.

Parameters:

T (Array of shape (N, n_X_components) or (N, A) or None, optional, default=None) – Scores of predictor variables.
U (Array of shape (N, n_Y_components) or (N, A) or None, optional, default=None) – Scores of response variables.

Returns:

X_reconstructed (Array of shape (N, K)) – If T is not None, returns the reconstructed X.
Y_reconstructed (Array of shape (N, M)) – If U is not None, returns the reconstructed Y.

Raises:

NotFittedError – If the model has not been fitted before calling inverse_transform().

See also

transform: Transforms X and Y to their respective scores.
fit_transform: Fits the model and returns the scores of X and Y.

Computes B with stateless_fit. Then computes Y_pred with stateless_predict. Y_pred is an array of shape (A, N, M). Then evaluates and returns the result of metric_function(Y_val, Y_pred).

Parameters:

X_train (Array of shape (N_train, K)) – Predictor variables.
Y_train (Array of shape (N_train, M) or (N_train,)) – Response variables.
A (int) – Number of components in the PLS model.
weights_train (Array of shape (N_train,) or None) – Weights for each observation. If None, then all observations are weighted equally.
X_val (Array of shape (N_val, K)) – Predictor variables.
Y_val (Array of shape (N_val, M) or (N_val,)) – Response variables.
weights_val (Array of shape (N_val,) or None) – Weights for each observation. If None, then all observations are weighted equally.
metric_function (Callable receiving arrays Y_val (N_val, M), Y_pred (A, N_val, M), and, if weights is not None, also, weights_val (N_val,), and returning Any.) – Computes a metric based on true values Y_val and predicted values Y_pred. Y_pred contains a prediction for all A components.

Returns:

metric_function(Y_val, Y_pred, weights_val)

Return type:

Any.

See also

stateless_fit: Fits on X_train and Y_train using A components while

optionally, instead

stateless_predict: Computes Y_pred given predictor variables X and

regression

Performs cross-validation for the Partial Least-Squares (PLS) model on given data. preprocessing_function will be applied before any potential centering and scaling as determined by self.center_X, self.center_Y, self.scale_X, and self.scale_Y. Any such potential centering and scaling is applied for each split using training set statistics to avoid data leakage from the validation set.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
folds (Array of Int of shape (N,)) – An array defining cross-validation splits. Each unique Int in folds corresponds to a different fold.
metric_function (Callable receiving arrays Y_val (N_val, M), Y_pred (A, N_val, M), and, if weights is not None, also, weights_val (N_val,), and returning Any.) – Computes a metric based on true values Y_val and predicted values Y_pred. Y_pred contains a prediction for all A components.
metric_names (list of str) – A list of names for the metrics used for evaluation.
preprocessing_function (Callable or None, optional, default=None,) – A function that preprocesses the training and validation data for each fold. It should return preprocessed arrays for X_train, Y_train, X_val, and Y_val. If Callable, it should receive arrays X_train, Y_train, X_val, Y_val, and, if weights is not None, also weights_train, and weights_val, and returning a Tuple of preprocessed X_train, Y_train, X_val, and Y_val.
show_progress (bool, default=True) – If True, displays a progress bar for the cross-validation.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.

A

Number of components in the PLS model.

Type:: int

Returns:: metrics – A dictionary containing evaluation metrics for each metric specified in metric_names. The keys are metric names, and the values are lists of metric values for each cross-validation fold.
Return type:: dict[str, Any]
Raises:: ValueError – If weights are provided and not all weights are non-negative.
Warns:: UserWarning. – If at any point during iteration over the number of components A, the residual goes below machine epsilon.

See also

_inner_cv: Performs cross-validation for a single fold and computes evaluation

metrics.

_update_metric_value_lists: Updates lists of metric values for each metric and

fold.

_finalize_metric_values: Organizes and finalizes the metric values into a

dictionary

stateless_fit_predict_eval: Fits the PLS model, makes predictions, and

evaluates

Notes

This method is used to perform cross-validation on the PLS model with different data splits and evaluate its performance using user-defined metrics.