ikpls.numpy_ikpls

This file contains the PLS Class which implements partial least-squares regression using Improved Kernel PLS by Dayal and MacGregor: https://doi.org/10.1002/(SICI)1099-128X(199701)11:1%3C73::AID-CEM435%3E3.0.CO;2-%23

The PLS class subclasses scikit-learn’s BaseEstimator to ensure compatibility with e.g. scikit-learn’s cross_validate. It is written using NumPy.

This file also contains the _R_Y_Mapping class which is a concrete Mapping to store the PLS weights matrix to compute scores U directly from original Y for different numbers of components.

Author: Ole-Christian Galbo Engstrøm E-mail: ocge@foss.dk

Classes

PLS(algorithm, center_X, center_Y, scale_X, ...)

Implements partial least-squares regression using Improved Kernel PLS by Dayal and MacGregor: https://doi.org/10.1002/(SICI)1099-128X(199701)11:1%3C73::AID-CEM435%3E3.0.CO;2-%23

class ikpls.numpy_ikpls.PLS(algorithm: int = 1, center_X: bool = True, center_Y: bool = True, scale_X: bool = True, scale_Y: bool = True, ddof: int = 1, copy: bool = True, dtype: type[floating] = <class 'numpy.float64'>)

Bases: BaseEstimator

Implements partial least-squares regression using Improved Kernel PLS by Dayal and MacGregor: https://doi.org/10.1002/(SICI)1099-128X(199701)11:1%3C73::AID-CEM435%3E3.0.CO;2-%23

Parameters:

algorithm (int, default=1) – Whether to use Improved Kernel PLS Algorithm #1 or #2.
center_X (bool, default=True) – Whether to center X before fitting by subtracting its row of column-wise means from each row.
center_Y (bool, default=True) – Whether to center Y before fitting by subtracting its row of column-wise means from each row.
scale_X (bool, default=True) – Whether to scale X before fitting by dividing each row with the row of X’s column-wise standard deviations.
scale_Y (bool, default=True) – Whether to scale Y before fitting by dividing each row with the row of Y’s column-wise standard deviations.
ddof (int, default=1) – The delta degrees of freedom to use when computing the sample standard deviation. A value of 0 corresponds to the biased estimate of the sample standard deviation, while a value of 1 corresponds to Bessel’s correction for the sample standard deviation.
copy (bool, default=True) – Whether to copy X and Y in before potentially applying centering and scaling. If True, then the data is copied before fitting. If False, and dtype matches the type of X and Y, then centering and scaling is done inplace, modifying both arrays.
dtype (type[np.floating], default=numpy.float64) – The float datatype to use in computation of the PLS algorithm. This should be numpy.float32 or numpy.float64. Using a lower precision than float64 will yield significantly worse results when using an increasing number of components due to propagation of numerical errors.

Raises:

ValueError – If algorithm is not 1 or 2.

Notes

Any centering and scaling is undone before returning predictions to ensure that predictions are on the original scale. If both centering and scaling are True, then the data is first centered and then scaled.

fit(X: ArrayLike, Y: ArrayLike, A: int, weights: ArrayLike | None = None) → Self

Fits Improved Kernel PLS Algorithm #1 on X and Y using A components.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.

A

Number of components in the PLS model.

Type:: int

max_stable_components

Maximum number of components that can be used without the residual going below machine epsilon.

Type:: int

B

PLS regression coefficients tensor.

Type:: Array of shape (A, K, M)

W

PLS weights matrix for X.

Type:: Array of shape (K, A)

P

PLS loadings matrix for X.

Type:: Array of shape (K, A)

Q

PLS Loadings matrix for Y.

Type:: Array of shape (M, A)

R

PLS weights matrix to compute scores T directly from original X.

Type:: Array of shape (K, A)

R_Y

Mapping from number of components to PLS weights matrix to compute scores U directly from original Y. Keys range from 1 to A. Values are arrays of shape (M, n_components) where n_components is the key. Values are computed lazily and cached upon first access. See Notes for more information.

Type:: Mapping[int, Array]

T

PLS scores matrix of X. Only assigned for Improved Kernel PLS Algorithm #1. IMPORTANT: If weights are provided, these are NOT the scores of X but instead weighted scores. In this case, scores can be computerd using transform.

Type:: Array of shape (N, A)

X_mean

Mean of X. If centering is not performed, this is None. If weights are used, then this is the weighted mean.

Type:: Array of shape (1, K) or None

Y_mean

Mean of Y. If centering is not performed, this is None. If weights are used, then this is the weighted mean.

Type:: Array of shape (1, M) or None

X_std

Sample standard deviation of X. If scaling is not performed, this is None. If weights are used, then this is the weighted standard deviation.

Type:: Array of shape (1, K) or None

Y_std

Sample standard deviation of Y. If scaling is not performed, this is None. If weights are used, then this is the weighted standard deviation.

Type:: Array of shape (1, M) or None

Returns:

self – Fitted model.

Return type:

PLS

Raises:

ValueError – If weights are provided and not all weights are non-negative.
ValueError – If weights are provided, and scale_X or scale_Y is True, and the number of non-zero weights is not greater than ddof.

Warns:

UserWarning. – If at any point during iteration over the number of components A, the residual goes below machine precision for np.float64.

See also

transform: Transforms X and Y to their respective scores.
fit_transform: Fits the model and returns the scores of X and Y.

Notes

R_Y is provided for convenience only as it is not required to derive B. Therefore, every value in R_Y is computed lazily and only actually evaluated when accessed by its key for the first time after a call to fit - either by the user or because it is needed by transform. After a value is computed, it is cached for fast future retrieval. R_Y is implemented as a concrete Mapping.

predict(X: ArrayLike, n_components: int | None = None) → ndarray[tuple[Any, ...], dtype[floating]]

Predicts on X with B using n_components components. If n_components is None, then predictions are returned for each individual number of components.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
n_components (int or None, optional, default=None.) – Number of components in the PLS model. If None, then each individual number of components is used.

Returns:

Y_pred – If n_components is an int, then an array of shape (N, M) with the predictions for that specific number of components is used. If n_components is None, returns a prediction for each number of components up to A.

Return type:

Array of shape (N, M) or (A, N, M)

Raises:

NotFittedError – If the model has not been fitted before calling predict().

Transforms X and Y to their respective scores using n_components components. If n_components is None, then scores for all components up to A are returned.

Parameters:

X (Array of shape (N, K) or None, optional, default=None) – Predictor variables.
Y (Array of shape (N, M) or None, optional, default=None) – Response variables.
n_components (int or None, optional, default=None.) – Number of components in the PLS model. If None, then scores for all components up to A are returned.

Returns:

T (Array of shape (N, n_components) or (N, A) or None) – X scores of X. If n_components is an int, then the scores up to n_components is used. If n_components is None, returns scores for all components up to A.
U (Array of shape (N, n_components) or (N, A) or None) – Y scores of Y. If n_components is an int, then the scores up to n_components is used. If n_components is None, returns scores for all components up to A.

Raises:

NotFittedError – If the model has not been fitted before calling transform().

See also

fit_transform: Fits the model and returns the scores of X and Y.
inverse_transform: Reconstructs X and Y from their respective scores.

Notes

If multiple calls to transform are made with Y provided and a previously seen value of n_components, then self.R_Y[n_components] is only computed once and stored for future use.

Any centering and scaling of X`and `Y is carried out before computation of the scores and is undone afterwards.

fit_transform(X: ArrayLike, Y: ArrayLike, A: int, weights: ArrayLike | None = None) → Tuple[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]]

Fits Improved Kernel PLS Algorithm #1 on X and Y using A components and returns T and U which are the scores of X and Y, respectively.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.

Returns:

T (Array of shape (N, A)) – PLS scores matrix of X.
U (Array of shape (N, A)) – PLS scores matrix of Y.

See also

transform: Transforms X and Y to their respective scores.
inverse_transform: Reconstructs X and Y from their respective scores.

inverse_transform(T: ArrayLike | None = None, U: ArrayLike | None = None) → ArrayLike | Tuple[ArrayLike, ArrayLike] | None

Reconstructs X and Y from their respective scores.

Parameters:

T (Array of shape (N, n_X_components) or (N, A) or None, optional, default=None) – Scores of predictor variables (X).
U (Array of shape (N, n_Y_components) or (N, A) or None, optional, default=None) – Scores of response variables (Y).

Returns:

X_reconstructed (Array of shape (N, K)) – If T is not None, returns the reconstructed X.
Y_reconstructed (Array of shape (N, M)) – If U is not None, returns the reconstructed Y.

Raises:

NotFittedError – If the model has not been fitted before calling inverse_transform().

See also

transform: Transforms X and Y to their respective scores.
fit_transform: Fits the model and returns the scores of X and Y.

cross_validate(X: ArrayLike, Y: ArrayLike, A: int, folds: Iterable[Hashable], metric_function: Callable[[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]], Any] | Callable[[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]], Any], preprocessing_function: None | Callable[[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]], Tuple[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]]] | Callable[[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]], Tuple[ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]], ndarray[tuple[Any, ...], dtype[floating]]]] = None, weights: ArrayLike | None = None, n_jobs: int = -1, verbose: int = 10) → dict[Hashable, Any]

Performs cross-validation for the Partial Least-Squares (PLS) model on given data. preprocessing_function will be applied before any potential centering and scaling as determined by self.center_X, self.center_Y, self.scale_X, and self.scale_Y. Any such potential centering and scaling is applied for each split using training set statistics to avoid data leakage from the validation set. If weights are provided, then these will be provided in preprocessing_function and metric_function and used for any centering and scaling in the cross-validation procedure.

Parameters:

X (Array of shape (N, K)) – Predictor variables.
Y (Array of shape (N, M) or (N,)) – Response variables.
A (int) – Number of components in the PLS model.
folds (Iterable of Hashable with N elements) – An iterable defining cross-validation splits. Each unique value in folds corresponds to a different fold.
metric_function (Callable receiving arrays Y_val (N_val, M), Y_pred (A, N_val, M), and, if weights is not None, also, weights_val (N_val,), and returning Any.) – Computes a metric based on true values Y_val and predicted values Y_pred. Y_pred contains a prediction for all A components.
preprocessing_function (Callable or None, optional, default=None,) – A function that preprocesses the training and validation data for each fold. It should return preprocessed arrays for X_train, Y_train, X_val, and Y_val. If Callable, it should receive arrays X_train, Y_train, X_val, Y_val, and, if weights is not None, also weights_train, and weights_val, and returning a Tuple of preprocessed X_train, Y_train, X_val, and Y_val.
weights (Array of shape (N,) or None, optional, default=None) – Weights for each observation. If None, then all observations are weighted equally.
n_jobs (int, optional default=-1) – Number of parallel jobs to use. A value of -1 will use the minimum of all available cores and the number of unique values in folds.
verbose (int, optional default=10) – Controls verbosity of parallel jobs.

Returns:

metrics – A dictionary mapping each unique value in folds to the result of evaluating metric_function on the validation set corresponding to that value.

Return type:

dict of Hashable to Any

Raises:

ValueError – If weights are provided and not all weights are non-negative.

Notes

This method is used to perform cross-validation on the PLS model with different data splits and evaluate its performance using user-defined metrics.