Multiset sparse partial least squares path modeling

Multiset sparse Partial Least Squares path modeling (msPLS) is a multivariate approach for modeling the relationship between Q related data sources (X₁,...,X_q,...,X_Q), with the help of latent variables (LVs). Each data source contains p_q number of manifest variables (MVs), measured on the same n samples (i.e. Xq∈ℝn×pq), each data source is assigned to its corresponding LV (ζ₁,...,ζ_q,...,ζ_Q). The LVs are linear combinations of their MVs (ζq=Xqwq, where ζq∈ℝn×1 and wq∈ℝpq×1). The relationship between the data sources is encoded in a connectivity matrix, like in Partial Least Squares path modeling (PLS-PM), and modelled through a multiple regression model between the LVs;

where ∑m=1Mqζm→q denotes the sum of M_q LVs that are explanatory for ζ_q, θ_qm is the coefficient capturing the effect of the mth ζ_m→q on ζ_q, and v_q is white noise, following the notation of [22, 24] for PLS-PM. A full description for the PLS-PM algorithm can be found in [24] (Algorithm 6). The weight vectors w_q are estimated as

or as

depending on the mode of the regression. PLS-PM denotes Eq. (2) as Mode A and Eq. (3) as Mode B regression. For msPLS, Mode A (i.e. multiple univariate regression) is used for the weight vectors of MVs that do not have any response MVs, and Mode B (i.e. multivariate regression) is used for the weight vectors of MVs that do have response MVs. The descriptions of the objective functions of PLS-PM can be found in [22, 24] and the objective function for msPLS is given by Eq. (5) in the “General case” section.

In a high dimensional setting (i.e. p_q>>n), the covariance matrix of X_q in Eq. (2) is non-invertible. To solve this problem, we propose to replace Eq. (2) with Elastic Net (ENet) penalization. Replacing the ordinary least square estimator in Eq. (2) with ENet penalisation has two advantages; not only we overcome the multicollinearity issue encountered in a high dimensional setting, but ENet also enforces sparse variable selection, which ease the interpretability of the final model. Equation (2) then becomes

where λ₁ denotes the LASSO penalty and λ₂ denotes the Ridge penalty parameters [27].