Partial Least Squares Generalized Linear Regression

Question

Are there any plans to extend the PLS model to what I see called PLS-GLM.  In this model, the response can be categorical.  The model is a hybrid of PLS and a regularization method that allows the response to remain truly, say, binary.  That is, the response is not recoded as 0/1 and then PLS is applied as if it were numeric.  PLS-GLM has maximum likelihood as the objective function instead of maximum covariance.  Presently I am just reading the literature so I have little more to offer yet.

gene · Answer

OK, so here is where I landed on this issue.  I coded what is called the PLS-GLM algorithm from the paper "PLS Generalized Linear Regression", Bastien, Vinzi and Tenenhaus, Computational Statistics and Data Analysis 48 (2005).  In this paper they present an algorithmn that uses maximum likelihood to solve for the set of orthogonal PLC components.  I compared my results from PLS-DA in JMP Pro against those from PLS-GLM.  They are not exactly the same, no surprise since the math is not the same.  However the models they produce are very close.  So close that I come away with the belief that coding the response as continuous is probably OK.  At least I can say that I did the purist approach, allowing the response to retain its native distribution, and the differences between the two methods was hardly noticeable.

It would still be pretty cool to have PLS-GLM added to the analysis suite.

gene · Answer

Thanks Dan,

This is exactly what I do at the moment.  However I have been told that JMP codes the binary response as 0/1 and does regular PLS and treats the response as numeric.  In the paper PLS Generalized Linear Regression, Bastien, Vinzi, Tenenhaus in Computational Statistics and Data Analysis 2005, tey discuss how to do PLS with a binary response and make use of the link function.  Instead of maximizing Cov(X * beta, Y) we maximize the likelihood of Y given X * beta.  There are several other papers that do this PLS-GLR.  So they make explicit use of the fact that the response is not numeric (could also be Poisson or anything from the exponential family I suppose).  The end result is still an othhogonal basis for the span of X, but is is derived without forcing the response vector to reside in an iner product space with X *beta.

To me, It comes down to the question of whether PLS-GLM gives better results than PLS-DA.  I don't know the answer, but it is more difficult for me to defend the PLS-DA as my modeling technique when I know I'm ignoring the native distribution of the response.  It's sort of like fitting binary response data using OLS, which we know is wrong.

Dan_Obermiller · Answer

I think you are looking at what is called PLS-DA (PLS - Discriminant Analysis). Use Fit Model, put a nominal target in as the Y and your predictors into the model. From the Standard Least Squares drop-down, choose Partial Least Squares.