nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Principal component regression

ehchandlerjr

Hello - I have been working with and using PCA in JMP for about four years now. Nothing fancy, but I feel very comfortable with a lot of the functionality. I've been using the PCs as factors for DoE, thinking I was some kinda smartie and came up with a new use case for PCA. Low and behold, I stumble across a Wikipedia page last week on principal component regression. Ha.

Anyway, I have looked and looked, and don't see any reference, much less functionality, in JMP for PCR (or integration with DoE for that matter). Am I missing something, and does this functionality exist in JMP, even if only in JSL form? If not, does anyone have experience with it? Would you just use the inverted correlation matrix on the PCs, basically the inverse of the PC formula column?

Thanks!

Accepted answers

All comments

Thierry_S

Hi,

I dabbled with the same idea and was told that was nonsense on this board. Still, there may be a labor-intensive way to get what you want.

Calculate the Principal Components from your data set (Multivariate Methods > Principal Components)
In the report, go to Save Columns > Save Principal Components Values. The number of PCs you save depends on your data, and I tend to use PCs that capture at least 80% of the data variation
Then, treat the PCs as any variables in your model (e.g., Response Screening, Fit Model,..)

Note: This approach is directly inspired by the workstream in R as explained on this page (LINK)

Best,

ehchandlerjr

Thanks for the reply! So what is people's beef with PCR? Is it a fundamental statistical issue, or is it something else? Like i have a paper in using that takes 700 solvents across 100 properties. Seems highly unlikely there are 100 truly independent, fundamental aspects to the solvent space, and PCR seems like a reasonable way to reduce it to a manageable size. I'm sure there are many other places too for zeolites, minerals, etc. And that's just chemical.

ehchandlerjr

I guess I'm also asking what's the best way to invert the regression model that you get back into terms of the original column space? I guess full disclosure: I'm using these PCs as factors in an experimental design. Do you make the model in terms of the PCs and apply some inverse correlation matrix to it or do you just then do some sort of fit to all of the factors and see which fit best to get a more physically based model?

Victor_G

Could a Partial Least Squares regression be a more simple solution ?

Dan_Obermiller

I'm not sure that people have "beef" with PCR, but it has some limitations. Ultimately, you must be certain about what you want from your modeling effort.

PCR is a biased regression technique. This is not necessarily bad, but your model would be predictive only. It is not intended to provide any insights into what CAUSES things to happen like a designed experiment would. It is truly just built on correlations, with no claims on causation.

When you perform PCA on the X's as @Thierry_S suggests, only keeping the first few principal components is only considering the X's, not the target or response variable, Y. So you may not end up with the best model. The last principal component might have the best predictive ability of the target, so you should keep ALL principal components to build your PCR model. Once you fit a model with all PCs, you can then remove the insignificant ones to get a final model.

Your last post gets at the major issue with PCR: how do you translate back to the original X space. There are several possible approaches. One approach is to keep the factor that has the highest loading from each of the significant PCs. A second approach is to remove the factor that has the lowest loading from each of the significant PCs. And there are other possibilities. Which approach you choose will depend on what you want from a model.