Data analysis when much more columns than rowsHello everybody
I would like to ask for your help because I have a data set with:
- in columns, process parameters of a galvanized steel production line
- in rows, coils that have been processed on this production line
The objective is to correlate a quantity of defects appearing on a coil to one or several process parameters.
Unfortunately, I have a data set with many more columns (407) than rows (63).
Do you have any advice on how to proceed?
AnalyticsRe: Data analysis when much more columns than rowsHello @stephanegeorges,
There are a lot of options, depending if you are on JMP or JMP Pro.
Before going to analysis directly, I would visualize the correlations between process parameters and between process parameters and response (defaults), as this could give an indication about correlated variables and influential inputs for the analysis of defaults (platform "Multivariate" in Analyze -> Multivariate Methods).
Then, some options to consider :
- JMP :
- A first approach could be to do a Principal Component Analysis (PCA), and then try to model the default response with these PCA parameters. But this approach will make you lose some explainability about which factors impact the defaults, as they will be combined in PCA variables.
- Another option would be to try Partial Least Squares (PLS), a very efficient approach when you deal with a lot of correlated variables and very few observations. It is for example used in spectral analysis, where you have few observations but a lot of input data (transmittance/intensity for each wavelength for example).
- What may also be interesting is to try Predictor Screening to have an overview about which factors are the most important.
- JMP Pro :
- Instead of Predictor screening, you can try "Random Forest", where you'll have a lot more infos on the model and possibility to save formula. Since RF uses features bagging for each tree, you can be confident that even in case of multicollinearity and more parameters than observations, each parameter will have the same probability to be selected in a tree, so you'll have a good view on which factors impact the defaults.
- You can also look at Generalized Regression models with penalized estimation methods like Lasso, Ridge or Elastic Net.
These are the models/techniques I would try first before trying more complex methods.
I let experts in this community complete my answers 
I hope this will help you,
