"Very wide" PLS data set?<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />Does anyone have a "very wide" dataset that would lend itself to PLS modelling?
A customer has a spectral dataset with 3500 columns (wavelengths), 70 rows, 1 response to predict. They tried to build a PLS model in JMP and it was extremely slow (10 minutes). I would like to investigate but customer cannot give me the data (even anonymised!). Does anyone have a dataset of comparable size? All my stock examples are a lot smaller.
Please let me know if you have any ideas regarding the speed of fitting. The customer's other software fitted the model in << 1 minute. Would Pro be any different? Are there any options that would help?
Thanks,
Phil
Re: "Very wide" PLS data set?Thanks, Scott.
That is one of the datasets that I looked at. I agree it is a good one. It just wasn't as big as the customer's data set though as it has hundreds of columns rather than thousands.
Thanks,
Phil
Re: "Very wide" PLS data set?It makes sense as we are now asking JMP to fit models for all those different numbers of factors over all the 7 folds in the data.
Re: "Very wide" PLS data set?Phil and Peter,
Just and FYI:
Another good wide data set for PLS is the Gasoline Octane NIR data set.
It was one of the first we used for PLS back in the early JMP days, but still is very good and representative of a common laboratory wide data situation.
Just put it out under the JMP Technical Reference Library under the Data Table folders: D50 Gasoline Octane NIR (Gardner)
Haven't tried out the JMP Pro PLS features on this one yet (such as validation and the newer anlaysis options), but think it would work as well.
Scott
Re: "Very wide" PLS data set?Yes, the validation technique plays a huge role in response time. Just for giggles I chose KFold = 7 rather than None and it took about 14 minutes to resolve.
Re: "Very wide" PLS data set?Hi Peter,
Great suggestion for that data set, thanks.
Having played about (I used the "age" column as my Y) I can also fit a 15 factor model in about the same time. Although, I find that with only 8 GB of RAM in my machine (must talk to IT about that!) I can't use all 11,000 covariates. About half of them is fine though.
Whether or not you use validation has a big effect. If I ask for LOO validation everything grinds to halt. I think this is what the customer was doing in JMP. And possibly the model the customer fit in their other software did not use validation.
This has given me some useful avenues to explore.
Thanks again,
Phil
Re: "Very wide" PLS data set?OK...I just created a simple numeric response that was one of the covariates x 3 so a straight line relationship so there was SOMETHING to find...used all of the almost 11,000 covariates, JMP Pro, my machine, None for validation...and I think this one thing is the key...using the PLS personality in Fit Model, no standardize X, and it took about 15 seconds to find a 15 factor solution.
Re: "Very wide" PLS data set?Phil:
One dataset that comes immediately to mind is the Micro Array data set that is central to our JMP Pro version 12 demos that has almost 11,000 numeric covariate columns, 4 categorical responses. It takes maybe 4 seconds for that to run on my Windows machine for one response. I've never tried multiple responses. In fact I'm going to run a PLS oriented Mastering Event October 15 demonstrating this specific data set with only one response just to keep it simple. A bit of a different data set and it runs in the Discriminant platform, not the PLS platform but this might help?
Here's the specs on my Windows laptop if this helps:
