Application for Functional Data on Infrared Spectroscopy and Chromatography

Question

I have an inquiry about using Functional Data Explorer on infrared spectroscopy and chromatography.  Example waveforms are pasted below and I'm trying to get the customer to send me a data table.  The application is to use these as responses from DOE.  For infrared spectrography, the presence of peaks at specific wavenumbers is interesting.  For chromatography, the area under the curve at points in time (the x-axis is time) is interesting.  There was also discussion about aligning the x-axis and subtracting a basis response.

https://www2.chemistry.msu.edu/faculty/reusch/virttxtjml/spectrpy/infrared/infrared.htm

https://www.researchgate.net/figure/High-performance-liquid-chromatography-HPLC-chromatograms-of-the-flavonoids-in-CN30_fig1_281614653

Bill_Worley · Answer

Hey Martin,

Your suggestion P-Splines is something I have been playing with using FDE.  I have not tried FT as of yet, but I am also skeptical that it will do any better than a spline fit....at least for spectral data.  One thing I have found that seems to work well is to use the Rows as Functions instead of stacking the data.  I have had good results with NIR and Mass Spec data the Rows as Functions and P-Splines.  I did run into some difficulty with NMR data, but I need to revisit that data to try some new things.

Ultimately though, I use PLS or Gen Reg to fit the predictive models and I am using the scanning Elastic Net in Gen Reg to find some really good predictive models that are on par with or better than PLS models for both continuous and binomial response data.

Phil_Kay · Answer

I believe that JMP Development have recently had some good discussions with customers with data like this. So they have a good understanding of why spectral and chromatography data (and similar such as XRD) are difficult for FDE as it is now. Hopefully they will be able to find solutions that could open this up to new applications and new users of JMP Pro!

martindemel · Answer

Though already sa long time ago:

Just to add on Phil/Mike/Bill's comments: This kind of data will be difficult to fit with FDE standard knots. you either want to try fourier (though this could overswing on one side) or P-Splines where you adjust the knots closer around the peak variations. This will help to eliminate the effect of smoothing. Often this is not important but if you are interested in the different positions of the peaks (hight, auc, ...) this might become important.

From my past work in with seismographic and signal processing customers filtering the noise was important, there FDE still lacks of good filtering techniques. This would help in the data processing a lot I believe.

However, I would give it a try and see how good you can be (currently). Though I like nonlinear and curfe fit platform I'm not that convinced they will do a decent good job either