534004_en-US

liliconcom

C64 Case Study Chemometric Analysis of Spectral Data

Chemometric Analysis of Spectroscopic Data

 

Overview

This case study will demonstrate the analytic workflow for analyzing spectral data using both JMP and JMP Pro platforms.  The use of spectroscopic analysis techniques for characterizing raw materials, intermediates, and finished products is widespread in all of Chemistry.  From college chemistry labs to Research & Innovation labs, as well as the production environment. Spectral analysis is virtually everywhere.  The hardest part of spectral analysis is not the collection of the spectra, but the analysis of the data the comes with the spectra.  Over the last few years JMP has been working towards built-in multivariate spectral analysis techniques also known as chemometric analysis. 

Wiki definition for chemometrics:

“Chemometrics is the science of extracting information from chemical systems by data-driven means. Chemometrics is inherently interdisciplinary, using methods frequently employed in core data-analytic disciplines such as multivariate statistics, applied mathematics, and computer science, in order to address problems in chemistry, biochemistry, medicine, biology and chemical engineering.

Chemometrics is applied to solve both descriptive and predictive problems in experimental natural sciences, especially in chemistry. In descriptive applications, properties of chemical systems are modeled with the intent of learning the underlying relationships and structure of the system (i.e., model understanding and identification). In predictive applications, properties of chemical systems are modeled with the intent of predicting new properties or behavior of interest. In both cases, the datasets can be small but are often very large and highly complex, involving hundreds to thousands of variables, and hundreds to thousands of cases or observations.”

The JMP Analytic Workflow for the Analysis of Spectral Data

1. Getting spectral data into JMP is the first challenge.  There is an add-in for pulling in *.csv and *.jdx files.  The .jdx files must be in the standard format and not compressed.  Alternately Multiple File Import can be used but will require initial set up.
2. Spectral data can be analyzed in both the stacked format and the wide format.  Use the stacked format (by row) with Graph Builder when first reviewing the data and for some initial pre-processing.  Parallel Plots in Graph Builder work well for wide format data. 
   1. When using parallel plots drag the wavelength data to the x-axis then right click, hover over Combine Scales, and use Parallel Merged to get an idea of differences in baseline shift.

 

 

3. Pre-processing of the data may be needed. 
   1. The first step is to use stacked wavelength data to calculate the Standard Normal Variate (SNV).  SNV can be found in the add-in. 
   2. Another pre-processing option is using the Savitzy-Golay smoother/derivative filter.  You can take the first or the second derivative.  The first derivative cleans up additive variations and the second derivative cleans up both additive and multiplicative data.  Taking the derivative changes the structure of the data and can make interpretation harder. 
   3. Sometimes you will need to do both Savitzy-Golay and SNV.  Do Savitzy-Golay first and then do SNV.  *Soon you will be able to do all pre-processing within the FDE platform in JMP Pro 17. 
4. The use of many multivariate techniques including MDMVCC, PCA, Multidimensional Scaling and clustering can and should be used to show multicollinearity and sub-grouping.
5. Modeling in regular JMP:
   1. Stepwise Regression in regular JMP has been shown to work nicely with Near Infrared data and is a good way to find important wavelengths (variables)
   2. Decision Trees work well but be mindful that it may be better to do the splits manually as opposed to letting JMP do all the splits.
   3. You can use saved Principal Components to do Principal Component Regression in the Fit Model platform.  It is more difficult to determine the most important wavelengths this way, but it is possible with Assess Variable Importance using the Prediction Profiler.
   4. Partial Least Squares (PLS) the most prolific of all chemometric modeling tools.  PLS is built to handle the highly correlated and typically very wide data sets related to spectral and chromatographic data.  PLS can be used for variable selection to determine which variables can be attributed to the overall best fit via PLS.  That said, if the data is well conditioned through pre-processing, then there is typically no need to use PLS for variable selection unless you are looking at a mixture of component constituents.
6. Moving to JMP Pro for modeling:
   1. Partial Least Squares (PLS) – See above, PLS is also available in regular JMP.
   2. Generalized Regression (Gen Reg) – penalized regression models are good for variable selection and because there can be hundreds if not thousands of columns per spectra/chromatogram Elastic Net works very well to determine the most important variables (wavelengths/peaks).  It is also advised to scan the range of Elastic Net Alpha possibilities to avoid missing important variables.  For example, the default Elastic Net Alpha is 0.99.  If you go with the default, you will likely get a good model, but you will also very likely miss out on some very important variables that will improve your predictive capability.  If you do not have time to scan the range of Elastic Net Alpha you can pick something in the mid-range (0.56) to see if that gives a reasonable model.
   3. Functional Data Explorer (FDE) – FDE is not necessarily a predictive modeling platform as much as it can be used to analyze data on continuum and determine which and how many functional Principal Components are related to the shape of a set of curves.  The use of Functional DOE using supplemental variables makes FDE that much powerful and you can get an inverse calibration using FDOE. 
   4. Neural Nets, Bootstrap Forest, Support Vector Machines, XG Boost, can all be used in the analysis of chemometric data.  With JMP Pro you can use Model Screening to determine which is the best modeling platform to use for your data.

Best PracticesCase StudiesRe: C- 41 Case Study Chemometric Analysis of Spectral Data

Chemometrics is on the way to becoming a trend in Pharma and HLS. Please find below some LinkedIn articles of mine on the topic espousing JMP

FDA’s NIR Guidance for Industry | LinkedIn

Emerging Analytical Techniques in QbD – PAT Paradigm | LinkedIn

JMP® - a primer to AQbD | LinkedIn