JMPers who are familiar with JMP know that JMP has always been committed to the development and application of experimental design. Looking back at the history of experimental design, from the 1920s (1920), when Fisher first proposed the statistical concept of experimental design, to today, experimental design has developed multiple statistical methods, such as response surface, Taguchi design, mixture design, and final design. Optimal design, uniform design, supersaturated experimental design, etc. Bradley Jones, chief research and development expert at JMP, also twice won the Brembo Award for individuals from the American Society for Quality for his innovative deterministic screening design.
After more than 30 years of development, the JMP experimental design platform has included all the experimental design methods mentioned above. At the same time, it also provides experimental design auxiliary functions such as experimental design evaluation, design plan comparison, and sample efficacy evaluation to help more users use experiments. Design researchers choose the most appropriate and effective solution among various experimental design methodologies.
JMP Experimental Design Platform at a Glance
Compared with the big data that is in full swing today, experimental design is suitable for small sample data. In the case of small samples, how to better select the optimal model so that it can effectively predict without overfitting?
In recent years, many methods have been applied, such as the common AICc (Akaike Information Criterion), Bayesian Inference, etc. This time we are going to introduce another brand-new method, namely: the automatic verification Fractionally Weighted Bootstrap (FWB) technology introduced in JMP Pro 16.
JMP Pro introduces automatic verification and model building for small sample data, as two new and practical and effective technologies. No new experiments are needed to DOE Data for predictive modeling . This technology forms a verification set based on the concept of original training data and Bootstrapping random sampling.
Before introducing JMP's new automatic verification FWB technology, let's briefly review the application thinking of cross-validation in big data machine learning.
First divide the data into training set and validation set, or even test set. In the process of determining the model complexity, the model when the SSE (i.e., the sum of squared differences between groups) reaches the minimum in the validation set is selected as the best model, as shown in the figure.
Model cross validation
For experimental design data or small sample data, is it possible to construct two unrelated SSEs based on a limited sample size and use one of them as a validation set? Under this assumption, the automatic verification FWB technology launched by JMP Pro 16 is aimed at small sample data.
Using Bootstrap technology, construct multiple (customizable) pairs of random coefficients with the same distribution and negative correlation for each experimental design sample:
Apply the generated sets of random coefficients to the following calculation formula:
After derivation and calculation, this method is used to obtain:
The two tend to be uncorrelated, that is, they can achieve similar effects of cross-validation. Using such a technique, JMP can build multiple sets of models of different complexity based on multiple sets of pairwise random coefficients, while recording the non-zero proportion of model variable parameter estimates.
However, new problems have arisen.
Multiple pairs of random coefficients are generated. For example, if there are 2500 pairs of random coefficients, 2500 models will be established. How to finally confirm which model variables are significant variables? How to define non-zero scale limits to help determine the final model?
Here, JMP introduces null variables that are independent and unrelated to the prediction object and randomly distributed. When building the model, this empty variable is introduced into the modeling at the same time. In the final multi-group model, if the proportion of non-zero parameter estimates of an independent variable is higher than that of the null variable, it is considered that the independent variable needs to be included in the model. The above text description may not be able to fully express it, so we use a real case to briefly explain it.
Case Studies
The goal of this case is Reduce aggregation issues during protein purification , Adjustable experimental and process parameters, including protein content, formaldehyde, etc. 6 main factors . In this actual case, the deterministic screening design under the JMP "Experimental Design" platform was applied, which contains 6 main factors, as well as interaction factors and higher-order terms, for a total of 27 factors.
After collecting the experimental results, first use the automatic verification function to randomly generate the first set of paired coefficients and null variables for each sample. Then use JMP's "Fitting Model" function to introduce null variables and model simultaneously. On this basis, 250 pairs of random coefficients were generated again to establish the model.
Finally, according to the generated non-zero proportions, only variables with non-zero proportions higher than the null variable (protein concentration, temperature, formaldehyde concentration, the interaction term of formaldehyde concentration and protein concentration, p<0.05, as shown in the figure below) are regarded as significant models items, included in the scope of the final model.
Next, let's take a detailed look at how to implement model building in JMP, and use FWB to automatically verify calculations to find significant model factors.
In this real case, through the model results, we found that only three main factors affect and positively affect the final degree of protein aggregation. Among the many interaction factors and higher-order terms, only one, the interaction term between formaldehyde and protein concentration, has a significant impact on the final degree of protein aggregation. This result greatly narrows the range of parameters that need to be optimized in the process and clarifies the important parameters in the process of process adjustment. After optimizing all parameters, the protein polymerization degree was optimized from the initial average of 14.8% to the best 2.5%, which greatly improved the safety and reliability of the protein product.
In this case, we also found that although as many as 27 factors were considered at the beginning of the experimental design, only 4 factors ultimately significantly affected the process products through FWB technology screening. This also verifies the principle of effect sparsity that we often talk about in experimental design.
original: https://mp.weixin.qq.com/s/BqiTKQcb9wJmFJweUqRrOA
