We’ve created a holistic DOE demo for a JMP day and would appreciate feedback and edits for improvement. Primary goal is to illustrate the value of JMP Pro using bootstrap forest vs JMP using imputation and stepwise selection methods for variable selection to aid design of next experiment. This builds upon a couple of demos Phil Kay developed recently. Please unzip the attached zip file and open the journal Holistic DOE Case Study. Open the Messy Production Data outline to see the flow of holistic DOE demo:
- Statistical Modelling: Mixture of EDA, Statistics and Experience
- Messy data complicates the issue and the outline Problem of Correlated Xs presents a dataset created by Phil with one Y and two highly correlated Xs showing that traditional stepwise methods select one X vs Bootstrap Forest selecting both Xs as potentially impacting the Y.
- Holistic DOE outline then builds upon the first two outlines with anonymised messy data from the banding example. It is a random sample of rows with anonymised columns of the banding example. It is a sample of just 20 rows to try and mimic the size of data that might be typical in R&D. After clicking on first data link the rest of the links give the demo flow to illustrate the extent of missing data, correlation of continous Xs, then compare X variables selected by stepwise regression, stepwise regression with imputation, decision tree with informative missing which is also a stepwise variable selection method, and bootstrap forest with informative missing. Bootstrap forest with informative missing gives a larger list of potential Xs that may impact Y and therefore gives greater assurance that we do not miss an important X. At this point it may be worth summarising AB Agri story (without assigning problem/solution to AB Agri): A leading developer and producer of animal feedstuff wasted a R&D cycle by missing an important X when selecting from a set of correlated Xs by using traditional stepwise selection methods. This meant they had to repeat a R&D cycle at considerable expense, were six months late to market which resulted in lost sales and reduced market opportunity. They now avoid repeating this problem by using Bootstrap forest and related variable selection methods that identify all potential Xs from a set of correlated Xs irrespective of whether or not some of your data cells are missing.
- DSD sub-outline then performs a DSD on the six variables selected by bootstrap forest and via stepwise selection methods (which is now valid due to zero or near zero correlation structure of the model effects) results in a model involving 5 of the 6 factors. If we had used stepwise variable selection methods (with or without missing imputation) we would have missed 4 of the 5 important Xs and would have required further cycles of development adding to the time to market, cost of development and reducing the opportunity to sell product early when often a premium can be charged.
It would be great to get feedback for improving the demo wrt brevity, simplicity, compellingness …
Malcolm
