Hi All,
I'm hoping someone from JMP might be able to provide an explanation as the the differences/similarities of the Simulate vs Bootstrap approaches when evaluating model estimates. (JMP Pro)
Some links to previous Discovery Summit presentations from Gotwalt and others can be found here, here, and here. One clear difference between the methods is that Simulate requires you to swap out a column formula with another column formula, whereas bootstrapping does not. I understand the concept of bootstrapping when recalculating the parameter estimates, but it's unclear to me why the fractional random weighted column needs to be swapped out when simulating the same data. See for example the first link above and corresponding presentation that explains this approach using an autovalidation setup (in this case they were working with DOE results).
What I'd like to better understand is the benefits/drawbacks of either approach and when is one approach better than another. I have a wide and tall data table where I have several process variables and product quality measurements. I want to narrow down the set of possible process variables that might be influencing the outcome to provide production some idea of where to investigate. To do this, I take the Null Factor idea from their presentations and include that as a truly random variable in the model. I then run a very basic GenReg model on the factors and Y output, and then right click the column of estimates and run either the bootstrap or simulate option on the parameter estimates.
I do not get the same results running both ways, and I'd like to understand why. Also, is one approach more appropriate over another since this isn't a DOE?
Examples of the results are below on a Pareto Plot, where any factors that are introduced into the model fewer times than the Null Factor are simply removed. The first image shows the results of using the Simulate method, where the factor on the far right of the Pareto plot is the Null Factor. Similarly, the second image shows the results using the Bootstrap method, where again the Null Factor is at the far right.
Why do I obtain two very different results for the two methods? Is one more "trustworthy" than the other? Why would I use one method over the other? Is one method more robust than the other? If you do get different results, why does JMP Pro have the two options?
Any insights to this are much appreciated.
Thanks!,
DS
