How to determine the Length t-ratio corresponding to p-value of 0.05 from screening design?

Question

Hi,

I am a new user of JMP12.

I am having a hard time to interpret the results of a screening design (please see the attached screenshot). I know JMP makes this split-plot by showing the Lenth t-ratio of each effect as the length of each bar. Also, the blue vertical line indicates the effects with individual p-value of 0.1.

Now, I was wondering if there is any way to know the Length t-ratio corresponding to an individual p-value of 0.05 from this result.

Thanks :)

Mark_Bailey · Answer

Yes, the Screening platform uses a two-sided t test with Lenth's PSE for the denominator.

Yes, bootstrapping sample statistics was established in the 1970s as a valid non-parametric approach to inference.

No, you cannot use the parametric t distribution to obtain the p-values for the Lenth t ratio. That is the whole point of using the Monte Carlo approach. It would be much easier to use the distribution model but the p values would not be accurate.

Sham · Answer

Thanks again!

Just for clarification, JMP by default uses the two-tailed t-test, right?

So, for example, if my screening design matrix has a degree of freedom 15, can I use the simulated t-ratios from JMP to manually calculate the p-values from a standard two-tailed t-distribution table? Will my calculated p-value will match with that from the JMP? I understand my calculated p-values might not be exactly the same, but is this a valid approach based on the algorithm JMP uses?

Mark_Bailey · Answer

The standard approach in bootstrapping is to sort the simulated statistics, then find the position of your sample statistic in the list. If 90% of the statistics were less than your estimate, the p-value is 0.1.

That is exactly the same as finding the p-value for a sample t statistic with a parametric t-distribution.