Determine significance of difference between percent differences

Question

I have four sets of data from two experiments: one set of "before" measurements and one set of "after" measurements for each experiment. For each experiment, I have calculated a percent difference between the mean of "before" data and the mean of "after" data. I would like to know if the difference between the percent difference calculated for Experiment 1 and the percent difference calculated for Experiment 2 is significant. Can this be done in JMP? If so, can anyone indicate how this might be done?

For context/example, let's say that the percent difference between the means of before and after data in Experiment 1 is 30% and the percent difference between the means of before and after data in Experiment 2 is 33%. My sense is that 33% difference is not significantly different from 30% difference, but is this even a valid statistical question? I have tried searching Google for an answer to this and have come up short, but a colleague recommended JMP for this type of analysis.

gzmorgan0 · Answer

Mark Bailey is absolutely correct, you should work with the observations' data and not just an aggregated summary statistic.

The attached file is a journal file with notes of paired analyses using the JMP Sample Data table Dogs.jmp. Since I did this in JMP 14, I am also attaching a .pdf file, in case you have an older version of JMP and it will not open.

Read the notes. Hopefuly this will give you some ideas on which JMP graphs and analyses to use.

Good luck.

Dogs_Example.pdf

Dogs_Example.jrn

Mark_Bailey · Answer

You are using some terminology in an ambiguous way but that's OK. Without seeing your design or your data, I can only give a few thoughts.

Generally, it is better to work with the raw data instead of the reduced data. For example, analyze the measurements, not the percent change. For example, analyze the measurements, not their means.

The analysis of such data usually involves a linear model. If you have one factor, you can use Analyze > Fit Y by X. If you have more than one factor, then use Analyze > Fit Model.

See Help > Books > Design of Experiments for examples of experiments and analysis. Also, see Help > Books > Basic Analysis or Fitting Linear Models for more details on the regression analysis.