nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How do I calculate p-values for sensitivity analyses?

Hello,

I am trying to run a sensitivity analysis and wanted to know if JMP has a way of calculating p-values for this.

How it goes is I have a cohort with a mean value of μ0. I then run another analysis where I remove 30 participants from the cohort and re-calculate the mean value of this reduced cohort (μ1). My question is can I calculate a p-value that describes whether μ0 and μ1 are significantly different or not, and if so, how I go about doing that?

Thank you in advance.

Accepted answers

Thierry_S

Hi,
If I understand the fundamental question you are trying to address is whether a characteristic of the tested people has an influence on the test results. If I'm correct, your approach (i.e. subgroup analysis for "sensitivity) is questionable at best. Therefore, may I suggest that you look into a linear model that includes ALL your subjects and where the characteristics of you "certain people" would be included as a variable(s). Let us know if this makes sense and if you need help setting up the analysis (you may want to read about the Fitting Linear Models in the HELP > BOOKS.
Best,
TS

All comments

Georg

I think the t-test does excactly this. Having the two samples (the group with µ0 and µ1 in different rows marked by a feature, you can run it in the fit y by x platform (oneway analysis), and activate it via red triangle (t-test).

Another way would be to test the mean in the distribution platform. So having called it for the data of µ1 and knowing µ0, you can call it via red triangle in the distribution "test mean". In both cases you get your p-value regarding both distributions having the same mean. This assumes normally distributed data, however there are several ways to also handle different distributions.

Mark_Bailey

I'm not sure if that approach is valid. The t-test or F-test in Oneway is for two independent samples from two populations. He wants to compare a mean statistic for a sample and a sub-sample. It is not just the matter of how to set up the calculation but also the independence of the data.

What do you mean by a "sensitivity analysis?" The mean is an unbiased estimator. Is this test a question of power? Precision? Is there a precedence for it? Is there a reference about this comparison?

Apologies, I should have provided this detail in the initial question.

So I have a cohort of 200 people, and all 200 sat a test which provides each of them with a score from 0-10. The μ0 is the mean test score for these 100 people.

However, 30 of these people have a certain trait, and I want to see if removing that trait from the cohort changes the mean test score and by how much. So I remove these 30 people from the cohort, and re-calculate the mean test score for the remaining 70 people to calculate μ1.

My question is if I can calculate a p-value to assess whether the difference between μ0 and μ1 is statistically significantly or not.

Apologies there's a typo in the 5th line: "recalculate the mean test score for the remaining 170 people"