Sample size & Statistical power

Question

Hi,

I have done a finite element model validation study by comparing the a certain quantity of interest to experimental data. I have conducted this study for 18 different samples in order to evaluate the mean prediction error of the model. The real & simulated samples are dependent.

I want to assess if 18 was a statistically significant number of samples.

How can i calculate the power?

Thanks for your help

MRB3855 · Answer

Hi @TraceMatrices73 : Excellent, that is something we can work with! To make sure we are all on the same page…for a one-way random effect ANOVA, the ICC is defined as follows.

Let V(b) = Variance between groups/classes/whatever.

Let V(w) = Variance within groups/classes/whatever.

Let V(t) = Total Variance = V(b) + V(w).

Then ICC = V(b)/V(t).

And your hypotheses are then:

H0: ICC <= 0.9

Ha: ICC > 0.9

Are we on the same page? If so, what are the groups/classes? I.e., what is the categorical factor/effect in the one-way ANOVA?

TraceMatrices73 · Answer

Hi MRB,

Thank you for your message.

What i want to prove is that the predictions obtained by the model correlate well with experimental results. For example, this could be demonstrated with an intraclass correlation coefficient greater than 0.9. Alternative hypothesis is that predictions by the model do not correlate well with experimental results.

MRB3855 · Answer

Hi @TraceMatrices73 . For me, some clarity is needed here. You say power > 0.8 is what you are after. 
Power = Probability(reject H0, given Ha is true). What are the null and alternative hypotheses, H0 and Ha, respectively? Practically speaking, Ha is what you want to “prove”, and H0 is opposite of Ha.