How to correctly import data points that has their own uncertainty and subsequently conduct statistical test?

Question

Normally one treats your data points as if they are exact (which is kind of untrue to my honest opinion). But if I have an extra step of multiple measurements per sample, it would create an uncertainty of estimation of the measured value associated with that sample. in fact, a lot of quantities are generated via an estimation process which has an associated uncertainty.

On top of that, I have different experimental treatments and each treatment has multiple replicates/samples (so there is another level of uncertainty). My end goal is to compare the treatment averages by statistical tests. If I want to account for the uncertainty in the estimation of my individual data points, as well as the uncertainty of treatment replicate variability, how do I go about it right from the start (from data importing)? I dont mind manually calculating the total propagated SE and type it in a column, but I dont know how to incorporate this information in an analysis.

I failed to find anything useful on the Internet, not just on JMP. Your help is very appreciated.

ricardor · Answer

@Mark_Bailey Thanks for your reply. I know in my case (subsampling) mentioned above, it can be analyzed with a nested model. However, this becomes unsuitable as the estimation process is deviated from simply averaging the subsamples' values (which I have to deal with). I am not talking about using exact individual values as response variables, which is of course, easy to do.

As you said, a first step modeling exercise of original measured data points should be done but that modeling exercise only produced a statistical summary i.e. it does not save the estimates and its associated uncertainty as a variable, all you can do is tabulate them. (And the require model could be very niche- and mechanistic-specific that takes on something other than a statistical modeling approach provided by statistical softwares). Now, this is the question: I now then want to do a second statistical analysis based on the estimates generated by the prior modeling exercise, which I found hard to find an easy approach.

Mark_Bailey · Answer

Please see JMP > Help > JMP Document Library. The two books, "Essential Analysis" and "Fitting Linear Models" should provide a good source of information with explanations and examples. All the examples are installed with JMP.

Mark_Bailey · Answer

JMP and other mature statistical software deals with this problem through modeling the individual observations. In other words, we do not analyze a summary of the observations (e.g., mean response) and try to include a separate estimate of the uncertainty (e.g., standard deviation or standard error) but instead include the original individual observations (e.g., replicates). The exact nature of the 'accounting' of this uncertainty depends on the 'experimental units' and the chosen modeling technique, but in general it can be done.

So in a simple case of 'one-way analysis of variance,' the individual responses are entered in the Y role and the treatment levels are entered in the X role. The sums of squares estimate the 'within' and the 'between' treatment variability to be compared using the F ratio with a hypothesis tests or multiple comparison procedures. (This one-way ANOVA is not the only possible analysis but just to serve as an example.)