Best Practices for Qualifying new Operators

Question

Interesting question came up yesterday during a Measurement Systems Analysis discussion with a biotech company.

Let's say you've conducted a MSA, looking specifically at different operators, and concluded that the system is working fine. What happens when new operators come on board? Should you run a new MSA study, or is there another way to qualify the new operator?

I'd be curious to hear from those with industry experience here. Any guidelines here?

Thanks,

Jordan

benfrancis · Answer

When I was working in Unilever we were looking at a measurement system for friction in hair products. They wanted to know who was the "best" operator, which prompted a lot of questions from me as to what was best.

We never fully got to a solution as the data was from different operators working on different products at different times. I suggested that they'd need a standard product that all operators should test in as similar conditions as possible to provide some form of comparison. This dataset could be analysed with quantile regression to establish the measurement range over time (or whatever longitudinal measure). Then anyone new could do the test with the product and see how they sat in the quantiles.

These were my musings, not sure how helpful they are...!

DonMcCormack · Answer

If they're following the ISO standard, then they'll include operator as one of the sources of reproducibility. In practice, there are never enough operators to get a decent estimate of operator variability (typically 10+). Ultimately, it gets down to whether they consider operator a fixed or random effect. If fixed, they will technically have to rerun the study. If random, all depends on how well they've quantified V(operator).