Thoughts on Measurement System Evaluation - Destructive Testing

Peter_Bartell

Pretty much every time I deliver the "Evaluating and Monitoring Your Process Using..." Mastering JMP event I get asked the question..."What should I do when the object being measured is destroyed during the measurement process...making repeat measurements of the same object impossible?" Some SEs have also sought my thoughts on this issue privately as well. Frankly the literature isn't much help beyond, "Try to find parts/objects that you are at least willing to assume are identical and then treat those measurements as the repeat measurements." During my Kodak tenure I was never too happy with this idea...so here's how we sometimes handled the situation:

Feel free to add your thoughts as well.

 

 

Byron_JMP

Peter brings up some good points, and I'm pretty sure we should start a completely separate to talk about measuring consistency in anything manufactured in rolls. Paper, films, fabrics (woven and non-woven) and metals are produced by a number of our customers  and they way they try to measure consistency across the width and length of the rolls is fairly inconsistent but the problems of autocorrelated samples, repeatability, and sample sizes are common to all.

Back to Peter's question: I see a little different version of the same question.  In the biopharm world, there is usually the ability to take a large enough representative sample that multiple tests can be performed. So, while each test consumes material, there is generally enough material to repeat the test. On in the case of a filled product, there is a sample of vials from a lot, and there is an expectation that the vial to vial difference is much smaller than the test to test variation and a lot of this is due to the complexity of the test methods (reference 1 for example.)This first issue generally revolves around the reproducibility of the test method (precision). Large molecules in a complex matrix (dirty background, not particularly pure) are often tough to measure quantitatively, especially when the quantitation is of the biologically active material in the sample.  In order to deal with precision issues test methods are carefully validated to characterize the accuracy, precision, linearity and range of the method. (see references below for definitions)  After validation, once we have a good grip on the intermediate precision of the assay, we can start to talk about the assay format, or in statistical terms how to take advantage of the central limits theorem to increase the accuracy and precision of the reportable result. The general idea is to do multiple runs of multiple tests (replicates) to reduce the variation the reported test result.

Reference 1

http://www.bioprocessintl.com/upstream-processing/assays/assessing-similarity-with-parallel-line-and-parallel-curve-mode…

Regulatory and Guidance documents about test method validation

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Quality/Q2_R1/Step4/Q2_R1__Guideline.pdf​

http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm386366.pdf

http://www.fda.gov/ucm/groups/fdagov-public/@fdagov-drugs-gen/documents/document/ucm368107.pdf

Useful, review-like documents

https://www.agilent.com/cs/library/primers/Public/5990-5140EN.pdf

http://www.biopharminternational.com/assay-development-and-method-validation-essentials​

The big book of assay development

http://www.ncbi.nlm.nih.gov/books/NBK53196/pdf/Bookshelf_NBK53196.pdf