in the previous issue "JMP Measurement System Analysis (MSA) series one In , we introduced the analysis process of the entry-level traditional metrological measurement system, and when the output characteristics of the process are count data, that is, the measurement value is a limited set of classification numbers, the most common such as qualified / Unqualified, first-class product / second class / third class /… The analysis methods for such measurement systems will obviously be different. At present, for the counting measurement system, the consistency ratio and Kappa value to analyze, the focus of this article is to introduce how to apply JMP to assist in the assessment.
Among them, the consistency ratio is the most commonly used to measure the consistency of classification measurement results, and it is also the most easy-to-understand statistic. It is described by the formula:
And Kappa value ( kappa ) is another statistic that measures the consistency of classification measurement results, and the more common one is Cohen κ, which is suitable for measuring the degree of agreement when two variables have the same number of bins and bin values. Described with a formula that is:
In the formula, P 0 is the actually consistent ratio; P e Consistent with expectations ( random guess ) The ratio.
if you feel kappa If the formula is somewhat difficult to understand, you may wish to understand it qualitatively as follows. Suppose we want to evaluate two raters against 30 The consistency of the measurement results of whether a part is qualified or not, imagine that even if two graders do not strictly follow the standard operating procedure, they just give them at will. 30 A rating measurement result, then the consistency of the two can also be obtained to a certain extent, then, the so-called kappa The value is the consistent result after deducting this part of random uncertain components, so it is relatively more accurate. Here, as an applicator, we don't have to be too entangled Kappa value calculation process, because in JMP With the assistance of modern statistical analysis software, the corresponding calculation results can be directly given, and you only need to interpret and judge the results. In general, -1 ≤κ≤ 1 , when κ =1 When , it means that the two are completely consistent; when κ =-1 When , it means that the two are completely opposite; when κ =0 When , it means that the degree of consistency between the two is similar to that of blindness. In addition, in AIAG MSA In the manual, the recommended reference standards are shown in the table -1 shown.
Table-1 Kappa General Judgment Criteria for Value
Next, to avoid unnecessary disagreement, we directly quote the AIAG MSA manual ( No. 4 version, English version ) , Chapter III, Section C , Attribute Measurement Systems Study , P131-140 The case in the paper demonstrates the operation process of counting measurement system analysis and briefly analyzes the output results.
"50 a representative part, 3 raters, each component tested 3 The count data of times is summarized in the table -2 shown.
- "1 ” indicates that the part is qualified or accepted
- "0 ” indicates that the part is rejected or rejected
- The sampling details of representative parts are actually quite critical, such as 50% The left and right components should come from " grey " zone ( That is, the critical zone near the upper and lower limits of the specification ) , and the remaining parts should be clear qualified products and half non-conforming products
- The tester should carry out blind test to avoid human subjective factors affecting the authenticity of the measurement data”
Table 2 Attribute Measurement System Analysis Dataset
JMP The analysis operation steps for the counting measurement system in this example:
- First, when the source data is imported into JMP After that, the data structure can be easily converted to the graph by using the stacking platform and a little tidying -1 The form shown, and the final variable type should be set to nominal variables.
figure 1 Input and sorting of analysis data of counting measurement system
- According to Figure-2 Settings, start "Analysis > quality and process > variability / Counting Gauge Chart" platform.
figure 2 "Variability / Counting Gage Diagram" launch platform setup
- After clicking OK, the main follow-up work is to analyze the series of analysis output reports of counting measurement systems, according to AIAG MSA The guideline of the manual, which mainly includes the following aspects:
- Based on Kappa Value, to evaluate whether there is good consistency between the raters, the results are shown in the figure -3 shown. It can be seen from this that due to Kappa value all >0.75 , so there is good agreement among raters.
image 3 between raters Kappa value summary
- Based on Kappa value, to assess whether there is good agreement between the rater and the reference standard, the results are shown in Fig. -4 shown. It can be seen from this that due to Kappa value all >0.75 , so that there is also good agreement between the raters and the reference standard.
Figure 4 between different raters and reference standards Kappa value summary
- After evaluating the agreement between raters and between raters and reference standards, each rater's measurement validity, false positive rate, and false negative rate can also be assessed based on agreement ratios.
First, as shown in Figure-5 shown in AIAG MSA In the manual, this section evaluates the following 4 Aspects:
- Across all measures, the rater's own consistency
- Consistency of all measurements and reference standards for raters
- Consistency between all raters themselves and other raters
- Consistency between all raters themselves and other raters, and with reference standards
Figure-5 AIAG MSA Summary Table of Analytical Validity Studies for Handbook Attribute Measurement Systems
And in JMP , the corresponding output report is shown in the figure -6 shown. It is worth mentioning that in JMP In , the effectiveness of evaluation item II is defined as the ratio of the number of correct decisions to the total number of decision opportunities. As an example, suppose the rater A for a part 3 second rating, with 1 If one decision is incorrect (e.g. pass, pass, fail), then the other two decisions are still counted as correct. This definition of validity differs from AIAG MSA manual, because in AIAG MSA As defined in the manual, raters A All of the parts 3 All decisions will be counted as incorrect. In fact, there is no absolute right or wrong between these two definitions. Relatively speaking, calculating all rating decisions alone can provide more information about the entire measurement process. For the results of the IV evaluation items, JMP Although not directly given, it was also not found that AIAG MSA The manual sets clear reference standards for this assessment item.
Figure-6 JMP Relevant output report for analysis effectiveness research of attribute measurement system
Secondly, in terms of the false positive rate and false negative rate of different raters, the summary results are shown in Figure-7 shown. The false positive rate and false negative rate here refer to:
Figure-7 Summary Report of False Positive and False Negative Rates for Different Raters
Ultimately, at the AIAG MSA In the manual, examples of reference standards for some quantitative indicators in this evaluation link are given, as shown in Table -3 As shown, and the summary of the actual evaluation results of the raters in this link, as shown in the table -4 shown.
table 3 Examples of Rater Effectiveness, False Negative and False Negative Rate Reference Standards
Table 4 Summary of Rater Effectiveness, False Negative Rate, and False Negative Rate Assessment Results
Through the comparison, it is not difficult to see that if the current rater effectiveness, false negative rate and false positive rate reference standards are used as the basis, then no rater is in the 3 is perfectly acceptable on this dimension, and no rater is 3 It is completely unacceptable in this dimension. The problem mainly lies in the indicator evaluation of the false negative rate, which seems to be somewhat different from the previous Kappa The statistical conclusions are contradictory. Therefore, as a research group for the analysis of this measurement system, on the one hand, it can re-examine whether the current standard definition of validity, false negative rate and false negative rate is reasonable, or whether the raters need better training ( especially the rater representatives c) , the measurement system needs further improvement? On the other hand, it is also possible to communicate and discuss with internal and external customers, how do they view the current analysis results, what are their expectations and requirements, and whether the two parties can jointly bear the current measurement system risks reflected? Because of all the analysis results, the current measurement system is not unconditionally acceptable.
Also, in addition to the above with the AIAG MSA In addition to the analytical content corresponding to the manual, JMP Some other visualization charts are also given, such as Fig. -8 The gage count plot shown. In the gage count chart, you also don't have to be too entangled Y The source of the axis value can be firstly clarified in the first figure, " Consistency Percentage " The lower the value of the part, the more inconsistent the raters gave the measurement results; in the second plot, the " Consistency Percentage " The lower the value of the rater, the less consistent with other raters or self-rating results. That is to point out the direction for you to further analyze the count-type measurement system, and it can also be mutually corroborated with other analysis results.
Figure-8 JMP Gage Count Chart
The above is all the content of the analysis of counting measurement systems that we are going to introduce in this issue. I hope this article can help you learn more about JMP exist MSA specific applications in the field.
