Hello
My name is Hiroaki Ono, and I work as a tester in the JMP Japan division of SAS Institute Japan Co., Ltd.
The Cochran-Mantel-Haazel test for comparing mean scores between groups can be performed in JMP using "Bivariate Relationships."
However, please note the following:
(1) Even if you only have one layer, you need to add a layer variable.
(2) If you want to compare average scores, you need to specify in advance in the data what scores you want to give.
By default, the scores are numbered 1, 2, 3, 4, 5, ...
(3) Both the X and Y variables should be set as categorical variables (nominal or ordinal scale).
Incidentally, I personally believe that the CMH test explained below is not intended to confirm "the absence of age bias" (which I interpret as meaning that "even if age is not included in the analysis, there will be no confounding bias in the relationship between the groups and the outcome variable").
(1) In general, in many statistical tests, “non-significance” simply means “there is not enough evidence to reject the null hypothesis.” In other words, “absence of evidence is not evidence of absence.”
(2) Even if a result is “significant,” it may only have been significant in this one instance by chance.
(3) Even if a statistical test can "prove" that there is a relationship between "age" and "group", this does not necessarily mean that "age" is a confounding factor (in the sense of satisfying the DAG backdoor criterion).
Personally, I think that whether "age" is a confounding factor should be decided by talking to experts in the field, not by a test comparing groups (I think Hernán and Robins' What If: Causal Inference also takes the position that confounding factors should not be decided by tests). Furthermore, I think that you should talk to experts in the field (ideally before looking at the data) to decide whether there are any other confounding factors besides age that need to be considered. However, I think there are various ways of thinking about it.
Here I would like to explain using the attached pseudo data "Example data.jmp" (because the copyright of the attached data was unclear). The operation is quite complicated, so if you have any questions, please reply again.
Assume that the purpose of the test is to perform a Cochran-nMantel-Hazel test to compare ordinal-scale responses (a, i, u, e, o, ka) between two groups (A and
. The "Frequency" column contains the frequency of each combination. We will explain the case where the scores are numbered consecutively and the case where they are average ranks.
(As mentioned in (c) at the end, the CMH test using mean ranks as scores is equivalent to a Wilcoxon test. If you already have data on a continuous scale and want to use mean ranks as scores, it is easier to perform a Wilcoxon test than the CMH test using the following procedure. Also, if the data is large, the results will be similar to a two-sample t-test using those scores.)
■ When using consecutive numbers as the score
If you want to assign consecutive numbers (1, 2, 3, 4, 5, 6) to the responses (a, i, u, e, o, ka) and compare the average scores between groups, you would follow the steps below.
(a-1) Add a “layer” column to the data where all values are the same.
(a-2) Select [Analyze] → [Bivariate Relationships].
And assign the following roles to each column:
Set the Y variable to [Y, response variable]. In this case, the Y variable must be on a nominal or ordinal scale.
X variables (variables for group 2) are [X, explanatory variables]
Click [Block] to select the "Layer" column created in (a-1) above.
If you have a frequency column, select it as [Frequency]
Once you have finished configuring all the settings, click OK.
(a-3) The "Cochran-Mantel-Haenszel test" is output at the bottom of the report. Among them, "Score comparison between X" is the Cochran-nMantel-Haenszel test, which compares the mean scores between groups.
■ When the average ranking is used as the score
If you would like to give the responses (a, i, u, e, o, ka) a score other than a sequential number, you will need to actually give them the scores yourself before running the data.
For example, if you want to use the average rank as the score, the steps would be as follows:
(b-1) If the Y variable is a string, select [Columns] → [Recode] to convert it to numbers with the serial numbers 1, 2, 3, 4, 5, 6, ....
In this case, after creating a new column with Recode, right-click on the recoded column, select Column Info, and set Data Type to Numeric and Scale to Continuous.
(b-2) Select [Analyze] → [Univariate Distribution]. Assign the following roles to each column:
The recoded numeric column created in (b-1) above is [Y, column]
If there is a frequency column, select it as Frequency
Once you have finished setting up all the options, click [OK].
(b-3) Click the red triangle button on the column name outline above the histogram,
Select [Save] → [Average Ranks]. This will save the average ranks to the data table.
(b-4) Right-click on the column of average ranks, select "Column Information", and change "Scale" to "Ordinal scale".
After this, follow the same steps as (a-1) to (a-3) in "When using serial numbers as scores" above. However, for (a-2) [Y, Objective variable], specify the average ranking columns created in (b-2) and (b-3).
■The Cochran-nMantel-Haazel test, which compares mean scores between two groups (hereinafter referred to as the ``mean score CMH test''), is related to several tests known by other names.
(c-1) When the sample size is large, the "score mean CMH test" produces results similar to those of the two-sample t-test on the scores. In particular, the results are asymptotically the same.
(c-2) The "Score Mean CMH Test" produces results similar to those of the Cochran-Armitage Test. Since the Cochran-Armitage Test is equivalent to the score test of logistic regression for ordinal responses, when the sample size is large, the "Score Mean CMH Test" produces results similar to those of ordinal logistic regression.
(c-3) When using mean ranks as scores, the "Score Mean CMH Test" produces the same results as tests known as the "Wilcoxon Test", "Mann-Whiteney Test", "Wilcoxon-Mann-Whiteney Test", etc. (When a Wilcoxon test is performed in JMP, results are output for cases with and without 1/2 continuity correction. These results are for the "Kruskal-Wallis Test (Chi-square approximation)" without 1/2 continuity correction. JMP's CMH test does not perform 1/2 continuity correction.)
I remember that these relationships (c-1) to (c-3) are written in a textbook somewhere, but I can't find it at the moment. If you are interested, please contact me and I will try to find a textbook that contains the explanations.
