My name is Ono and I am a senior tester at the JMP Japan division of SAS Institute Japan.
I may be mistaken and off the mark, but below I would like to state what I personally think at this stage.
■Regarding the first question in particular, since you said that the results are different each time, I think something strange is going on. (Note that in the "Logistic Plot" with only one continuous explanatory variable, random numbers are used to determine the Y coordinate. However, at least in JMP15 and later, a fixed random seed value is used, so the plot should not look different each time it is run. Also, random numbers are not used for other numerical results of logistic regression, and for mathematical reasons, at least the maximum likelihood point estimates should be the same in any statistical software if the data and settings are the same, except for the two situations described below.) If the reply below is irrelevant, could you tell me which numerical results in the report are different each time? Also, could you confirm whether the numerical results are different each time even if you rerun the analysis using a script?
■Generally, when fitting a model in JMP, you should pay attention to the following points:
Point 1 : If your question is whether JMP's results are different from those of other statistical software, please note that JMP uses a method called effect coding.
Point 2 : Please note that (part of) the JMP results will change depending on the "scale" set for the column.
Point 3 : Please note that (part of) the JMP results may change depending on the "order in which values are displayed."
Point 4 : By using scripts, you can rerun the same analysis without using the mouse.
＜Details＞
Logistic regression for binary, ordinal, or nominal responses has only one global solution, except in two cases:
(1) When there is linear independence between explanatory variables
For example, when the relationship between three variables X1, X2, and X3 is X1 + X2 = X3, or when the sample size is smaller than the number of explanatory variables, the parameter estimates cannot be uniquely determined.
(2) When separation exists
In this case, the parameter estimates will be either +∞ or -∞, i.e., they do not exist.
Conversely, in situations other than these two, logistic regression analysis for binary, ordinal, and nominal response variables has a unique solution. Therefore, if the data and model settings are the same, in situations other than the two above, even if different statistical software is used (*1), at least the point estimates (point estimates obtained by maximum likelihood estimation) should and will be the same within the range of numerical error. In addition, at least JMP logistic regression analysis does not use random numbers in the optimization process (*2), so in that respect, if the data and settings are the same and the same version of JMP is used, they should and will be exactly the same (*3).
(*1) However, some routines that perform "logistic" in other software use penalized likelihood with L1 norm or L2 norm by default instead of normal logistic regression analysis. Also, as described in (b) below, parameter estimates may differ depending on the coding of categorical explanatory variables. Furthermore, there are three types of tests for maximum likelihood estimates: the Wald test, the likelihood ratio test, and the score test, and the tests and confidence intervals differ depending on which test is used.
(*2) However, in the "Logistic Plot" drawn when there is only one continuous explanatory variable, random numbers are used to determine the Y coordinate. However, at least in JMP version 15 and later, a fixed random number seed value is used, so the results should not change each time you run it. The help for the current version (18.2) can be viewed at the following URL.
(*3) However, depending on the time it takes to process the data, the default test for the effect may be the likelihood ratio test or the Wald test. For the current version (18.2), more details are provided at the following URL.
However, there are three points (pitfalls) that you should be careful of (a) to (c).
(a) In JMP, even if the numerical values of the data are the same, the results of graphs and statistical analyses will change depending on the scale (continuous, ordinal, or nominal) used in the data table.
(b) In JMP, the coding of explanatory variables (especially for nominal scales) is done in a way that is often used in manufacturing and industrial statistics. This coding is called "effect coding." For example, for male and female, in the case of nominal scales, they are coded as follows:
Male 1
Male 1
Female -1
Female -1
Female -1
or,
Male -1
Male -1
Woman 1
Woman 1
Woman 1
For example, in the case of a three-value nominal scale, it would be coded as follows:
Ehime 1 0
Ehime 1 0
Kagawa 0 1
Tokushima -1 -1
Tokushima -1 -1
Tokushima -1 -1
(When this type of effect coding is used for nominal scale variables, the parameter estimates represent deviations from the "intermediate setting." In manufacturing and industrial statistics, many nominal scale variables are originally continuous, and deviations from the settings in the current process are often an issue, so this type of coding is likely used frequently.)
A different coding is used for the ordinal explanatory variables.
(c) When the data is sorted, the last level will be "-1" (in nominal scale coding). You can also change the last level by using the "Value Display Order" column property. (In the above example, Tokushima is the last category displayed.)
The parameter estimates change depending on the difference between (b) and (c). However, the fit to the data as a whole ("Full Model Test", "Details of Fit", and "Lack of Fit (LOF)") does not change even if the settings in (b) and (c) are different.
In JMP, you can save your analysis in a "script" or what is called "JMP Scripting Language (JSL)." You can save a script from the red triangle menu at the top of a report. In JMP version 18, you can use the red triangle menu in the report > Save Script and select To Data Table to save it to the data table. This saves the script in the upper left corner of the data table. This allows you to reproduce your analysis without having to use your mouse.
(In particular, JMP18 added a new feature called "Workflow Builder," which allows a wider range of mouse operations to be recorded.)
■2
Somers' D will also be negative if there is a so-called "negative correlation coefficient" (just as Pearson's and Kendall's correlation coefficients are negative). For example, the following data has negative Somers' D and Kendall's τb:
dt1 = New Table("Test1",
New Column("X", Ordinal, Values([1,1,1,2,2,2,3,3,3])),
New Column("Y", Ordinal, Values([1,2,3,1,2,3,1,2,3])),
New Column("度数", Values([9,8,7, 6,5,2, 4,3,1]))
);
dt1 << Contingency(
Y( :Y ),X( :X ), Freq( :度数 ),
Measures of Association( 1 )
);
To run this script, first select File > New > JSL Script from the JMP pull-down menu (for JMP 18 for Windows). Then, copy and paste the above script into the script window that is called. After copying and pasting, select Edit > Run Script from the pull-down menu.
This is also related to (c) in 1 above, but the results will vary depending on the order in which the data is sorted. In the default state where nothing is done, "smallest" and "largest" are defined by the order in which the data is sorted (A,B,C if the data is alphabetical, or 1,2,3 if the data is numeric).
You can change this order in the column properties "Value Display Order." (In older versions, this "Value Display Order" was named something else.)
(1) In the data table, right-click the column header and select Column Information.
(2) Click the Column Properties combo box and select Value Display Order.
(3) If the values are not sorted from smallest to largest, select a value and then use the up and down arrows (↑ or ↓) to change the order.
(4) After changing the display order, click the OK button in the upper right corner of the screen to close the column information window.
For example, the data below is exactly the same as the data above, but because the order of the values is different (X variables are defined as "3" being the smallest and "1" being the largest), Somers' D and Kendall's τb are positive, unlike the results above. (The following script was created using the current JMP 18. It may not work as expected in older versions.)
dt2 = New Table( "Test2",
New Column( "X", "Ordinal",
Set Property( "Value Order", {Custom Order( {3, 2, 1} )} ),
Values( [1, 1, 1, 2, 2, 2, 3, 3, 3] )
),
New Column( "Y","Ordinal",
Set Property( "Value Order", {Custom Order( {1, 2, 3} )} ),
Set Values( [1, 2, 3, 1, 2, 3, 1, 2, 3] )
),
New Column( "度数",
Set Values( [9, 8, 7, 6, 5, 2, 4, 3, 1] )
)
);
dt2 << Contingency(
Y( :Y ),X( :X ), Freq( :度数 ),
Measures of Association( 1 )
);
Somers' D can be negative (its minimum value is -1).
Also, to check if your analysis is incorrect, look at the contingency table in the report and make sure the values are sorted from smallest to largest. If they are not sorted as you expect, you need to change the order of the values in the column properties of the data table.
