KMeans Clustering Comparison Report with Optimal CCC lower than other clusters

Question

Hi,

a customer asked why the Optimal CCC specified in the cluster comparison report for kmeans clustering sometimes is lower than the CCC of other clusters, which is contradictory to the documentation.

I could reproduce this behaviour using the nicardipine data set and kmeans clustering as attached (run the script for clustering)

is there another criteria involved in the decision not mentioned in the documentation or why does this happen?

NicardipineCluster.jmp

Dan_Obermiller · Accepted Answer

I think that there may be several things that cause this anomaly. If one of these is causing it, the documentation should mention it, so I think more detail is needed in the doc.

My thoughts:

The nicardipine variables you used are really categorical, not continuous. That could be contributing to the problem. The discreteness of the data typically leads to the CCC choosing too many clusters.

Two of the variables are constant values. Even the doc says that they are ignored, so I don't think that is the problem.

Every cluster from 6 on up has a cluster with only a single observation. That may be why those clusters are not "optimal" (but I really don't believe this either, but it is possible).

Looking at the CCC across clusters, the CCC drops dramatically and goes negative. Negative CCC values usually indicate outliers. Because there are 5 cluster sizes in a row with a lower CCC, I believe anything with a higher CCC than the 3 cluster model would indicate an overfitting. Could JMP be applying a similar logic?

If you run KMeans with clusters from 4 to 12, instead of 3, you get the results you expect, so perhaps this is a bug??

For completeness, the JMP doc references a SAS technical paper, I have included the conclusion from that paper for completeness. The full doc is here: https://support.sas.com/kb/22/addl/fusion_22540_1_a108_5903.pdf

Certainly an interesting situation. I would love to hear the real reason for 3 being chosen!

jiancao · Accepted Answer

JMP uses the largest CCC before it falls under 2 for the first time in labeling the # of cluster as the "Optimal CCC". I agree that the JMP documentation stating "Larger values of CCC indicate better fit" needs to change.