No option for goodness of fit test for normal 2 and 3 mixture in distribution

Question

I have a customer that needs to confirm what they see visually with a GOF test for any distribution they use. The normal 2 and 3 mixture distributions don't include an option for a GOF test in the distribution platform.

Is there a different way to get a p-value to test the GOF of these two distributions?

DonMcCormack · Answer

Kolmogorov-Smirnoff GOF is the only method I can think that can be applied to something like this. We already use it in Oneway, but in a different context. Since it's nonparametric, I assume multimodal distribution are fair game. That said, I haven't read the research literature in a very long time, so I'm not completely certain. The Wiki entry suggests it is and its application can be found in the R package KSgeneral.

That said, someone would have to write JSL to implement it.

MarilynWheatley · Answer

This came up in a demo; they wanted to see how to test all possible distributions to find the one that fits the best. Their procedure says to confirm the fit with a test and a p-value. In this case, the 2 mixture distribution was the best fit using AICc when we fit all distributions. My suggestion was to use a regular normal distribution, since the AICc was almost the same as the 2 mixture, and it is easier/most people understand it. From what he explained, he'd like to use the distribution that JMP recommends as the best fit, but he can'd do that unless he can get a GOF test for the best distribution.

To be clear, he'd never used normal 2 mixture distribution before and did not say that he was able to get a GOF from other software for that particular distribution, but he did say that he can test all the available distributions in his other/previous package. We were both a bit surprised that there was no way to test the fit for the distribution that JMP identified as the best, just because the option is available for the other distributions.

SamGardner · Answer

I am curious why they want to confirm the mixture distribution?  I think a GOF test for a mixture normal is tricky, and I am not aware of other software that has that capability.