Different mixture experiment analysis results JMP 17 & 18Did something in JMP 18 change in analysis of mixture experiment results. Below is shown that analysis results from exactly the same DOE data sets are different
1. DOE three component mixture
JMP 17 results
JMP 18 Results
2. DOE three mixture components + one process effect
JMP 17 results
JMP 18 results
In JMP 18 analysis above the model is different!
What has changed and why? Remark: despite of different plots & model graphs of JMP 17 - 18 prediction profilers are identical.
Design of ExperimentsRe: Different mixture experiment analysis results JMP 17 & 18Hi @frankderuyck,
In the first case, can you provide anonymized dataset ? Are the columns properties the same ?
In the second case, the models are different in the two captures. You should have similar results by having the same models.
Re: Different mixture experiment analysis results JMP 17 & 18My first three component case study above was similar to the one presented in a known earlier JMP webcast vinegar/oil/water case study below, you can see that the plot is also similar to mine but not the same in JMP 18
Re: Different mixture experiment analysis results JMP 17 & 18I remeber that the explanation for the wide confidence interval was the strong collinearity among the mixture effects which makes sense; this does not show up anymore in the JMP 18 plot (?)
Re: Different mixture experiment analysis results JMP 17 & 18The link to the webinar is https://community.jmp.com/t5/JMP-Blog/Summer-DOE-Series-Mixture-DOE/ba-p/669012
I got green light to use this data set for my DOE training
Re: Different mixture experiment analysis results JMP 17 & 18Hi @frankderuyck,
I would compare the outcomes of the first use case on something else than p-values, due to multicollinearity in mixture designs.
Again, without any dataset to test, it's difficult to help you since we have only a partial view at this comparison.
For the first use case, if the model are set up correctly and in the same way between JMP 17 and 18, try to look at the Parameter estimates (and standard deviations), at metrics focussed on predictivity and optimization (like RMSE) and residuals/diagnostic plots.
Since you mentioned the Profiler has similar results between versions, I would guess that the differences you are seeing are mostly due to the use of Mixture designs instead of a calculation change between versions.
Re: Different mixture experiment analysis results JMP 17 & 18 Find in attachment the DOE analysis data set
During my september training I still used JMP 17, trainees got JMP 18, it was a big surprise and annoying that my 17 training results were different! Mine are similar (same data) to those presented in webinar above, think also with JMP 17.
Re: Different mixture experiment analysis results JMP 17 & 18Strange, I used your data table and analyzed in both 17 and 18. The only difference I found was the confidence limits on the Actual by Predicted plots (leverage). All of the other statistics were identical (I used your scripted fit model).
JMP18
JMP17
Re: Different mixture experiment analysis results JMP 17 & 18Indeed question is why?
Re: Different mixture experiment analysis results JMP 17 & 18The jmp 17 plot is should be more accurate because of the very strong collinearty among mixture components; limits from jmp 18 are to my opinion too narrow?
Re: Different mixture experiment analysis results JMP 17 & 18Yes, but in your initial post, the statistics were different as well. That does not appear to be a JMP issue.
Re: Different mixture experiment analysis results JMP 17 & 18I'm not sure the correct word is accurate...different, yes. It appears the JMP18 version looks more like a leverage plot than the one from JMP17. Typically those plots are narrowest in the center and expand near the ends. Collinearity is typically a cluster of points near the middle.
https://www.jmp.com/support/help/en/18.1/#page/jmp/effect-leverage-plots.shtml?os=mac&source=application%23ww184200
Re: Different mixture experiment analysis results JMP 17 & 18The DOE analysis script is 100% reproducible both in JMP 17 and 18. When running second DOE (mixture + process effect) analysis script, developed in jmp 17, in jmp 18 besides the prediction plot also model is different (?) The prediction profilers always are equal.
Re: Different mixture experiment analysis results JMP 17 & 18As in case 1 the case 2 data set and analysis script are exactly the same, indeed now results are also different..
Only difference is the two jmp versions
Re: Different mixture experiment analysis results JMP 17 & 18I'm confused. I posted the output models for both 17 and 18 above. They are identical. The only difference is the leverage plot. You must have some preferences set differently in the 2 versions.
Re: Different mixture experiment analysis results JMP 17 & 18Hi @frankderuyck,
I might have found a clue about the difference in plots.
- In JMP 17, when you extend the ranges of "Actual by Predicted" plot, it seems the plot is constructed by using a response mean of 0 (confidence band is symetrical around 0):
So maybe the modeling without intercept of mixture models was (incorrectly ?) translated into a modeling with intercept = 0, hence the confidence band being large and non-symetrical around the response mean.
- In JMP 18, you don't observe this situation as confidence band are symetrical around the response mean (5,88) :
Curious to know if any developers might confirm or inform about this difference in plots. @Ryan_Lekivetz ?
Best,
Re: Different mixture experiment analysis results JMP 17 & 18Hi @frankderuyck ,
For any question like this I would recommend to contact JMP Technical Support (support@jmp.com). Send example data with saved scripts so they can most easily recreate the analysis exactly as you have done it. They will also need information about the version(s) of JMP that you are using.
Regards,
Phil
Re: Different mixture experiment analysis results JMP 17 & 18OK Phil, I informed technical support about this discussion
Re: Different mixture experiment analysis results JMP 17 & 18Tech support will certainly get to the bottom of this. However, I do believe that @Victor_G might be on to something. I ALWAYS suggest to people fitting mixture models to look at the overall ANOVA report (Regression Reports > Analysis of Variance). It is closed by default , but I wish it were open and visible for your analyses.
If you are fitting a mixture model, that report will have the message: Tested against reduced model Y=mean. This is what you SHOULD see when fitting a Scheffe mixture model.
If it is a no-intercept model, the report will have the message: Tested against reduced model Y=0. We do NOT want this for a Scheffe mixture model. This might occur if the mixture components do not always add to exactly 1 or perhaps some of the components do not have the correct column properties assigned.
Again, I know tech support will get to the correct answer, but I wanted to chime in on this item that is often overlooked when fitting a Scheffe mixture model.
Re: Different mixture experiment analysis results JMP 17 & 18Agreed, I think Victor figured it out. The JMP 18 leverage plots look "more correct" IMHO.
Re: Different mixture experiment analysis results JMP 17 & 18Thanks all for your comments/inputs, Victor you nailed it again!
Will the jmp 18 approach be kept in next jmp versions? In training sessions not al participants use the same version
Re: Different mixture experiment analysis results JMP 17 & 18Find below feedback from JMP technical support, this is in agreement with the anwer of Vicotor above:
The difference you noted between JMP 17 and 18 was due to the Actual by Predicted plot not appropriately accounting for the "hidden" intercept in mixture models. To answer your last question in the community thread, the modified calculation of the confidence interval will persist into future versions.
Re: Different mixture experiment analysis results JMP 17 & 18Sorry Victor, mistake in your name above
