Creating a simulation model and experimental model and comparingSo I have some experimental data and some simulation data and I would like to compare the sensitivities of the factors, and see how they line up.
This is similar to: https://community.jmp.com/t5/Discussions/How-do-I-make-a-prediction-model-based-on-my-old-experiment-data/td-p/14029
I'm new to JMP and so hopefully I'm doing this right.
Attached is the simulation (2) and experimental data (3).
1. Is using the Fit Model command on the two data sets and comparing the parameter esimates the correct approach?
2. If so, when running both sets of data, I get the same values for the Std Errors for X1 X2 X3. I was under the impression that the standard errors are relate to the fit quality of the parameter estimate, but I don't see how they could be the same for all 3. Is there something I'm missing here?
Thanks
DOERe: Creating a simulation model and experimental model and comparingDisclaimer: I have not looked at the data, models, or any analysis that one would create for your problem.
I'll focus most of my commentary on your first question...with many questions and some thoughts.
1. Can you share more about the practical problem at hand? I believe any analysis and conclusions need to be considered and filtered through that lens vs. just looking at plots, data, and statistics.
2. Blind numeric comparison of parameter estimates is one, and only one way to compare these two systems. I can all but guarantee they will differ. Heck, rerun the empirical experiment again, and I'll all but guarantee a numeric difference. But are these differences important from a practical point of view? See question #1. above.
3. How much overlap within the factor space exists between the simulation data and the empirical data? Are there areas of interest that are not consistent?
4. How much overlap is there within the response space from each? Are there areas of interest that are not consistent?
5. What is the degree of consistency across the residual space for each model? After all...this is an estimate of unexplained variation for each model...is the degree of inconsistency problematic?
6. Have you tried simulation for each model to see what the sensitivities are over the modeling space?
I invite others to comment and add their thoughts. But at the end of the day I think just looking at parameter estimates is a very, very narrow view of 'what's going on in the system?'
Re: Creating a simulation model and experimental model and comparingI don't understand your question. What do you mean by "sensitivities of factors"? Are you trying to compare the 2 models from each data set? If so, as @P_Bartell indicates, there are many characteristics of the model that could be compared (R-squares, RMSE, coefficients (Estimates), Residuals (In fact you have unusual residuals in your DOE data). After briefly looking at the 2 data sets, I'm not sure what analysis you are doing, but I do not get the same std errors for the parameter estimates, nor do I get the same statistically significant parameters in the models.
How was the simulation created? What model was used for the simulation?
Re: Creating a simulation model and experimental model and comparing1. Sure, so essentially I have a simulation of circuit performance varying some parameters. I also have measured results of the same circuit with those same parameters varied. I would like to show that the measured results either matched or did not match the simulation.
2. The parameter estimate in this case would be the circuit parameter that was varied, and so I think it would be important from a practical point of view.
3. For now its the same, but the simulation model should probably have more points.
4-6, Not sure I understand.
Ultimately, I'm just trying to show that simulation accurately predicts the fabricated devices.
