Gaussian model prediction equation not very accurate.

Question

Hi all,

I'm using the Gaussian process model to fit a few hundred computer simulation data points. The computer simulations are a deterministic system, so the response never changes if the same inputs are used. I used the space filling Fast Flexible Filling design to generate the table. There are 7 different factors and 3 of those are entered categorically, the other 4 are continuous. The problem is that no matter how large I make the training design, the prediction equation generated is not very accurate when tested against a validation set of another couple hundred runs. The error in the training set used for the fit is less than 1%, but the error in the validation set can be as high as 4.5%. I'm looking to get around at most an error of 1-1.5%.

I've also tried using the Fit Model with a response surface model effect, and after crossing the factors a couple more times the actual vs predicted plot becomes a near perfect 45 degree angle, but the prediction equation isn't very accurate either. The Gaussian model is supposed to be the best choice for computer simulation data.

Does anyone know what limitations the Gaussian model has? Is my 1-1.5% error too unrealistic? The validation data set is generated in the same way, fast flexible filling. There's nothing unusual that stands out in the validation data. Many of the points that have 4% error have factor values that are very similar to other data points that have less than 1% error. I would think that with a smoothing fit, which is what the Gaussian model does, that the equation would predict everything very well.

Also, not sure how significant it is but I'm using version 12 of standard JMP. Is the Gaussian model in JMP Pro more powerful? Are there other statistical analysis software packages that can handle Gaussian modeling better than JMP?

Thanks for any help I appreciate it!

krln · Answer

Thanks for the answer, billw@JMP.

I have been looking into the books in the Help section, yet the description of the correlation functions is quite brief.

I thought that, no matter what correlation structure is used, the Gaussian process model always interpolates the design points perfectly. Apparently this is not the case for the Gaussian correlation structure, but I could not find a clear explanation for this in the Help section books... I will look into other sources to clarify this.

Thanks for your help!

Bill_Worley · Answer

Krin,

I would refer you to the books section under Help and a clearer explanation of the differences between Gaussian and Cubic.  Below are a few things I found there.

Note: The estimated parameters can be different due to different starting points in the

minimization routine, the choice of correlation type, and the inclusion of a nugget parameter.

Correlation Type Choose the correlation structure for the model. The platform fits a spatial

correlation model to the data, where the correlation of the response between two

observations decreases as the values of the independent variables become more distant.

Gaussian restricts the correlation between two points to always be nonzero, no matter the

distance between the points.

Cubic allows the correlation between two points to be zero for points that are far enough

apart. This method is a generalization of a cubic spline.

HTH

krln · Answer

Hi all,

I have a question related to this post. I am setting up a Latin hypercube design and fitting a Gaussian Process. When comparing both the Gaussian and Cubic correlation structures, and comparing the predicted values at the design points with the simulated value at each point, these values are exactly the same for the Cubic interpolation function, but not for the Gaussian. Is this explained by the bias as in bllw@jmp's post? Or is this due to any roundoff errors? In theory there should be perfect interpolation for both types of correlation functions, or am I wrong on this one?

Any help would be highly appreciated!