Model MetricsHi
in order to predict an electrical parameter by using an equipment parameters at a particular production step of our semiconductor Fab we used a Random Forest. If we look at the relative error (%) of the model we can see that is is very small being around 0.5% (see attached picture). The RMSE is low too, but R2 is only 52%.
So if we look at the percentage error we can say that the model is fine while this seems to be not true if we look at R2. How to solve?
Thanks Felice
Re: Model MetricsThe RMSE is an estimate of the standard deviation of the response. It is assumed to be constant over the entire range of the response in the model. The R square estimates how much of the response variance is not the random error component. So your model might be significant, but it will not be very accurate.
Re: Model MetricsI triyed to calculate the systematic error MBE in percentage and this is only 0.01%, that is pratically negligible, and the absolute error is only 0.54%. If we use Accuracy=100-MAE=99.46% that is really a good value. Same for RSME that in percentage is only 0.007% by indicating a very small average deviation from true values. Once again seems that R2 is not giving the complete picture.
What do you think?
Felice
Re: Model MetricsCan you show a plot of Actual by Predicted?
Re: Model MetricsHi
in attached the correlation requested.
Felice
Re: Model MetricsThank you for providing that plot.
It shape of the markers looks like a rhombus. I am not sure what to expect but it is unusual.
It also demonstrates why the R square is about 0.5 and why predictions will not be very accurate. Imagine predicting the response at an input of 131. I assume that the predicted response is the abscissa. It appears to be unbiased but with high variance. There is a lot of data so the confidence interval on the mean response might be relatively narrow but the prediction interval would be much wider.
