nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Model Metrics

FR60

in order to predict an electrical parameter by using an equipment parameters at a particular production step of our semiconductor Fab we used a Random Forest. If we look at the relative error (%) of the model we can see that is is very small being around 0.5% (see attached picture). The RMSE is low too, but R2 is only 52%.

So if we look at the percentage error we can say that the model is fine while this seems to be not true if we look at R2. How to solve?

Thanks Felice

Accepted answers

Mark_Bailey

Thank you for providing that plot.

It shape of the markers looks like a rhombus. I am not sure what to expect but it is unusual.

It also demonstrates why the R square is about 0.5 and why predictions will not be very accurate. Imagine predicting the response at an input of 131. I assume that the predicted response is the abscissa. It appears to be unbiased but with high variance. There is a lot of data so the confidence interval on the mean response might be relatively narrow but the prediction interval would be much wider.

All comments

Mark_Bailey

The RMSE is an estimate of the standard deviation of the response. It is assumed to be constant over the entire range of the response in the model. The R square estimates how much of the response variance is not the random error component. So your model might be significant, but it will not be very accurate.

FR60

I triyed to calculate the systematic error MBE in percentage and this is only 0.01%, that is pratically negligible, and the absolute error is only 0.54%. If we use Accuracy=100-MAE=99.46% that is really a good value. Same for RSME that in percentage is only 0.007% by indicating a very small average deviation from true values. Once again seems that R2 is not giving the complete picture.

What do you think?

Felice

Mark_Bailey

Can you show a plot of Actual by Predicted?

FR60

in attached the correlation requested.

Felice

Picture2.png