Hi @Peter_Hersh. I appreciate it if you can answer this question: after we pick the important variables by comparing them to the null factor, do we need to go back to the original data set prior to auto-validation and use the chosen variables (main or interaction) and run the model again to find the predictive equation? or we can use the data set with weighted validation column and no null factor to do so? or we can use the simulated table? If the latter can be used, how to get rid of the null factor in the final equation?
I am kind of stuck here or am missing a point.
Thanks.
@Amir_H, Great question. For the auto-validation technique, Phil and I show off here the idea is variable selection. If you would like to make a predictive model using this technique you should use Self-Validating Ensemble models (SVEM). The basic idea with that is that you make many different models each time you re-calculate the weights and save the model results. After making many of these models you take the average of all of the models and typically get good results. Below is a link to a talk we gave that covers that technique and has an example script.
Re-Thinking the Design and Analysis of Experiments? (2021-EU-30MP-776)
Thanks, @Peter_Hersh. As far as I am learning about this technique, it's very useful when the dataset is very small and for variable selection there as you said. It has not helped me much yet because my dataset is not too small. Also, based on my experience, sometimes you may have to use other techniques to get rid of large VIFs and GenReg may not help. I am still trying to learn how to make the best out of either the Add-in (which only uses GenReg) or method (script).
