38330_en-US

liliconcom

Penalized Regression simulation studies

Hi,

 

A customer (a very competent statitician supporting pre-clinical in pharma) asked me if I know of any studies that have been done to determine how effective Elastic Net and Lasso are compared with standard techniques. Do they work better in certain situations? I think they are particularly interested in the application in the context of DoE.

 

Do you know of any publications that I can point them to? Any internal studies that we have done that can be shared?

 

Thanks,
Phil

Re: Penalized Regression simulation studies

Thanks, Clay. I think my customer will be very happy with that answer.

Phil

Re: Penalized Regression simulation studies

A lot of the research papers on penalized regression that I've encountered tend to be focused on observational data rather than designed experiments. Here is one paper that may be helpful:

http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0107957

 

Evaluating a variable selection method can be pretty tricky because there are so many things to consider. Do we tune using cross-validation or an information criteria? For a penalized method, we also have to consider how the penalty weights are chosen. And the results can depend on whether the goal is to minimize the prediction error or if it is to capture the variables that truly influence the response.

 

I've attached slides from a presentation I did at QPRC in 2015 related to using penalized regression for analyzing screening experiments. My goal was to see how well the lasso would perform in supersaturated designs. That was before the ERIC criteria around, which I think has a lot of potential for these cases.

 

Re: Penalized Regression simulation studies

Also, maybe:

 

https://community.jmp.com/t5/Discovery-Summit-2015/Model-Selection-Strategies-for-Definitive-Screening-Designs/ta-p/22854

 

Ian

Re: Penalized Regression simulation studies

Phil,

I've started to look into model selection, in particular, which techniques work best out of ad-hoc selection (fit everything, remove everything above p=0.05, then do it again), stepwise with BIC, and Lasso. Although I haven't been able to get terribly far because of time constraints, I've come to understand that the selection technique will be very dependent on the design. So, for something like a full factor, technique won't matter, but with a nearly saturated custom design, it will. I'm hoping to spend more time on this if things slow down. Clay Barker and Ryan Lekivetz have also done some work in this area.

Don