Adaptive vs. Non Adaptive for Lasso/Elastic Net

cweisbart

Hi all,

 

I was wondering under which situations would I check the Adaptive box under generalized regression that appears for Lasso and Elastic net estimation methods? Is there a reason to not have it be adaptive? Any theory or background would be greatly appreciated.

 

jiancao

They are discussed in the JMP documentation https://www.jmp.com/support/help/en/15.2/#page/jmp/estimation-method-options.shtml:

“Adaptive Lasso This option uses the MLEs to weight the l1 penalty. MLEs cannot be computed when the number of predictors exceeds the number of observations or when there are strict linear dependencies among the predictors…...

Adaptive Elastic Net This option uses the MLEs to weight the l1 penalty. MLEs cannot be computed when the number of predictors exceeds the number of observations or when there are strict linear dependencies among the predictors…...”

 

In other words, do not use the adaptive versions when n < p or the predictors are strongly correlated because they rely on the MLE estimates to achieve the so-called "oracle property".

 

To go deeper here are two highly cited papers:

http://users.stat.umn.edu/~zouxx019/Papers/adalasso.pdf  (This JASA paper is cited by JMP)

http://users.stat.umn.edu/~zouxx019/Papers/aenet.pdf