Lasso singularity details

gene

I am applying lasso to a design matrix that has a couple of singularity issues.  There are a couple of columns that are linear combinations of other columns.  I have run lasso in two cases.  The first I just let lasso reprt the singularities.  The second case is where I remove the two columns that generate the singularities.  The models are a bit different.  The other thing that occurs is that when I launch lasso with the full column rank matrix it automatically does logistic regression and I have to do an extra step to make it do lasso.  Is there a general protocol to remove singularities if they are evident?

Mark_Bailey

Why would you enter a variable that is known to be a linear combination of other variables used in the model? Would you expect the models to be the same?

 

What is the modeling type of your response variable? If it is categorical, then JMP is trying to help and it selects logistic regression. But you can over-ride that behavior, as you discovered.