P.Mean: LASSO talk (created 2012-11-05). News: Sign up for "The Monthly Mean," the newsletter that dares to call itself average, www.pmean.com/news.

I volunteered to give a talk for the Kansas University Medical Center journal club about the Lasso (Least Absolute Shrinkage and Selection Operator). Here are some resources I will use in the talk.

The classic paper on the Lasso is Robert Tibshirani. Regression Shrinkage and Selection Via the Lasso Journal of the Royal Statistical Society, Series B. 1994; 58: 267-288. We propose a new method for estimation in linear models. The "lasso" minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly zero and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described. Available at: http://www-stat.stanford.edu/~tibs/lasso/lasso.pdf.

I have a PDF version of some R output using the glmnet library. This library was written by Jerome Friedman, Trevor Hastie, and Rob Tibshirani.

Here are some graphics that I will talk about.

Figure 1. Density function for a double exponential distribution (solid line) with density function for a comparable normal distribution shown for reference (dotted line).

Figure 2. Animation of hard thresholding.

Figure 3. Animation of soft thresholding.

Figure 4. Plot of glmnet object.

Figure 5. Plot of glmnet object with extra annotation.

Figure 6. Plot of crossvalidation for a glmnet.

This page was written by Steve Simon and is licensed under the Creative Commons Attribution 3.0 United States License. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Incomplete pages.