Regression and Variable Selection via A Layered Elastic Net

Year: 2017       Vol.: 66       No.: 2      

Authors: Michael Van B. Supranes and Joseph Ryan G. Lansangan

Abstract:

One approach in modeling high dimensional data is to apply an elastic net (EN) regularization framework. EN has the good properties of least absolute shrinkage selection operator (LASSO), however, EN tends to keep variables that are strongly correlated to the response, and may result to undesirable grouping effect. The Layered Elastic Net Selection (LENS) is proposed as an alternative framework of utilizing EN such that interrelatedness and groupings of predictors are explicitly considered in the optimization and/or variable selection. Assuming groups are available, LENS applies the EN framework group-wise in a sequential manner. Based on the simulation study, LENS may result to an ideal selection behavior, and may exhibit a more appropriate grouping effect than the usual EN. LENS results to poor prediction accuracy, but applying OLS on the selected variables may yield optimum results. At optimal conditions, the mean squared prediction error of OLS on LENS-selected variables are on par with the mean squared prediction error of OLS on EN-selected variables. Overall, applying OLS on LENS-selected variables makes a better compromise between prediction accuracy and ideal grouping effect.

Keywords: regression, variable selection, variable clustering, high dimensional data, elastic net, grouping effect

Download this article: