Further processing options
Cover Image of NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
available via Open Access

NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY

Saved in:

Bibliographic Details
Journal Title: The Annals of Statistics
Authors and Corporations: Zhang, Cun-Hui
In: The Annals of Statistics, 38, 2010, 2, p. 894-942
Media Type: E-Article
Language: English
published:
Institute of Mathematical Statistics
author_facet Zhang, Cun-Hui
Zhang, Cun-Hui
author Zhang, Cun-Hui
spellingShingle Zhang, Cun-Hui
The Annals of Statistics
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
author_sort zhang, cun-hui
spelling Zhang, Cun-Hui 0090-5364 Institute of Mathematical Statistics https://www.jstor.org/stable/25662264 <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p> NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY The Annals of Statistics
facet_avail Online
Free
format ElectronicArticle
fullrecord blob:ai-55-aHR0cHM6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8yNTY2MjI2NA
id ai-55-aHR0cHM6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8yNTY2MjI2NA
institution DE-14
DE-15
DE-Ch1
DE-D13
imprint Institute of Mathematical Statistics, 2010
imprint_str_mv Institute of Mathematical Statistics, 2010
issn 0090-5364
issn_str_mv 0090-5364
language English
mega_collection JSTOR Arts & Sciences I Archive
JSTOR Mathematics & Statistics
match_str zhang2010nearlyunbiasedvariableselectionunderminimaxconcavepenalty
publishDateSort 2010
publisher Institute of Mathematical Statistics
recordtype ai
record_format ai
series The Annals of Statistics
source_id 55
title NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_unstemmed NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_full NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_fullStr NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_full_unstemmed NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_short NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_sort nearly unbiased variable selection under minimax concave penalty
url https://www.jstor.org/stable/25662264
publishDate 2010
physical 894-942
description <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p>
collection sid-55-col-jstoras1
sid-55-col-jstormaths
container_issue 2
container_start_page 894
container_title The Annals of Statistics
container_volume 38
format_de105 Article, E-Article
format_de14 Article, E-Article
format_de15 Article, E-Article
format_de520 Article, E-Article
format_de540 Article, E-Article
format_dech1 Article, E-Article
format_ded117 Article, E-Article
format_degla1 E-Article
format_del152 Buch
format_del189 Article, E-Article
format_dezi4 Article
format_dezwi2 Article, E-Article
format_finc Article, E-Article
format_nrw Article, E-Article
_version_ 1799323718067945474
geogr_code not assigned
last_indexed 2024-05-17T17:59:52.389Z
geogr_code_person not assigned
openURL url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fvufind.svn.sourceforge.net%3Agenerator&rft.title=NEARLY+UNBIASED+VARIABLE+SELECTION+UNDER+MINIMAX+CONCAVE+PENALTY&rft.date=2010-04-01&genre=article&issn=0090-5364&volume=38&issue=2&spage=894&epage=942&pages=894-942&jtitle=The+Annals+of+Statistics&atitle=NEARLY+UNBIASED+VARIABLE+SELECTION+UNDER+MINIMAX+CONCAVE+PENALTY&aulast=Zhang&aufirst=Cun-Hui&rft.language%5B0%5D=eng
SOLR
_version_ 1799323718067945474
author Zhang, Cun-Hui
author_facet Zhang, Cun-Hui, Zhang, Cun-Hui
author_sort zhang, cun-hui
collection sid-55-col-jstoras1, sid-55-col-jstormaths
container_issue 2
container_start_page 894
container_title The Annals of Statistics
container_volume 38
description <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p>
facet_avail Online, Free
format ElectronicArticle
format_de105 Article, E-Article
format_de14 Article, E-Article
format_de15 Article, E-Article
format_de520 Article, E-Article
format_de540 Article, E-Article
format_dech1 Article, E-Article
format_ded117 Article, E-Article
format_degla1 E-Article
format_del152 Buch
format_del189 Article, E-Article
format_dezi4 Article
format_dezwi2 Article, E-Article
format_finc Article, E-Article
format_nrw Article, E-Article
geogr_code not assigned
geogr_code_person not assigned
id ai-55-aHR0cHM6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8yNTY2MjI2NA
imprint Institute of Mathematical Statistics, 2010
imprint_str_mv Institute of Mathematical Statistics, 2010
institution DE-14, DE-15, DE-Ch1, DE-D13
issn 0090-5364
issn_str_mv 0090-5364
language English
last_indexed 2024-05-17T17:59:52.389Z
match_str zhang2010nearlyunbiasedvariableselectionunderminimaxconcavepenalty
mega_collection JSTOR Arts & Sciences I Archive, JSTOR Mathematics & Statistics
physical 894-942
publishDate 2010
publishDateSort 2010
publisher Institute of Mathematical Statistics
record_format ai
recordtype ai
series The Annals of Statistics
source_id 55
spelling Zhang, Cun-Hui 0090-5364 Institute of Mathematical Statistics https://www.jstor.org/stable/25662264 <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p> NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY The Annals of Statistics
spellingShingle Zhang, Cun-Hui, The Annals of Statistics, NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_full NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_fullStr NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_full_unstemmed NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_short NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
title_sort nearly unbiased variable selection under minimax concave penalty
title_unstemmed NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
url https://www.jstor.org/stable/25662264