Eintrag weiter verarbeiten
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY
Gespeichert in:
| Zeitschriftentitel: | The Annals of Statistics |
|---|---|
| Personen und Körperschaften: | |
| In: | The Annals of Statistics, 38, 2010, 2, S. 894-942 |
| Medientyp: | E-Article |
| Sprache: | Englisch |
| veröffentlicht: |
Institute of Mathematical Statistics
|
| author_facet |
Zhang, Cun-Hui Zhang, Cun-Hui |
|---|---|
| author |
Zhang, Cun-Hui |
| spellingShingle |
Zhang, Cun-Hui The Annals of Statistics NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| author_sort |
zhang, cun-hui |
| spelling |
Zhang, Cun-Hui 0090-5364 Institute of Mathematical Statistics https://www.jstor.org/stable/25662264 <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p> NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY The Annals of Statistics |
| facet_avail |
Online Free |
| format |
ElectronicArticle |
| fullrecord |
blob:ai-55-aHR0cHM6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8yNTY2MjI2NA |
| id |
ai-55-aHR0cHM6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8yNTY2MjI2NA |
| institution |
DE-15 DE-14 DE-Ch1 DE-D13 |
| imprint |
Institute of Mathematical Statistics, 2010 |
| imprint_str_mv |
Institute of Mathematical Statistics, 2010 |
| issn |
0090-5364 |
| issn_str_mv |
0090-5364 |
| language |
English |
| mega_collection |
JSTOR Arts & Sciences I Archive JSTOR Mathematics & Statistics |
| match_str |
zhang2010nearlyunbiasedvariableselectionunderminimaxconcavepenalty |
| publishDateSort |
2010 |
| publisher |
Institute of Mathematical Statistics |
| recordtype |
ai |
| record_format |
ai |
| series |
The Annals of Statistics |
| source_id |
55 |
| title |
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_unstemmed |
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_full |
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_fullStr |
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_full_unstemmed |
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_short |
NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_sort |
nearly unbiased variable selection under minimax concave penalty |
| url |
https://www.jstor.org/stable/25662264 |
| publishDate |
2010 |
| physical |
894-942 |
| description |
<p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p> |
| collection |
sid-55-col-jstoras1 sid-55-col-jstormaths |
| container_issue |
2 |
| container_start_page |
894 |
| container_title |
The Annals of Statistics |
| container_volume |
38 |
| format_de105 |
Article, E-Article |
| format_de14 |
Article, E-Article |
| format_de15 |
Article, E-Article |
| format_de520 |
Article, E-Article |
| format_de540 |
Article, E-Article |
| format_dech1 |
Article, E-Article |
| format_ded117 |
Article, E-Article |
| format_degla1 |
E-Article |
| format_del152 |
Buch |
| format_del189 |
Article, E-Article |
| format_dezi4 |
Article |
| format_dezwi2 |
Article, E-Article |
| format_finc |
Article, E-Article |
| format_nrw |
Article, E-Article |
| _version_ |
1792362524629270549 |
| geogr_code |
not assigned |
| last_indexed |
2024-03-01T21:55:04.391Z |
| geogr_code_person |
not assigned |
| openURL |
url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fvufind.svn.sourceforge.net%3Agenerator&rft.title=NEARLY+UNBIASED+VARIABLE+SELECTION+UNDER+MINIMAX+CONCAVE+PENALTY&rft.date=2010-04-01&genre=article&issn=0090-5364&volume=38&issue=2&spage=894&epage=942&pages=894-942&jtitle=The+Annals+of+Statistics&atitle=NEARLY+UNBIASED+VARIABLE+SELECTION+UNDER+MINIMAX+CONCAVE+PENALTY&aulast=Zhang&aufirst=Cun-Hui&rft.language%5B0%5D=eng |
| SOLR | |
| _version_ | 1792362524629270549 |
| author | Zhang, Cun-Hui |
| author_facet | Zhang, Cun-Hui, Zhang, Cun-Hui |
| author_sort | zhang, cun-hui |
| collection | sid-55-col-jstoras1, sid-55-col-jstormaths |
| container_issue | 2 |
| container_start_page | 894 |
| container_title | The Annals of Statistics |
| container_volume | 38 |
| description | <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p> |
| facet_avail | Online, Free |
| format | ElectronicArticle |
| format_de105 | Article, E-Article |
| format_de14 | Article, E-Article |
| format_de15 | Article, E-Article |
| format_de520 | Article, E-Article |
| format_de540 | Article, E-Article |
| format_dech1 | Article, E-Article |
| format_ded117 | Article, E-Article |
| format_degla1 | E-Article |
| format_del152 | Buch |
| format_del189 | Article, E-Article |
| format_dezi4 | Article |
| format_dezwi2 | Article, E-Article |
| format_finc | Article, E-Article |
| format_nrw | Article, E-Article |
| geogr_code | not assigned |
| geogr_code_person | not assigned |
| id | ai-55-aHR0cHM6Ly93d3cuanN0b3Iub3JnL3N0YWJsZS8yNTY2MjI2NA |
| imprint | Institute of Mathematical Statistics, 2010 |
| imprint_str_mv | Institute of Mathematical Statistics, 2010 |
| institution | DE-15, DE-14, DE-Ch1, DE-D13 |
| issn | 0090-5364 |
| issn_str_mv | 0090-5364 |
| language | English |
| last_indexed | 2024-03-01T21:55:04.391Z |
| match_str | zhang2010nearlyunbiasedvariableselectionunderminimaxconcavepenalty |
| mega_collection | JSTOR Arts & Sciences I Archive, JSTOR Mathematics & Statistics |
| physical | 894-942 |
| publishDate | 2010 |
| publishDateSort | 2010 |
| publisher | Institute of Mathematical Statistics |
| record_format | ai |
| recordtype | ai |
| series | The Annals of Statistics |
| source_id | 55 |
| spelling | Zhang, Cun-Hui 0090-5364 Institute of Mathematical Statistics https://www.jstor.org/stable/25662264 <p>We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of p ≫ n, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in l r balls. We use the SURE method to derive degrees of freedom and C p -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level. For full rank designs and general sub-quadratic penalties, we provide necessary and sufficient conditions for the continuity of the penalized LSE. Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed method.</p> NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY The Annals of Statistics |
| spellingShingle | Zhang, Cun-Hui, The Annals of Statistics, NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title | NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_full | NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_fullStr | NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_full_unstemmed | NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_short | NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| title_sort | nearly unbiased variable selection under minimax concave penalty |
| title_unstemmed | NEARLY UNBIASED VARIABLE SELECTION UNDER MINIMAX CONCAVE PENALTY |
| url | https://www.jstor.org/stable/25662264 |