DC Field | Value | Language |
---|---|---|
dc.contributor.advisor | Yang, Eunho | - |
dc.contributor.advisor | 양은호 | - |
dc.contributor.author | Yun, Jihun | - |
dc.date.accessioned | 2021-05-13T19:32:25Z | - |
dc.date.available | 2021-05-13T19:32:25Z | - |
dc.date.issued | 2020 | - |
dc.identifier.uri | http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=911002&flag=dissertation | en_US |
dc.identifier.uri | http://hdl.handle.net/10203/284671 | - |
dc.description | 학위논문(석사) - 한국과학기술원 : 전산학부, 2020.2,[iii, 27 p. :] | - |
dc.description.abstract | We study high-dimensional estimators with the trimmed $l_1$ penalty, which leaves the $h$ largest parameter entries penalty-free. While optimization techniques for this non-convex penalty have been studied, the statistical properties have not yet been analyzed. We present the first statistical analyses for M-estimation, and characterize support recovery, $l_{\infty}$ and $l_2$ error of the trimmed $l_1$ estimates as a function of the trimming parameter h. Our results show different regimes based on how h compares to the true support size. Our second contribution is a new algorithm for the trimmed regularization problem, which has the same theoretical convergence rate as difference of convex (DC) algorithms, but in practice is faster and finds lower objective values. Empirical evaluation of $l_1$ trimming for sparse linear regression and graphical model estimation indicate that trimmed $l_1$ can outperform vanilla $l_1$ and non-convex alternatives. Our last contribution is to show that the trimmed penalty is beneficial beyond M-estimation, and yields promising results for two deep learning tasks: input structures recovery and network sparsification. | - |
dc.language | eng | - |
dc.publisher | 한국과학기술원 | - |
dc.subject | High-dimensional Statistics▼aLearning with Sparsity▼aRegularization | - |
dc.subject | Optimization▼aDeep Learning | - |
dc.subject | 고차원 데이터 분석▼a희소 학습▼a정규화▼a최적화▼a딥 러닝 | - |
dc.title | Trimming the $l_1$ regularizer | - |
dc.title.alternative | 가지친 $l_1$ 정규화 : 통계적 이론 분석, 최적화, 그리고 딥러닝 문제에 대한 적용 방법 | - |
dc.type | Thesis(Master) | - |
dc.identifier.CNRN | 325007 | - |
dc.description.department | 한국과학기술원 :전산학부, | - |
dc.contributor.alternativeauthor | 윤지훈 | - |
dc.title.subtitle | statistical analysis, optimization, and applications to deep learning | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.