Adversarial dropout for deep neural networks = 딥 뉴럴 네트워크 학습 향상을 위한 적대적 드롭아웃

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 253
  • Download : 0
Deep neural networks have demonstrated their usefulness in a wide range of applications such as vision, voice, natural language processing. A deep neural network provides high capacity for better modeling the input-to-output transition of real data. However, the high capacity potentially occurs an over-fitting problem that indicates a large gap between the training error and generalization error. To alleviate this problem, this dissertation proposes a novel regularization technique. Specifically, adversarial dropout identifies a subnetwork that performs worse or provides output far from the output of the origin network, even though most of the neurons in the origin network are contained in the subnetwork. We developed a regularization that reduces the gap between the output of the intentionally perturbed network and the correct target, or the output of the origin network. Additionally, we theoretically proved that the regularization was the upper bound of the gap between the training and the inference phases of the random dropout. Our experiments demonstrated the applicability of adversarial dropout to two types of deep neural networks such as feed-forward neural networks and recurrent neural networks.
Advisors
Moon, Il-Chulresearcher문일철researcher
Description
한국과학기술원 :산업및시스템공학과,
Publisher
한국과학기술원
Issue Date
2018
Identifier
325007
Language
eng
Description

학위논문(박사) - 한국과학기술원 : 산업및시스템공학과, 2018.8,[v, 63 p. :]

Keywords

Deep Neural Networks▼aregularization▼aoptimization▼aadversarial training▼asupervised learning▼asemi-supervised learning▼asequence modeling▼adropout; 인공지능▼a뉴럴네트워크▼a정규화▼a지도학습▼a반지도학습

URI
http://hdl.handle.net/10203/264734
Link
http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=827894&flag=dissertation
Appears in Collection
IE-Theses_Ph.D.(박사논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0