DC Field | Value | Language |
---|---|---|
dc.contributor.advisor | 양홍석 | - |
dc.contributor.author | Kim, Taeyoung | - |
dc.contributor.author | 김태영 | - |
dc.date.accessioned | 2024-07-25T19:31:21Z | - |
dc.date.available | 2024-07-25T19:31:21Z | - |
dc.date.issued | 2023 | - |
dc.identifier.uri | http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=1045942&flag=dissertation | en_US |
dc.identifier.uri | http://hdl.handle.net/10203/320710 | - |
dc.description | 학위논문(석사) - 한국과학기술원 : 전산학부, 2023.8,[iii, 68 p. :] | - |
dc.description.abstract | In this paper, we extend the Neural Tangent Kernel analysis to incorporate the Jacobian Regularisation, which includes additional term in the training objective. This additional regulariser forces the neural network to be flat around its training data, which makes the network to be robust against the adversarial attack. We first show that the Jacobian of neural network converges to the Gaussian Process jointly with the output as the widths of neural network tends to infinity, and characterise the kernel of this GP. We then show that the training under Jacobian regularisation is described by the inner products between the gradients of outputs and Jacobians with respect to the parameters, which we named this inner product as Jacobian Neural Tangent Kernel due to its similarity of NTK in literatures. We show that the properties of NTK also hold in JNTK, that at initialisation the JNTK converges to the deterministic asymptotic counterpart, and it stays constant during training if the width of neural network is sufficiently large. We finally prove that the Jacobian regularised training dynamics coincide with kernel regression with JNTK. Using this correspondence to kernel regression, we analyse how Jacobian regularised training enhances the robustness, and when is it possible | - |
dc.language | eng | - |
dc.publisher | 한국과학기술원 | - |
dc.subject | 신경 기울기 커널▼a야코비안 정칙화▼a무한 너비 인공 신경망▼a심층 학습 이론 | - |
dc.subject | Neural tangent kernel▼aJacobian regularisation▼aInfinite width neural network▼aDeep learning theory | - |
dc.title | (An) infinite-width analysis on the Jacobian-regularised training of a neural network | - |
dc.title.alternative | 무한 너비 인공 신경망의 야코비안과 로버스트한 훈련 | - |
dc.type | Thesis(Master) | - |
dc.identifier.CNRN | 325007 | - |
dc.description.department | 한국과학기술원 :전산학부, | - |
dc.contributor.alternativeauthor | Yang, Hongseok | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.