DC Field | Value | Language |
---|---|---|
dc.contributor.advisor | 윤세영 | - |
dc.contributor.author | Eom, Seong-Ha | - |
dc.contributor.author | 엄성하 | - |
dc.date.accessioned | 2024-07-30T19:30:38Z | - |
dc.date.available | 2024-07-30T19:30:38Z | - |
dc.date.issued | 2024 | - |
dc.identifier.uri | http://library.kaist.ac.kr/search/detail/view.do?bibCtrlNo=1096063&flag=dissertation | en_US |
dc.identifier.uri | http://hdl.handle.net/10203/321358 | - |
dc.description | 학위논문(석사) - 한국과학기술원 : 김재철AI대학원, 2024.2,[iii, 20 p. :] | - |
dc.description.abstract | Contrastive language-image pre-training (CLIP) has demonstrated remarkable zero-shot classification ability, namely image classification using novel text labels. Existing works have attempted to enhance CLIP by fine-tuning on downstream tasks, but these have inadvertently led to performance degradation on unseen classes, thus harming zero-shot generalization. This paper aims to address this challenge by leveraging readily available image-text pairs from an external dataset for cross-modal guidance during inference. To this end, we propose X-MoRe, a novel inference method comprising two key steps: (1) cross-modal retrieval and (2) modal-confidence-based ensemble. Given a query image, we harness the power of CLIP's cross-modal representations to retrieve relevant textual information from an external image-text pair dataset. Then, we assign higher weights to the more reliable modality between the original query image and retrieved text, contributing to the final prediction. X-MoRe demonstrates robust performance across a diverse set of tasks without the need for additional training, showcasing the effectiveness of utilizing cross-modal features to maximize CLIP's zero-shot ability. | - |
dc.language | eng | - |
dc.publisher | 한국과학기술원 | - |
dc.subject | 제로샷 분류▼a교차 모달 검색▼a앙상블 | - |
dc.subject | Zero-shot classification▼aCross modal retrieval▼aEnsemble | - |
dc.title | Cross-modal retrieval meets inference: improving zero-shot classification with cross-modal retrieval | - |
dc.title.alternative | 교차 모달 검색과 추론의 만남: 교차 모달 검색으로 제로샷 분류 개선 | - |
dc.type | Thesis(Master) | - |
dc.identifier.CNRN | 325007 | - |
dc.description.department | 한국과학기술원 :김재철AI대학원, | - |
dc.contributor.alternativeauthor | Yun, Se-Young | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.