Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 187
  • Download : 0
DC FieldValueLanguage
dc.contributor.authorKim, Dongjunko
dc.contributor.authorShin, Seungjaeko
dc.contributor.authorSong, Kyungwooko
dc.contributor.authorKang, Wanmoko
dc.contributor.authorMoon, Il-Chulko
dc.date.accessioned2022-11-17T06:01:29Z-
dc.date.available2022-11-17T06:01:29Z-
dc.date.created2022-09-14-
dc.date.created2022-09-14-
dc.date.issued2022-07-18-
dc.identifier.citationThe 39th International Conference on Machine Learning, ICML 2022-
dc.identifier.urihttp://hdl.handle.net/10203/299794-
dc.description.abstractRecent advances in diffusion models bring state-of-the-art performance on image generation tasks. However, empirical results from previous research in diffusion models imply an inverse correlation between density estimation and sample generation performances. This paper investigates with sufficient empirical evidence that such inverse correlation happens because density estimation is significantly contributed by small diffusion time, whereas sample generation mainly depends on large diffusion time. However, training a score network well across the entire diffusion time is demanding because the loss scale is significantly imbalanced at each diffusion time. For successful training, therefore, we introduce Soft Truncation, a universally applicable training technique for diffusion models, that softens the fixed and static truncation hyperparameter into a random variable. In experiments, Soft Truncation achieves state-of-the-art performance on CIFAR-10, CelebA, CelebA-HQ 256×256, and STL-10 datasets.-
dc.languageEnglish-
dc.publisherInternational Conference on Machine Learning-
dc.titleSoft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation-
dc.typeConference-
dc.identifier.wosid000900064901014-
dc.type.rimsCONF-
dc.citation.publicationnameThe 39th International Conference on Machine Learning, ICML 2022-
dc.identifier.conferencecountryUS-
dc.identifier.conferencelocationBaltimore, Maryland-
dc.contributor.localauthorKang, Wanmo-
dc.contributor.localauthorMoon, Il-Chul-
dc.contributor.nonIdAuthorSong, Kyungwoo-
Appears in Collection
MA-Conference Papers(학술회의논문)IE-Conference Papers(학술회의논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0