Color generation for paragraph level of text

Abstract

Colors reflect the semantics of language. There have been several studies to try to learn the relationship between color and word/phrase/sentence level of text. In this paper, I expand that relationship to go beyond word/phrase/sentence to paragraph. As a dataset of paragraphs and colors, I use books descriptions and colors of cover images. I modify the recurrent neural networks with hierarchical attention to generate color distributions for paragraph. Three types of studies are conducted on the results of my model: quantitative study, qualitative study, and human annotation study. I show that my model performs better than other baseline models in predicting the color distributions from the quantitative study. In the qualitative study, I visualize the results generated from my model and analyze how the model understands the main objects and the atmosphere in the given text. I carry out a human subject study and show that the subjects prefer the results of my model compared to the baseline model for representing the colors of a book based on the description. Additionally, by visualizing color palettes of word/phrase/sentence level of text, the proposed model shows that it is also possible to generate colors in different levels of text. I also apply my model to movie dataset and show the scalability of my research on various types of color-text datasets.