Polymodal embeddings for the multifaceted nature of word meanings

Abstract

A distributed representation has become a popular approach to capturing a word meaning. However, while humans tend to use more diverse sources such as perception, cognition, and emotion in developing a word meaning, it is hardly captured by current amodal or bimodal approaches. In this paper, we propose polymodal embeddings for such multifaceted nature of word meanings. In particular, we attempt to integrate several aspects of word meanings using additional data based on the human cognition model. We show that the proposed method outperforms the state-of-the-art baselines in word similarity task and hypernym prediction task. Also, we investigate which aspects of word meanings are not sufficiently reflected by the embedding and suggest a solution. Finally, we computationally show the different characteristics of concrete words and abstract words and the difference between the word similarity and the word relatedness.