Domain adaptation in sentiment classification based on probabilistic models

Abstract

Sentiment classification is a task to determine overall contextual polarity of a review document. Sentiment classification can be used for a company to check the problem of their products or services from the large data. It also can be used for customer to decide the products or services they would consume. There are two main difficulties dealing with sentiment classification. First, the documents are usually represented as a bag-of-words model and the dimension of such document data is very large, so we need methods to extract or reduce the number of dimension. Secondly, if the domain is different for training data and testing data, the performance decreased severely. However, it is hard to get the labeled data for the all the domain we are interested in. To extract or reduce the dimension, we tried three methods: principal component analysis (PCA), conditional entropy (CE), and independent component analysis (ICA). We can reduce the dimension using PCA without any loss of information. By changing the estimation of probability a little bit, we are able to achieve more balanced estimation of CE, which gives robust recognition through different number of features we selected. ICA can make the features independent, so that it was expected to give better result when we used it with CE. However, experiments suggest that ICA is not useful for CE. To resolve the problem of domain difference, we propose domain adapting Boltzmann machine algorithm. The big difference between domains comes from the word dictionary used for each domain. So we take the approach to generate target domain words that are not appearing in source domain, and vice versa. In this thesis, we first applied this idea to simple toy problem and then real world problem. We improved the classification accuracy using our algorithm.