Personalized image aesthetic quality assessment by joint regression & ranking

Abstract

We propose an image aesthetic quality assessment algorithm, which considers personal taste in addition to generally perceived preference. This problem is formulated by a combination of two different learning frameworks based on support vector machines - Support Vector Regression (SVR) and Ranking SVM (R-SVM), where SVR learns a general model based on public datasets and R-SVM adjusts the model to accommodate personal preference obtained from user interactions. The combined framework, called R-SVR, is represented by a single objective function, which is optimized jointly to learn a model for personalized image aesthetic quality assessment. For the optimization, we use only a small subset of public dataset identified by k-nearest neighbor search instead of using all available training data. This strategy is useful in practice because it reduces training time significantly and alleviates data imbalance problem between regression and ranking. The proposed algorithm is tested through simulation and user study, and we present that our interactive learning algorithm by R-SVR is effective to increase user's satisfaction and improve prediction performance.