(A) study on image quality estimation based on deep neural networks using visual quality perception characteristics

Abstract

In image quality prediction, it is the most important to understand how humans perceive the processed images since human observers are the ultimate receivers of the images. Thus, objective image quality assessment (IQA) methods which predict the quality of images based on the human visual quality perception have been extensively studied. However, after the successful usage of deep convolution neural networks (CNNs), many IQA methods have applied the black box approach, in which CNN automatically learns useful information from the IQA database and performs predictions without considering any human visual perception characteristics. In this paper, we propose a novel deep neural network that performs the image quality prediction based on human visual perception characteristics. The proposed deep neural network is called Deep HVS-IQA Net. Different from previous CNN based IQA models, we directly apply human psychophysical characteristics to the training of proposed Deep HVS-IQA Net using a distortion detection probability map from a just noticeable difference (JND) model and a saliency map which extracts important objects from the image. Also, we trained Deep HVS-IQA Net by applying a newly proposed rank loss which adds additional losses if predicted quality score orders are different from reference quality score orders so that the image quality prediction is stably performed. Furthermore, we applied the channel attention mechanism to our network so that the network can focus on more informative channels in the feature maps. We evaluate the proposed Deep HVS-IQA Net on the large IQA databases and show the state-of-the-art quality prediction accuracy among all IQA models.