Resolution improvement of infrared images using high-resolution visible image information

Abstract

Infrared (IR) images provide valuable information for many applications such as surveillance and remote sensing. However, compared to a visible image, the quality of an IR image is usually poor and its spatial resolution is limited. The main reason for the quality and resolution degradation of an IR image is blurring effects due to non-ideal optics and finite detector size. Therefore, improving the quality and/or the spatial resolution of IR images is an important issue. This dissertation deals with a resolution improvement algorithm of an IR image. To improve the quality and resolution of an IR image, conventional restoration-based algorithms model an IR image acquisition process mathematically and solve an inverse problem. Therefore, the quality of the restored image relies on the modeling accuracy of the degradation process that is usually difficult to achieve. In addition, some high-frequency (HF) components cannot be recovered and ringing artifacts near strong edges become noticeable even with the use of a constraint term in the restoration process. Meanwhile, in some applications such as medical and range imaging, to improve an image captured from a sensor, an image obtained from a different sensor is also utilized by examining their statistical relationships between intensity values or the co-occurrence of discontinuities. This dissertation presents a new IR image resolution improvement framework based on the information of the aligned high-resolution visible image. In the proposed framework, we first estimate HF patches for edge regions in an IR image, which have a strong correlation with the aligned edges in the visible image. The estimation is performed on the basis of local intensity correlations between two images. By utilizing the relationship between the IR patches for correlated edges and their estimated HF patches, we then determine multiple HF patch candidates for uncorrelated edges. Finally, we suggest an optimization procedure to improve re...