Active forensic method for spherical panorama and passive forensic method for deepfake

Abstract

Novel types of media content are emerging with constant advancements in technology. The development of deep learning technology has given rise to deepfake technologies that synthesize highly natural human faces. The spherical panorama is receiving much attention as next-generation content, due to improvements in computing and network speeds. However, compared to the quantitative and qualitative growth of new types of content, such as deep fakes and spherical panoramas, technology that addresses the social problems caused by these new developments is lagging far behind. Deepfakes are spawning social problems, including pornography, fake news, and destruction of evidence. As such, various techniques to detect them are currently being investigated. Deepfakes are still difficult to detect, but easy to bypass the detection. Spherical panorama content has high value compared to existing contents. However, there are not many studies on technology that can protect the copyrights of spherical panorama content. Furthermore, leakage and protection scenarios have not yet been analyzed. This paper focused on analyzing the leakage and protection scenarios that may occur in the process of providing a spherical panorama content service. A watermarking method, which is an active forensic technique for protecting the copyrights of such content, is proposed. Spherical panoramic image watermarking technology must be capable of inserting a watermark in the source image as well as be able to detect the watermark in the perspective image that is rendered from the source image. To bypass various distortions between the source and perspective images, the watermark was detected after the perspective image was restored into an equirectangular formed image. In this paper, a deep learning model based on a convolutional neural network was used to restore the perspective image to an equirectangular formed image without the original source image. For the insertion and detection of the watermark, the magnitude coefficient of the discrete Fourier transform was used as the domain. The magnitude coefficient of the discrete Fourier transform has the following property: The coefficient value is not altered by the horizontal and vertical movements of the image. A robust watermark technique that protects against viewpoint desynchronization is proposed based on this unique property. Identifying the accurate viewpoint from the perspective image is a huge advantage in the spherical panoramic image watermarking technique. In addition, the technique’s robust defense against viewpoint desynchronization makes it possible to use a continuous viewpoint system to detect a watermark in a spherical panoramic image. A template method is also proposed to improve the system’s resistance against spherical angle translation attacks. This paper also proposes a passive forensic method that detects deepfakes without any prior information being inserted. Various deepfake detection techniques have been introduced

however, detecting all types of deepfake images with a single model remains a challenge. A technique for detecting various types of deepfake images is proposed. This technique involves the utilization of three common traces generated by deepfake: residual noise, warping artifacts, and blur effects. A network designed for steganalysis was adopted to detect pixel-wise residual noise traces. Moreover, landmarks, which are locations on the face where unnatural deformations primarily occur in deepfake images, to capture high-level features. Lastly, because the effect of a deepfake is similar to that of blurring, features from various image quality measurement tools that can capture traces of blurring were applied. The experimental results demonstrated that each detection strategy is efficient

the performance of the proposed network is stable as well as superior to that of existing detection networks on various deepfake type datasets.