Although quantitative and qualitative growth of spherical panorama content has been achieved, watermarking methods for copyright protection are still insufficient. The spherical panorama image watermarking technique must be able to detect a watermark from the viewed images, but it is difficult to detect the watermark directly from the images due to various distortions. We propose a method for recovering the watermark-detection target viewed image to an equirectangular-formed source image and detecting the watermark after recovery. For recovery, we propose combining the scale-invariant feature transform point matching and the shift and rotation estimation method using a Euclidean transformation matrix. For watermark embedding and detection, a discrete Fourier transform magnitude coefficient is used. With the shift-invariant characteristic, our proposed method is robust against viewpoint desynchronization. Because finding an accurate viewpoint is difficult, our method offers a substantial advantage in the spherical panorama image watermarking scheme. Furthermore, it allows watermark detection on a continuous viewpoint spherical panorama system. In addition, our experiments reveal that our method is robust against signal attacks, such as JPEG compression, Gaussian filter blurring, noise addition, and histogram equalization.