Positron emission tomography (PET) images usually suffer from low spatial resolution mainly because of the finite dimension of crystals. To improve the spatial resolution based on wobble scanning, we previously proposed a sinogram-based super-resolution (SR) algorithm using a space-variant blur matrix. However, the algorithm may cause unwanted resolution loss owing to an inevitable interpolation process for preparing evenly spaced projections. In this article, we propose a novel one-step line of response (LOR)-based SR framework for 3D PET images. In the framework, we efficiently determine a large number of space-variant point spread functions (PSFs) in the image space by using the scanner symmetries and the proposed PSF interpolation scheme based on nonrigid registration. Furthermore, to minimize the resolution degradation in the evenly spaced parallel-beam rebinning and to reduce the computational time in the graphics processing unit (GPU) implementation, we introduce parallel-friendly LOR reconstruction based on cone-beam reordering. We then obtain a high resolution image via a one-step super-resolved 3D PET image reconstruction with the determined PSFs. The proposed framework provides noticeable improvement on the spatial resolution of PET images with a considerable reduction of computational time.