Dynamic positron emission tomography (PET) is widely used to measure changes in the bio-distribution of radiopharmaceuticals within particular organs of interest over time. However, to retain sufficient temporal resolution, the number of photon counts in each time frame must be limited. Therefore, conventional reconstruction algorithms such as the ordered subset expectation maximization (OSEM) produce noisy reconstruction images, thus degrading the quality of the extracted time activity curves (TACs). To address this issue, many advanced reconstruction algorithms have been developed using various spatio-temporal regularizations. In this paper, we extend earlier results and develop a novel temporal regularization, which exploits the self-similarity of patches that are collected in dynamic images. The main contribution of this paper is to demonstrate that the correlation of patches can be exploited using a low-rank constraint that is insensitive to global intensity variations. The resulting optimization framework is, however, non-Lipschitz and non-convex due to the Poisson log-likelihood and low-rank penalty terms. Direct application of the conventional Poisson image deconvolution by an augmented Lagrangian (PIDAL) algorithm is, however, problematic due to its large memory requirements, which prevents its parallelization. Thus, we propose a novel optimization framework using the concave-convex procedure (CCCP) by exploiting the Legendre-Fenchel transform, which is computationally efficient and parallelizable. In computer simulation and a real in vivo experiment using a high-resolution research tomograph (HRRT) scanner, we confirm that the proposed algorithm can improve image quality while also extracting more accurate region of interests (ROI) based kinetic parameters. Furthermore, we show that the total reconstruction time for HRRT PET is significantly accelerated using our GPU implementation, which makes the algorithm very practical in clinical environments.