This letter presents a fast motion vector refinement (MVR) for video downsampling in the discrete cosine transform (DCT) domain. To reduce the computational complexity of the MVR, we propose a fast search algorithm, based on minimizing the number of required checkpoints, and a computationally efficient method for extracting the motion-compensated DCT (MC-DCT) block. Compared to the known extraction method of the MC-DCT block, the proposed method requires fewer than one-half the multiplications and additions.