Phonons at gigahertz frequencies interact with electrons, photons, and atomic systems in solids, and therefore, have extensive applications in signal processing, sensing, and quantum technologies. Surface acoustic wave (SAW) resonators that confine surface phonons can play a crucial role in such integrated phononic systems due to small mode size, low dissipation, and efficient electrical transduction. To date, it has been challenging to achieve a high quality (Q) factor and small phonon mode size for SAW resonators at gigahertz frequencies. We present a methodology to design compact high-Q SAW resonators on lithium niobate operating at gigahertz frequencies. We experimentally verify designs and demonstrate Q factors in excess of 2x10(4) at room temperature (6x10(4) at 4 Kelvin) and mode size as low as 1.87 lambda(2). This is achieved by phononic band structure engineering, which provides high confinement with low mechanical loss. The frequency Q products (fQ) of our SAW resonators are greater than 10(13). These high-fQ and small mode size SAW resonators could enable applications in quantum phononics and integrated hybrid systems with phonons, photons, and solid-state qubits.