Caudal keels, a pair of lateral keel-like structures in tunas along the caudal peduncle, are a remarkable specialization. Although various hypotheses about the function of caudal keels have been proposed, our understanding of their underlying hydrodynamic mechanism is still limited. The penalty immersed boundary method was used to explore the hydrodynamics of a self-propelled flexible plate with a keel-like structure on the leading edge of the plate in an effort to understand the role of the caudal keel in nature. The clamped leading edge of the flexible plate was forced into a prescribed harmonic oscillation in the vertical direction but was free to move in the horizontal direction. For comparison, simulations without a keel were also performed. Vortical structures and pressure distributions were visualized to characterize the hydrodynamic benefits of the keel. The keel generates streamwise vortices that result in negative pressure and enhance the average cruising speed and thrust. The underlying propulsion mechanism was analyzed in detail by examining the phase of the heaving stroke. The average cruising speed and the propulsion efficiency are increased by more than 11.0% and 6.7%, respectively, by the presence of the keel. A parametric study was performed to determine the set of parameters of the keel that maximizes the propulsion efficiency eta as a function of the reduced length (l/L) and the reduced height (h/L) of the keel.