Disclosed are a method and an apparatus for controlling a quadrature amplitude modulation-filter bank multi-carrier (QAM-FBMC) system. A method of controlling interference is performed by a reception apparatus of the QAM-FBMC system, wherein the reception apparatus is paired with a transmission apparatus. The method includes receiving a pre-coded data symbol; and removing residual interference caused due to a non-orthogonal filter from the pre-coded data symbol by using a decision feedback equalizer.