Application specific instructions play an important role in reducing the required code size and increasing performance in embedded DSP systems. This paper describes a new approach to generate application specific instructions for DSP applications. The proposed approach is based on a modified subset-sum problem and supports multicycle complex instructions, as well as single-cycle instructions, while the previous state-of-the-art approaches generate only the single-cycle instructions or just select instructions from the fixed super-set of possible instructions. In addition, the proposed approach can also be applied to the case that instructions are predefined. Experimental results on real applications show that various given constraints can be met by the generated set of application specific instructions without attaching special hardware accelerators.