A hybrid optimization method of evolutionary and gradient search

Abstract

This article proposes a hybrid optimization algorithm, which combines evolutionary algorithms (EA) and the gradient search technique, for optimization with continuous parameters. Inheriting the advantages of the two approaches, the new method is fast and capable of global search. The main structure of the new method is similar to that of EA except that a special individual called the gradient individual is introduced and EA individuals are located symmetrically. The gradient individual is propagated through generations by means of the quasi-Newton method. Gradient information required for the quasi-Newton method is calculated from the costs of EA individuals produced by the evolution strategies (ES). The symmetric placement of the individuals with respect to the best individual is for calculating the gradient vector by the central difference method. For the estimation of the inverse Hessian matrix, symmetric Rank-1 update shows better performance than BFGS and DFP. Numerical tests on various benchmark problems and a practical control design example demonstrate that the new hybrid algorithm gives a faster convergence rate than EA, without sacrificing the capability of global search.