Global Optimization Techniques Based on Swarm-intelligent and Gradient-free Algorithms

Abstract

The need for solving nonlinear optimization problems is pervasive in many fields. Particle swarm optimization, advantageous with the simple underlying implementation logic, and simultaneous perturbation stochastic approximation, which is famous for its saving in the computational power with the gradient-free attribute, are two solutions that deserve attention. Many researchers have exploited their merits in widely challenging applications. However, there is a known fact that both of them suffer from a severe drawback, non- effectively converging to the global best solution, because of the local “traps” spreading on the searching space. In this article, we propose two approaches to remedy this issue by combined their advantages. In the first algorithm, the gradient information helps optimize half of the particles at the initialization stage and then further updates the global best position. If the global best position is located in one of the local optima, the searching surface’s additional gradient estimation can help it jump out. The second algorithm expands the implementation of the gradient information to all the particles in the swarm to obtain the optimized personal best position. Both have to obey the rule created for updating the particle(s); that is, the solution found after employing the gradient information to the particle(s) has to perform more optimally. In this work, the experiments include five cases. The three previous methods with a similar theoretical basis and the two basic algorithms take participants in all five. The experimental results prove that the proposed two algorithms effectively improved the basic algorithms and even outperformed the previously designed three algorithms in some scenarios.