Using the Cuckoo Search for Generating New Particles in Particle Swarm Optimization Algorithm

Corresponding Author: Fariaa Abdalmajeed Hameed Technical College of Informatics, Sulaimani Polytechnic University, Sulaimani, Iraq Email: fariaa.hameed@spu.edu.iq Abstract: This study is focused on as Cuckoo Search (CS), one of the current meta-heuristic optimization algorithm. The CS algorithm is useful in generating and searching for the most optimum particles of important meta-heuristic optimization algorithm, known as the Particle Swarm Optimization (PSO), to enhance its performance. This optimization is confirmed through a benchmark online optimization and actual problems. The PSO algorithm performance is also compared with differing algorithms representative of the area. The CS optimal solutions outperform alternative current solutions as CS has distinct search features. The study findings have implications for future studies and practice.


Introduction
In the current business landscape, there exist several optimization opportunities, with some of the top being the optimizing schedules and workflow for the mitigation of resources (cost and time) and maximization of output.
In this regard, simple exhaustive search constitute one of the pioneering optimizer types, with every potential combination presented to obtain the best one. This type of optimization is known for its accuracy as they best combination are eventually obtained but its efficiency is quite low because if there are more than a few thousand combinations, it takes considerable time to examine all of them. This is the reason behind the limiting of exhaustive search optimizers of the number of variables employed, or limiting the number of variable values-but the best alternative is by using Cuckoo Search (CS). Specifically, CS is deemed to an evolutionary optimization algorithm developed and presented by Yang and Deb (2009;Yang, 2011). CS primarily refers to the cuckoo bird, where the actual bird takes advantage of some other bird species by laying its eggs in their nests (Payne et al., 2005). In this case, every egg is a representation of a new solution-with the objective being the alternative use of new and potentially better solutions than those in existing ones. CS algorithm is prominent and popular owing to its simplicity.
Contrastingly, conventional optimization algorithms entail the use of traditional methods like the dynamic programming, branch-and-bound and gradient-based techniques whereas their modern counterparts involve meta-heuristics searching. Meta-heuristics algorithms include simulated annealing, evolutionary computation EC, colony optimization, among others.
In the past few years, evidence shows that metaheuristic algorithm has been understudied although its combination with other optimization methods could create a robust system that could handle actual large scale problems. This holds true for the meta-heuristic algorithm branch known as the Particle Swarm Optimization (PSO). PSO refers to a stochastic search procedure that has its basis on social behavior observations (e.g., flocking birds and schools of fish). This algorithm type evaluates search space in a simultaneous manner through the use of global information, which is why it has a higher likelihood to determine global solution to a specific issue.

Previous Studies on CS Application
Under this section the CS application findings in prior studies are presented and discussed. On the basis of the reviewed work, there are several categories using CS and they include the field of engineering (Vaisakh et al., 2009;Fan and Zahara, 2007;Omran et al., 2005;Price et al., 2005;Qin et al., 2009), pattern recognition (Kennedy et al., 2001;Potter et al., 2010;Xin et al., 2011;Storn and Price, 1997), software testing and data generation Storn and Price, 1997;Hao et al., 2007), networking (Shelokar et al., 2007;Pant et al., 2008), job scheduling (Fu et al., 2010;El Dor et al., 2012) as well as data fusion and wireless sensor networks (Xin et al., 2010;Akbari and Ziarati, 2008).
More specifically, a new diagnosis technique that used CS in engineering was adopted in (Yang and Deb, 2010) to resolve issues of engineering design optimization, with the inclusion of springs design and welded beam structures. The optimization objective in this case is to mitigate the spring weight and the overall cost of fabrication. The findings were benchmarked with evolutionary methods (e.g., GA and PSO) and were evidenced to be quite efficient and superior in almost the entire issues tested. In (Vazquez, 2011), the spiking neuron's accuracy and performance was measured using pattern recognition/classification fitted with CS Algorithm and a comparison was later conducted between CS and DE algorithms performance. Based on the outcome, spiking neuron model trained with CS outperformed in the adjustment of synaptic neuron weights.
Moving on to (Bacanin, 2012), the author focused on process parameters including, maximum cycle number of nests, runtime, number of parameters and probability and their role in optimization of process performance measures. The study employed CS in object-oriented software to address issues of unconstrained optimizationcombinational and numeric optimization issues in JAVA programming language. Based on the outcome of the benchmark test, the proposed software had good performance, indicating the readiness of the system to be used to solve new problems. Natarajan and Subramanian (2012), the author brought forward Enhance Cuckoo Search for Optimization of Bloom Filter in Spam Filtering. The study used an Enhanced Cuckoo Search (ECS) algorithm to mitigate the total membership invalidation cost of BFs, within which the optimal false positive rates are determined along with the number of elements within each of the bins. Based on the experimental outcome, for CS and ECS outperformed in numbers of bins ECS. In a related study, (Burnwal and Deb, 2012) brought forward Cuckoo Search for the purpose of scheduling optimization of flexible manufacturing system by decreasing the penalty cost and increasing the machine use time as used performance. The finding was compared to evolutionary techniques findings (e.g., GA) and it indicated the efficient and better performance of CS when solving tested problems.
Moreover, the Cuckoo Based Particle Approach (CBPA) was also used in (Dhivya et al., 2011) to realize energy efficiency in Wireless Sensor Network and multimodal objective functions. CS was used to cluster head selection and form clusters among the sensor nodes, after which the authors measured the decrease in energy of Wireless Sensor Networks (WSNs) and the increase in the lifetime through performance. The CBPA was then compared with the standard LEACH protocol and HEED protocol. The results of the simulation illustrated that CBPA generated comparable results owing to the process of optimal search in cluster formation and appropriation of paths in transmitting sensed data.
Along the same line of study, (Yildiz, 2012) made use of CS for the selection of optimal parameters in the context of milling operations, (Chifu et al., 2012) optimized the composition processes of semantic web service with the assistance of CS, while (Kumar and Chakarverty, 2011) realized optimal design for reliable integrated system. Lastly, Walton et al. (2011) modified CS to solve non-linear issues (e.g., mesh generation).

Overview of Cuckoo Search Algorithm
Cuckoo Search (CS) is described as a novel metaheuristic algorithm created to issues relating to optimization, aligned with its namesake, which is the parasitic behavior of cuckoo species, Levy flight behavior and fruit flies. Specifically, in CS, the cuckoo's walking steps are determined by the flights, with each egg representing a new solution and the objective being to utilize optimum ad better solutions replacing the ineffective ones in the nest, with each nest having one egg. It is possible to extend the algorithm to more complicated cases, within which each nest has many eggs (i.e., many solutions). For instance, when producing new solutions x (t+1) for cuckoo I, a Levy flight is performed using the following equation: In the above equation, α > 0 represents the step size that is related to the examined problem scales, where in majority of cases α = 1 can be used. The product ⊕ represents entry wise multiplications, whose product is similar to the PSO, with the difference being that the random walk through Levy flight works more efficiently in terms of exploration of search step as the length of the step is longer in long-term. In essence, the Levy flights provide a random walk represented as follows: The above equation is an infinite variance, having an infinite mean. Figure 1 demonstrates the pseudo code of CS. Parameters: n represents the number of hosts nests, P a represents the probability of discovering an alien egg, MaxIter represents the maximum number of iterations, Initialization, Generate initial n host, X i ): Replace (x (T) ) with (x (t+1) ) Discard a fraction of p a worse nests, Develop new nests with Levy flights, Identify and keep the best solutions

Overview of Particle Swarm Optimization (PSO)
The Congress on Evolutionary Coputation (Kennedy and Eberhart, 1995) presented a paper on PSO in 1995, triggering waves of publications in the past decade on the different applications success of PSO to resolve issues concerning optimization, as inspired by the flocking and foraging behavior of birds and fish (Brownlee, 2011). Such attributes are highly desirable, easily understandable and implemented. PSO is mainly used for its timely convergence, particularly when pitted against other optimization algorithms (e.g., simulated annealing and genetic algorithms) as illustrated in (Abraham et al., 2006).
The appeal of PSO stems from its simple conceptual framework and the birds flocking analogy facilitating conceptual visualization of the search process. In particular, a solution in PSO is displayed through a particle, with the population of solutions referred to as swarm of particles and each particle having two major properties, namely position and velocity. Added to this, each particle shifts to a new position with the help of velocity and once they are settled in a new position, the best position of the particle and swarm are made current as required. Each particle velocity is modified on the basis of its experiences.
In other words, the velocity (V i ) of each particle is updated with the help of the following equation: begin Objective function f(x) Generate initial population of n host nest Evaluate fitness and rank eggs while (t > MaxGeneration) or stop criterion t = t +1 Get a cuckoo randomly/generate new solution by lèvy flights Evaluate quality/fitness, F i Choose a random nest j if (F i > F j ) Replace j by the new solution end if Worst nest is abandoned with probability P a and new nest is built Evaluate fitness and rank the solutions and find current best end while Post process results and visualization end Repetitive steps of the process are carried out until a criterion is met. In the first step, PSO is initialized, where the initial swarm of particles is produced and each particle initialized at random position as well as velocity, after which each particle is evaluated for their fitness. Each calculation of the fitness value is compared against the prior optimum one of the particle and the prior optimum value of the whole swarm. From this, the personal best and global best positions are updated accordingly.
In case a stopping criterion falls short of being met, the velocity and position are adjusted to create a new swarm and the personal and global best positions and old velocity used in the update of velocity. The two main PSO operations are velocity update and position update, with the former being based on three components, namely, the old velocity constituting inertia/momentum term, experience of individual particle constituting cognitive or self-learning term and experience of the whole swarm constituting group/social learning term. In each term, a weight constant is allocated and for the fundamental PSO algorithm, there are three required constants. Notably, PSO algorithm is not in need of sorting of fitness values of solutions in any step. The pseudo-code of PSO is depicted in Fig. 2.

Particle Swarm Optimization/CS: PSO/CS and Frameworks
The proposed algorithm primarily aims to improve the PSO algorithm performance to achieve optimum solutions in comparison to standard PSO and CS algorithms with lower execution periods compared to standard PSO. Both CS and PSO algorithms have a stochastic nature and as such, are invaluable in achieving global optimum compared to their gradient descent counterpart. However, it is simple for the former to drop into the local optima with unsatisfactory convergence accuracy.  30 [-30,30] In order to leverage the advantage using the best of CS and PSO to enhance optimization performance, the new design preservers the PSO algorithms. It uses CS to produce the initial values of the particles and the velocity vectors rather than randomly generating them with high cost and significant period of time, or to face the risk of being trapped in the local optima-thus, the CS strategy initialization of the PSO particles and velocity is conducted to improve PSO performance. The usual PSO algorithm proceeds right after. The pseudo code of CS/PSOE is demonstrated in Fig.  3, where it is notable that the particles positions are produced through CS algorithm after which they are updated normally using PSO foe each particle, evaluated on the basis of their fitness value. Lastly, the optimum solution is identified by the algorithm.

Experimental Results and Discussion
The CS/PSO performance in light of minimization and maximization benchmark functions are chosen, with the inclusion of 4 actual problems (section 6.1 details the benchmark).  x

Results and Discussion
The algorithm brought forward in this study is compared with both CS and PSO standard algorithms, with the parameters of PSO established as number of particles and number of iterations during the run C1, C2 = 1.19, R1, R2 = 1. With regards to the CS parameters, they were established as population size and number of iteration and problem dimension during the run, with maximum population size of 100, Cr = 0.5 and F = 0.7. Each problem has a distinct average of the best value for 20 run times and maximum number of functions evaluations (Nb.evals) (refer to Tables 2 and 3).
On the basis of the experiments, it is notable that CS/PSO managed to achieve the optimum outcomes on majority of the problems posed, with the algorithm leading to significant developments compared to prior PSO and CS, with CS/PSO evidently outperforming all other algorithms tested on Figures 4, 5, 6 and 7. Figure 8 illustrates the optimum solution for CS/PSO algorithm, with iteration 4000 and 500 and domain (n = 30), for SpShpere function. Figures 9 and Fig. 10 illustrate the minimum result for the proposed algorithm CS/PSO, with iteration 3000 and 4000 and domain (n = 30), for both the Rastrigin function and Ackley function. The best value was obtained with iteration 1500 for Rn Rosenbrock, with domain {n = 30). Thus, performance was good in the overall prior functions.

Conclusion
The performance of PSO can be enhanced by achieving the optimum values and accordingly, we used the CS algorithm in this study to provide a balance between exploitation and exploration. The new method proposed, which is CS/PSO enhances the performance of particle swarm optimization through the incorporation of CS algorithm. The primary objective is to use the proposed algorithm to produce PSO particles, with the help of CS algorithm to improve the PSO searching abilities. The approach achieved outstanding results on 8 well-known benchmark functions, with the inclusion of 4 actual life problems. The results of the computations indicated that the proposed approach provided accurate outcomes in comparison to its evolutionary counterparts.