A SURVEY: PARTICLE SWARM OPTIMIZATION-BASED ALGORITHMS FOR GRID COMPUTING SCHEDULING SYSTEMS

Bio-inspired heuristics have been promising in solving complex scheduling optimization problems. Several researches have been conducted to tackle the problems of task scheduling for the heterogeneous and dynamic grid systems using different bio-inspired mechanisms such as Genetic Algorithm (GA), Ant Colony Optimization (ACO), Particle Swarm Optimization (PSO). PSO has been proven to have a relatively more promissing performance in dealing with most of the task scheduling challenges. However, to achieve optimum performance, new models and techniques for PSO need to be developed. This study surveys PSO-based scheduling algorithms for Grid systems and presents a classification for the various approaches adopted. Metatask-based and workflow-based are the main categories explored. Each scheduling algorithm is described and discussed under the suitable category


INTRODUCTION
Grid computing emerges to address the need to provide users, such as engineers, scientists, city planners, with more computing power (Foster and Kesselman, 2004). It creates a collaborative infrastructure where users with their computational facilities, distributed in different geographical locations, are interconnected via a high-speed internet. The goal this infrastructure is to share resources among the participating users, which enabling them to access high computing power that cannot be provided ordinarily by an individual. Consequently, the high computing power is applied to solve sophisticated problems in various disciplines (Zhong et al., 2005;Lecca et al., 2011;Tao et al., 2011a;Kayser et al., 2011;Valdivia et al., 2012).
The fundamental pillar of Grid efficiency is its task scheduling subsystem, whose function is to efficiently assign tasks to resources. Grid environment is dynamic as resources join and leave the Grid and the resources are heterogeneous as they are from different computing domains and subject to different policies. These dynamism and heterogeneity factors impose additional challenges to scheduling and hence the conventional techniques cannot be directly use. Moreover, grid applications are basically categorized in to two (Sinnen, 2007); meta-task applications and DAG applications. The former category describes independent tasks which have no priority relations and data communication among them while the later is otherwise known as workflow applications and describes applications that are represented as Directed Acyclic Graph (DAG), in which the nodes of the graph represent the tasks and the directed edges represent the dependences among the tasks. Scheduling DAG tasks is more challenging as the dependencies among the tasks have to be considered (Yu and Buyya, 2005;Cao et al., 2003).
Generally, the goal of mapping computational tasks to grid resources is to achieve optimal schedule and this problem has been proven to be NP-complete (Ullman, 1975) which means the execution of the algorithms

PARTICLE SWARM OPTIMIZATION (PSO)
PSO was originally introduced in (Kennedy and Eberhart, 1995). It is a population-based heuristic strategy for solution search. It belongs to the class of swarm intelligence and share with Genetic algorithm the feature of starting with random solutions. However, it is distinct in that each of its solutions, known as particle, is associated with position and velocity vectors. In PSO, the randomly generated particles are initialized with random positions and velocities. Each of the particles flies through multidimensional solution-search space and, at eachiteration, its velocity is dynamically adjusted according to its previous best position and best positions of others. Unlike Genetic Algorithm, each particle remembers its best position so far. In other words, each particle's best position is stored in its memory, which is regarded as experience. The experience is replaced when new experience is discovered in its search process. PSO was initially designed to simulate flock of birds seeking for food. A scenario of group of birds randomly searching for food in an area demonstrates the PSO process (Raja and Baskar, 2012). In this analogy, there is only one piece of food in the area being searched. The birds do not know where the food is. But they know how far the food is and their peer's positions. So what is the best strategy to find the food? An effective strategy is to follow the bird which is nearest to the food. The goal of each particle is to attain the global best position before convergence. The PSO optimization process is driven by the velocity vector (Engelbrecht, 2005), which is modeled as follows Equation 1 and 2: x (t)) c r (y (t) x (t)) Where: w = The inertia weight t = The time step c 1 and c 2 = Are two positive constant r 1 and r 2 = Are random numbers v ij = The velocity of particle i in dimension j x ij = The position of particle i in dimension j y ij = The particle i's best position so far in dimension j ŷ j = The best position so far among the swarm in dimension j

PSO-BASED SCHEDULING TECHNIQUES IN GRID
Generally, the problem of grid task scheduling is finding an assignment scheme that maps grid jobs to heterogeneous grid resources while achieving certain desired objective(s). This problem has been proven to be an NP-complete and can therefore be best dealt with using meta-heuristics algorithsms.
Due to its fast convergence, simplicity and ease of implementation, PSO has drawn much attention from various fields. The style of its operation has made it suitable for dealing with task scheduling problem. A substantial number of PSO-based grid scheduling works have been proposed. In this study, we surveyed the scheduling works from the last 8 years. The annual number of published papers (based on IEEE, Elsevier and Springer databases), in which PSO is applied in various areas of this field, is depicted in Fig. 1.

Classification of PSO-based Scheduling Techniques
The PSO-based scheduling techniques proposed in literature can be generally classified based on the particle encoding, particle updating method and scheduling objective or fitness evaluation. In this section, we discuss the different classifiications outlined.

Solution Representation
To apply PSO in scheduling, typically the first step is to randomly generate the initial scheduling solutions, which are then encoded to form the initial PSO swarm. In other words, each scheduling solution is represented by a particle in the initial swarm. The representation of the particles is one of the key determining factors for the successful application of PSO. Hence, many algorithsms vary in the way they represent scheduling solutions as particles.

Particle Updating
Originally, the PSO was developed for continuous numeric optimization problems and hence cannot be directly applied to solve discrete optimization problems such as grid scheduling. That is due to the fact that, in original PSO, the particles' positions are real-valued. The values would therefore need to be converted to integers or the algorithsm must be reengineered to produce integer values, which would be valid for representing number of grid resources. Moreover, many of the algorithsms under study differ in the way the velocities of particles are parameterically modified and updated. Selection of velocity parameters is also critical to the performance of PSO. Another important factor which distinguishes many of the algorithsms is the technique used to assist PSO escape premature convergence which is detrimental to the solution quality of the search. The employed technique is required to provide tradeoff between exploration and exploitation searches. The former explores new regions and once it finds a good one, the later search the local optima.

Scheduling Objectives or Fitness Evaluation
The algorithms proposed in the literature also vary in their scheduling objectives. The objectives are usually set in PSO's fitness function. After every PSO iteration, each task-resource mapping solution is evaluated based on the metrics in the fitness funtion until the fittest solution cannot be improved further. Scheduling problem is usually single, dual or multi-objective. However, most of the recent proposals in the literature are multiobjective. One of the criteria for measuring the strength of an scheduling algorithm is its ability to deal with multi-objective problems with less complexity. Some of the scheduling objectives in the studied literautre are minimizing makespan, improving reliability, minimizing energy consumption, load balancing. The distribution of these scheduling objectives is shown in Fig. 2.

PSO-BASED SCHEDULING ALGORITHMS IN GRID
In this paper, some of the PSO based grid-task scheduling heuristics are discussed. The aim is to describe each algorithm and explore the various techniques adopted in the use of PSO to solve task scheduling problems. In the next subsections, the scheduling algorithms are discussed.

PSO-based
Meta-task Scheduling Algorithms Liu et al. (2010) proposed a scheme to solve metatask scheduling problem. In this approach, fuzzy matrices are used to represent positions and velocities of particles in PSO. The elements in each matrix represent fuzzy relations between grid nodes and jobs.

Fig. 2. Distribution of Scheduling Metrics
In other words, each element in the matrix denotes the degree of membership that the grid node would process the job in the feasible schedule solution space. Therefore each matrix encodes a potential solution and represents a particle's positions. Each position matrix has corresponding velocity matrix which denotes velocities of the particles. The PSO position and velocity models are used to update positions and corresponding velocities in the matrices. In the first time step of the algorithm one position matrix is generated using LJFR-SJFR heuristic to minimize the makespan and the flowtime. The fuzzy approach outperformed GA and SA when compared. However, studies in (Torkestani, 2012) reported that the fuzzy PSO is less efficient especially in large scale assignment. Izakian et al. (2010) proposed a new matrix based encoding scheme to address the problems of the fuzzy approach. In this scheme, an m x n matrix represents assignment solutions. The position matrix of each particle has the characteristics that all its elements have either 0 or 1 and in each column, only one element is 1, all others are 0. The idea is, each column represents a job allocation and each row represents allocated jobs in a node. Velocities are also represented in matrices and updated using a modified velocity model. This way of encoding is referred as 'indirect'. Additionally, a 'direct' encoding method is proposed which represents particles positions as 1 x n vectors. The two methods are used alternatively depending on certain laid down conditions. It is however indicated that the direct method is more efficient as it is faster. Moreover, it is noted that it will incur system time to convert particles from one encoding to the other. A way out method is provided with which positions can be updated without necessarily undergoing conversion.
Abdelhalim (2008) adopted Re-Excited PSO (Abdelhalim et al., 2007) to solve meta-task assignment problem on heterogeneous processors with energy consideration. The idea of Re-excited PSO is that, the final particle positions, obtained from the PSO run, are reused to start a new run, re-initializing all other variables such as inertia weight, random velocities. As the algorithm runs, the velocity value of each particle is truncated to the nearest integer. This approach is claimed to have the advantage of improving the solution quality as a result of the improved exploration of the solution space. However, result of the study of the algorithm conducted in (Kang and He, 2011) reported that the algorithm's efficiency decreases as the size of the task assignment problem increases due to the updating mechanism. Moreover, the swarm size of 100 as used in the study, is beyond the limit suggested in some studies (Das and Abraham, 2006;Ho et al., 2008), where it is indicated that even though there is a slight improvement to the optimal solution with increasing swarm size, it increases the number of function evaluations to converge to an error limit. Prescilla and Selvakumar (2013) proposed energy aware strategy. The approach addresses the shortcoming of Binary Particle Swarm Optimization (BPSO) which update particle using only its velocity. In their approach, the BPSO is modified by introducing particle's current position as additional factor used to in updating the particle. According to result of their study, the MBPSO performed well in terms of minising energy consumption. However, the algorithm takes a lot of time to execute. Kang et al. (2008) presented a novel approach to address the problems of Re-Excited PSO. In this proposal, a new method is proposed for updating the positions of particles based on the premise that the objective of updating particle is to disturb its own position, to fly towards its personal best and to fly toward the global best. Their idea is further optimized in (Kang and He, 2011) with the use of extension of VNS, Variable Neighborhood Decent (VND) heuristic, for exploration and exploitation process with the goal to improve solution quality. The global best found, after PSO run, is used as input in the VND algorithm and the algorithm attempts to improve it. Moreover, when more and more of the population converge around the global best, thereby leaving other areas unexplored, the approach employed migration mechanism for more exploration. The approach outperformed the Re-Excited PSO. The comparison was, however, against makespan only. Therefore, the performance need to be further evaluated in multi-objective problems.  enhanced the no-velocity PSO scheme to solve meta-task scheduling problem. Novelocity PSO approach is a form of constriction method in which particles do not modify their positions using their associated velocities. Rather, their positions are adjusted using the velocities of their respective personal best positions, velocity of the global best position and the velocity of the swarm's center. In this enhancement, subgroup best position and subgroup's center are adopted instead of the personal best position and swarm's center respectively. Xu-Huai et al. (2009) proposes an Immune Particle Swarm Optimization (IPSO) algorithm. The basic idea of the IPSO is to record the particles with a higher fitness in the evolution process and make the new particles which satisfy neither the assumption nor the constraint condition replaced by the recorded ones. In addition, immune regulation should be done to maintain the species diversity while it decreases. The approach addresses the problem of satisfying user tasks. However, when load is high the strategy is deficient in terms making tradeoff between user task satisfaction and scheduling time. Vidya et al. (2012) presented a strategy based on constriction factor where the constriction is to eliminate the inertia weight. However, this approach deficient in terms of its encoding scheme as particle is represented as entire solution set. Nikkhah et al. (2009) proposed fault tolerant PSO based scheduling scheme in which task replication strategy is employed for fault tolerance. Each particle, in this approach, is encoded as 3D hypercube. The first dimension denotes number of tasks, the second dimension denotes number of resources and the last, whose value is 5, denotes the maximum number of task copies. Each cell in the hypercube indicates the probability of assigning a special copy of a task to a resource. Pooranian et al. (2013) proposed a hybrid mechanism in which Gravitational Emulation Local Search (GELS) algorithm is embedded in PSO, PSO-GELS. GELS is used to address the weakness of PSO in its local search heuristic in order to avoid local optimality. However, the version of GELS used in this work is no longer the better alternative. For instance, the randomized version proposed in (Balachandar and Kannan, 2007) addressed some shortcomings of GELS. Prado et al. (2010) presented a learning strategy based on Fuzzy Rule-Based Sytem that incorporates PSO-based knowledge acquisition which is referred as KASIA. The proposed scheduling system characterizes the grid systems based on some laid down features so as to determine the future state of the grid systems and the result is included in next scheduling decisions. The aim is to improve makespan and in their further work in Prado et al., 2012), the robustness of the approach is studied under other optimization criteria. It is also shown in their comparative study in (Prado et al., 2011), that the approach slightly outperformed Differential Evolution algorithm. However, particle is represented as entire solution set (matrix). This encoding method may incur system memory to store all the matrix components for each particle and accessing such components could be time consuming. Hence, the PSO could be better if a simpler and more efficient encoding approach is used. In the same vein, Hu et al. (2012) presented grid scheduling system that fuse together resource monitoring and prediction subsystems to support instantaneous resource information acquisition. PSO is applied to optimize the accuracy and efficiency of the prediction subsystem where hybridization of continuous and binary versions of PSO is proposed. In their approach, hyperparameter selection and feature selection which are two components of the prediction model are coded as hybrid vector consisting of real and binary numbers respectively representing the two components. The vector is represented as particle during each iteration the hybrid is jointly updated. Zhang et al. (2010) proposed a security aware strategy in which node reliaibility evaluation model is constructed based on the historical behavior of grid node. Based on the node reliability model, a task security evaluation model is established. The PSO features are redefined to smooth application to the Science Publications JCS problem. Huang et al. (2013) also discussed reliability aware algorithm which integrates mutation mechanism into PSO. The mutation strategy is used to help PSO avoid getting trapped in local optima. The authors claim that the algorithm is efficient and reliable. Sadhasivam and Meenakshi (2009) proposed a grouping based load balancing mechanism. Jobs are grouped first and PSO is called to determine to determine optimal schedule of groups and jobs are then dispatched parallelly to their corresponding resources. In the PSO operation, SPV is used to convert contnuos position vector to permutation vector from which operation vector is obtained. Kiyarazm et al. (2011) proposed a strategy to solve load balancing scheduling problem for meta-task. In this strategy, a greedy algorithm which is executed once in their method, is used as optimizer for the initial randomly generated solutions. This is inorder to speedup convergence and improve optimality. Yuan et al. (2010) proposed the use of Quantum PSO to address the problem makespan degradation due to load imbalance. In the Quantum PSO, velocity is eliminated and particles have no fixed positions in the quantum time space. Coimbatore, 2010 integrates GA in PSO and incorporates MapReduce framework to address the problem of unbalanced load in Hadoop clusters. The studies in (Liu et al., 2011) however, indicates that their approach lacks, among other things, accuracy and load balancing efficiency. Moallem and Ludwig (2009) and Ludwig and Moallem (2011) proposed an approach to solve grid load balancing scheduling problem. In this strategy, grid nodes are encoded as particles, compute load as positions and velocity of a particle is determined by the load difference it has compared to its neighbor. The larger the load difference between two particles, the more attractive they are to each other. Hence the best particle is one which possesses the largest load size and other particles are therefore attracted to it. Particles move toward each other by exchanging loads.The past experience of each particle is cancelled to reflect the dynamism of the grid nodes. This approach, however, does not use any strong mechanism to avoid PSO's premature convergence. Moreover, although the velocity cancellation approach, adopted in this approach (and other approaches discussed above), serves for its employment purpose, the resulting slow convergence will still manifest in the system and may cripple the net performance of the system especially in large scale scheduling problems when the system will need to work in real-time.

JCS
The problem of mobile node resource scheduling is studied by Runze et al. (2011). Their study considered the factors that affect distribution scheme and employed PSO to obtain faster and better allocation plan. During each PSO iteration, the approach integrates genetic and cross factors in order to expand search scope and obtain better solution. Chen et al. (2006) proposed an approach to solve problem of scheduling workflow tasks where scheduling solutions are expressed as Task-Resource Assignment Graphs (T-RAG) and hence the task scheduling problem is turned into a graph optimal selection problem. Each T-RAG is encoded as a particle and the longest path of the graph is considered as fitness value. The algorithm used Depth-First Search (DFS) heuristic to determine paths in each DAG application. Pure DFS is known with strength in terms of memory efficiency. However, it is some times trapped infinitely extending the search, which leads to a costly performance outcome (Korf, 2010;Duff et al., 2011). Moreover, heterogeneity of grid nodes is not considered in this approach. Yin et al. (2007a) presented a hybrid approach, HPSO, to provide reliable workflow task assignement solution. It is used in (Yin et al., 2007b) to solve multiobjective problem. The strategy embedded hill-climbing heuristic into PSO. At the end of each iteration, values associated with each particle are scanned for possible updating with aim to improve the solution quality. To cause PSO escape from local optimality, they employed constriction factor velocity model which allows particles to adjust their positions and velocities only by referring to their personal and swarm best positions. In this approach, particle's position represents a processing node and the particle's dimensions represent the grid tasks. The drawback of this proposal is that the hillclimbing heuristic scans each particle which renders local search computation complex (Engelbrecht, 2005). Tao et al. (2009) proposed RHDPSO and RCPSO (Tao et al., 2011b) to solve multi-QoS constrained grid workflow scheduling problem. PSO particles represent DAG task graphs; dimensions of a particle represent the tasks in the corresponding graph. In this work, new discretization method is proposed based on Rotary Discrete rule. Particle's behavior in searching for new position, which is controlled by the action of velocity, is viewed as a rotation of a wheel driven by the action of velocity. Each segment on the wheel is considered as particle's position in a dimension. Velocity drives the

JCS
wheel to rotate and determine new positions. But when the new velocity is very small and less than a defined value, the wheel does not rotate and hence the position does not change. Moreover, rapid change of velocity direction results to poor exploration. One way they considered as remedy is to employ constriction strategy i.e., to cancel the history of velocity, particle's velocity is therefore controlled by personal and global best positions. Chaotic logistic sequence operator is used to disturb the particle's personal and global best positions in order to overcome the premature convergence and local optimum.
Pei (2010) adopted the non-dominated sort strategy for multi-objective workflow scheduling problem with QoS constraints. The proposed strategy is tested against two cases: Failure without self-adaptation and No Failure. The author claims that not only opportunistic placement of workflow tasks is possible but also significant performance gains are achievable. Garg and Singh (2011) also employed non-dominated sort strategy to obtain pareto optimal solutions for multi-objective workflow scheduling problem. The 'indirect' encoding method proposed in (Izakian et al., 2009b) is used but personal and global best are updated based on nondominated comparisons on the population. In multiobjective problem, non-dominated solution is one that is best at least in one of the objectives with respect to others and pareto optimal solutions is a set of all nondominated solutions. However, it is indicated in (Charlotte et al., 2008), that generating non-dominated solutions takes large amount of computational time and is memory demanding.
Chen and Wang (2011) expressed dependent task scheduling problem as in . In this work, a drawback associated with the original PSO is identified and disclosed. In this work, a drawback associated with the original PSO is identified and disclosed. It is argued that generating initial assignment solutions randomly degrades PSO performance. To address this problem, Best Performance Resource (BPR) heuristic is embedded in PSO to generate initial assignment efficiently. Least Finish Time (LFT) is also used to prioritize tasks in the task queue. Chen and Zhang (2012) proposed a set-based PSO approach to address workflow scheduling problem. Assignment solution-i.e., PSO position-is encoded as a set and the dimensions of the position are the tuples of the set. Based on the encoding, the position and velocity vectors are redefined. Table 1 shows the classification and summary of the various PSO-based scheduling algorithms.

CONCLUSION
As an algorithm, PSO is fast, simple and with ease of implementation. It has been an enabling mechanism for achieving more satisfactory solution results for task scheduling problem in grid systems. This study discussed the various PSO based scheduling proposals and the techniques employed to suit the application of PSO in this field for achieving optimal performance. The body of scheduling work done so far with the use of PSO and the level of performance achieved indicate that the heuristic is a good option for solving grid task scheduling problem. However, as more and more new techniques are being discovered day-by-day, scheduling schemes with optimum performance are yet to be achieved.