Towards Solving the Problem of Virtual Machine Placement in Cloud Computing: A Job Classification Approach

: Cloud Computing is a paradigm that delivers services by providing an access to wide range of shared resources which are hosted in cloud data centers. One of the recent challenges in this paradigm is to enhance the energy efficiency in these data centers. In this study, a model that identifies common patterns for the jobs submitted to the cloud is proposed. This model is able to predict the type of the job submitted and accordingly, the set of users’ jobs is classified into four subsets. Each subset contains jobs that have similar requirements. In addition to the jobs’ common pattern and requirements, the users’ history is considered in the jobs’ type prediction model. The goal of job classification is to find a way to propose useful strategy that helps to improve power efficiency. Based on the process of jobs’ classification, the best fit virtual machine is allocated to each job. Then, the virtual machines are placed on the physical machines according to a novel strategy, called Mixed Type Placement strategy. The core idea of the proposed strategy is to place virtual machines of the jobs of different types in the same physical machine whenever possible. The placement process is based on Multi Choice Knapsack Problem which is a generalization of the classical Knapsack Problem. This is because different types of jobs do not intensively use the same compute or storage resources in the physical machine. This strategy minimizes the number of active physical machines which, in turn, leads to major reduction in the total energy consumption in the data center. The total execution time and the cost of executing the jobs submitted are considered in the placement process. To evaluate the performance of the proposed strategy, the CloudSim simulator is used with a real workload trace to simulate the cloud computing environment. The results show that the proposed strategy outperform both Genetic Algorithm and Round Robin from energy efficiency perspective.


Introduction
With the rapid growth in the cloud computing model, the reduction in the total energy consumed by the cloud data centers has become essential. This reduction can be achieved by observing how the power is delivered to computing resources and how these resources are utilized to serve the jobs which request these services.
The continuous growth in the cloud computing model poses a challenge for existing works to enhance the energy efficiency in cloud data centers. Hence, the need for improving the existing resource allocation and management algorithms in cloud data centers and proposing new ones is highly recommended. In this study, a new model is suggested to predict the job type in the workload based on common behaviors and patterns of the job. If there is a history for the user who submits the job to the cloud, it will also be considered in predicting the job type. If not, the job requirements are the parameters which are considered in the classification process. According to the job type, the best fit Virtual Machine (VM) is allocated to job. Then, VMs are placed to the Physical Machines (PMs) considering that no VMs of the same type of jobs will consolidate on the same PM relying on the Knapsack Problem (KP). High Performance Computing (HPC) jobs can be effectively combined with Data Intensive (DI) jobs on the same PM. HPC jobs mostly request compute resources, whereas DI jobs request storage and network bandwidth resources. In other words, HPC jobs and DI jobs do not intensively use the same resources. This can potentially lead to more efficient resource provisioning and reduction in the number of switched-on PMs and, consequently, higher energy efficiency. If there is no available PM for combining VMs of different types of jobs, then the KP algorithm to place the VMs is used. To note, Knapsack is not the only possible way for solving the VM placement problem, there are other methods, for example the bin packing algorithm used in (Beloglazov, 2013). KP is used in this study because it has different variants; one of them is called the Multi Choice Knapsack Problem (MCKP). It is used when the items should be chosen from different sets to be combined in one knapsack (Pisinger, 1995). This suits the proposed model of combining two VMs from two different sets into the same PM.
Thus, reducing energy consumption can be achieved through two situations addressed in this study.
First situation: Minimizing the number of switchedon PMs by combining the VMs of different kind of users' jobs and placing them on the same PM. Second: Minimizing the CPU frequency as much as possible for the VMs of the DI jobs using the Dynamic Voltage Frequency Scaling (DVFS) technology. To note, minimizing frequency is a very crucial issue. Minimizing frequency is done only when the VM of the DI job works in space-shared policy to prevent any effect on the other VMs hosted on the same PM. In addition, minimizing frequency must not violate the DI job's Quality of Service (QoS).
This work assumes that: Upcoming jobs are organized as Bag-of-Tasks (BoT) where jobs are independent and they can be executed in any order and one VM is allocated to each job.
This work tackles the problem of high consumed power in cloud data centers by proposing the following key contributions: • An online job classification model, which divides the workload into subsets. Each subset utilizes different kind of resources. This can potentially lead to more efficient resource provisioning and reduce the number of switched-on PMs which, in turn, results in higher energy efficiency • A Near-optimal energy-efficient scheduling algorithm in heterogeneous virtualized cloud data centers The rest of this paper is organized as follows. The next section presents the related work. Section 2 is a background to the concept of virtualization. Section 3 describes the components of the system model proposed to submit users' jobs to the cloud. It explains how these jobs are classified. In section 4, the details of the proposed model and the idea of the new proposed Mixed Type Placement (MTP) strategy are described. Experimental results are discussed and analyzed in section 5. Finally, conclusions listed in section 6.

Related Work
Many works tried to improve the power efficiency in data centers. The authors in (Horvath et al., 2007;Wang and Lu, 2008;Wang et al., 2009;Li et al., 2011) used DVFS techniques to save energy. Some works like the one presented in (Lawson and Smirni, 2005) dynamically adjust the number of CPUs in a cluster to operate in ''sleep'' mode when the utilization is low, others like in (Lang and Patel, 2010;Heo et al., 2011;Chakravarty and Sinha, 2013) switch the PMs from on to off. Bradley et al. (2003;Guenter et al., 2011;Bobroff et al., 2007), the authors statistically analyze the workload data and examine how to minimize the power consumption using workload history. The studies in (Garg et al., 2011;Tesauro et al., 2007;Zhang et al., 2014a) optimized multiple aspects of data center behavior such as consumed power, service cost and the overall performance. The authors in (Chase et al., 2001) proposed a cost approach to manage shared server resources such that the service requests for these resources are represented as a function taking into account the energy consumption. The authors in (Burge et al., 2007) scheduled tasks to heterogeneous machines according to the energy costs of each one to maximize the profit. In (Deore and Patil, 2013), the research argued energyefficient job scheduling and allocation scheme to minimize the number of switched-on PMs, so the amount of consumed power can be reduced. The thesis in (Feller, 2013) focused on the IaaS cloud service model whose goal is to offer a computing infrastructure by provisioning VMs on-demand. The thesis presented in (Beloglazov, 2013), presented some algorithms for distributed dynamic consolidation of VMs in virtualized cloud data centers. The VM placement strategy used in this thesis is based on bin packing algorithm.
Various optimization methods such as, Ant Colony Optimization (ACO) (Gao et al., 2013), Particle Swarm Optimization (PSO) (Zhang et al., 2014b) and Genetic Algorithms (GA) (Portaluri et al., 2014;Dong et al., 2014) were proposed to do the process of VM placement aiming to reduce energy consumption in data centers.
However, to the best of our knowledge, no work from the literature investigate the effects of combining different types of jobs on the same PM on energy efficiency.

Background
This section presents a background on the virtualization concept.

Virtualization
Virtualization is creating a virtual version of something (e.g., operating system, CPU, storage device, or network device). A single PM, which is the real hardware, can host one or more VM. A VM is a piece of software running on PM that emulates the properties of a separated PM.
Supplementary, the concept of virtualization breaks the traditional model of the PM that host a single Operating System (OS). It creates several VMs which are hosted on one PM, each VM may have its own OS. This concept is organized using hypervisor technology. A hypervisor (Josyula et al., 2011), also called Virtual Machine Manager (VMM), is a software that controls all PM resources, allocate resources needed by each operating system, monitors the utilization of the resources in turn and makes sure that the guest operating systems of the VMs cannot disrupt each other. Figure 1 illustrates the virtualization architecture.

VM Management
VM Management (Josyula et al., 2011) is the process of coordinated provisioning of the virtualized resources, as well as the runtime of such provisioning. This feature includes the mapping of the virtual resources to the physical ones and also overall management capabilities such as capacity, billing and Service Level Agreement (SLA) contract.
An important issue in VM management is the VM migration process, which is the process of transferring a VM from source to destination PM. Basically there are two kinds of VM migrations: Live and offline.

System Components
The proposed system is based on the Cloud computing environment paradigm, whereby Cloud users are able to request the services offered by the Cloud providers to execute their jobs. Indeed, this proposed system consists of two main components: A cloud user and a cloud provider, as shown in Fig. 2.
The cloud provider has an essential node called global scheduler. This node acts as an interface between users and the cloud infrastructure. It profiles and analyzes the service requirements of the submitted jobs and decides whether to accept or reject them based on the availability of resources. It selects the data center that execute the jobs such that the power consumption can be reduced, while the QoS requirements of the submitted jobs are met. As data centers are located in different geographical regions, they have different energy costs depending on regional constraints. Each data center is responsible for updating this information to the global scheduler in order to achieve an energy-efficient scheduling.
The two components, Cloud user and Cloud provider, are discussed in details below.

Cloud User
The cloud user submits the job (or set of jobs) to the cloud provider to be executed. Job can be defined as a task that is executed by the resources of the cloud data center, typically with an implied QoS requirement. A Job can be considered as a finer abstraction of an application service being hosted in the VM. This work proposes a job submission model as shown below: QoS includes the deadline and budget for the user jobs: • DL i : The deadline of job i • B i : The maximum budget of job i

Fig. 2. System architectural components
Cloud users can submit their jobs (code and data) to be executed and manipulated in a specific cloud data centers. The job submission process is done via cloud provider.

Cloud Provider
Cloud provider component includes two models: global scheduler and data center (s). The function of this component is to receive the users' jobs, execute them on a specific data center and then send back the result of the jobs execution to their users.

Global Scheduler Model
The cloud provider has an essential node called the global scheduler. This node receives the users' jobs and profiles them. The profile process contains sufficient information about the jobs. This information is very useful in executing these jobs. Global scheduler model operates in three main phases: Prediction phase, SLA phase and mapping phase.

Profiling and Prediction Phase
This phase is in charge of profiling users' jobs and deciding their types before the VM allocation process. Four types of jobs result from this phase: High Performance Computing (HPC), Data Intensive (DI), High Performance Computing and Data Intensive (HPC-DI) and Normal.
Firstly, Job profiling results in the following infrastructure required to execute the jobs within their QoS requirements: However, in reality, accurate job requirements such as execution time are not readily available in cloud environments. It is possible to examine and anticipate such requirements by executing part of the code. Many studies assumes that execution time with specific processing elements is a known quantity or estimated by the cloud user (Garg et al., 2011;Lee, 2012). To estimate the job execution time, four major solutions were proposed: • Code analysis (Nudd et al., 2000;Reistad and Gifford, 1994) • Analytic benchmarking/code profiling (Yang et al., 1993) • Historical/Statistical prediction (Sanjay and Vadhiyar, 2008;Iverson et al., 1996) • Empirical analysis (Berman et al., 2005) Using these solutions, or by executing part of the code, other information related to user jobs can be obtained. In addition, there exists many studies which proposed models to estimate specific parameters (e.g., Cycle Per Instruction (CPI) (Chen and John, 2011;Intel Corporation, 2008), Memory Access Per Instruction (MPI) (Zhang and Chang, 2014) and estimated bandwidth (Zhu et al., 2012).
After profiling, this phase utilizes a mix of statistical information and some input features for the jobs to predict the job type. This is done in two stages: User Id Checker (UIC): The users in cloud environments, most probably, submit the same type of jobs multiple times to be executed or served. Therefore, information of such users and their jobs can be collected by the provider they are dealing with and saved in log files. This work suggests creation of a Log File (LF) to help and support the decision of specifying the types of cloud users and their frequent jobs. So, the LF of the executed jobs contains statistical information about the jobs and their users. LF could be described as a list of records, one record for each job. The record is of the form: When users submit their jobs to the cloud provider, UIC checks if there is a match between the upcoming jobs and an existing record in LF. If there is a match, the type of the job will be predicted in this stage according to the history of the user who submits this job. This work supposes that even when there is a match, it is up to the provider to specify the job type depending on this match and to the user history, or by considering some features that are related to the new job submitted. This is because the user may change the requirements associated with the job at the subsequent submission, which may lead to a wrong type prediction. If there is no match for the job in this stage, the next one will perform the job type prediction.
Job Features Checker (JFC): If there is no available record in LF for the submitted job, the provider cannot depend on the job history when specifying the job type. Instead, the provider relies on some information associated with each job (which is obtained after profiling the job or given by the user) in the process of job type prediction. The features and or parameters associated with jobs and examined to help in job type prediction are: Total execution time of the job, Cycle Per Instruction (CPI), Memory Access Per Instruction (MPI), Size of the job and Bandwidth. JFC benefits from these values (which are listed in Table 1) to guess the type of the job. This work proposes four algorithms to predict the users' types considering the history of such users. The prediction depends on the type of the jobs submitted (either all jobs or last n jobs submitted by a specific user), the type of jobs submitted by the user within a past period of time, or depending on both. The algorithms are: i. All Submitted Jobs (allSJ) Algorithm: This algorithm predicts the user type depending on the type of all the jobs submitted by this user. To make accurate prediction of the user type, the cloud provider must perform a statistical analysis to choose the values of nJOB (which represents n jobs submitted by a specific user) and mDAY (which represents m days of jobs' submission by a specific user) to be used in the four algorithms illustrated above. These values may vary over time depending on the users who are dealing with the provider and how they deal.
In case of the absence of user's history, the provider can predict the user type based on the requirements associated with the jobs. Equations 1 to 4 are proposed to calculate the total requirements for all jobs of a user as below: Equations 1 to 4 calculate the total values requested for time execution, memory, storage and bandwidth respectively, which are required to execute jobs of a specific user. The resulting values specify the total requirements for every user.
By comparing these requirements with previous requirements (which stored and considered as benchmarks by provider), the user type can be predicted.
In job type prediction process, this work assigns weights to the proposed checkers in the prediction phase. Equations 5 to 8 are proposed to be used in this study: Where: w1, w2 and w3 are weight values, such that: w1+w2+w3 = 1 This work suggests that it is up to the cloud provider to assign the weight values. w1 is assigned to the history of the user who submits the job (if any), w2 is assigned to the values related to HPC jobs, which represents the total execution time, CPI and MPI. Usually, these values are high in HPC jobs. Finally, w3is assigned to the values related to DI jobs, which are job size and bandwidth. In general, these values are high in DI jobs. These thresholds are chosen by examining real workload trace. A histogram is sketched for each one of the above five parameters to select the proper thresholds. Based on these histograms, two methods are used: The average and median. Figure 3 is an example of the histogram sketched for selecting CPI threshold for 100 jobs.
If two or more values are equal and there is history for the user that submit the job, the requirement associated with the job can help in job type prediction (as in Equation 1 to 4).

The SLA Phase
In this phase, an agreement about the offered services between the user and the cloud provider is achieved.
SLA is a legal contract between participants (user and the cloud provider) to ensure that the QoS requirements of the users are met. If any party violates the SLA terms, the defaulter has to pay a penalty or do legal actions according to the clauses defined in this SLA (Wu and Buyya, 2011;Moustafa, 2015). In this study, the SLA phase decides whether the provider can execute the user's job or not depending on the job requirements and the available resources. If the provider is not able to execute the job with the required QoS (deadline and budget), user will be informed in this phase. Otherwise, the job will pass to the mapping phase. The form for the SLA contract between the user and the provider is presented in Table 2.

Mapping Phase
After specifying the types of jobs for all users and the SLA between the users and the providers is confirmed, this phase is concerned with selecting the best data center to serve the jobs (mapping jobs to a data center). The selected data center is the one that consumes the minimum amount of power when executing the users' jobs. The global scheduler interacts with the local schedulers of each data center to execute the jobs submitted to it. The CPU availability at particular times in the future and all information about the free time slots are available to the global scheduler based on the information provided by the local schedulers. The global scheduler receives the detailed information periodically from each cloud data center. These information includes the available resources and some other values like the amount of the consumed power resulted when executing the jobs on the available resources of the data center.
In the mapping phase, the provider maps the jobs to a specific cloud site based on the information provided by the local schedulers. The mapping process must consider the SLA constraints. At the selected cloud site, the local scheduler will perform the jobs scheduling based on the proposed MTP strategy.

The Data Center Model
In addition to the global scheduler. The cloud provider has a number of data centers. Each data center consists of n heterogeneous PM s refer to Fig. 4. Each PM i is equipped with multicore processors and characterized by the configuration shown below:   Based on the model presented in (Beloglazov, 2013), each data center has a local scheduler, which performs the scheduling process for all jobs received by the data center. In addition, the data center has a local manager (resides on each PM) and a global manager (maintains the overall system's resource utilization of a set of PMs by interacting with their local managers).

The Proposed Model
In cloud computing service models, different users submit their jobs to the cloud provider to be executed by some heterogeneous VMs. Each job will be mapped to a VM which satisfies its requirements. The mapping process requires an efficient strategy for allocating resources (heterogeneous CPU, storage, memory and network bandwidth) needed to execute a huge number of jobs. The resource allocation problem is NP-hard (Zhang et al., 2014a;Duan et al., 2007).
The problem even becomes more challenging when trying to effectively schedule many jobs in distributed, heterogeneous and virtualized cloud systems. So, this work proposes a model presented in the next sections to solve this problem.

Model Representation
The proposed system, S, can be modeled as fourtuple (D, PM, VM, J), such that: • D is a set of data centers, each element D d ∈D represents a single data center in the system • PM is a set of physical machines in the data center; PM pm,d ∈PM each element represents a single PM pm in D d • VM is a set of virtual machines associated with physical machines in the data center; each element VM vm,pmd ∈VM represents a single VM vm on single in data center D d • J is a set of jobs; each element job j ∈J represents a single job

Multi Objective Function
This paper proposes a multi-objective optimization scheduling algorithm, which aims to minimize the energy consumption, while maintaining the SLA specifications.
The power consumed by PMs in data centers usually determined by the CPU, disk storage, memory, network interfaces (Beloglazov et al., 2012). Among these components, the CPU consumes the most amount of energy. So, in this study, only the energy consumed by the CPU is considered. However, the consumption of the other components becomes very costly as a result of establishing numerous data centers round the world. Each data center contains thousands of PMs composed of these components. Equation 13 is used to find the total power of each PM: Where: P CPU = The power consumed by the CPU P Memory = The power consumed by the memory P Storage = The power consumed by the storage disk The power consumption model of the CPU, which is generally composed of CMOS circuits, is the sum of both the CPU static power (P CPU-static ) and the CPU dynamic power (P CPU _ dynamic ). Equation (14) is used to compute the power consumed by the CPU (Brooks et al., 2000;Elnozahy et al., 2003;Chaudhry et al., 2015): where, (P CPU-Static ) is a constan t, say ω and (PCPU-Dynamic) as in (15): Where: A = An activity factor that accounts how frequency gates switch C = The total capacitance at the gate outputs V = The voltage of the CPU and f is the operating frequency Voltage V can be expressed as a linear function in frequency, V = af, such that a is constant. All constants (ω, A and C) can be combined together in a constant, sayβ. So, Equation 14 that compute power consumption model of CPU can be written in (16): Frequency f is the only variable value in (16). Thus, DVFS technique is considered in this study, which usually results in a linear power-to-frequency relationship for a data center. DVFS modulates a processor's clock frequency and supply voltage in lockstep as programs execute. The premise is that a processor's workloads vary according to job requirements. When the processor has less work, it can be slowed down without affecting performance adversely. The architecture of the CPU supports different frequencies, so when the type of job is DI, this work minimizes the CPU frequency to a minimum level.
The PMs on/off switching technique is also considered in this study. In general, idle PM consumes approximately 70% of the power consumed when this PM running at the full CPU speed (Beloglazov, 2013). This fact justifies the technique of switching idle servers off to reduce the total power consumption.
Let n represents the total number of jobs, the total power consumption for executing these jobs (T energy ) can be measured using Equation 17: Where: x i = Equal to 0 if the machine PM i = Off and equal to 1 if it is on.
So, the main objective of the proposed system is to minimize the value of T energy .
In addition to the total consumed energy, an important value to be measured is the total execution time of the set of all jobs (T time ). The execution time for any job is the sum of its data stage in and stage out, adding to the actual time needed to execute the job.
The execution time In fact, T time is not the total time of executing all jobs because jobs are executing in parallel. The maximum finish time among all jobs in any workload (makespan) is the actual finish time of jobs' execution. It is an important value to be measured. This value reflects the accuracy of the jobs to VMs allocation process from the execution time point of view. Usually, minimizing the makespan value leads to minimizing T time . Makespan can be measured as in (20) Another important value to be calculated is the total cost for executing the set of jobs (T cost ), which can be measured using (21) The objectives of the proposed system are to minimize the values of consumed power, time and cost which is expressed in (17), (20) and (21) respectively.
The new allocation strategy (suggested in this study) to allocate jobs to VMs is discussed in the next section.

VM Allocation and VM Placement Strategy
In cloud computing environments, cloud users are served by providing VMs to execute their jobs. Cloud providers provide such services.
Cloud services can be delivered in the following sequence: • A cloud user submits the job (or set of jobs) to a cloud provider • A cloud provider analyzes the submitted jobs and predicts their types and requirements • According to the job requirements, the provider performs the VM provisioning to user's job (i.e., VM allocation) using best fit algorithm. Two main policies for the VMs to jobs allocation in cloud computing environments (Calheiros et al., 2011) can be used • Space-shared policy: The result is a VM with one or more cores • Time-shared policy: The result is a core that holds two or more VMs Every job is allocated to a VM with specific frequency. Since the CPU architecture supports different frequencies for the same CPU, it is possible to scale the frequency of the CPU cores up or down using DVFS technology. In the proposed work, when DI jobs are allocated to VMs in a space shared policy, the core frequency is minimized to a minimum frequency level if this will not affect the job's QoS requirements. This leads to better energy efficiency due to the cubic relation existing between energy and frequency as illustrated in (16). Usually, DI jobs do not need to utilize the maximum frequency of the compute power resources. Instead, they utilize storage and bandwidth resources. After VM provisioning, the provider performs the VM placement, which is the process of placing the VM on the proper PM. The VM placement process is implemented using the MCKP model (Pisinger, 1995). Knapsacks in this case represent the PMs of the data center. The aim is hosting the requested VMs on the minimum possible number of knapsacks. To note, the Job type is the essential factor in this placement. In this study, two VMs which belong to the same type of jobs are not placed on the same PM. This leads to further minimizing the number of switched-on PMs which is guaranteed by KP. However, when there is no available PM for combining VMs of different types of jobs, the classical KP algorithm is used to place the VMs.
The above sequence represents the job life cycle as illustrated in Fig. 5.

Performance Analysis
This section presents the evaluation of the proposed model. The CloudSim simulator, which is an extensible simulation toolkit that enables modeling and simulation of cloud computing systems and application provisioning environments, is used. The CloudSim supports both system and behavior modeling of cloud system components such as data centers, PMs, VMs and resource provisioning policies (Calheiros et al., 2011). A real workload trace to simulate the cloud computing environment is used. The jobs information is based on real data provided by (Google, 2015). The Google workload traces are collected from large cloud systems (about 12,500 compute nodes over 29 days). The traces consist of different types of jobs. Real workload traces can provide a very high level of realism when used directly in performance evaluation experiments. More details about this data are available in (Reiss and Wilkes, 2013). Three parameters are applied in the experiments: Number of PMs, number of VMs, number of jobs in the workload. The PMs and VMs configurations are as those provided by Amazon cloud data centers (Amazon, 2015). Table 3 illustrates the VMs instance types, called M3, offered by Amazon and used in the experiments in this study.

Results
Extensive experiments are carried out and repeated ten times for different number of jobs. Then, the results are compared with some strategies from the literature, e.g., Round Robin (RR) and Genetic Algorithm (GA). The results show that the proposed strategy outperforms both RR and GA as detailed in the next sections.

Energy Consumption
The evaluation in this section is based on the total consumed energy as a measurement. The experiments are done and repeated 10 times for different numbers of PMs, VMs and jobs.
In this study, the sets of jobs which are tested consist of 50, 200, 400, 600, 800 and 1000 jobs. To evaluate the proposed MTP strategy, a comparison with many classical job scheduling and placement strategies (e.g., Round Robin (RR) and Genetic Algorithm (GA)) are done. GA is a general purpose optimization technique inspired by the biological evolution. The initial population is produced randomly in the experiment. Then, it evolves to better approximate solutions from generation to generation iteratively (The number of iterations is set to 100) based on a specific fitness function (the consumed power in the experiment). The individual VMs cross by the genetic operators and combine in PMs. Each new population represents a new solution of VMs to PMs placement. Similar work can be found in (Dong et al., 2014).
In RR, VMs are placed and distributed to PMs in the data center sequentially in a circular manner, like the one in Eucalyptus, which is an open source software for building private and hybrid clouds (Eucalyptus, 2015).
The comparison of the proposed algorithm with the GA and RR approaches in terms of the total consumed energy resulting from executing different number of jobs is presented in Fig. 6.
The energy consumption of each set of jobs increased as the number of jobs increased. This is because the total execution time of a job will increase as more jobs are submitted. Within the same set of jobs and from energy efficiency perspective, MTP outperforms both GA and RR as illustrated in Fig. 6. MTP is better in enhancing energy efficiency because it involves minimum number of PMs in executing the set of jobs (as shown in Fig.  10). Minimizing the number of involved PMs is the main factor in enhancing the energy efficiency in cloud data centers, because even the completely idle PMs consume about 70% of their peak power. Another reason is that the resources of the involved PMs using MTP are better utilized than when using RR and GA (as shown in Fig. 7).

Total Execution Time
The total time required to execute different numbers of jobs using RR, GA and MTP is listed in Fig. 8.
In most scenarios, the total time required to execute users' jobs using MTP is more than the time of using RR and GA. This is because the VM to PM placement process in MTP spends time searching the best PM to place VM on it. The best PM is the PM that is already switched on and also hosts a VM allocated for a job of different type from the job served by the VM to be placed on it. Forcing jobs to be executed in minimum number of PMs will increase the turnaround time of these jobs. Consequently, the jobs' total execution time (makespan) will increase.
In GA, VMs are crossed and combined on PMs according to genetic operators. So, no time to spend in searching for specific PMs before the process of VM placement as in MTP. In RR, VMs are placed on PMs sequentially in a circular manner. No time to spend neither in cross and combine operations as in GA, nor in searching for specific PMs before the process of VM placement as in MTP. So, the total time for jobs execution is better in RR.

Cost
The experiments related to cost depend on real cost values offered by Amazon (2015). The cost per hour values are illustrated in Table 4.
As illustrated in Fig. 9, the tests show that the proposed MTP outperforms RR and GA from the cost perspective. The reason is that MTP utilizes the minimum resources that meet the QoS requirements and execute users' jobs.
MTP do not involve any VM with resources that exceed the actual jobs' requirements which, in turn, avoids users from paying any extra costs.

Impact of Using MTP
The proposed MTP has some impacts which are detailed in the next sections.

Impact on the Number of Active PMs
The proposed MTP strategy results in the minimum possible number of active PMs that are able to execute the total jobs compared to RR and GA. This outcome is due to the combination of high compute intensive jobs with data intensive jobs in the same PM, as the high compute intensive job mostly relies on the CPU performance, whereas the data intensive job utilizes disk storage.  The MTP strategy forces the scheduler to combine the VMs allocated to different types of jobs on the same PM. There is no exception unless if there is an SLA violation, or there is no available resources in an active PM to serve the VM. Figure 10 illustrates the differences in the number of active PMs using RR, GA and MTP for different number of jobs. The reason is that the compute and storage resources of each PM is utilized in an optimal way. Such utilization prevent the idle state for the resources (which is not guaranteed in GA and RR). In turn, the total number of switches on PMs needed to execute each set of jobs is minimized.

Impact with DVFS on the Type of Job
Since the CPU architecture supports different frequencies, the proposed algorithm minimizes the CPU frequency to a minimum level unconditionally when the type of job is data intensive. This does not affect the system performance and at the same time reduces the power consumption by an acceptable percentage. The percentage value differ from workload to another depending on the number of data intensive jobs it contains. Table 5 shows the reduction in power consumption when applying the frequency scale down to MTP strategy in executing different number of jobs.

Conclusion
In this study, a model that identifies common patterns for the jobs submitted to the cloud is presented. This model predicts the type of the job submitted; and accordingly, the set of users' jobs is classified into four subsets. Each subset contains jobs that have similar requirements. In addition to the jobs' common pattern and requirements, the users' history is considered in the jobs' type prediction model. The goal of job classification is to find a way to propose useful strategy that helps to improve power efficiency. Following the process of jobs' classification, a new VM placement strategy, called Mixed Types Placement (MTP), is proposed. Its core idea is to place the VMs of the jobs of different types in the same PM. The evaluation of MTP shows promising results in reducing the total energy consumed in the cloud data centers. The reduction in the consumed energy using MTP results in minimizing the number of PMs involved in executing users' jobs and the resources of the involved PMs are utilized optimally.
As a future work, we are working to subdivide the HPC and DI subsets based on new features and rely on the new sub-subsets in the process of VM placement. The HPC subset is divided into two main categories: CPU intensive and Memory intensive. Similarly, the DI subset is divided into two main categories: Hard disk intensive and Bandwidth intensive.
Moreover, the VM management approaches, such as VM migration and consolidation, will be applied to the proposed model to make it more integral to work in cloud computing paradigm.