A Cost-Effective Resource Provisioning Framework
using Online Learning in IaaS Clouds
Xiaohu Wu
Aalto University, Finland xiaohu.wu@aalto.fi
Patrick Loiseau
Eurecom, France patrick.loiseau@eurecom.fr
Esa Hyytia
University of Iceland, Iceland esa@hi.is
Abstract—Cloud vendors often offer three types of instances: reserved, on-demand and spot instances; a premier challenge for all users is to determine the proportion of various instances so as to minimize the cost of completing all jobs while satisfying their response-time targets. A previous online learning framework enables dynamically acquiring on-demand and spot instances from the cloud, and two open problems remain to be addressed in this framework: (i) extending it to the case incorporating reserved instances and the instances an organization itself possesses, and (ii) characterizing the expected cost-optimal scheduling principles to determine the proportion of various instances, which is determinative of the performance of this framework. In this paper, based on the pricing of cloud instances, we define the rules by which each job utilizes instances, and characterize the effect of pricing on the instance utilization state and cost. We also assume that the expected cost of utilizing spot instances is no more than the cost of utilizing on-demand instances alone. We analyze the proposed model here and solve the above two problems. The correctness and effectiveness of the proposed conclusions and solutions are also validated by simulations.
I. INTRODUCTION
A. Background
Cloud computing holds exciting potential of elastically scaling computation capacity up and down to match the time-varying demand of an organization, thus eliminating the need of purchasing servers of its own to satisfy the peak demand. Infrastructure as a Service (IaaS) is seeing a fast growth and nowadays has become the second-largest public cloud subsegment [1], [2], accounting for almost half of all the data center infrastructure shipments. Cost management in IaaS clouds is therefore a premier concern for users and has received significant attention [3].
In particular, there are three purchasing models available to users1: (i) on-demand instances, (ii) reserved instances, and (iii) spot instances. On-demand instances are always available to tenants and enable tenants to pay only for the period in which instances are consumed on an hourly rate. Alternatively, tenants can have a long-term commitment to
reserved instances and are billed for every hour during the entire prescribed term that the tenant selects, regardless of whether the instance is running or not. The term is typically long (e.g., 1 or 3 years). Tenants can also bid for spot instancesand run them as long as their bid is above the spot market price; however, spot prices be above or below the
1In this paper, we will use ”users” and ”tenants” interchangeably.
Fig. 1. Interaction of users with IaaS clouds: each user completes its jobs arriving over time and cost-effectively utilize and purchase multiple types of cloud instances under the control of some policy.
on-demand price. Compared to on-demand instances alone, reserved instances can have a significant discount up to 75% and spot instances can save up to 66% [4]. Facing these cloud instances with attractive features in different aspects, a fundamental decision problem for all tenants is how to determine their optimal proportion so as to complete the jobs of a tenant cost-effectively.
B. Motivation, Concern and Contribution
Recently, Menache et al. [6], [7] considered the application of online learning to utilize spot and on-demand instances from clouds, where jobs arrive over time and are specified by an arrival time, a deadline, a workload, and a parallelism bound. In this paper, we assume all the jobs are worth processing and consider an objective of minimizing the cost of completing all jobs by their deadlines while utilizing cloud instances2.
As illustrated in Figure 1, this paper aims to provide a complete understanding of the application of online learning to cost-effective resource provisioning in IaaS clouds; more specifically, it will address the following two concerns. Concern i. The approach in [6] where only on-demand and spot instances are considered can be applied to the case of a startup company who has a limited budget and whose future success is not clear, given the fact that reserved instances have a long-term commitment. Hence, one concern of this paper is 2This objective corresponds to a special case in [6] where the value of each job is larger than the cost of completing it.
Fig. 2. t∈[1, T]andJtdenotes the jobs arriving att, whereJ=∪T t=1Jt;
w is a distribution overP andd = maxj∈J{dj}. As process 2 repeats,
the optimal scheduling policies are identified, i.e., the ones with the highest probabilities.
to extend that framework to the case incorporating reserved instances, as pointed out in [6]. This extension is central to several important scenarios. The business of a startup company may turn successful and stable, and it will have the need to consider purchasing reserved instances to further minimize the cost. Consider another case where a matured organization has its own computing instances such as a private cloud. With the expansion to the original business or the introduction of new business, it needs consider utilizing on-demand and spot instances together with its own instances to process the growing daily workloads. Once the business becomes stable, it needs to consider again purchasing reserved instances to minimize the cost.
Concern ii. Another concern of this paper is on scheduling policies. In the framework, there is a set J of jobs that arrive over time and a set P of n scheduling policies. A policy defines the proportion of various instances to be utilized for completing a job j and thus determines its cost. The online learning framework is illustrated in Figure 2 and has the following property: when J is large, the actual cost of completing J is close to the cost of completing all the jobs using the best policy πinP. Hence, the performance of this framework depends on P and π and, it is determinative to
characterize the cost-optimal setP∗ of scheduling policies in which the cost-optimal policy π∗ is located; here, we have that, for anyP, the cost of the bestπ∈ P is no less than the cost ofπ∗ ∈ P∗.
As stated in [6], the whole set of feasible policies can be huge and only a best-effort approach is used to propose a part of practical policies; here a policy can use a fixed proportion of spot and on-demand instances to completej. However, we never know how far the performance of the best such policy is from the performance of the optimal policy π∗.
Contributions. The contributions of this paper are to resolve the above two concerns effectively and summarized as follows.
Modeling. Based on the pricing of cloud instances, we define the rules by which each job utilizes instances, and characterize the effect of pricing on the instance utilization state and cost. Constrained by the information available to us and since spot prices often vary unpredictably over time [5], we can only use a simple assumption that the expected cost of utilizing spot instances is cheaper than that of on-demand instances.
1 2 3
On-demand Yes Yes Yes
Spot Yes ? Yes
Long-term Yes Yes ?
Fig. 3. Yes (resp. ?) in thei-th row andj-th column represent thej-th class of works have (resp. have not) addressed the cost-effective resource provisioning problem where thei-th instances are taken into account.
Scheduling Principles. An analysis of the above model brings the characterization of the optimal principles in expectation to utilize the instances possessed by an organization itself, and to utilize spot, on-demand instances under the assumption that the arrival and execution process of jobs is independent of the change of spot prices over time, also made in [12].
Practical Policies. When online learning is applied, there is no exact statistical knowledge of the jobs as well as spot prices; hence, the practical scheduling policy can be defined to be a function of the current characteristics of each job and the current amount of instances possessed by an organization alone. We therefore propose a policy to realize the principle for long-term instances approximately; we also propose a policy to utilize spot and on-demand instances that is also robust against the special case where spot prices vary with the specific execution process of substantial jobs from an user [12]. Here, the coefficients of the function specifying the scheduling policy will be determined by the dynamics of the arriving jobs and spot prices and can be learned by online learning.
Applicability in the General Case. The results on scheduling policies enable us to directly extend the framework in [6] to the case incorporating the instances possessed by an organization; we further identify the condition under which it is advisable to consider purchasing reserved instances and propose a solution to the more general case incorporating reserved instances.
Simulations. Finally, the effectiveness of the proposed solu-tions and the correctness of the proposed conclusions in this paper are validated through simulations.
C. Related Work
We now discuss the related works on cost-effective resource provisioning in IaaS clouds. In methodologies,the first class of works[8]–[12]assumes the exact statistical knowledge of the entire workload of executed jobs over time a priori or the workload can be accurately predicted in the long-term future; then various optimization techniques can be applied. In [8]–[10], stochastic programming techniques are be applied to achieve the cost-optimal acquisition of reserved, spot and on-demand instances. In [11], the proposed algorithm iteratively makes decisions of purchasing reserved and on-demand in-stances based on the workload predictions. In [12], the optimal strategy for the users to bid for the spot instances are derived, given a predicted distribution over spot prices. The second class of works does not require any statistical knowledge
of workload and the technique of competitive analysis is applied [14] to purchase reserved or on-demand instances. In particular, a deterministic algorithm is proposed with a performance no more than2−αtimes the optimal cost, where
αis the discount of reserved instances. We refer to the works in this paper and [6] as the third class of works, where the online learning approach is applied, and, it does not require the strong assumption of knowing the exact statistical feature of workloada priori. Finally, Table 3 summarizes these related works to highlight what remain to fill the gap in this problem.
II. PROBLEMDESCRIPTION ANDMODEL
In this section, we introduce the jobs, the pricing models in IaaS clouds, the rule by which every job utilizes various instances, and the optimization objective.
A. Online Batch Jobs
This paper focuses on an organization whose batch jobs arrive over time; their arrival is monitored every time slot of
L minutes (e.g., L = 5) and time slots are indexed by t = 1,2,· · ·. Each job j is characterized by four parameters: (i)
arrival slotaj: If jobjarrives at a certain time point in[i(t0−
1), it0), thenaj=t0; (ii)deadlinedj ∈ Z+(measured in time
slots): every job must be completed by the time slot aj + dj −1; (iii) job size zj (measured in CPU time slots that
need to be utilized); (iv) parallelism constraint δj: the upper
bound on the number of instances that could be simultaneously assigned to a job. Thus, a job j is described by the tuple
{aj, dj, zj, δj}. The job j is said to be active at a time slot τ if less than dj time slots have passed since its arrival aj,
and the integration of the number of instances assigned to j
over time so far is less than zj. Denote by lenj = zj δj the
minimum execution time required for completing a jobj and by sj =
dj
lenj the slackness of j, which implies the level of
time flexibility of allocating resources to j. The organization plans to rent computing instances from the cloud to process its jobs and aims to minimize the cost of completing all the jobs by their deadlines.
B. Resource Provisioning and Pricing in the Cloud
Now, we begin to introduce the pricing models in the cloud.
On-demand Instances.The price of an on-demand instance is charged on an hourly basis and it is fixed and denoted by p. Even if partial hour of on-demand instances is consumed, the tenant will be charged the fee of the entire hour.
Spot Instances.Tenants can bid a price for spot instances and
spot prices are updated at regular time intervals (e.g., every 5 minutes in Amazon) [12]. They are assigned to a job and run until the spot price exceeds the bid price, and a tenant is charged based on the spot price. Spot price may change unpredictably over time [5] implying that spot instances can get lost suddenly. Once the spot price exceeds the bid price of a job, its spot instances will be terminated immediately by the cloud. Here, the tenant will be charged for the maximum integer hours of execution and the partial hour of execution is not to be charged. However, once a job is completed and
spot instances are terminated by the tenant, the consumption of these instances will be charged for the full hour even if the job finishes earlier.
Long-term Instances.Reserved instances and the instances that an organization itself possesses are called long-term instances. Reserved instances can only be reserved for a fixed long term (e.g., 1 or 3 years) and tenants have to pay for these instances in every hour of that term even if they are not being used. The hourly costs of reserved and the other type of instances are assumed to bep2andp1no matter whether they are being utilized, wherep1< p2< p. The later runs almost all the time for processing the continuously arriving jobs, and we assume in this paper that it is the cheapest to use such instances and the costp1 is without loss of generality assumed to be 0.
C. Rules of Allocating Resource to Jobs
Defined by the pricing models, we propose the rules of allocating instances to jobs in this subsection.
Firstly, we only consider the allocation of on-demand and spot instances. Each jobjis allocated instances to completezj
workload by the deadlinedj. To meet the deadline, we assume
that whenever a jobj arrives ataj, it will immediately begin
to acquire instances. Given the facts that the tenant is charged based on hourly boundaries and that there is a parallelism bound on the jobs, a general rule of resource allocation is that (i) the allocation of on-demand and spot instances to each active job is updated simultaneously every hour. In the i-th allocation, the number of on-demand instances allocated to j
is fixed in the time interval[aj+ 60(i−1)/L, aj+ 60i/L−1]
and they will be utilized for an hour; this number is denoted byoi
j; if any of the spot instances is terminated by the tenant
during the execution, the entire hour of fee will be charged by the cloud. So, we assume that (ii) the tenant will bid the price bi
j for a fixed number siij of spot instances at the i-th allocation.Since there is no exact statistical knowledge of how spot prices changes under the online learning framework of this paper and we only know that spot instances are on average cheaper than its on-demand counterpart, we assume that (iii)at every allocation the tenant is to bid for the maximum number of spot instances under the parallelism constraint. Here, we therefore havean important questionof how to determine the proportion of on-demand and spot instances that are acquired from the cloud and allocated toj.
Secondly, once the rules to utilize spot and on-demand instances are fixed as above, we assume that (iv)the allocation of long-term instances to a job can be updated at most once at every allocation update of that job. We denote byri
jthe
num-ber of long-term instances assigned toj at thei-th allocation. The parallelism constraint translates tooi
j+siij+rij=δj. D. Characterizing the Cost and Objective
In this subsection, we characterize the cost of utilizing various instances by the jobs, where reserved instances are not considered.
LetLen= 60/LandDjdenote the total number of
alloca-tion updates ofj until its completion. Let Xi
{0,1} denote the random variables indicating whether or not the spot instances are obtained successfully at the i-th allocation, whereXi
j,1+Xj,i2+Xj,i3= 1. In the case that the
i-th allocation is not the last one, if the spot instances run for an entire hour at this allocation and the spot price is charged, then set Xj,i1 to 1, and Xj,i2 and Xj,i3 equal 0. In the case that thei-th allocation is the last one, if the spot instances run until the completion of j and the spot price is charged, then setXi
j,2 to 1, and Xj,i1 andXj,i3 equal 0; letTj,i1 denote the random number of time slots during which j is utilizing spot instances, where 0≤Ti
j,1 < Len. However, if the spot price is not charged and the instances are interrupted by the cloud or are even not obtained at the beginning of this allocation, then setXi
j,3 to 1, and Xj,i1 andXj,i2 equal 0; letTj,i2 denote the random number of time slots during whichjis utilizing these spot instances. Except the above cases, set Xi
j,1, Xj,i2, Xj,i3 to 0. Then, we have for all the jobs j ∈ J that the cost of completing j is Dj X i=1 siij·pij·(Xj,i1+Xj,i2) +oij·p, (1) subject to Dj X i=1 (Xj,i1·Len+Xj,i2·Tj,i1+Xj,i3·Tj,i2)·siij +Len·oij+Len·rij=zj, (2) where P
j∈JLen·rji =α·r·T andpij is the average spot
price to be charged at the i-th allocation of j. The total cost of completing all the jobs is
X j∈J Dj X i=1 siij·pij·(Xj,i1+Xj,i2) +oij·p. (3)
A mainobjectiveof this paper is to minimize Equation (3). We set p0 to P j∈T PDj i=1si i jp i j(X i j,1+X i j,2) P j∈J PDj i=1si i jp i j(X i j,1Len+X i j,2T i j,1+X i j,3T i j,2) .
In the long run,p0is the average cost of utilizing spot instances and we make the following assumption:
0< p0 < p, (4)
where pis the on-demand price. Such an assumption is one motivation of bringing spot instances into the cloud market and has been validated by many experimental works [12]. Form of Scheduling Policy.We now identify what scheduling policies can be designed in the scenario of this paper. Con-strained by the online learning approach, we assume in this paper that there is no exact statistical knowledge on the jobs and spot prices, as well asXi
j,1,Xj,i2,Xj,i3,Tj,i1andTj,i2.At the very beginning of the i-th allocation of j, we denote by
zi
j the remaining workload ofj to be processed, i.e.,zjminus
the workload ofjthat has been processed, wherez1
j =zj, and
bysij =(dj−(i−1)·Len)·δj zi
j
the current slackness ofj. At every
allocation ofj, only the current characteristics ofj(i.e.,zi j,δj, aj, anddj) and the amount of available long-term instances are
definitely known. The rules of allocating resource to jobs in Section II-C determines the form of scheduling policy. Based on these observations, we make the following definition.
Definition 1: At the i-th allocation to j, the policy is a function F : Yi
j × N → (r, f), where Y i
j is a set of the
current characteristics of j, N is the amount of the current available long-term instances, andf ∈[0,1].
Here, at the i-th allocation of j, it will be allocated r long-term instances, acquire(1−f)·(δj−r)on-demand instances,
and bid some price forf·(δj−r)spot instances.In this paper, we aims to study the specific form of the functionF so as to minimize Equation (3) as much as possible.
The main notation of this paper is also listed in Table III in the Appendix.
III. SCHEDULINGPOLICIES IN THECLOUD
In this section, we study the expected cost-optimal policy (i.e., the F in Definition 1) to utilize spot and on-demand instances and the instances possessed by an organization.
A. Optimal Structure to Utilize Three Types of Instances
Due to inequality (4) and the constraint in Equation (2), increasing the utilization α of long-term instances and then consuming as many spot instances as possible can minimize Equation (3). Built on this, we conclude that
Principle 1: The scheduler manages to (i) make the long-term instances fully utilized, and (ii) utilize long-long-term in-stances in a way so as to maximize the opportunity for the remaining workload of jobs to utilize spot instances on the whole.
Principle 2: In the case that long-term instances have been used up and only spot and on-demand instances are left available to process the remaining workload of jobs, the scheduler manages to on the whole maximize the opportunity for the jobs to utilize spot instances.
The above principles identify (i) the optimal order to utilize the three types of instances, and (ii) the objective that the scheduling policies for different instances should achieve.
B. Scheduling Policy for Long-term Instances
At any time slott, denote bysj(t)the current slackness of
a job j that is defined to be the ratio of (aj+dj−1) times δj to the remaining workload of j to be processed. Roughly,
whensj(t)>1(resp.sj(t) = 1), there is the flexibility (resp.
no flexibility) for j to utilize unstable spot instances under the deadline constraint; according to Principle 1, the policy should be capable of increasing the average slackness of all the jobs. We denote byN(t0)thecurrentnumber of long-term instances that have not been allocated out at a time slott0; let
mt0(t00) = min{N(t0),· · · , N(t00)}, where t0 ≤t00.
We now propose a simple yet effective policy under which, upon the arrival of a jobj att=aj, it willfirstbe allocated rj = (1− min{β·sj, 1})·min{zj/dj, mt(aj +dj−1)}
Fig. 4. The coordinates (X, Y)=(the number of long-term instances allocated toj, the slackness ofj), whereβ =0.2, 0.1, and 0 respectively in the red, green, and blue lines andδj= 20.
β ∈(0,1) (as in [6], the issue of rounding the allocations of a job to integers is ignored for simplicity). As illustrated in Figure 4, under the sameβ, the jobs with a smaller slackness will be allocated a larger proportion of long-term instances; under the same slackness of jobs, the smallerβ is, the larger the proportion of long-term instances is. Since we prioritize long-term instances for utilization, this policy realizes the first point of Principle 1. After the allocation toj, the completion of
jcan be viewed as processing a new jobj0whereδ
j0 =δj−rj, aj0 =aj,dj0 =dj, andzj0 =zj−rj·djwho will utilize spot
and on-demand instances alone to be completed. Here, when only spot and on-demand instances are available to process the jobs,j0 has more flexibility (with a larger slackness) than
j to utilize spot instances since we have1≤δj·dj zj ≤
δj·dj−ν zj−ν ,
whereν=rj·dj. Hence, the policy realizes the second point
of Principle 1 approximately.
As we will see, the above policy also enables analyzing the optimal principle to utilize spot and on-demand instances in a common case and bringing a stable cost structure to propose a solution for reserved instances.
C. Cost-Effective Policy to Utilize Spot Instances
Theoretical Results. We first identify the expected optimal principle of utilizing spot and on-demand instances to mini-mize Equation (1). The analysis is based on the assumption in [12] where i.i.d. job arrivals induce i.i.d. spot prices; so the distributions overXi
j,1,Xj,i3,Tj,i2 are irrelevant ofj andifor all j∈ J andi∈[1, Dj−1].
Once a job j is allocated rj reserved instances at all t ∈ [aj, aj+dj −1], it can be viewed as a new job where
spot and on-demand instances alone are utilized. So, without loss of generality, we are to consider the case with on-demand and spot instances alone to simplify the analysis. Let P(xi
j,1, xij,2, xij,3, tij,1, tij,2) denote the probability when
Xj,i1 = xij,1, Xj,i2 = xij,2, Xj,i3 = xij,3, Tj,i1 = tij,1, and Ti
j,2 = tij,2 at the i-th allocation of j. This probability depends only on the bid strategy and spot prices that vary unpredictably. In the allocations to j excluding the last one, we have PLen−1
t=1 P(0,0,1,0, t) +P(1,0,0,0,0) = 1; in the last allocation, we assume that the instances will have run for
τ time slots at the completion time ofj, where0< τ ≤Len, and we have Pτ−1
t=1P(0,0,1,0, t) + P(0,1,0, τ,0) = 1. The expected workload processed by spot instances at i-th allocation istij·sii
j, where the expected processing time by spot
instancestij =PLen−1
t=1 t·P(0,0,1,0, t)+Len·P(1,0,0,0,0) at the allocations excluding the last one (i.e., i∈[1, Dj−1])
and tij = Pτ−1
t=1 t·P(0,0,1,0, t) +τ·P(0,1,0, τ,0) at the last allocation (i.e.,i=Dj).
We set β1 to
PDj
i=0t
i j
Len·(Dj−1)+τ. The case that PDj
i=0t
i j =lenj
where Len·(Dj −1) +τ ≤ dj equals the case that β1·
dj ≥lenj, i.e., β1·sj ≥1. According to inequality (4) and
Equations (1) and (2), we therefore conclude that:
Proposition 1: In the case that β1·sj ≥ 1, the expected
optimal proportion of spot instances allocated toj is 1. We now consider the other case that β1 · sj < 1. If zji/δj ≥dj−Len·(i−1), we have to turn to totally utilize
the safe on-demand instances to meet the deadline; assume that the above condition is first satisfied at the (i0 + 1)-th
allocation. Here, we note that if such i0 does not exist, the optimal proportion of spot instances at every allocation is 1. Then, we have the following relation for completing j:
Pi0 i=1t i j·siij + Pi0 i=1Len·o i j + (dj −Len·i0)·δj = zj, where oi
j + siij = δj. Further, we have that dj · δj = zj+Pi
0
i=1(Len−t
i
j)·siij. Since the distributions overXj,i1,
Xi
j,3,Tj,i2 are the same at the different allocations of j and irrelevant ofiandj wherei∈[1, Dj), we have that allt
i j can
be set to be the same valueω ∈[1, Len]. Hence,in the case that the abovei0exists, the expected total workload processed by spot instances(i.e., Pi0
i=1t i j·si i j=ω· Pi0 i=1si i j)is fixed, irrelevant of the proportion of spot instances allocated toj at every allocation.Interestingly, we conclude that
Proposition 2: In the case that β1·sj < 1, the expected
optimal proportion of spot instances allocated toj is 1. Based on Proposition 1 and 2, we conclude that Principle3: Assume that the distributions overXi
j,1,Xj,i3,
Ti
j,2 are irrelevant of j and i where 1 ≤ i ≤ Dj −1.
Despite the value of β1·sj, the expected optimal proportion
of spot instances at every allocation is 1 untilj has to turn to totally utilize safe on-demand instances to satisfy the deadline requirement.
Robust Policies.Sometimes, the assumption in Principle 3 can be violated since temporal correlation in spot prices can be induced by the temporal correlation in cloud workloads from all tenants [12]. Due to the diversity of tenants, for substantial jobsj of a tenant, it can happen thatXj,i1= 0andtij,2 in the initial allocations of j are much smaller than the later ones, which is different from the case in Principle 3. In this case, if sii
j is set toδj, the job will quickly get into a situation in
which it has to totally utilize on-demand instances to satisfy the deadline requirement, and has no opportunity to utilize the cheaper spot instances in the later slots. Further, based on Principles 3, we propose the following more robust policy to determine the proportion of spot instances at every allocation:
f(sij) = min
(β1·sij)
β2, 1 , (5)
whereβ1∈(0,1]andβ2∈[0,+∞). In particular, ifsij ≥
1
β1,
j will bid for δj spot instances; if sij <
1
β1 andβ2 is large enough, f(si
j) is close to 0. Hence, for jobs with relatively
small slackness, selecting appropriateβ1and large enoughβ2 can guarantee that, j will totally utilize (or utilize a large
Algorithm 1: Dynalloc
Input :at every time slott,mtjobs to be processed, eachjwith {aj, dj, z0j, δj}and its policy{β, β1, β2, b}
1 forj←1tomt do 2 ifaj=tthen
// allocate long-term instances to j
3 fort0←ajtoaj+dj−1do 4 rj, rj(t0)←− 5 (1− min{β·sj, 1})·min{ zj dj, mt(aj+dj−1)}; 6 forj←1tomt do 7 i← lt−aj Len m + 1; 8 if tLen−aj =i−1then
// at every allocation update 9 ifd z
0
j
δj−rj + 1e ≤aj+dj−tthen
// have flexibility for spot instances 10 call Algorithm 2;
11 ifthe spot instances ofjare terminated att−1then 12 ifd
zj0
δj−rje ≥dj−Len·ithen
// give up utilizing spot instances 13 ifaj+dj−1≥aj+Len·ithen
14 o0←min{oj(t−1),d z0j dj−Len·ie};
15 acquireo0on-demand instances for the time interval
h aj+Len·i, aj+Len· d dj Lene −1 i ; 16 fort0←aj+Len·itoaj+dj−1do 17 oj(t0)←min{zj0, o1}; 18 o00← d z 0 j aj+dj−te;
19 acquireo00on-demand instances fordaj+dj−t
Len ehours; 20 fort0←ttoaj+dj−1do
21 allocatemin{zj0, o00}more instances toj;
proportion of) on-demand instances at the early allocations, and, as its current slackness becomes larger, it will utilize a larger proportion of spot instances in the later allocations. Here, we still have that1≤zµi
j
≤ zµi−ν j−ν
, whereµ=δj·(dj−
(i−1)·Len)andν=oij·Len.
Finally, the scheduling policy for spot and on-demand instances is proposed in Algorithm 2.
D. Scheduling Framework
As described above, a general policy is defined by a tuple {β, β1, β2, b} and determines the amount of long-term instances and the proportions of spot and on-demand instances allocated to a job, and the bid price for spot instances. This specifies the function F in Definition 1. The complete framework to utilize long-term, spot and on-demand instances is presented as Algorithm 1. To avoid a lengthy description of Algorithm 1, during its execution, zj0 always denotes the remaining workload of jobjthat remains to be allocated more resource for full completion with its current allocation. We let
rj(t), oj(t) and sij(t) denote the number of long-term,
on-demand and spot instances allocated tojatt∈[aj, aj+dj−1],
where allrj(t)at different slots have the same valuerj.
Algorithm 2: Proportion(j,β1,β2,b) 1 fort0←ttot+60 L −1do 2 sij(t0)←f(sij)·min δj−rj(t0), z0j
min{dj−Len·(i−1),Len}
;
3 oj(t0)←δj−sij(t0)−rj(t0); 4 bij←b;
5 At thei-th allocation update, bid forsij(t)spot instances using a bid
pricebi j;
Given a set P of such tuples {β, β1, β2, b}, as more and more jobs j are processed, the performance of an online learning algorithm is close to that of the best policy defined by some tuple in P. Here, this best policy is capable of well adapting to the statistical features of Xj,i1, Xj,i2, Xj,i3,
Ti
j,1 and Tj,i2 (that are determined by the dynamics of the arriving jobs and spot prices) so as to minimize Equation (3). In other words, for the policies defined by P, it realizes Principles 1 and 2 in the most cost-effective way.
IV. THEORETICALFRAMEWORK OFONLINELEARNING In this section, we introduce the online learning framework in which all the four types of instances are considered.
A. Online Learning for Optimal Resource Utilization
In this subsection, we extend the online learning approach in [6], presented as Algorithm 3 in the Appendix, to the case where an organization possesses R1 instances. Its main idea is also illustrated in Figure 2.
Algorithm 3: OptiLearning
Input : a setPofnpolicies, eachπbeing parameterized so that
π∈ {1,2,· · ·, n}; the set of jobsJtthat arrive at time slot
t;
1 initialize the weight vector of policies of dimensionn:
w1={w1,1,· · ·, w1,n}={1/n,· · ·,1/n}; 2 fort←1toT do
// upon the arrival of job j
3 ifJt6=∅then
4 for eachj∈ Jt, pick a policyπwith a probabilitywj,π,
being applied toj; 5 ift≤dthen 6 wj+1←wj; 7 else 8 whileJt−d6=∅do 9 ηt← q 2 logn d(t−d); 10 get a jobjfromJt−d; 11 forπ←1tondo 12 w0j+1,π←wj,πexp−ηjcj(π); 13 forπ←1tondo 14 wj+1,π← wj0+1,π Pn i=1w0j+1,i; 15 Jt−d← Jt−d− {j};
Let J denote a set of jobs, and T = maxj∈J{aj}. To
make various instances utilized cost-effectively (i.e., realize Principles 1 and 2), the cost cj(πj) of completing a job j
Algorithm 4: ResAmount
Input :the information on spot prices, on-demand price,J0
0, the
best policyπ∗fromP
1 without loss of generality, the starting time slot is set to 1;
2 use Algorithm 1 to compute the costcj0(π∗)of completingj0∈ J0
0
under the policyπ∗;
3 c(J0 0)← P j0∈J0 0cj 0(π∗); 4 R∗2←arg min 1≤R2≤R{p2·R2·T+c(J 0 0)};
under some policyπj ∈ Pis defined by Equation (1), i.e., the
cost incurred by the consumed spot and on-demand instances excluding long-term instances. As proved in [6], we have that
Proposition 3: Let N0 = | ∪T
t=d+1Jt|. For all δ∈ (0,1),
it holds with a probability at least 1−δover the random of online learning that
maxπ∈P n P t∈∪T t=d+1Jt cj(πj)−cj(π) N0 o ≤9 q 2dlog (n/δ) N0 . B. Case Incorporating Reserved Instances
Reserved instances have a long-term commitment for at least 1 year andare considered for purchasing on the condition that a set of jobs J0 recurs in a periodof length T0, which is the case of many scenarios [15]. This condition enables a stable cost structure to predict the expected cost of utilizing spot and on-demand instances. Denote byR the total number of two types of long-term instances and these long-term instances are allocated using Algorithm 1. The number of long-term instances allocated to every job j ∈ J0 at every
t∈[aj, aj+dj−1]is a constantrj, which depends only onR
andJ0, and, the completion ofj can be viewed as processing a new job j0 with δj0 = δj −rj, aj0 = aj, dj0 = dj, and zj0 =zj−rj·dj, where only spot and on-demand instances are
available for processing. Here, long-term instances have a fixed utilization of (P
j∈J0rj·dj)/(R·T) every period. Let J
0
0 denote the set of all the new jobsj0, whereJ0
0depends onJ0 andR. Assume thatRis an upper bound of reserved instances that need to be purchased, e.g.,R= max1≤t≤T{Pj∈J(t)δj},
whereJ(t) ={j ∈ J |t∈[aj, aj+dj−1]}.
A study shows that the spread of the past spot prices does not change significantly every year [13] and the performance of online learning is close to that of the best policy; so the final solution is presented as Algorithm 4 to approximately determine the optimal amount of reserved instances R∗2 to be purchased. Here, c(J0
0) is used as the expected cost of utilizing spot and on-demand instances alone to completeJ0
0; the allocation of long-term instances is irrelevant to spot prices and, givenR, has a stable effect on the feature of the jobs (i.e.,
J0
0) to processed by spot and on-demand instances. OnceR∗2is determined by Algorithm 4, we can run Algorithm 3 to process the jobs that arrive in the time to come; using Algorithm 3 to choose policies for jobs is also more robust against the cloud market dynamics than using the best policy alone for all jobs and Algorithm 3 will again approach the performance of the best policy.
V. EVALUATION
In this section, we validate the effectiveness or correctness of the proposed policies, solutions and related conclusions through simulations. The behaviors and convergence of Al-gorithm 3 over real datasets basically coincide with its coun-terparts over synthetic datasets [6] and synthetic datasets can provide the evaluation with diverse workloads and spot prices so as to validate the results of this paper; so we only take the simulations on synthetic datasets with the limit of pages.
A. Parameter Settings
The on-demand price is p= 0.25per hour while reserved instances bring a discount of 50% compared to on-demand instances [4]. All the jobs in the experiments have a parallelism bound of 20.The first type of jobs and spot pricesis generated in a time interval of length T = 30000 with around 15000 jobs as follows: (i) the job’s arrival is generated according to a poisson distribution with a mean of 10 minutes. The sizezjof
every jobjis chosen uniformly from the interval[0,600]. The job’s deadline is generated asx·zj/δj, wherexis uniformly
distributed over [1,3]. (ii) spot prices are updated every time slot [12]. Their values are chosen independently as 0.15 + max{x2,0}, where x2 is a normal random variable with a mean µ= 0and a varianceσ= 1. Here, the change of spot prices is independent of the arrival of jobs over time, which is the case in Principle 3.
The second type of jobs and spot prices to be evaluated is generated in a time interval of length T = 216600 with 4000 jobs as follows: (i) jobs are generated periodically: the sizez2i+1, arrival timea2i+1and deadlined2i+1of(2i+ 1)-th job are 780, 96i+ 1 and48; the z2i+1, a2i+2 and d2i+2 of (2i+ 2)-th job are set to 480, 96i+ 49 and 48 for all i = 0,1,2,· · ·. (ii) for each of the first 12 time slots respectively in[a2i+1, a2i+1+d2i+1−1]and in[a2i+2, a2i+2+d2i+2−1], the spot price is set to0.21 + 0.05·max{0, x3}, wherex3is a norm random variable withµ= 0andσ= 3; For each of the rest time slots, it is set to0.05 + 0.05·max{0, x4} wherex4 is a norm random variable with µ= 0and σ= 1. Here, the change of spot prices is correlated to the arrival of jobs over time, which is an example of the case for the robust policies in Section III-C.
The parameters β, β1, β2, and b define the schedul-ing policies. β, β1, β2 are chosen respectively from
{0,0.25,0.35,0.4}, {0.4,0.8} and {0,1,20,400}; the bid price b ∈ {0.16,0.19,0.22,0.25}. Here, {β, β1 = 0.4 or 0.8, β2= 0, b}define the same policy where f(sij)
is 1; so there is a total of 112 policies to be evaluated. Moreover, in [6], the algorithm will randomly select a pa-rameter θ ∈ {0.0, 0.2, 0.4, 0.6, 0.8, 1.0} and acquire a fixed proportionθ of spot and on-demand instances from the cloud for every processed jobj.
B. Results
Experiment 1.This experiment aims tovalidate the correctness of Principle 3and is taken under the first type of input where spot and on-demand instances are considered alone.
We refer to the ratio of the total cost of utilizing various instances to the total workload processed by them as the average unit cost. The minimum unit cost of the proposed policies specified by {β1, β2, b} is 0.1921; the best policies are defined by {β1 = 0.4 or 0.8, β2 = 0, b = 0.22} where
f(sij) = 1. This validates Principle 3.
Fig. 5. The average unit cost of each policy under the first type of input.
Experiment 2. This experiment aims to validate the effective-ness of the proposed robust policies in Section III-C and is taken under the second type of input where R= 0.
Fig. 6. The average unit cost of each policy under the second type of input: the blue stars and green circles respectively correspond to the ones of the policies proposed in this paper and [6].
The average unit cost of each policy is illustrated in Figure 6. The minimum unit cost of the proposed policies specified by {β1, β2, b} is 0.1144; the optimal policy is the 27-th policy where {β1, β2, b} = {0.8, 400, 0.25} and
f(sij) = (0.8·sij)400. Using the policies in [6], the best policy is the 24th policy where {θ, b} ={1,0.25} with an average unit cost of 0.1762. The performance of the best policy of this paper improves the one in [6] by 35.07%. Starting with an initial distribution{1/n,· · · ,1/n}over them, Algorithm 3 achieves an average unit cost of 0.1218 using the n = 32 polices of this paper while it achieves an average unit cost of 0.1813 using the n = 24 polices in [6]. The proposed policies of this paper improve the performance of Algorithm 3 by 32.82%. With more and more jobs (about 4000 jobs) processed, the policies with the best performance (i.e., the 6th, 13th, 20th, and 27th policies in Figure 6) are assigned the highest probabilities; so Algorithm 3 has a performance close to the best policy in hindsight.
Experiment 3. This experiment aims to show how well the scheduling policy in Section III-B for long-term instances can realize Principle 1 and is taken under the first type of input. We assume that an organization itself possesses 50 instances, i.e., R1= 50.
After the allocation of long-term instances to a set of jobs
J under a policy defined by β, the jobs to be processed by spot and on-demand instances alone can be viewed as a new set of jobs J0. The experimental results over 60000
β% 0.00 0.25 0.35 0.40 γ& 96.38% 92.36% 84.70% 78.11% α0& 0.1991 0.1982 0.1909 0.1881 α% 0.1469 0.1498 0.1507 0.1541 TABLE I POLICYPERFORMANCE γ α0 α β= 0 96.36% 0.2075 0.1531 TABLE II ACTUALPERFORMANCE
time slots are shown in Table I. Here, γ, α0, and α denote the utilization of long-term instances, the minimum average unit cost of policies (defined by different {β, β1, β2, b}) to process J0, the minimum average unit cost of policies to
process J respectively when β = 0, 0.25, 0.35, and 0.4. In the experiment, the larger β is, the smaller α0 is. Given the limited long-term instances, choosing a larger β can reduce the allocation of them to jobs with larger slackness so as to allocate more of them to jobs with smaller slackness, which leads to that most of the jobs in J0 have relatively
larger slackness and more opportunity to utilize cheaper spot instances. However, a larger β can also lower the utilization of long-term instances greatly when the jobs arrive sparsely (every 10 minutes), thus leading to a largerα. The best policy is defined byβ = 0,f(si
j) = 1, andb= 0.22, and, the same
as the one in Experiment 1, ignoring the policy for long-term instances.
We set β to 0 and run Algorithm 3 over 90000 slots to show the performance of the scheduling framework in Algorithm 1; the experimental results are shown in Figure 7 and Table II. The performance of online learning is 1.0422 times the performance of the best policy.
Fig. 7. The circles, pluses, and stars denote the distribution over 28 policies while executing Algorithm 3 respectively over 5000, 15000, 35000 time slots.
Experiment 4. This experiment aims to validate the proposed solution for reserved instances in Section IV-B and is taken under the first type of input where R1= 50.
As illustrated in Figure 8, running Algorithm 4 over 90000 time slots, the estimated number of reserved instances to purchased is 122 and the average unit cost is 0.1267. Both Principle 3 and our experiment over the past data show that the best policy can be defined by β = 0, f(si
j) = 1 and b= 0.22, wherebdetermines the probability that a job obtains spot prices. However we still run Algorithm 3 over 60000 time slots to see how much is saved when purchasing 122 reserved
Fig. 8. The utilization and average unit cost when purchasing x reserved instances under the best policy.
instances; then, the average unit cost is 0.1303. This is close to the performance of the best policy and save the average unit cost by 14.89%, compared with the average unit cost of Table II in Experiment 3.
VI. CONCLUSION
The problem of cost-effective resource provisioning in IaaS clouds is a premier concern for all users. In this paper, we aim to provide a complete understanding of the application of online learning to this problem. In particular, we extend the online learning approach, where on-demand and spot instances had ever been addressed, to the case incorporating reserved instances and instances possessed by an user itself. Through modeling the effect of the pricing models on the instance utilization state, we characterize the expected cost-optimal scheduling principle on average to utilize spot and on-demand instances and the instances possessed by an user itself, which is determinative of the performance of this approach. Finally, with the limit of pages, simulations over synthetic datasets are done to validate the proposed theory and solutions. In the future, we are to add more simulations over synthetic datasets with more extensive parameter settings and real datasets.
REFERENCES
[1] ”Cisco Maintains Lead in Public Cloud Infrastructure while HP Leads in Private.” https://www.srgresearch.com/articles/cisco-maintains-lead-public-cloud-infrastructure-while-hp-leads-private.
[2] ”Sizing the Public Cloud Computing Market.” http://softwarestrategiesblog.com/2011/06/01/sizing-the-public-cloud/. [3] S. S. Manvi, and G. K. Shyam. ”Resource management for Infrastructure
as a Service (IaaS) in cloud computing: A survey.” Journal of Network and Computer Applications 41 (2014): 424-440.
[4] ”Featured AWS case study.” https://aws.amazon.com/ec2/purchasing-options/.
[5] O. A. Ben-Yehuda, M. Ben-Yehuda, A. Schuster, and D. Tsafrir. ”De-constructing Amazon EC2 spot instance pricing.” ACM Transactions on Economics and Computation 1, no. 3 (2013): 16.
[6] Ishai Menache, Ohad Shamir and Navendu Jain. ”On-demand, Spot, or Both: Dynamic Resource Allocation for Executing Batch Jobs in the Cloud.” In 11th International Conference on Autonomic Computing. USENIX Association, 2014.
[7] N. Jain, I. Menache, and O. Shamir. ”Allocation of Computational Resources with Policy Selection.” U.S. Patent 20,130,246,208, issued September 19, 2013.
[8] Y. Hong, J. Xue, and M. Thottethodi. ”Dynamic Server Provisioning to Minimize Cost in an IaaS Cloud.” In Proceedings of the ACM SIGMET-RICS joint international conference on Measurement and modeling of computer systems. ACM, 2011.
[9] S. Chaisiri, B. Lee, and D. Niyato. ”Optimization of Resource Pro-visioning Cost in Cloud Computing.” IEEE Transactions on Services Computing, no. 2 (2012): 164-177.
[10] S. Chaisiri, R. Kaewpuang, B. Lee, and D. Niyato. ”Cost minimiza-tion for provisioning virtual servers in amazon elastic compute cloud.” In Proceedings of International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems. IEEE, 2011. [11] R. Van den Bossche, K. Vanmechelen, and J. Broeckhove. ”IaaS
reserved contract procurement optimisation with load prediction.” Future Generation Computer Systems 53 (2015): 13-24.
[12] L. Zheng, C. Joe-Wong, C. W. Tan, M. Chiang and X. Wang. ”How to Bid the Cloud?” Annual Conference on Special Interest Group on Data Communication (SIGCOMM). ACM, 2015.
[13] P. Sharma, D. Irwin, and P. Shenoy. ”How Not to Bid the Cloud.” In the 8th Workshop on Hot Topics in Cloud Computing. USENIX, 2016. [14] W. Wang, B. Li, and B. Liang. ”Optimal Online Multi-Instance
Ac-quisition in IaaS Clouds.” IEEE Transactions on Parallel and Distributed Systems, vol. 26, no. 12 (2015): 3407-3419.
[15] A. Khan, X. Yan, S. Tao, and N. Anerousis. ”Workload characterization and prediction in the cloud: A multiple time series approach.” In Pro-ceedings of IEEE International Symposium on Network Operations and Management. IEEE, 2012.
TABLE III NOTATION
Symbol Explanation
L length of a time slot (e.g., 5 minutes)
Len the number of time slots in an hour, i.e., 60L
J a set of jobs that arrive over time
j a job ofJ
aj arrival time ofj
dj relative deadline:a jmust be completed by a deadline j+dj−1
zj job size ofj, measured in CPU×time slots
δj
parallelism bound ofj, determining the maximum number of machines that can be simultaneously utilized byj
lenj
the minimum execution time ofjwhen allocated the maximum number of instances from its execution until completion, i.e., zj
δj sj the slackness, i.e.,
dj
lenj
Dj the total number of allocations tojuntil its
com-pletion
ri
j(resp.rj(t))
the number of long-term instances allocated to it at thei-th allocation ofj(resp. at time slott)
siij(resp.
sij(t))
the number of spot instances that are bid for, at the
i-th allocation ofj(resp. at time slott)
oij(resp.oj(t))
the number of on-demand instances allocated to it, at thei-th allocation ofj(resp. at time slott)
bi j
the bid price for spot instances at thei-th allocation ofj
pij
the average price of spot instances paid by a tenant at thei-th allocation ofj
zi
j(resp.zj(t))
the remaining workload ofjto be processed at the very beginning of thei-th allocation toj (resp. at the very beginning of time slott)
si
j(resp.sj(t))
the slackness at the very beginning of thei-th allo-cation toj(resp. at the very beginning of time slot
t), i.e., (dj−(i−z1)·Len)·δj
j(t) (resp.
(aj+dj−t)·δj
zj(t) ) p the price of on-demand instances
p1 the price of using an instance possessed by the
organization in a unit of time
p2 the price of a reserved instance
R1 the total number of computing instances possessedby an organization
R2 the number of reserved instances to be purchased
R the total number of long-term instances, i.e.,R1+ R2 Xi j,1, Xij,2, Xi j,3, Tj,1i , Ti j,2
random variables characterizing the state in which spot instances are utilized byjat thei-th allocation (see Paragraph 2 of Section II-D for more explana-tion.)
T the number of time slots, i.e.,maxj∈J{aj}
P a set of policies, each indexed byπ
{β, β1, β2, b}
a tuple of parameters that defines a policy and determines the allocation of various instances toj
at thei-th allocation
f(si j)
a function defined by{(β1·sij)β2, 1}and
deter-mining the proportion of spot instances allocated to
j
T0 the length of a period in which a set of jobsrecurs J0
rj the number of long-term instances allocated to a jobjat everyt∈[a j, aj+dj−1]
J0
0
after the allocation of long-term instances to J0, the virtual jobs to be processed by spot and on-demand instances alone, each with an arrival time
aj, a deadlinedj, a workloadzj−rj·dj, and a
