ENERGY EFFICIENCY
Traffi
c-Aware Resource
Provisioning for
Distributed Clouds
Dan Xu, AT&T Labs
Xin Liu, University of California, Davis
Athanasios V. Vasilakos, Lulea University of Technology, Sweden
Examining important cloud traffic characteristics and optimizations
produced fine-grained traffic-awareness approaches that can more
efficiently reduce energy costs for distributed clouds with dynamic,
diverse traffic.
loud-computing-based traffi c has been rapidly growing in recent years. Cisco forecasted that an-nual global datacenter IP traf-fi c will reach 7.7 zettabytes by the end of 2017, with its cloud IP traffi c reaching 5.3 zettabytes.1 Correspondingly, the service providers, including Google, Microsoft, Facebook, and AT&T, are building and expanding their datacenters nationwide and worldwide. Such geographically distributed datacenters are often re-ferred to as Internet datacenters (IDCs) and we use
cloud as a general term that refers to an IDC’s col-lection of hardware, software, and services. An IDC typically consumes many megawatts of power, which imposes a signifi cant electricity cost to its operator. For example, Google’s datacenters consume nearly 300 million watts annually.2
To reduce energy costs, researchers have pro-posed load-aware server provisioning schemes, in which the number of active servers is controlled dynamically based on the load.3–6 When the load is low, extra servers can be scheduled in sleeping mode. In this paradigm, obtaining traffi c volume information is a challenging issue. As the “Related Work in Resource Provisioning” sidebar describes, many researchers have worked on traffi c-aware cloud resource provisioning. However, considerable room for achieving a fi ne-grained traffi c-awareness remains. In the load-aware server provisioning schemes,4–7 researchers typically consider traffi c dynamics in a large time scale only, such as tens of minutes, during which traffi c demand (that is, input to the server provisioning algorithms) is usually as-sumed static given the current time interval for serv-er provisioning. Howevserv-er, as we obsserv-erved from a real 30 I E E E C L O U D C O M P U T I N G P U B L I S H E D B Y T H E I E E E C O M P U T E R S O C I E T Y 2 3 2 5 - 6 0 9 5 / 1 5 / $ 3 1 . 0 0 © 2 0 1 5 I E E E
datacenter traffi c trace—and as Theophilus Benson and his colleagues indicated8—cloud traffi c demand also varies over a small time scale, such as hundreds of milliseconds, and often shows burstiness.
Here, we’ll discuss the key challenges caused by traffi c dynamics in a small time scale, such as server overloading. In considering traffi c dynamics in both large and small time scales, we propose a framework of solutions that can guarantee service quality with a low cost for distributed clouds. Our solutions le-verage joint server provisioning (dam building) for large time scales and cross-IDC load shifting (water balancing) for small time scales, which are similar to water fl ood control. We discuss how cross-IDC load shifting can be designed to effi ciently and cost-effectively alleviate traffi c dynamic issues in a small time scale. Like existing work,6,9 our cross-IDC load-shifting schemes can leverage electricity price diversity to reduce energy costs.
To achieve a desirable balance between costs and service quality, it’s also important to exploit the heterogeneity of cloud jobs. Cloud jobs can be classifi ed by two general categories: delay sensitive jobs (DSJs), such as interactive jobs, and delay toler-ant jobs (DTJs), such as batch jobs and analytical
jobs. Most existing work considers only interactive jobs.4,6,7 Although some work considers both inter-active and batch jobs,10 joint resource provision-ing for both DSJs and DTJs hasn’t been studied in-depth. In this article, we further examine cloud traffi c diversity and consider joint DSJ and DTJ provisioning.
Cloud Traffi
c Dynamics and Diversity
In this section, we examine traffi c dynamics and di-versity of clouds. We fi rst show the traffi c dynamics of an IDC in different time scales.
Traffi c Dynamics and Time Scales
User request traffi c at an IDC varies over time. For example, research shows that the login rate of Win-dows Live Messenger changes signifi cantly at differ-ent hours.4 Dynamic traffi c leads to a time-varying load demand. The current traffi c or load-aware serv-er provisioning schemes often use load prediction or estimation methods to capture the variation. How-ever, those schemes address only large-time-scale traffi c variation, such as predicting the future load approximately every half hour.4 Small-time-scale traffi c variation is rarely considered.
any researchers have explored traffi c-aware cloud resource provisioning. Common approaches include prediction- or estimation-based approaches.
For example, Gong Chen and his colleagues propose statistical models to forecast Microsoft serv-ers’ load of connection-intensive requests (such as messenger login in), and design server provisioning schemes accordingly.1 Minghong Lin and his col-leagues propose basing their online server provi-sioning design on an estimate of the load size at the beginning of each time interval.2
Other work gives workload as the inputs of the resource provisioning algorithms.3,4 In those schemes, server provisioning is often jointly designed with load dispatching—that is, by determining the load at each server. To guarantee service quality, some work explic-itly considers a QoS constraint, such as a delay.3
1. G. Chen et al., “Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services,” Proc. 5th USENIX Symp.
Net-worked Systems Design and Implementation (NSDI
08), 2008, pp. 337–350.
2. M. Lin et al., “Dynamic Right-Sizing for Power-Pro-portional Data Centers,” IEEE/ACM Trans.
Network-ing, vol. 21, no. 5, 2013, pp. 1378–1391.
3. L. Rao et al., “Minimizing Electricity Cost: Opti-mization of Distributed Internet Data Centers in a Multi-Electricity-Market Environment,” Proc. IEEE
INFOCOM, 2010, pp. 1145–1153.
4. J. Tu et al., “Dynamic Provisioning in Next-Genera-tion Data Centers with On-Site Power ProducNext-Genera-tion,”
Proc. 4th Int’l Conf. Future Energy Systems
32 I E E E C L O U D C O M P U T I N G W W W . C O M P U T E R . O R G / C L O U D C O M P U T I N G
ENERGY EFFICIENCY
We analyze the traffic of an IDC operated by a major US service provider. We extract the traffic in-formation from the IDC’s Hadoop Distributed File System (HDFS) log and plot a one-second traffic cap size for a three-minute snapshot (see Figure 1). As the figure shows, the one-second traffic exhibits an obvi-ous on/off pattern. Other research shows the on/off traffic pattern for a Microsoft datacenter,8 in a time granularity of 15 and 100 milliseconds, respectively.
Small-time-scale traffic burstiness is difficult to address for two reasons. First, it’s hard to predict traffic variation in a small time scale. Second, it’s im-possible to perform dynamic server provisioning ac-cording to the small-time-scale traffic variation due to the large time delay in turning on servers. Thus, server provisioning can be performed only in a large time scale, such as every tens of minutes to hours.
Traffic burstiness might incur both congestion and server idleness at different times. Sometimes, traffic spikes impose a load that’s larger than the available capacity. This overloading degrades service performance, such as by incurring a large service delay. It shouldn’t happen frequently because most Internet requests, such as search and Web brows-ing, tolerate only a small time latency. When traffic goes lower—that is, it goes into off cycles—servers become idle and power is wasted.
To address these issues, we view the IDCs as rivers or lakes and overloading as flooding. We de-sign joint server provisioning and cross-IDC load shifting, with server provisioning performed in a large time scale and considered as dam building. In a small time scale, cross-IDC load shifting is
performed to reduce traffic burstiness such that a smaller number of servers (a lower dam) is needed at each IDC. Cross-IDC load shifting is like water bal-ancing among rivers, lakes, and reservoirs. We dis-cuss efficient load-shifting policies that can reduce total energy costs for distributed clouds.
The traffic we consider here mainly refers to in-teractive jobs. In a cloud, many other types of jobs might require provisioning in different ways, as we now describe.
Cloud Traffic Diversity
As we noted earlier, IDC traffic comes from both DSJs and DTJs. Resource provisioning to DTJs should be different from DSJs. DSJs have a small service delay and thus must be served promptly. In contrast, DTJs can be buffered in queues. DSJs have a higher service priority than DTJs, so capac-ity resource for DSJs must be guaranteed first. The remaining server capacity, which varies instanta-neously due to dynamic DSJ traffic demand, can then serve DTJs via dynamic speed scaling. This paradigm is called valley filling—that is, using DTJs
to fill the DSJ load-demand valley (see Figure 2). Our discussion of interactive job traffic refers to DSJ traffic. When traffic is low, some servers become idle. In this case, they can be allocated to process DTJs rather than shut down. The cross-IDC load shifting for DSJs also fills the valley of some IDCs. But this load shifting doesn’t cause valleys to disappear; there’s still some remaining capacity to utilize. DTJs can also be shifted from one IDC to another, such as to exploit more efficient
serv-0 100 200 300 400 500 600 700 800 900 1000 1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97 103 109 115 121 127 133 139 145 151 157 163 169 175
Traffiic size (Mbit)
Time (in seconds of a three-minute snapshot)
Time granularity:1 sec
FIGURE 1. Datacenter traffic burstiness. Traffic trace is collected from the datacenter of a major US service provider.
ers or a lower electricity price in another location. Also, DSJs or DTJs can be further divided by dif-ferent classes according to job types and service re-quirements. For example, search requests and Web browsing requests can be considered as two different DSJ classes, whereas user information backup and weblog analysis can be two different DTJ classes.
Building Dams and Balancing Water
Using “water” and “damming” as metaphors for IDC resource provisioning applies to DTJ provisioning, which is like filling a river valley: the water (of DSJs) is below the height of the dam (an IDC’s total active server capacity). Efficient valley filling is challeng-ing, however. On one hand, DTJ load demand is high, which often incurs a large energy cost. On the other hand, DTJs also desire as small a delay as possible. Moreover, valley filling must be jointly designed with DSJ provisioning. Here, we discuss joint server provi-sioning, cross-IDC load shifting, and valley filling for distributed clouds. Our design goal is to minimize the total costs, including energy costs and load-shifting costs, while satisfying DSJ QoS constraints and assur-ing a desirable delay performance for DTJs. Given the problem’s complexity, we first focus on DSJs only, dis-cussing joint server provisioning and cross-IDC load shifting; we then address how valley filling can jointly work with server provisioning and load shifting. Measuring Server Overloading
As discussed earlier, the small-time-scale DSJ traf-fic burstiness can lead to overloading—that is, a situation in which load demand exceeds the capacity provisioned. To measure overloading, we introduce a QoS metric called overloading probability, which is
defined as the probability that the DSJ load demand is larger than the available server capacity.
As Figure 2 shows, IDC overloading can be tem-porally monitored. We can set a small time interval, such as hundreds of milliseconds, to track over-loading and then set a measurement period, such as hours, to calculate overloading probability. We can monitor overloading probability by first count-ing the number of time intervals with overloadcount-ing, and then dividing them by the total number of small time intervals during the measurement period. In practice, we can use a slide window of measurement periods to update an IDC’s overloading probability. We can directly translate overloading probability into a DSJ’s QoS requirements (that is, its service latency requirements) because the DSJ load demand is implicitly based on a given service latency require-ment. A smaller service latency requirement leads to a larger load demand given the same amount of traffic. We can consider overloading probability as the percentage of time that the service latency re-quirement can’t be guaranteed. A small overloading probability requirement is desirable for guaranteeing DSJ service quality.
To reduce the overloading probability, we can increase the available capacity by provisioning more active servers. This is like building a higher dam to prevent flooding. However, turning on many addi-tional active servers isn’t cost effective. To address this issue, traffic burstiness at each IDC must be reduced first so a lower capacity can meet the same overloading probability requirement.
Cross-IDC Load Shifting
To reduce traffic burstiness, we designed cross-IDC load-shifting schemes to leverage traffic’s statisti-cal multiplexing gain. Although existing work has considered cross-IDC load shifting for exploiting electricity price diversity,9 how to use load shifting
Overloading Overloading Overloading Overloading Overloading
Time
FIGURE 2. Delay sensitive jobs (DJBs) overload and fill valleys by delay tolerant jobs (DTJs). The DSJs’ load demand exceeds total available capacity at four time intervals. If the measurement period starts from the first interval and lasts to the last interval (that is, the period is 34 time intervals), the overloading probability is calculated as 4/34 =11.7 percent.
34 I E E E C L O U D C O M P U T I N G W W W . C O M P U T E R . O R G / C L O U D C O M P U T I N G
ENERGY EFFICIENCY
to efficiently reduce traffic burstiness and further achieve cost effectiveness hasn’t been investigated. By load shifting, an IDC’s excessive traffic can be processed by another IDC that has lower traffic de-mand. Thus, load demand at each IDC is smoothed. Here, we consider different policies for load shifting. First, we consider a policy in which each IDC instantaneously shifts its traffic to another one according to a fixed ratio. Consider Di as the traf-fic volume of IDC i, which is a time-varying random variable. IDC i shifts its traffic to IDC j according to the current Di and a fixed ratio rij (that is, rijDi). Any pair of IDCs can shift traffic to each other. Thus, we call this scheme ratio-based load swapping. In RBLS, traffic shifting ratio rij is a control variable to optimize. We must consider many factors when optimizing rij. For example, if IDC j has a large maximum server capacity, rij is likely to be large. If the current electricity price at IDC j is low, rij might also be large to let IDC i shift more traffic to IDC j. Thus, RBLS can exploit various factors to reduce energy costs and explore server capacity more effi-ciently; nevertheless, it also uses statistical multi-plexing gain to smooth traffic at each IDC.
However, RBLS can be expensive to implement because the load-shifting numbers can be large— that is, O(N2), where N is an operator’s number of IDCs. We thus consider offloading polices, in which each IDC shifts traffic only to a selected offloader. In offloading, each IDC can shift traffic based on a fixed ratio—that is, ratio-based offloading. In RBO,
distribution of the remaining traffic demand at each IDC still follows the same (scaled-down) distribu-tion as the original load demand. Alternatively, each IDC can shift the load beyond the available capac-ity to the offloader; this is called threshold-based offloading. Intuitively, TBO is more efficient than
RBO in terms of decreasing overloading probability and saving energy costs. This is because, in RBO, each IDC’s load demand is still random. A large server capacity is still needed to constrain server overloading, especially when traffic demand at each IDC is heavy-tail distributed. Although in TBO, the “excessive” traffic of all IDCs is aggregated at the offloader, we can expect a higher statistical mul-tiplexing gain. TBO is similar to discharging flood from rivers to a reservoir—that is, only the amount of water (traffic) that is beyond the dam (maximum active server) capacity is offloaded.
Server Provisioning and Load Shifting
Cross-IDC load shifting is performed in a small time scale to capture the small time scale’s traffic varia-tion. In TBO, configuring load shifting—such as
determining traffic-splitting ratios in RBLS or deter-mining each IDC’s number of servers (that is, the threshold for offloading—is coupled with server pro-visioning, which is performed in a large time scale. For a relatively large time interval, load shifting fol-lows the same configuration.
The joint server provisioning and load-shifting schemes need each IDC’s traffic information. To calculate overloading probability, we need the DSJ traffic’s mean and variance (before load shifting). Assuming a certain distribution of the DSJ traffic, we can calculate the overloading probability based on the DSJ traffic’s mean and variance. We require an overloading probability constraint—that is, it must be smaller than a predefined value—to guaran-tee service quality. Our objective is to minimize all IDCs’ costs, including energy costs and load-shifting costs. An IDC’s energy cost is the product of the electricity price and the total power consumption, which is proportional to that the servers consume. Further, server power consumption can be modeled by an increasing function of both the number of ac-tive servers and the average load.
Problem 1
Joint sever provisioning and load shifting for DSJs can be formulated by the following optimization problem (Problem 1). The main inputs for Problem 1 are each IDC’s traffic statistics (mean and variance). The objective is to minimize the total energy and load-shifting costs. Problem 1 has three constraints:
• An overloading probability constraint at each IDC.
• A server allocation constraint at each IDC (the number of active servers at each IDC is upper limited by the total number of servers).
• A load-shifting constraint between two IDCs (the traffic amount shifted is bounded by band-width between two IDCs).
Problem 1’s outputs are the number of active servers at each IDC and the parameters of traffic shifting.
Although DSJ traffic is dynamic, Problem 1 is a deterministic optimization problem based on traffic statistics information that remains static for a large time interval. The variation of traffic statistics right-ly represents the large-time-scale traffic dynamics we discussed earlier. Thus, Problem 1 captures traf-fic dynamics in both the large and small time scales. We formulated Problem 1 with different cross-IDC load-shifting schemes—RBLS, RBO, and TBO—using sophisticated convex optimization models.11,12 Figure 3 compares the performance of
the different schemes, each of which jointly works with server provisioning. In the simulations, we con-sider 15 IDCs, each of which has a maximum of 100 servers. Each server’s maximum CPU frequency is normalized as 1. Server capacity or load is defined as the total CPU frequency of all active servers. The number of active servers, constrained by 100, is a control variable to optimize. Load demand (before load shifting) at each IDC follows an exponential distribution, with a mean of 10. We randomly select-ed each IDC’s electricity price from one to five. The figure’s x-axis shows the overloading probability
con-straint at each IDC. Clearly, a smaller overloading probability constraint leads to more active servers by the proposed optimization model for each scheme, and thus a higher energy cost. In RBO and TBO, we selected one IDC as the offloader.
As the figure shows, RBLS always leads to the smallest cost, because it can achieve traffic statisti-cal multiplexing gain at every IDC. TBO is second. Although RBO performs the worst, it’s still much better than no load shifting. Our results show that it’s important to reduce traffic burstiness using an efficient cross-IDC load-shifting scheme. In prac-tice, with operation complexity considered, TBO is a desirable load-shifting scheme with dynamic DSJ traffic demand.
Filling Valleys: Joint Resource Provisioning
for DSJs and DTJs
In terms of resource usage, DSJs have a higher pri-ority than DTJs. When the current DSJ load demand is smaller than the available capacity, we provision the remaining capacity to DTJs, which is like filling a river valley.
Beyond a Best-Effort Scheme
Intuitively, it seems that valley filling provides best-effort provisioning to DTJs, because they can use only the capacity remaining. However, valley fill-ing isn’t simply best effort; DTJ provisionfill-ing must be carefully optimized together with DSJs. As we indicated earlier, DTJs contribute to energy costs and also require reasonable service latency. Thus, when performing server provisioning—that is, when determining total capacity—we can’t consider only the DSJ overloading issue; we also must consider the DTJ performance delay. Otherwise, even DTJ queue stability can’t be guaranteed. Thus, valley filling must be jointly configured with server provisioning in a large time scale to determine how much average capacity DTJs should receive. In a small time scale, following the configuration, instantaneous DTJ ca-pacity allocation is performed according to current DSJ load demand. Figure 4 shows the joint server provisioning, cross-IDC load shifting, and DSJ/DTJ capacity allocation scheme.
Configuring Valley Filling
As Figure 4 shows, without DTJs, server provision-ing in Problem 1 is deterministic and performed independently for each (large) time interval. This is because the overloading probability constraint for DSJs must be satisfied within each large time interval. Otherwise, for some large intervals, DSJs will experience poor performance. With DTJs, joint server provisioning and valley-filling configuring aren’t independent for different large time intervals. In this case, DTJ delay performance, such as queue stability, is measured over a long time period, which might cross many large time intervals. Also, DTJs
0 200 400 600 800 1,000 1,200 0.1 0.05 0.01 0.005 Total cost ($)
Overloading probability constraint at each IDC
FIGURE 3. Simulation results of three joint server provisioning schemes based on convex optimization models: ratio-based load swapping (RBLS), ratio-based offloading (RBO), and threshold-based offloading (TBO). Costs at an Internet datacenter (IDC) are the product of its power consumption and electricity price.
36 I E E E C L O U D C O M P U T I N G W W W . C O M P U T E R . O R G / C L O U D C O M P U T I N G
ENERGY EFFICIENCY
can be buffered in a long queue, whereas DSJs must be served in real time. Moreover, configuring valley filling must incorporate both load variation of DSJs and price temporal dynamics. For example, if the electricity price is potentially lower in the next large time interval, less capacity can be allocated to DTJs in the current large time interval. Historic informa-tion can also affect the current decision, such as by affecting the DTJ queue.
Problem 2
We now consider Problem 2, a stochastic problem of joint server provisioning, load shifting, and val-ley filling, that we designed based on Problem 1. Here, we can shift the load of DTJs from one IDC to another. For DTJ load shifting, the purpose isn’t to exploit statistical multiplexing gain, because DTJs don’t have issues with small-scale traffic dynamics. For DTJs, traffic can be smoothed by buffering. DTJ load shifting can exploit a lower electricity price and a higher service rate in another IDC. Given their different purposes, the structures of DSJ and DTJ load shifting are also different.
First, the objective of Problem 2 is to minimize the total costs, which also include energy costs con-tributed by DTJs. Unlike Problem 1, the total costs here aren’t for each single (large) time interval, but rather the average costs over all large time intervals. Second, there are additional constraints. The sum of load demand of DSJs and DTJs must be smaller than the total available capacity, which is the capac-ity allocation constraint. To guarantee DTJ queue
stability, the average queue length should be up-per bounded. Or, alternatively, the average service rates for DTJs should be larger than their average arrival rates. We determine the service rate at each IDC for DTJs by the capacity allocated. Neverthe-less, the overloading probability, server allocation, and load-shifting constraints also must be satisfied. Thus, Problem 1’s constraints constitute a subset of Problem 2’s constraints.
Leveraging DTJ queue information. Problem 2 is dif-ficult because DSJs and DTJs must be jointly consid-ered, along with the traffic dynamics of DSJs and the stochastic properties of DTJ provisioning. We
devel-oped several different sophisticated algorithms that solve Problem 2.13,14 Our proposed algorithms barely leverage system statistical information—which is
difficult to obtain in practice—yet nonetheless guar-antee the cost and delay bounds. One important
dif-ference among our schemes is whether DTJ queue
backlog information at each IDC is leveraged or not;
note that when a class of DTJs is shifted to another
IDC, a sub-queue is created for that DTJ class in the
destination IDC.
We evaluate performance of a DTJ queue-based valley-filling scheme that leverages the current DTJ queue information to adjust capacity for DTJs. The DTJ queue-based scheme’s basic idea is to achieve a tunable tradeoff between DTJ queue length and IDC energy costs. The scheme’s objective function is to minimize the sum of two functions of the control variable vector—that is, the server capacity assigned
Stochastic
Correlated with other time intervals Deterministic
Independent of other time intervals
Large time interval (scale) Small time interval (scale)
Configuring Cross-IDC load shifting Server provisioning Estimate DSJ traffic statistics Instantaneous
load shifting Instantaneousvalley filling
Configuring valley filling Problem 2 Estimate DSJ traffic statistics Problem 1
FIGURE 4. Joint server provisioning, load shifting, and valley filling at different time scales. Without delay tolerant jobs (DTJs), server provisioning in Problem 1 is deterministic and performed independently for each (large) time interval.
to each class of DTJs at each IDC. One function is increasing with the control variable vector, such as the energy cost function, which is weighted by a control parameter V. The other one is decreasing
with the control variable vector, such as a weighted linear sum of server capacity for each DTJ class at each IDC, multiplied by –1, where the server capac-ity’s weight for each class of DTJs at each IDC is the current queue length of the DTJ class.
Clearly, when queue length of a DTJ class be-comes large, the decreasing function has a large weight, and a high server capacity will be assigned to this DTJ class due to the minimization objective; this in turn reduces the queue length of the DTJ class. The control parameter V is to tune the
trad-eoff between energy costs and DTJ queue length. Setting a large value of V gives the increasing
func-tion a large weight, which leads to a small server capacity due to the objective of minimization, and thus reduces energy costs. When V is set small, the
decreasing function has a large weight and thus a large server capacity will be assigned, which further reduces DTJ queue length. The scheme’s constraints are the same as those in Problem 2.
We compared the queue-based scheme to a service-rate based scheme that doesn’t leverage any queue length information but does guarantee long-term queue stability. The service-rate based scheme almost achieves the minimum cost with DTJ queue-stability assured. As Figure 5 shows, the queue-based scheme achieves a significantly smaller queue delay for a slightly larger cost. This result implies the
impor-tance of leveraging DTJ queue information in valley filling. Details of this study are available elsewhere.13
The DTJ queue-based provisioning scheme in-tuits that capacity allocation for each class of DTJs at each IDC is correlated with the queue length of the class of DTJs. When a DTJ queue in an IDC is long, a larger server capacity is assigned to reduce queue length and vice versa. In practice, we can tune queue-based algorithms to achieve a desirable tradeoff between energy costs and DTJ queue delay.
Coupling load shifting to capacity allocation. Intui-tively, the classic stochastic optimization method— back-pressure routing15—can be a good candidate for DTJ load shifting among IDCs because it guar-antees DTJ queue stability. Back-pressure routing’s basic idea is to shift more of a DTJ class’s traffic to an IDC, which has less traffic than the class of DTJs. However, we found that back-pressure routing isn’t optimal in reducing energy costs in the dis-tributed clouds scenario. This is because, in back-pressure routing, the decision of cross-IDC DTJ load shifting is mainly based on the difference between the amount of DTJ class traffic in both the original IDC and each destination IDC. Cross-IDC load shift-ing isn’t directly correlated with capacity allocation in each IDC. Thus, back-pressure routing doesn’t lever-age electricity price location diversity well.
To address this issue, we designed a cross-IDC DTJ load-shifting scheme that’s closely coupled to server capacity allocation at each IDC. The scheme’s basic idea is that the amount of a DTJ class load
0 50 100 150 200 250 300 350 400 0 5 10 15 20 25 30 10 50 70 90
DTJ queue delay (mins)
Total cost ($)
x 10,000
Percentage of DTJ load demand out of total load demand
FIGURE 5. Performance of valley filling that leverages queue information of delay tolerant jobs (DTJs). Here, five Internet datacenters (IDCs) are considered, with 15 classes of delay sensitive jobs (DSJs) and 10 classes of DTJs; load demand between the DSJs and DTJs is comparable.
38 I E E E C L O U D C O M P U T I N G W W W . C O M P U T E R . O R G / C L O U D C O M P U T I N G
ENERGY EFFICIENCY
that’s shifted to another IDC depends only on how much server capacity in the destination IDC is as-signed for the DTJ class. As Figure 6 shows, our scheme achieves a much smaller energy cost and a much smaller DTJ queue delay compared to the cross-IDC DTJ load-shifting scheme based on back-pressure routing.15
any challenges and opportunities exist related to efficient cloud management:
• Traffic dynamics might need to be considered in different time granularities.
• Applications with different resource require-ments—such as CPU, I/O, disk, and memory— must be better coordinated and provisioned. • Multiplexing different applications by
virtual-ization could further improve efficiency.
• Jobs with different QoS or service deadlines must be further prioritized.
• Accurately modeling server energy costs and load-shifting costs is challenging.
We will consider all of these issues in our future work.
References
1.
Cisco Global Cloud Index: Forecast andMethod-ology, 2012–2017, white paper, Cisco Systems,
2013.
2.
J. Glanz, “Power, Pollution and the Internet,”New York Times, 22 Sept. 2012; www.nytimes
.com/2012/09/23/technology/data-centers-waste -vast-amounts-of-energy-belying-industry-image .html.
3. Y.Y. Chen et al., “Managing Server Energy and
Operational Costs in Hosting Centers,” Proc. ACM SIGMETRICS Int’l Conf. Measurement and Modeling of Computer Systems, 2005, pp.303–314.
4. G. Chen et al., “Energy-Aware Server
Provi-sioning and Load Dispatching for Connection-Intensive Internet Services,” Proc. 5th USENIX Symp. Networked Systems Design and Implemen-tation (NSDI 08), 2008, pp. 337–350.5. M. Lin et al., “Dynamic Right-Sizing for
Power-Proportional Data Centers,” IEEE/ACM Trans. Networking, vol. 21, no. 5, 2013, pp. 1378–1391.6. L. Rao et al., “Minimizing Electricity Cost:
Opti-mization of Distributed Internet Data Centers in a Multi-Electricity-Market Environment,” Proc. IEEE INFOCOM, 2010, pp. 1145–1153.7. J. Tu et al., “Dynamic Provisioning in
Next-Generation Data Centers with On-Site Power Production,” Proc. 4th Int’l Conf. Future Energy Systems (e-Energy 13), 2013, pp. 137–148.8. T. Benson et al., “Understanding Data
Cen-ter Traffic CharacCen-teristics,” ACM SIGCOMM Computer Comm. Rev., vol. 40, no. 1, 2010, pp.92–99.
9. R. Stanojevic and R. Shorten, “Distributed
Dy-namic Speed Scaling,” Proc. IEEE INFOCOM, 2010, pp. 426–430.10. H. Xu and B. Li, “Temperature Aware Workload
Management in Geo-Distributed Datacenters,”0 200 400 600 800 1,000 1,200 1,400 1,600 1,800 70,000 75,000 80,000 85,000 90,000 95,000
Joint back-pressure routing based cross-IDC DTJ load shifting and capacity allocations scheme
Closely coupling cross-IDC DTJ load shifting to capacity allocation
DTJ queue delay (sec)
Total cost ($)
Total costs (Dollar) DTJ queue delay (sec)
FIGURE 6. Simulation results comparing the cross-IDC load-shifting scheme based on back-pressure routing with our scheme of closely coupling cross-IDC delay tolerant job (DTJ) load shifting with capacity allocation. The simulations considered five IDCs and 10 classes of DTJs. The result shows the importance of closely coupling cross-IDC DTJ load shifting with capacity allocation in cloud resource provisioning.
Cost for Internet-Scale Datacenters with Dy-namic Traffic,” Proc. IEEE 19th Int’l Workshop on Quality of Service (IWQoS), 2011, pp. 1–2.
12. D. Xu, X. Liu, and B. Fan, “Efficient Server
Provisioning and Offloading Policies for Inter-net Datacenters with Dynamic Load Demand,”IEEE Trans. Computers, vol. 64, no. 3,
pp.682-697, Mar. 2014; doi:10.1109/TC.2013.2295797.
13. D. Xu and X. Liu, “Geographic Trough Filling for
Internet Datacenters,” Proc. IEEE INFOCOM,
2012, pp. 2881–2885.
14. L. Georgiadis, M.J. Neely, and L. Tassiulas,
“Re-source Allocation and Cross-Layer Control in Wireless Networks,” Foundations and Trends in Networking, vol. 1, no. 1, pp. 1–149, 2006.15. D. Xu, X. Liu, and Z. Niu, “Joint Resource
Provi-sioning for Internet Datacenters with Diverse and Dynamic Traffic,” IEEE Trans. Cloud Comput-ing, preprint, doi:10.1109/TCC.2014.2382118.energy efficiency design, along with 3G/4G mobile networks performance modeling and analysis. Xu has a PhD in computer science from the University of Cali-fornia, Davis. Contact him at [email protected].
XIN LIU is a professor in the Computer Science De-partment at the University of California, Davis. Her research interests include wireless communication net-works. Liu has a PhD in electrical engineering from Purdue University. She’s a Chancellor’s Fellow and a member of IEEE. Contact her at [email protected].
ATHANASIOS V. VASILAKOS is a professor in the Department of Computer Science, Electrical, and Space Engineering at the Lulea University of Technol-ogy, Sweden. His research interests include cloud com-puting, cyber physical systems, and green computing/ networking. He’s a senior member of IEEE. Contact him at [email protected].
Advertising Personnel
Marian Anderson: Sr. Advertising Coordinator Email: [email protected]
Phone: +1 714 816 2139 | Fax: +1 714 821 4010 Sandy Brown: Sr. Business Development Mgr. Email [email protected]
Phone: +1 714 816 2144 | Fax: +1 714 821 4010
Advertising Sales Representatives (display)
Central, Northwest, Far East: Eric Kincaid
Email: [email protected] Phone: +1 214 673 3742 Fax: +1 888 886 8599
Northeast, Midwest, Europe, Middle East: Ann & David Schissler
Email: [email protected], [email protected] Phone: +1 508 394 4026 Fax: +1 508 394 1707 Southwest, California: Mike Hughes Email: [email protected] Phone: +1 805 529 6790 Southeast: Heather Buonadies Email: [email protected] Phone: +1 973 304 4123 Fax: +1 973 585 7071
Advertising Sales Representatives (Classified Line)
Heather Buonadies
Email: [email protected] Phone: +1 973 304 4123
Fax: +1 973 585 7071
Advertising Sales Representatives (Jobs Board)
Heather Buonadies
Email: [email protected] Phone: +1 973 304 4123
Fax: +1 973 585 7071
