In this report, we present the design of a performance-aware energy-efficient storage sys-tem for data centers that combines the following new ideas: (1) power-aware storage cache replacement algorithms and write polices that can help the underlying disk energy man-agement scheme save more energy and provide better response time; (2) a coarse-grain disk-speed selection algorithm (CR) that supports multiple data layouts; (3) an adaptive performance-directed energy-efficient data layout that requires no extra disks and sacrifices no reliability; and (4) a response-time performance guarantee method that can be coupled with many energy control algorithms. We also study the interaction of these different storage components and how future storage trends may affect the design of such systems.
Using both simulations with real system traces and emulated system evaluation, our re-sults show that our scheme is effective in overcoming the current limitations in energy man-agement for high-end storage systems, thereby making it practical in real systems. Specif-ically, our scheme is able to save the most amount of energy when compared to previous solutions, while still meeting the response-time performance requirements in data centers.
There are many potential areas where our results could be extended. For example:
performing real system energy measurement and performance evaluation by using commeri-cial multi-speed disks; broadening the workloads used, to include decision support systems (DSS), mixed workload types, and larger-scale ones; leveraging intelligent disk scheduling algorithms, such as free block scheduling [LSG+00], to improve the efficiency of disk re-organization and data migration; incooperating flash memory into our system design and studying the energy implications of such a hybrid system.
In summary, we believe that our design is an attractive energy management design for high-end storage system in data centers: one that offers significant energy savings, while meeting performance requirements, using simple, effective algorithms that themselves have wider applicability.
References
[AHK+02] Eric Anderson, Michael Hobbs, Kimberly Keeton, Susan Spence, Mustafa Uysal, and Alistair Veitch. Hippodrome: running circles around storage administration.
In Proceedings of the First USENIX Conference on File and Storage Technologies, 2002.
[Alb99] David H. Albonesi. Selective cache ways: on-demand cache resource allocation. In Proc. of the 32nd Annual International Symposium on Microarchitecture, 1999.
[Ame03] American Power Convension, Write Paper. Determining total cost of ownership for date centers and network room infrastructure.
ftp://www.apcmedia.com/salestools/CMRP-5T9PQG R2 EN.pdf, 2003.
[Bel66] L. A. Belady. A study of replacement algorithms for a virtual-storage computer.
IBM Systems Journal, 5(2):78–101, 1966.
[Bit04] BitMicro Networks. Flash ssds - inferior technology or closet superstar?
http://www.bitmicro.com/press resources flash ssd.php, 2004.
[Blo70] Burton H. Bloom. Space/time trade-offs in hash coding with allowable errors.
Commun. ACM, 13(7):422–426, 1970.
[BM01] R. Iris Bahar and Srilatha Manne. Power and Energy Reduction Via Pipeline Balancing. In ISCA-28, 2001.
[CD01] J. Chase and R. Doyle. Balance of power: Energy management for server clusters.
In Proceedings of the 8th Workshop on Hot Topics in Operating Systems(HotOS), May 2001.
[CDQ+05] Yiyu Chen, Amitayu Das, Wubi Qin, Anand Sivasubramaniam, Qian Wang, and Natarajan Gautam. Managing server energy and operational costs in hosting centers. In Proc. of the International Conference on Measurement and Modeling of Computer Systems, June 2005.
[CG02a] Dennis Colarelli and Dirk Grunwald. Massive arrays of idle disks for storage archives. In Proceedings of the 2002 ACM/IEEE conference on Supercomputing, pages 1–11. IEEE Computer Society Press, 2002.
[CG02b] Dennis Colarelli and Dirk Grunwald. Massive arrays of idle disks for storage archives. In Proceedings of the 2002 ACM/IEEE Conference on Supercomputing, pages 1–11, June 2002.
[CJZ06] Feng Chen, Song Jiang, and Xiaodong Zhang. Smartsaver: turning flash drive into a disk energy saver for mobile computers. In ISLPED ’06: Proceedings of the 2006 international symposium on Low power electronics and design, pages 412–417, 2006.
[CLG+94] Peter M. Chen, Edward K. Lee, Garth A. Gibson, Randy H. Katz, and David A.
Patterson. Raid: high-performance, reliable secondary storage. ACM Comput.
Surv., 26(2):145–185, 1994.
[CPB03a] E. Carrera, E. Pinheiro, and R. Bianchini. Conserving disk energy in network servers. In Proc. of the 17th International Conference on Supercomputing, June 2003.
[CPB03b] Enrique V. Carrera, Eduardo Pinheiro, and Ricardo Bianchini. Conserving disk energy in network servers. In Proceedings of the 17th annual international confer-ence on Supercomputing, pages 86–97. ACM Press, 2003.
[CPL01] ILOG CPLEX 7.5 User’s Manual. http://www.usherbrooke.ca/informatique/ressources/
documentation/data/cplex7.5/CPLEX75 userman.pdf, Nov 2001.
[CZL03] Zhifeng Chen, Yuanyuan Zhou, and Kai Li. Eviction-based cache placement for storage caches. In Proceedings of USENIX Technical Conference, San Antonio, Texas, June 2003.
[Des04] Hitachi Power and Acoustic Management - quietly cool. www.hitachigst.com /tech/techlib.nsf/techdocs/EBB67181ACB207C586256D340075B4DF/$file/WP -PowerAcoustic 25March.pdf, March 2004.
[DKB95] Fred Douglis, Padmanabhan Krishnan, and Brian Bershad. Adaptive disk spin-down policies for mobile computers. In Proc. 2nd USENIX Symp. on Mobile and Location-Independent Computing, 1995.
[Don04] Dong Li and Peng Gu and Hailong Cai and Jun Wang. EERAID: Energy-efficient Redundant And Inexpensive Disk Array. 11th ACM SIGOPS European Workshop, September 2004.
[EMC99] EMC Corporation. Symmetrix 3000 and 5000 Enterprise Storage Systems product description guide., 1999.
[FCAB00] Li Fan, Pei Cao, Jussara Almeida, and Andrei Z. Broder. Summary cache: a scalable wide-area Web cache sharing protocol. IEEE/ACM Transactions on Net-working, 8(3):281–293, 2000.
[GHL04] Chris Gniady, Y Charlie Hu, and Yung-Hsiang Lu. Program counter based tech-niques for dynamic power management. In 10th International Symposium on High Performance Computer Architecture, pages 24–35, February 2004.
[GIS+95] Richard A. Golding, Peter Bosch II, Carl Staelin, Tim Sullivan, and John Wilkes.
Idleness is not sloth. In USENIX Winter, 1995.
[GSKF03a] S. Gurumurthi, A. Sivasubramaniam, M. Kandemir, and H. Franke. DRPM:
Dynamic speed control for power management in server class disks. In Proceedings of the International Symposium on Computer Architecture, pages 169–179, June 2003.
[GSKF03b] S. Gurumurthi, A. Sivasubramaniam, M. Kandemir, and H. Franke. DRPM:
dynamic speed control for power management in server class disks. In Proc. of the 30th International Symposium on Computer Architecture, June 2003.
[GWP] Gregory R. Ganger, Bruce L. Worthington, and Yale N. Patt. The DiskSim simulation environment - version 2.0 reference manual. cite-seer.nj.nec.com/article/ganger99disksim.html.
[GZS+03] S. Gurumurthi, J. Zhang, A. Sivasubramaniam, M. Kandemir, H. Franke, N. Vi-jaykrishnan, and M.J. Irwin. Interplay of energy and performance for disk arrays running transaction processing workloads. In Proceedings of the International Symposium on Performance Analysis of Systems and Software, March 2003.
[Hil87] Mark D. Hill. Aspects of Cache Memory and Instruction Buffer Performance.
PhD thesis, Unversity of Berkeley, 1987.
[HLSS00] David P. Helmbold, Darrell D. E. Long, Tracey L. Sconyers, and Bruce Sherrod.
Adaptive disk spin-down for mobile computers. Mobile Networks and Applications, 5(4):285–297, 2000.
[HPC03] HP announces “smart” cooling solution for data centers.
http://www.hp.com/hpinfo/newsroom/press/2003/030304b.
html, March 4 2003.
[HPT03] HP Integrity RX5670 Cluster 64P Executive Summary.
http://www.tpc.org/results/individual results/HP/HP%20Integrity
%20rx5670%20Cluster%2064P ES.pdf, December 2003.
[HS89] M. D. Hill and A. J. Smith. Evaluating associativity in CPU caches. IEEE Transactions on Computers, 38(12), 1989.
[IBM] IBM hard disk drive - Ultrastar 36Z15. http://www.hgst.com/hdd/ultra/ul36z15.htm.
[IBM99] IBM Enterprise Storage Server. www.storage.ibm.com/hardsoft/products/ess/ess.htm IBM Corporation, 1999.
[ISG01] S. Irani, S. Shukla, and R. Gupta. Competitive analysis of dynamic power manage-ment strategies for systems with multiple power saving states. Technical report, UCI-ICS, Sept 2001.
[JS95] T. Johnson and D. Shasha. 2Q: A low overhead high performance buffer manage-ment replacemanage-ment algorithm. In Jorgeesh Bocca, Matthias Jarke, and Carlo Zan-iolo, editors, International Conference on Very Large Data Bases(VLDB), pages 439–450, Los Altos, CA 94022, USA, 1995. Morgan Kaufmann Publishers.
[JZ02] Song Jiang and Xiaodong Zhang. LIRS: an efficient low inter-reference recency set replacement policy to improve buffer cache performance. In Proceedings of the International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), pages 31–42. ACM Press, 2002.
[KCK+00] J. Kim, J. Choi, J. Kim, S. Noh, S. Min, Y. Cho, and C. Kim. A low-overhead high-performance unified buffer management scheme that exploits sequential and looping references. Proceedings of the Fourth Symposium on Operating System Design and Implementation, 2000.
[LAL+03] Y. Lu, T. Abdelzaher, C. Lu, L. Sha, and X. Liu. Feedback control with queueing-theoretic prediction for relative delay guarantees in web servers. In Proceedings of the 9th IEEE Real-Time and Embedded Technology and Applications Symposium, 2003.
[LD93] Scott T. Leutenegger and Daniel Dias. A modeling study of the TPC-C benchmark.
SIGMOD Record, 22(2):22–31, June 1993.
[LFZE00] Alvin R. Lebeck, Xiaobo Fan, Heng Zeng, and Carla Schlatter Ellis. Power aware page allocation. In ASPLOS, pages 105–116, 2000.
[LLD+04] Xiaodong Li, Zhenmin Li, Francis David, Pin Zhou, Yuanyuan Zhou, Sarita Adve, and Sanjeev Kumar. Performance directed energy management for main memory and disks. In Proc. of the 11th International Conference on Architectural Support for Programming Languages and Operating Systems, October 2004.
[LM01] Yung-Hsiang Lu and Giovanni De Micheli. Comparing system-level power man-agement policies. IEEE Design and Test of Computers, 18(2):10–19, March 2001.
[LSG+00] Christopher Lumb, Jiri Schindler, Gregory R. Ganger, David F. Nagle, and Erik Riedel. Towards Higher Disk Head Utilization: Extracting “Free” Bandwidth from Busy Disk Drives. In Proceedings of the 4th Symposium on Operating Systems Design and Implementation, 2000.
[MGST70] R. L. Mattson, J. Gecsei, D. Slutz, and I. L. Traiger. Evaluation techniques for storage hierarchies. IBM Systems Journal, 9(2):78–117, 1970.
[MM03] Nimrod Megiddo and D. S. Modha. ARC: A self-tuning, low overhead replacement cache. In Proc. 2nd USENIX Conference on File and Storage Technologies (FAST 03), San Franciso, CA, 2003.
[Moo02a] Bob Moore. Taking the data center power and cooling challenge. Energy User News, August 27th, 2002.
[Moo02b] Fred Moore. More power needed. Energy User News, Nov 25th, 2002.
[MSS+03] Grigorios Magklis, Michael L. Scott, Greg Semeraro, David H. Albonesi, and Steven Dropsho. Profile-based dynamic voltage and frequency scaling for a multi-ple clock domain microprocessor. ACM SIGARCH Computer Architecture News, 31(2):14–27, 2003.
[MT90] S. Martello and P. Toth. Knapsack problems: Algorithms and computer imple-mentations. J. Wiley, Ltd., 1990.
[MY] Arif Merchant and Philip S. Yu. Analytic modeling and comparisons of striping strategies for replicated disk arrays. IEEE Transactions on Computers, 44(3):419–
33.
[ONY00] K. Okada, N.Kojima, and K. Yamashita. A novel drive architecture of HDD:
multimode hard disc drive. In Proc. of the International Conference on Consumer Electronics, 2000.
[PB04] Eduardo Pinheiro and Ricardo Bianchini. Energy conservation techniques for disk array-based servers. In the 18th International Conference on Supercomputing, June 2004.
[PGG+95] R. Hugo Patterson, Garth A. Gibson, Eka Ginting, Daniel Stodolsky, and Jim Zelenka. Informed prefetching and caching. In Proceedings of the 15th ACM Symposium on Operating Systems Principles, 1995.
[PGK88] David A. Patterson, Garth Gibson, and Randy H. Katz. A case for redundant arrays of inexpensive disks (raid). In Proceedings of the 1988 ACM SIGMOD in-ternational conference on Management of data, pages 109–116. ACM Press, 1988.
[pow03] Dell poweredge 6650 executive summary. http://www.tpc.org/results/individual-results/Dell/dell-6650-010603-es.pdf, 2003.
[Pro] Panel computer hard disk drive precautions. http://www.pro-face.
com/support/technical/00apr3.htm.
[PS02] A. E. Papathanasiou and M. L. Scott. Increasing disk burstiness for energy effi-ciency. Technical report, University of Rochester, November 2002.
[PS04] Athanasios E. Papathanasiou and Michael L. Scott. Energy efficient prefetching and caching. In Proc. of the USENIX Annual Technical Conference, June 2004.
[Ran04] Parthasarathy Ranganathan. The power management challenge: Getting the next 100x. Keynote presentation at the 2nd workshop on optimizations for DSPs and Embedded Systems, March 12 2004.
[Rob07] Robin Harris. Flash is weirder than you know - so keep reading - and you´ll know.
http://storagemojo.com/?cat=5, 2007.
[RW93] Chris Ruemmler and John Wilkes. UNIX disk access patterns. In Proceedings of the Winter 1993 USENIX Conference, 1993.
[Sam07] Samsung. Flash ssd. http://www.samsung.com/global/business/semiconductor/
products/flash/Products FlashSSD.html, 2007.
[SCK05] S. W. Son, G. Chen, and M. Kandemir. Disk layout optimization for reducing energy consumption. In Proc. of the 19th International Conference on Supercom-puting, June 2005.
[SMW98] Elizabeth Shriver, Arif Merchant, and John Wilkes. An analytic behavior model for disk drives with readahead caches and request reordering. In SIGMETRICS
’98/PERFORMANCE ’98: Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, pages 182–191. ACM Press, 1998.
[SW96] Stefan Savage and John Wilkes. AFRAID – a frequently redundant array of inde-pendent disks. In USENIX Annual Technical Conference, pages 27–39, 1996.
[UPS+05] Bhuvan Urgaonkar, Giovanni Pacifici, Prashant Shenoy, Mike Spreitzer, and Asser Tantawi. An analytical model for multi-tier internet services and its applications.
In Proceedings of the 2005 ACM SIGMETRICS international conference on Mea-surement and modeling of computer systems, pages 291–302, 2005.
[WB90] Wen-Hann Wang and Jean-Loup Baer. Efficient trace-driven simulation method for cache performance analysis. In Proceedings of the 1990 ACM SIGMETRICS conference on Measurement and modeling of computer systems, pages 27–36. ACM Press, 1990.
[WBB02] Andreas Weissel, Bjoern Beutel, and Frank Bellosa. Cooperative I/O: a novel I/O semantics for energy-aware applications. In Proc. of the Fifth USENIX Symposium on Operating Systems Design and Implementation, December 2002.
[WGSS96] John Wilkes, Richard A. Golding, Carl Staelin, and Tim Sullivan. The HP AutoRAID hierarchical storage system. ACM Transactions on Computer Systems, 14(1):108–136, 1996.
[Wik07a] Wikipedia. Flash memory. http://en.wikipedia.org/wiki/Flash memory, 2007.
[Wik07b] Wikipedia. Hybrid drive. http://en.wikipedia.org/wiki/Hybrid Drive, 2007.
[Wik07c] Wikipedia. Ready boost. http://en.wikipedia.org/wiki/ReadyBoost, 2007.
[Wik07d] Wikipedia. Solid state drive. http://en.wikipedia.org/wiki/Solid state drive, 2007.
[WRPK02] An-I Wang, Peter L. Reiher, Gerald J. Popek, and Geoffrey H. Kuenning. Con-quest: Better performance through a disk/persistent-ram hybrid file system. In Proceedings of the General Track: 2002 USENIX Annual Technical Conference, pages 15–28, 2002.
[Wus02] Edward Wusterenhoff. Service level agreement in the data center.
http://www.sun.com/blueprints/0402/sla.pdf, April 2002.
[WW02] Theodore Wong and John Wilkes. My cache or yours? making storage more exclusive. In Proceedings of the Usenix Technical Conference, 2002.
[WZS91] Gerhard Weikum, Peter Zabback, and Peter Scheuermann. Dynamic file allocation in disk arrays. In SIGMOD ’91: Proceedings of the 1991 ACM SIGMOD interna-tional conference on Management of data, pages 406–415. ACM Press, 1991.
[YIY+00] H. Yada, H. Ishioka, T. Yamakoshi, Y. Onuki, Y. Shimano, M. Uchida, H. Kanno, and N. Hayashi. Head positioning servo and data channel for HDDs with multiple spindle speeds. IEEE Transactions on Magnetics, 36(5):2213–2215, September 2000.
[YN03] Wanghong Yuan and Klara Nahrstedt. Energy-efficient soft real-time cpu schedul-ing for mobile multimedia systems. In Proceedschedul-ings of the nineteenth ACM sympo-sium on Operating systems principles, pages 149–163. ACM Press, 2003.
[You95] Rachad Youssef. RAID for mobile computers. Master’s thesis, CMU, 1995.
[ZBJ+02] Y. Zhou, A. Bilas, S. Jagannathan, C. Dubnicki, J. F. Philbin, and K. Li. Expe-riences with VI communication for database storage. In ISCA’02, May 2002.
[ZCT+05] Qingbo Zhu, Zhifeng Chen, Lin Tan, Yuanyuan Zhou, Kimberly Keeton, and John Wilkes. Hibernator: helping disk arrays sleep through the winter. In Proceedings of the 20th ACM Symposium on Operating System Principles, Oct 2005.
[ZDD+04] Qingbo Zhu, Francis M. David, Christo F. Devaraj, Zhenmin Li, Yuanyuan Zhou, and Pei Cao. Reducing energy consumption of disk storage using power-aware cache management. In the Tenth International Symposium on High Performance Computer Architecture (HPCA-10), March 2004.
[ZPL01] Y. Zhou, J. F. Philbin, and K. Li. The multi-queue replacement algorithm for second level buffer caches. In Proceedings of the Usenix Technical Conference, June 2001.
[ZSZ04] Qingbo Zhu, Asim Shankar, and Yuanyuan Zhou. Power aware storage cache replacement algorithms. In the 18th International Conference on Supercomputing, June 2004.
[ZZ05] Qingbo Zhu and Yuanyuan Zhou. Power aware storage cache management. IEEE Transactions on Computers, 54(5):587–602, May 2005.