Optical Interconnects for Future Data
Center Networks
Optical Networks
Series Editor: Biswanath Mukherjee University of California, Davis Davis, CA
Broadband Access Networks: Technologies and Deployments Abdallah Shami, Martin Maier, and Chadi Assi (Eds.) ISBN 978-0-387-92130-3
Traffic Grooming for Optical Networks: Foundations, Techniques, and Frontiers Rudra Dutta, Ahmed E. Kamal, and George N. Rouskas (Eds.)
ISBN 978-0-387-74517-6
Optical Network Design and Planning Jane M. Simmons
ISBN 978-0-387-76475-7
Quality of Service in Optical Burst Switched Networks
Kee Chaing Chua, Mohan Gurusamy, Yong Liu, and Minh Hoang Phung ISBN 978-0-387-34160-6
Optical WDM Networks Biswanath Mukherjee ISBN 978-0-387-29055-3
Traffic Grooming in Optical WDM Mesh Networks Keyao Zhu, Hongyue Zhu, and Biswanath Mukherjee ISBN 978-0-387-25432-6
Survivable Optical WDM Networks Canhui (Sam) Ou and Biswanath Mukherjee ISBN 978-0-387-24498-3
Optical Burst Switched Networks Jason P. Jue and Vinod M. Vokkarane ISBN 978-0-387-23756-5
For further volumes:
Christoforos Kachris • Keren Bergman
Ioannis Tomkos
Editors
Optical Interconnects
for Future Data Center
Networks
Editors
Christoforos Kachris
Athens Information Technology Networks and Optical
Communications Lab Peania, Greece
Ioannis Tomkos
Athens Information Technology Networks and Optical
Communications Lab Peania, Greece
Keren Bergman Columbia University
Department of Electrical Engineering New York, USA
ISSN 1935-3839 ISSN 1935-3847 (electronic) ISBN 978-1-4614-4629-3 ISBN 978-1-4614-4630-9 (eBook) DOI 10.1007/978-1-4614-4630-9
Springer New York Heidelberg Dordrecht London
Library of Congress Control Number: 2012948542 © Springer Science+Business Media New York 2013
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
Printed on acid-free paper
Foreword
Recent advances in cloud computing and emerging web applications have created the need for more powerful data centers. These data centers need high-bandwidth interconnects that can sustain the heavy communication requirements between the servers in the data centers. Data center networks based on electronic packet switches will consume excessive power in order to satisfy the required communication bandwidth of future data centers. Optical interconnects have gained attention recently as a promising solution offering high throughput, low latency, and reduced energy consumption compared to current networks based on commodity switches.
This book presents the most recent and promising solutions that have been presented in the domain of the optical interconnects for data centers. First, it presents the requirements of future data center networks and how optical interconnects could support the data center scaling by providing the required bandwidth. The requirements of future data centers are provided for major data center owners and operators. The majority of the book presents most of the data center network architectures that are based on optical interconnects proposed by leaders in industry and academe. This book should be a valuable source of comprehensive information for researchers, professors, as well as network and computer engineers who are interested in high-performance data center networks and optical interconnects.
Department of Computer Science, Biswanath Mukherjee
University of California, Davis
Preface
The rise of cloud computing and web applications such as streaming video and social networks has created the need for more powerful data centers that can sustain much higher bandwidth than a few years ago. This trend requires high-performance interconnects for the data center networks that can sustain this bandwidth inside the data centers without consuming excessive energy.
In the last few years many researchers have pointed out the limitations of the current networks and have proposed as a viable solution data center networks based on optical interconnects. Data center networks based on optical interconnects are an interdisciplinary field which encompasses such fields as computer networks, computer architecture, hardware design, optical networks, and optical devices. Therefore it required a wide and in-depth knowledge of the above fields. This book collects the most recent and innovative optical interconnects for data center networks that have been presented in the research community by universities and industries in the last years.
This book is a valuable reference book for researchers, students, professors, and engineers who are interested in the domain of high-performance interconnects and data center networks. In addition this book will provide researchers and engineers who are working on high-performance interconnects invaluable insights into the benefits and advantages of optical interconnects and how they can be a promising alternative for the future data center networks.
Finally, we would like to thank all the authors who provided such interesting and valuable contributions and helped towards the realization of this book.
Peania, Greece Christoforos Kachris
New York, NY Keren Bergman
Peania, Greece Ioannis Tomkos
Contents
Part I Introduction to Data Center Networks
1 Introduction to Optical Interconnects in Data Centers . . . . 3 Christoforos Kachris, Keren Bergman, and Ioannis Tomkos
Part II Optical Interconnects in Data Center Networks
2 Optical Interconnects for Scale-Out Data Centers . . . . 17 Hong Liu, Ryohei Urata, and Amin Vahdat
3 Optical Interconnects in Next Generation Data Centers:
An End to End View . . . . 31 Madeleine Glick
4 Simulation and Performance Analysis of Data Intensive
and Workload Intensive Cloud Computing Data Centers . . . . 47 Dzmitry Kliazovich, Pascal Bouvry, and Samee Ullah Khan
Part III Optical Interconnects Architectures
5 The Role of Photonics in Future Datacenter Networks . . . . 67 Al Davis, Norman P. Jouppi, Moray McLaren,
Naveen Muralimanohar, Robert S. Schreiber, Nathan Binkert, and Jung-Ho Ahn
6 All-Optical Networks: A System’s Perspective . . . . 95 Nikolaos Chrysos, Jens Hofrichter, Folkert Horst,
Bert Offrein, and Cyriel Minkenberg
7 A High-Speed MIMO OFDM Flexible Bandwidth Data
Center Network . . . 119 Philip N. Ji, D. Qian, K. Kanonakis, Christoforos Kachris,
and Ioannis Tomkos
x Contents
8 A Petabit Bufferless Optical Switch for Data Center Networks . . . 135 Kang Xi, Yu-Hsiang Kao, and H. Jonathan Chao
9 Optically Interconnected High Performance Data Centers . . . 155 Keren Bergman and Howard Wang
Contributors
Al Davis HP Labs, Palo Alto, CA, USA,[email protected]
Amin Vahdat Google Inc., Mountain View, CA 94043, UC San Diego, USA
Bert Offrein IBM Research Zurich, S¨aumerstrasse 4, R¨uschlikon, Switzerland,
Christoforos Kachris Athens Information Technology, Peania, Athens, Greece,
Cyriel Minkenberg IBM Research Zurich, S¨aumerstrasse 4, R¨uschlikon, Switzerland,[email protected]
Dayou Qian NEC Laboratories America, Inc., 4 Independence Way, Princeton, NJ 08540, [email protected]
Dzmitry Kliazovich University of Luxembourg, 6 rue Coudenhove Kalergi, Luxembourg,[email protected]
Folkert Horst IBM Research Zurich, S¨aumerstrasse 4, R¨uschlikon, Switzerland,[email protected]
H. Jonathan Chao Polytechnic Institute of New York University, Brooklyn, NY, USA,[email protected]
Hong Liu Google Inc., Mountain View, CA, USA,[email protected]
Howard Wang Department of Electrical Engineering, Columbia University, New York, NY, USA,[email protected]
Ioannis Tomkos Athens Information Technology, Peania, Athens, Greece,
Jens Hofrichter IBM Research Zurich, S¨aumerstrasse 4, R¨uschlikon, Switzerland,[email protected]
xii Contributors
Kang Xi Polytechnic Institute of New York University, Brooklyn, NY, USA,
Keren Bergman Department of Electrical Engineering, Columbia University, New York, NY, USA,[email protected]
Konstantinos Kanonakis Athens Information Technology, Athens, Greece
Madeleine Glick APIC Corporation, Culver City, CA, USA,[email protected]
Moray McLaren HP Labs, Briston, UK,[email protected]
Naveen Muralimanohar HP Labs, Palo Alto, CA, USA,
Nikolaos Chrysos IBM Research Zurich, S¨aumerstrasse 4, R¨uschlikon, Switzerland,[email protected]
Norman P. Jouppi HP Labs, Palo Alto, CA, USA,[email protected]
Pascal Bouvry University of Luxembourg, 6 rue Coudenhove Kalergi, Luxembourg,[email protected]
Philip N. Ji NEC Laboratories America, Inc., Princeton, NJ, USA,
Robert S. Schreiber HP Labs, Palo Alto, CA, USA,[email protected]
Ryohei Urata Google Inc., Mountain View, CA, USA,[email protected]
Samee Ullah Khan North Dakota State University, Fargo, ND, USA,
Yu-Hsiang Kao Polytechnic Institute of New York University, Brooklyn, NY, USA,[email protected]
Acronyms
ADC Analog-to-Digital Converter AOC Active Optical Cable
ASIC Application Specific Integrated Circuit AWGR Arrayed Waveguide Grating Routing BPSK Binary Phase Shift Keying
BTE Bit-Transport Energy
CAGR Compound Annual Growth Rate CAWG Cyclic Arrayed Waveguide Grating
CMOS Complementary Metal-Oxide Semiconductor DCN Data Center Network
DDR Double Data Rate
DFB Distributed Feedback Laser DML Directly Modulation Laser DSP Digital Signal Processing
DWDM Dense Wavelength Division Multiplexing EARB Electrical Arbiter
EPS Electrical Packet Switch FEC Forward Error Correction FFT Fast Fourier Technology FIFO First In-First Out
FPGA Field-Programmable Gate Array FXC Fiber Cross-Connect
GHS Greenhouse Gases
HOL Head-of-Line
HPC High-Performance Computing
HVAC Heating, Ventilating and Air-Conditioning IT Information Technology
ITRS International Technology Roadmap for Semiconductors
LAN Local Area Network
MAN Metropolitan Area Network MEMS Micro-Electro-Mechanical Systems
xiv Acronyms
MIMO Multiple-Input, Multiple-Output
MMF Multi-Mode Fiber
MtCO2 Metric Tonne Carbon Dioxide NIC Network Interface Card
OFDM Orthogonal Frequency-Division Multiplexing
PD Photo Detector
PIC Photonic Integrated Circuit PON Passive Optical Network
QAM Quadrature Amplitude Modulation QoS Quality of Service
QPSK Quadrature Phase Shift Keying QSFP Quad Form-factor Plug-in
ROADM Reconfigurable Optical Add Drop Multiplexer SaaS Software-As-A-Service
SCC Single-chip Cloud Computer SDM Space Division Multiplexing SDN Software Defined Network SERDES Serializer-Deserializer SFP Small Form-factor Plug-in SOA Semiconductor Optical Amplifier
SMF Single-Mode Fiber
TCP Transmission Control Protocol ToR Top of Rack Switch
VCSEL Vertical Cavity Surface Emitting Laser
WAN Wide Area Network
WDM Wavelength Division Multiplexing WSC Warehouse Scale Computers WSS Wavelength Selective Switch WXC Wavelength Cross-Connect