The rapid growth of datacentre traffic due to cloud-computing applications and on-demand services motivates the deployment of high-performance datcentre interconnection networks. Optical net- working concepts and technologies present challenges and opportunities for satisfying current and future application needs in datacentres, owing to ultra-high bandwidth capacity and transparency to datarates, modulation formats and protocols. Of utmost importance is high-speed Optical Packet
Switching (OPS) which promises high network flexibility and bandwidth capacity to support ser- vice heterogeneity in datacentres. Remarkable technological advances in integrated photonic tech- nologies have been achieved, especially in AWG switching technology. It is indicated that AWG can scale up to 512 ports [47], highlighting the opportunity to fabricate high-radix optical AWG switches. The potentially wide deployment of high-radix AWG switches makes possible the idea of flattening current hierarchical datacentre networks.
The AWG-based OPS holds some attractive features including ultra-high bandwidth, high-speed switching, low power consumption and operational flexibility, thereby it offers an excellent solution to overcome the challenges in large-scale datacentres, and deserves continued attention. Nonethe- less, contention resolution in this type of switch is problematic, due to the absence of viable optical RAM technology. In the remainder of this thesis, various implementations of high-radix AWG switches employing advanced enabling optical technologies and contention resolution techniques are proposed, with the purpose of building large-scale flattened datacentre networks with improved energy efficiency and network performance. The performance evaluation and power estimation of the proposed optical packet switches will be a major focus of the work.
Bibliography
[1] Dennis Abts and John Kim, “High Performance Datacenter Networks: Architectures, Algo- rithms, and Opportunities,” Morgan & Claypool Publishers, Mar. 2011.
[2] Cisco Systems, “Cisco global cloud index: forecast and methodology 2012-2017,” White Paper, Oct. 2013. [Online]. Available: http://knoema.com/jlbtvff/cisco-global-cloud-index- 2012-2017.
[3] Cisco Systems, “Cisco global cloud index: forecast and methodology 2013-2018,” White Pa- per, Nov. 2014. [Online]. Available: http://www.cisco.com/c/en/us/solutions/collateral/service -provider/global-cloud-index-gci/Cloud Index White Paper.pdf.
[4] Cisco Systems, “Growth in the cloud,” 2014. [Online]. Available: http://www.cisco.com/c/da m/assets/sol/sp/network infrastructure/growth cloud infographic.html.
[5] W. E. Denzel, J. Li, P. Walker, and Y. Jin, “A framework for end-to-end simulation of high- performance computing systems”. in First International Conference on Simulation Tools and Techniques for Communications, Networks and Systems (SIMUTools 2008), Mar. 2008. [6] S. Kandula, S. Sengupta, A. Greenberg, P. Patel, and R. Chaiken,“Nature of datacenter traffic:
measurements and analysis,” in ACM SIGCOMM, 2009.
[7] T. Benson, A. Akella, and D. A. Maltz, “Network traffic characteristics of data centers in the wild,” in ACM SIGCOMM, 2010.
[8] M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat, “Hedera: dynamic flow scheduling for data center networks,” in Networked Systems Design and Implementation (NSDI) Symposium, Apr. 2010.
[9] P. Bodk, I. Menache, and M, Chowdhury, “ Surviving failures in bandwidth-constrained dat- acenters,” in ACM SIGCOMM conference on Applications, technologies, architectures, and
protocols for computer communication, Aug. 2012.
[10] FP Tso and D. Pezaros, “Improving data center network utilization using near-optimal traffic engineering,” IEEE Transactions on Parallel & Distributed Systems, vol. 24, no. 6, pp. 1139- 1148, June 2013.
[11] A. Roy, H. Zengy, J. Baggay, G. Porter, and A. C. Snoeren, “Inside the social networks (Dat- acenter) network”, in ACM SIGCOMM, pp. 123-137, Aug. 2015.
[12] R. Ramaswami, K. N. Sivarajan, and G. H. Sasaki, “Optical Networks: A Practical Perspec- tive,” Elsevier, USA, 2010.
[13] Andre DeHon, “Rent’s rule based switching requirements,” in International Workshop on System-level Interconnect Prediction, pp. 197-204, 2001.
[14] Tarek S. El-Bawab, “Optical Switching,” Springer, 2006.
[15] J. L. Hennessy and D. A. Patterson, “Computer Architecture: A Quantitative Approach,” Mor- gan Kaufmann Publishers, 2003.
[16] M. Al-Fares, A. Loukissas, and A. Vahdat, “A scalable, commodity data center network archi- tecture,” in ACM SIGCOMM, Aug. 2008.
[17] Charles Clos, “A study of non-blocking switching networks,” Bell System Technical Journal, vol. 32, no. 2, pp. 406-424, Mar. 1953.
[18] B. Mukherjee,“Optical WDM Networks,” Springer, 2006.
[19] M. Moazez, F. Safaei, and M. Rezazadeh, “Design and implementation of multistage intercon- nection networks for SoC networks,” International Journal of Computer Science, Engineering and Information Technology (IJCSEIT), vol. 2, no. 5, Oct. 2012.
[20] J. Kim, W. J. Dally, and D. Abts, “Flattened Butterfly: a cost-efficient topology for high-radix networks,” International Symposium on Computer Architecture (ISCA) , pp. 126-137, June 2007.
[21] Hewlett-Packard Development Company, “ Effects of virtualization and cloud computing on data center networks,” Oct. 2011. [Online]. Available: http://h20565.www2.hp.com/hpsc/doc/ public/display?docId=emr na-c03042885.
[22] R. N. Mysore, A. Pamboris, N. Farrington, N. Huang, P. Miri, S. Radhakrishnan, V. Subra- manya, and A. Vahdat, “PortLand: a scalable fault-tolerant layer 2 data center network fabric,”
in ACM SIGCOMM, pp. 39-50, Aug. 2009.
[23] A. A. Kist, and L. Brodie, “Quality of service, quality of experience and online learning,” Frontiers in Education Conference (FIE), pp. 1-6, Oct. 2012.
[24] Albert Greenberg, James R. Hamilton, Navendu Jain, Srikanth Kandula, and Changhoon Kim, “VL2: a scalable and flexible data center network,” in SIGCOMM, Aug. 2009.
[25] N. Farrington and A. Andreyev, “Facebooks data center network architecture,” in IEEE Opti- cal Interconnects Conference (OI), 2013.
[26] A. Andreyev, “Introducing data center fabric, the next-generation Facebook data center net- work,” Nov. 2014. [Online]. Available: https://code.facebook.com/posts/360346274145943/i ntroducing-data-center-fabric-the-next-generation-facebook-data-center-network.
[27] M. Alizadeh, and T. Edsall, “On the data path performance of Leaf-Spine datacenter fabrics,” High-Performance Interconnects (HOTI), pp. 71-74, Aug. 2013.
[28] Arista, “High Performance, Feature Rich 10/40GbE Systems: 40Tbps & 512×40GbE, 2048×10GbE, 768×10GBASE-T,” Nov. 2014. [Online]. Available: https://www.arista.com/e n/products/7300-series.
[29] C. Guo, H. Wu, K. Tan, L. Shiy, Y. Zhang, and S. Lu,“DCell: a scalable and fault-tolerant network structure for data centers,” in ACM SIGCOMM, Aug. 2008.
[30] C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, Y. Shi, C. Tian, Y. Zhang, and S. Lu, “BCube: a high performance, server-centric network architecture for modular data centers,” in ACM SIGCOMM, Aug. 2009.
[31] D. Li, C. Guo, H. Wu, K. Tan, Y. Zhang, and S. Lu, “FiConn: using backup port for server interconnection in data centers,” in IEEE International Conference on Computer Communica- tions, pp. 2276-2285, Apr. 2009.
[32] D. Li, C. Guo, H. Wu, K. Tan, Y. Zhang, S. Lu, and J. Wu, “Scalable and cost-effective interconnection of data-center servers using dual server ports,” IEEE/ACM Transactions on Networking, vol. 19, no. 1, pp. 102-114, Feb. 2011.
[33] CALIENT, “3D MEMS Optical Circuit Switching Demonstration,” Oct. 2013. [On- line]. Available: http://www.calient.net/2013/10/calient-3d-mems-optical-circuit-switching- demonstration.
[34] A. Neukermans, and R. Ramaswami, “MEMS technology for optical networking applica- tions,” IEEE Communications Magazine, vol. 39, no. 1, pp. 62-69, Jan. 2001.
[35] CALIENT, “S320 Photonic Switching”, 2013. [Online]. Available: http://www.enterprisema nagement360.com/wp-content/files mf/1377248645CalientS320DataSheetMarch12013.pdf. [36] O. Solgaard, A. A. Godil, R. T. Howe, L. P. Lee, Y. A. Peter and H. Zappe, “Optical MEMS:
from micromirrors to complex systems,” Journal of Microelectromechanical Systems, vol. 23, no. 3, pp. 517-538, June 2014.
[37] H. Du, F. S. Chau, and G. Zhou, “Mechanically-tunable photonic devices with on-chip inte- grated MEMS/NEMS actuators,” Micromachines, vol. 7, no. 4, pp. 1-24, Apr. 2016.
[38] International Telecommunications Union,“G.694.1 spectral grids for WDM applications: DWDM frequency grid” Series G: Transmission Systems and Media, Digital Systems and Networks, 2012.
[39] Alan Rogers, “Understanding Optical Fiber Communications,” Artech House, USA, 2001. [40] P. Urquhart, “Advances in Optical Amplifiers,” InTech, Rijeka, Croatia, 2011.
[41] A. K. Jaiswal and S. Verma, “Performance on SOA-based wavelength conversion at 10 GB/s by dual pump four wave mixing over a 50 nm,” International Journal of Advanced Research in Computer and Communication Engineering, vol. 2, no. 5, May 2013.
[42] Martin Maier, “Optical Switching Networks,” Cambridge University Press, Feb. 2008. [43] J. M. H. Elmirghani, and H. T. Mouftah, “All-optical wavelength conversion: technologies
and applications in DWDM networks,” IEEE Communications Magazine, vol. 38, no. 3, pp. 86-92, Mar. 2000.
[44] F. Bontempi, N. Andriolli, S. Faralli, J. Klamkin, E. Kleijn, T. de Vries, and G. Contestabile, “An InP monolithically integrated multi-frequency wavelength converter,” in 2014 Optical Fiber Communication Conference (OFC), Mar. 2014.
[45] G. Contestabile, Y. Yoshida, A. Maruta, and K. Kitayama, “Ultra-broad band, low power, highly efficient coherent wavelength conversion in quantum dot SOA,” Optics Express, vol. 20, no. 25, pp. 27902-27907, Dec. 2012.
[46] N. A. Awang, A. A. Latif, M. Z. Zulkifli, Z. A. Ghani, S. W. Harun, and H. Ahmad, ”S + C + L Band tunable wavelength conversion using FWM dual-wavelength fiber laser in a highly non-
linear fiber,” Microwave and Optical Technology Letters, vol. 55, no. 2, pp. 379-382, Feb. 2013.
[47] S. Cheung, T. Su, K. Okamoto, and S. J. B. Yoo, “Ultra-compact silicon photonic 512×512 25-GHz Arrayed Waveguide Grating Router,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 20, no. 4, pp. 310-316, Aug. 2014.
[48] D. M. Marom et al., “Wavelength-selective 1×4 switch for 128 WDM channels at 50 GHz spacing,” in Optical Fiber Communication Conference (OFC), pp. FB7-1-FB7-3, Mar. 2002. [49] R. Ryf et al., “Wavelength-selective switch for few-mode fiber transmission,” in 39th Euro-
pean Conference on Optical Communications (ECOC), 2013.
[50] M. Chen, H. Jin, Y. Wen, and V. C. M. Leung, “Enabling technologies for future data center networking: a primer,” IEEE Network, vol. 27, no. 4, pp. 8-15, Aug. 2013.
[51] G. Wang, D. G. Andersen, M. Kaminsky, K. Papagiannaki, T. S. Eugene Ng, M. Kozuch, and M. Ryan, “c-Through: part-time optics in data centers,” in ACM SIGCOMM, Oct. 2010. [52] N. Farrington, G. Porter, S. Radhakrishnan, H. H. Bazzaz, V. Subramanya, Y. Fainman, G.
Papen, and A. Vahdat, “Helios: a hybrid electrical/optical switch architecture for modular data centers,” in ACM SIGCOMM, Oct. 2010.
[53] H. H. Bazzaz, M. Tewari, G. Wang, G. Porter, T. S. Eugene Ng, D. G. Andersen, M. Kamin- sky, M. A. Kozuch, and A. Vahdat, “Switching the optical divide: fundamental challenges for hybrid electrical/optical datacenter networks,” in ACM Symposium on Cloud Computing (SOCC), pp. 1-8, Oct. 2011.
[54] D. Lugones, K. Katrinis, and M. Collier, “A reconfigurable optical/electrical interconnect ar- chitecture for large-scale clusters and datacenters,” in ACM International Conference on Com- puting Frontiers, pp. 13-22, May 2012.
[55] L. Xu, W. Zhang, H. L. R. Lira, M. Lipson, and K. Bergman, “A hybrid optical packet and wavelength selective switching platform for high-performance data center networks,” Optical Express, vol. 19, no. 24, Nov. 2011.
[56] K. Chen, A. Singla, A. Singh, K. Ramachandran, L. Xu, Y. Zhang, X. Wen, and Y. Chen, “OSA: an optical switching architecture for data center networks with unprecedented flexibil- ity,” Northwestern University ,Tech Rep, 2012.
project approach,” Journal of Lightwave Technology, vol. 16, no. 12, pp. 2117-2134, Dec. 1998.
[58] A. Carena, M. D. Vaughn, R. Gaudino, M Shell, and D. J. Blumenthal, “OPERA: an optical packet experimental routing architecture with label swapping capability”, Journal of Light- wave Technology, vol. 16, no. 12, pp. 2135-2145, Dec. 1998.
[59] D. K. Hunter et al., “WASPNET: a wavelength switched packet network,” Communications Magazine, vol. 37, no. 3, pp. 120-129, Mar. 1999.
[60] O. Liboiron-Ladouceur, P. G. Raponi, N. Andriolli, I. Cerutti, M. S. Hai, and P. Castoldi, “A scalable space-time multi-plane optical interconnection network using energy-efficient en- abling technologies,” Journal of Optical Communications and Networking, vol. 3, no. 8, pp. 1-11, Aug. 2011.
[61] S. D. Lucente, N. Calabretta, J. A. C. Resing, and H. J. S. Dorren, “Scaling low-latency optical packet switches to a thousand Ports,” Journal of Optical Communications and Networking, vol. 4, no. 9, pp. 17-28, Sept. 2012.
[62] N. Calabretta, R. P. Centelles, S. D. Lucente, and H. J. S. Dorren, “On the performance of a large-scale optical packet switch under realistic data center traffic,” Journal of Optical Com- munications and Networking, vol. 5, no. 6, pp. 565-573, June 2013.
[63] K. Xia, Y.H. Kaob, M. Yangb, and H. J. Chao, “Petabit optical switch for data center net- works,” Technical report, Polytechnic Institute of NYU, 2010.
[64] H. J. Chao and K. Xi, “Bufferless optical Clos switches for data centers,” in OSA/OFC/N- FOEC, 2011.
[65] X. Ye, Y. Yin, S. J. B. Yoo, P. Mejia, R. Proietti, and V. Akella, “DOS–a scalable optical switch for datacenters,” Architectures for Networking and Communications Systems (ANCS), pp. 1-12, Oct. 2010.
[66] Y. Yin, R. Proietti, X. Ye, C. J. Nitta, V. Akella and S. J. B. Yoo, “LIONS: an AWGR-based low latency optical switch for high-performance computing and data centers,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 19, no. 2, Mar. 2013.
[67] B. Kozicki, H. Takara, and M. Jinno, “Enabling technologies for adaptive resource allocation in elastic optical path network (SLICE),” in Communications and Photonics Conference and Exhibition (ACP), pp. 23-24, Dec. 2010.
[68] O. Gerstel, M. Jinno, A. Lord, and S.J.B. Yoo, “Elastic optical networking: a new dawn for the optical layer?,” Communications Magazine, vol. 50, no. 2, pp. 12-20, Feb. 2012.
[69] S. J. B. Yoo, Y. Yawei, and W. Ke, “Intra and inter datacenter networking: the role of optical packet switching and flexible bandwidth optical networking,” in Optical Network Design and Modeling (ONDM), pp. 1-6, Apr. 2012.
[70] N. Calabretta, “Scalable and low latency optical packet switching architectures for high per- formance data center networks,” Photonics in Switching (PS), July 2014.
[71] S. Peng, R. Nejabati, B. Guo, Y. Shu, G. Zervas, S. Spadaro, A. Pages, and D. Simeonidou, “Enabling multi-tenancy in hybrid optical packet/circuit switched data center networks,” in 40th European Conference and Exhibition on Optical Communication (ECOC), Sept. 2014. [72] Q. Huang, Y. K. Yeo, and L. Zhou, “Combining circuit and packet switching using a large
port-count optical cross-connect for data center networks,” Optics Communications, vol. 285, no. 21-22, pp. 4268-4274, Oct. 2012.
Chapter 2
A Novel Energy-Efficient High-Speed
Optical Packet Switching Datacentre
Network
2.1
Introduction
As stated in the previous chapter, high-capacity optical packet switching has the potential to support the communication requirements between ever increasing numbers of highly interdepen- dent servers in datacentres. An attractive candidate solution for optical packet switching is the wavelength-switched architecture, which adopts Tunable Wavelength Converters (TWCs) and an Arrayed Waveguide Grating (AWG) router for cross-connection. A key aspect of the AWG is that it is a passive optical device and can potentially scale to very high port counts [1]. Importantly, due to its fixed cyclic wavelength routing property, the AWG allows the selection of output ports based on the wavelength of the input optical signal [2]. Thus, by tuning the TWCs installed at input ports, an AWG can passively route the optical signals to any output ports, forming a high-speed, recon- figurable wavelength-routed switching fabric. Moreover, at switch output ports, the AWG allows multiple packets on different wavelengths to be detected concurrently, if multiple parallel receivers are located there. In particular, a contention-free N × N switch is possible provided that each AWG output has a 1 : N Demultiplexer (DEMUX) and N separate receivers [3]. However, it is generally not scalable to implement N receivers at each of N outputs, especially for high-port count AWGs, hence many fewer receivers are typically used and as such, the resulting contention needs to be
handled by some form of packet buffering. This makes contention resolution an important issue for the realisation of AWG-based optical switches.
Previous research, e.g. the Datacenter Optical Switch (DOS) [3] resolves conflicts by buffering con- tending packets in a single shared electronic buffering. However, due to considerable contention at output ports, especially under high load conditions, this design requires a high-capacity electronic buffer with the same port count and same port speed as the overall switch. Essentially, such an elec- tronic buffer amounts to a high port-count, high bandwidth switch, yielding a potentially high-cost, high power-consumption architecture overall. [4] makes extensive comparisons between electronic buffers and single-wavelength Fibre Delay Lines (FDLs) in the context of AWG switches. The authors conclude that the electronic buffer outperforms FDLs with respect to latency and through- put, though at the expense of higher power consumption. In related work, [5] and [6] studied the enhanced DOS architectures (a.k.a. LIONS) with three different loop-back electronic buffering designs, but did not further consider FDLs. In [7–10], AWG-based packet switches with N feed- back optical buffers are proposed. However, these architectures face network scalability and overall system complexity issues, due to the large number of FDL buffers utilised.
In comparison to these previous findings, the main novelty and advancement of the work presented in this chapter is that this work solves the contention resolution challenge in AWG-based packet switches by designing an efficient buffering hardware and packet scheduling scheme. More pre- cisely, a novel hybrid-buffered AWG architecture, which employs a relatively small number of multi-wavelength Fibre Delay Lines (FDLs), in combination with a very small electronic buffer, using an optimal method for integrating FDL wavelengths into the AWG switch, is proposed for deployment in building scalable, energy-efficient, high-performance interconnection networks. The motivations behind and the advantages of using such a hybrid-buffered AWG architecture are ex- amined by simulation work. The network dimensioning problem is also investigated.
This chapter is organised as follows. Section 2.2 presents details of the new architecture together with its contention resolution scheme. The format of the optical packet and the packet scheduling algorithm are also described in this section. Section 2.3 details the simulation setup and the ex- perimental work carried out to examine the network performance and power consumption of the proposed optical switch. Also investigated is the execution complexity of the packet scheduling scheme. In Section 2.4, some important conclusions are outlined.