Low Cost 100GbE
Data Center Interconnect
Where does 100GbE come into the data center
Challenges in the data center/equipment design
Evolution of 100GbE
• Digital interfaces • Fiber plant
• Modules • Embedded
San Jose, CA USA
Add Graphic Here Showing
Datacenter 100G placement
Challenges in Data Center
Equipment Design for 100Gb
Silicon capability outstrips 1U density capability
Power Consumption/Size
San Jose, CA USA
February 2012 4
Add picture of 1U switch
Size (LxWxH) Max Power
SFP 48x14x9 1.5W
QSFP 52x18x9 5W
CFP2 106x42x12 12W
Challenges in Data Center
Equipment Design for 100Gb
25G Signaling on PCB
Reach
IEEE Standardization Efforts for 100GE
2/15/2012 Page 6
0 to 30m: Active Optical Cable – Wavelength Agnostic
Reach limited by deployment practicality of AOC, not by the optical transceiver
0 to 70m?: Parallel Multi-Mode Fiber – 850nm?
0 to 2000m: Advanced Encoding – 1310nm
0 to 10,000m: WDM Single Mode Fiber – 1310nm 4x25G QSFP
4x25G QSFP with MPO
4x25G QSFP with LC Embedded Optics with LC
4x25G QSFP with LC Embedded Optics with LC
0 to 1000m: Parallel Single Mode Fiber – 1310nm
4x25G QSFP with MPO Embedded Optics with MPO
Longer Reach Possible
Longer Reach Possible SR10 - 0 to 300m
Parallel Multi-Mode Fiber
LR4 - 0 to 10,000m
Modules
MSA Module Evolution
• Supply chain advantages
– Multivendor – Supply – Cost
• Equipment vendor interoperability • Fiber plant flexibility
• Optics reliability
Single mode in the data center
Historically multi-mode has been used
• Transceivers were cheaper
• Fiber was more expensive but distance was short enough that it didn’t matter
Many aggressive data centers are now
moving to SMF
• Can support the largest datacenter reaches
• Offers future proofing (NRZ speeds, advanced encoding, multi-wavelength)
• Silicon Photonics friendly which drastically lowers transceiver cost
San Jose, CA USA
Fiber Cost Comparison
Density Fiber Type Relative
Cost
24f OM3 3.0
24f OM4 4.3
24f SMF 1
Duplex SMF 2.0*
Source: Jan 2012 quote from cable vendor LC or MTP Termination costs not included. * Cost to derive 12 - duplex pairs
24f Relative Fiber Cost
24-Fiber SMF cable is half the cost of 12 duplex SMF cables, 1/3 the cost of 24f OM3 and 1/4 the cost of 24f OM4
Fiber + Module Cost Comparison
@ 100m
Density Fiber Type Cable Cost(*) Module Type Module Cost Total Cost 24f OM3 3.0 SR4 4.5X 7.5X 24f OM4 4.3 SR4 4.5X 7.8X 24f SMF 1 MR4 4.5X-16X 5.5X-17X Duplex SMF 0.2 LR4 25X 25.2X 10Including the fiber cable, even at 100m MR4 is potentially the lowest cost overall solution.
This cost advantage increases with longer reach.
Evolution of Copper Backplanes
Copper backplane solutions historically transitioned from multi-chip to integrated solutions
• Start with switch + backplane copper transceiver PHY
• Followed by switch with integrated transceiver PHY
Similar transition cycle occurred at 3.125G, 6.25G and 12.5G
MSA Switch MSA N ports to network KR or SFI KR electrical MSA Switch MSA N ports to network KR or SFI
Switch Blade - M ports to backplane
Phase 1 – Multi-chip with backplane drivers Phase 2 – Integration
SFI KR
CPU
Server Blade – 2 or 4 ports to backplane
SFI KR PCI PHY PHY PHY PHY E m bedded PHY NIC 2x / 4x PHY CPU
Server Blade – 2 or 4 ports to backplane
KR PCI
NIC 2x / 4x
Switch Blade - M ports to backplane
C opper B ac k pl ane C opper B ac k pl ane 2/15/2012
Optical Evolution in Rack
2/15/2012 Page 12 MSA MSA Optical ports to networkSwitch Blade - M ports to backplane
4x28G/Port
Electrical Optical Switch
Switch
Switch and Blade Architecture with Optical I/O
Switch Optical
ports to network
Optical Switch Blade - M ports to backplane
Switch and Blade with Optical SoC Architecture
CONTEMPORARY:
MSA and AOC
EMERGING:
Embedded Optics
Future:
SoC Optics Integration
Opto-Electronic SoC Integration will enable new high-performance compute and networking architectures
What lies ahead for 100G and
beyond?
100G Serial
What lies ahead for 100G and
beyond?
ASIC Integration
San Jose, CA USA