Data Centre Link Survey

(1)

Data Centre Link Survey

Alan Flatman

Principal Consultant Principal Consultant LAN Technologies, UK LAN Technologies, UK [email protected] [email protected]

(2)

Survey Objectives

1.

establish length distribution requirements for

Higher Speed Ethernet links

2.

establish link speed & connectivity requirements

3.

focus on recent data centre designs/installations

4.

provide greater granularity for the first 30m of

horizontal/client links than existing studies

5.

assess level of standards compliance for cabling

6.

examine differences to existing cabling studies

(3)

Flatman Cabling Surveys for IEEE 802.3

July 1999 In-premises Optical Fibre Deployment to 2000

Jan 2003 Installed Copper Cabling Forecast to end 2005

July 2003 Installed Horizontal Cabling Length Distribution

(office & medium-sized data centre)

(4)

Data Centre Network Infrastructure

Storage Storage Servers

Clients

Access

Distribution

Core

LAN/NAS/SAN switches Routers/L3 switches Gateways/Firewalls

(5)

Data Centre Cabling Standards

¾

EIA/TIA-942:

Data Centre Cabling

» published in 2005

¾

CENELEC EN 50173-5:

Data Centre Cabling

» approved early 2007

¾

ISO/IEC 24764:

Data Centre Cabling

» based on CENELEC EN 50173-5

(6)

ISO/IEC 24764: Data Centre Cabling

Equipment ENI

MD

ZD

_(optional)

LDP

EO

network access cabling subsystem main distribution cabling subsystem zone distribution cabling subsystem equipment cabling ENI

LDP = Local Distribution Point EO = Equipment Outlet

ENI = Equipment Network Interface MD = Main Distributor

(7)

ISO/IEC 24764: Data Centre Cabling

(8)

Standard Data Centre Cabling Models

C Patch cord C Patch Panel Cross-connect Permanent Link Equipment Equipment Equipment cord Equipment cord Channel C C Patch Panel Cross-connect Patch cord C Patch cord C Patch Panel Cross-connect Permanent Link Equipment Equipment Equipment cord Equipment cord Channel C C Permanent Link Equipment Equipment Equipment cord Equipment cord Channel C

(9)

Data Centre Size Classification

IDC

₍₂₀₀₆₎₍₂₀₀₆₎

identifies 4 different types, distinguished by size:

Medium Data Centre

¾

~

20,000 ft

2

raised floor

¾

1,500-1,700 volume servers

¾

4-5 high end servers

Small Data Centre

¾

~

15,000 ft

2

raised floor

¾

350-500 volume servers

¾

1-3 high end servers

Large Data Centre

¾

~

35,000 ft

2

raised floor

¾

2,000-2,500 volume servers

¾

6-7 high end servers

Very Large Data Centre

¾

>100,000 ft

2

raised floor

¾

<25,000 volume servers

¾

>8 high end servers

(10)

Oct 2007 Data Centre Cabling Survey

Summary

¾

9 enterprise data centres from US, UK, Germany

¾

total data centre floor space was 715,000 sq. feet

¾

56k servers, 250k switch ports, 920k patch ports

¾

small, medium, large, v. large sizes (IDC classes)

¾ v. large had 74% links

¾ large had 20% links

¾ medium had 5% links

¾ small had 1% links

¾

data centre cabling mostly standards compliant

¾ some servers had direct connections to top of rack switches (i.e. no patch cords/patch panels involved)

(11)

Data Centre Link Speed – the next 5 years

100M 78% Client-to-Access Links Access-to-Distribution Links Distribution-to-Core Links 1G 100% 10G 11% 40G 0% 100G 0% 2007 (by site) 20% 80% 60% 0% 0% 10% 70% 80% 0% 0% 100M 11% Client-to-Access Links Access-to-Distribution Links Distribution-to-Core Links 1G 100% 10G 78% 40G 0% 100G 0% 2010 (by site) 0% 70% 90% 0% 10% 0% 60% 40% 10% 0% 100M 11% Client-to-Access Links Access-to-Distribution Links Distribution-to-Core Links 1G 78% 10G 80% 40G 30% 100G 10% 2012 (by site) 0% 40% 80% 30% 40% 0% 30% 80% 20% 60%

(12)

0 10 20 30 40 50 60 70 80 90 100 0-10m 11-2 0m 21-3 0m 31-4 0m 41-5 0m 51-6 0m 61-7 0m 71-8 0m 81-9 0m 91-1 00m

All V.Large Large Medium Small

% Links

Enterprise Data Centre Cabling Lengths

Client-to-Access Permanent Links

(13)

0 10 20 30 40 50 60 70 80 90 100 0-20m 21-3 0m 31-4 0m 41-5 0m 51-6 0m 61-7 0m 71-8 0m 81-9 0m 91-1 00m

All V.Large Large Medium Small

% Links

Enterprise Data Centre Cabling Lengths

(14)

0 10 20 30 40 50 60 70 80 90 100 0-30m 31-4 0m 41-5 0m 51-6 0m 61-7 0m 71-8 0m 81-9 0m 91-1 00m

All V.Large Large Medium Small

% Links

Enterprise Data Centre Cabling Lengths

Total of 250,000 links

(15)

Prior Data Centre Cabling Survey

(IEEE 802.3an DiMinico_01_1103)

Note: This is Permanent Link data

Total = 325,000 sq. feet 7 Data Centres

(16)

Prior Data Centre Cabling Survey

(IEEE 802.3an DiMinico_01_1103)

Note: This is Permanent Link data

Total = 448,000 sq. feet 11 Data Centres

(17)

0 10 20 30 40 50 60 70 80 90 100 0-30m 31-40m 41-50m 51-60m 61-70m 71-80m 81-90m 91-100m

Flatman All DiMinico J+M average DiMinico M+A average

% Links

Enterprise Data Centre Cabling Lengths

(18)

Enterprise Data Centre Cabling Lengths

0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

Flatman All DiMinico J+M Av. DiMinico M+A Av.

Cumulative % Links

Length (m) Shorter length distribution

for Flatman survey may be due to shift from pedestal to rack servers over 5 years (period between 2 surveys)

(19)

Enterprise Data Centre Cabling Lengths

0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

Flatman All Flatman V.Large Flatman Large Flatman Medium Flatman Small DiMinico J+M Av. DiMinico M+A Av.

(20)

0 10 20 30 40 50 60 0-10m 11-2 0m 21-3 0m 31-4 0m 41-5 0m 51-6 0m 61-7 0m 71-8 0m 81-9 0m 91-1 00m

Flatman All Permanent Links Flatman All Channels

% Links

Enterprise Data Centre Cabling Lengths

Client-to-Access Permanent Links & Channels

Average Perm Link = 15.8m Average Cordage = 4.4m Minimum Cordage = 3.0m Maximum Cordage = 9.0m Average Channel = 20.2m Min # Cords/Channel = 1.0 Max # Cords/Channel = 4.0 Avg # Cords/Channel = 1.6 used to calculate channel lengths

(21)

Enterprise Data Centre Cabling Lengths

Client-to-Access Channels

0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Flatman All Cumulative % Links Length (m) 33.3% 82.8% Total of 250,000 links

(22)

0 5 10 15 20 25 30 35 40 45 0-25m 26-50m 51-75m 76-100m 101-125m 126-150m 151-175m 176-200m 201-225m 226-250m 251-275m 276-300m

All V.Large Large Medium Small

% Links

Enterprise Data Centre Cabling Lengths

Access-to-Distribution Permanent Links

(23)

Prior Data Centre Cabling Study

(IEEE 802.3 HSSG Kolesar_01_0906)

(24)

(25)

0 5 10 15 20 25 30 35 40 45 0-25m 26-50m 51-75m 76-100m 101-125m 126-150m 151-175m 176-200m 201-225m 226-250m 251-275m 276-300m % Links

Optical Fibre Permanent Links; assumed to be switch-to-switch. Derived from cumulative length distribution & converted into m.

(26)

Prior Data Centre Cabling Study

(IEEE 802.3 HSSG Swanson_01_1106)

Note: Data based on several 1000’s Corning trunk cable shipments, simulation designed to represent actual data centre deployments (using concatenated trunks)

50% links have 2 trunks 20% links have 3 trunks 30% links are point-point

(27)

Prior Data Centre Cabling Study

(IEEE 802.3 HSSG Swanson_01_1106)

0 5 10 15 20 25 30 35 40 45 0-25m 26-50m 51-75m 76-100m 101-125m 126-150m 151-175m 176-200m 201-225m 226-250m 251-275m 276-300m % Links

Optical Fibre Permanent Links; assumed to be switch-to-switch. Derived from cumulative length distribution.

(28)

0 5 10 15 20 25 30 35 40 45 0-25m 26-50m 51-75m 76-100m 101-1 25m 126-1 50m 151-1 75m 176-2 00m 201-2 25m 226-2 50m 251-2 75m 276-3 00m

Flatman All Kolesar Swanson

% Links

Enterprise Data Centre Cabling Lengths

(29)

0 10 20 30 40 50 60 70 80 90 100 0 25 50 75 100 125 150 175 200 225 250 275 300

Flatman All Kolesar Swanson

(30)

Enterprise Data Centre Cabling Lengths

0 10 20 30 40 50 60 70 80 90 100 0 25 50 75 100 125 150 175 200 225 250 275 300

Flatman All Flatman V.Large Flatman Large Flatman Medium

Flatman Small Kolesar Swanson

Cumulative % Links Length (m) 1% links 1% links 99% links 99% links

(31)

Enterprise Data Centre Cabling Lengths

Access-to-Distribution Permanent Links & Channels

0 5 10 15 20 25 30 35 40 45 0-25m 26-50m 51-75m 76-100m 101-1 25m 126-1 50m 151-1 75m 176-2 00m 201-2 25m 226-2 50m 251-2 75m 276-3 00m

Flatman All Permanent Links Flatman All Channels

% Links

Average Perm Link = 46.1m Average Cordage = 10.4m Minimum Cordage = 3.0m Maximum Cordage = 24.0m Average Channel = 56.5m Min # Cords/Channel = 2.0 Max # Cords/Channel = 4.0 Avg # Cords/Channel = 2.7 used to calculate channel lengths

(32)

Enterprise Data Centre Cabling Lengths

Access-to-Distribution Channels

0 10 20 30 40 50 60 70 80 90 100 0 25 50 75 100 125 150 175 200 225 250 275 300 Flatman All Cumulative % Links Length (m) 89.0% Total of 16,000 links

(33)

0 10 20 30 40 50 60 70 80 90 100 0-25m 26-50m 51-75m 76-100m 101-125m 126-150m 151-175m 176-200m 201-225m 226-250m 251-275m 276-300m

All V.Large Large Medium Small

% Links

Enterprise Data Centre Cabling Lengths

Distribution-to-Core Permanent Links

(34)

Distribution-to-Core Permanent Links & Channels

0 10 20 30 40 50 60 0-25m 26-50m 51-75m 76-100m 101-1 25m 126-1 50m 151-1 75m 176-2 00m 201-2 25m 226-2 50m 251-2 75m 276-3 00m

Flatman All Permanent Links Flatman All Channels

% Links

used to calculate channel lengths

Average Perm Link = 37.6m Average Cordage = 11.1m Minimum Cordage = 4.0m Maximum Cordage = 12.0m Average Channel = 48.7m Min # Cords/Channel = 2.0 Max # Cords/Channel = 4.0 Avg # Cords/Channel = 2.1

(35)

Enterprise Data Centre Cabling Lengths

Distribution-to-Core Channels

0 10 20 30 40 50 60 70 80 90 100 0 25 50 75 100 125 150 175 200 225 250 275 300 Flatman All Cumulative % Links Length (m) 84.8% Total of 3,000 links

(36)

Data Centre Cabling Market Research

¾

detailed data centre cabling analysis by BSRIA

¾

research conducted in the USA during fall 2007

¾

assesses deployment trends 2007, 2008 & 2009

¾

report planned to be published 31 January 2008

¾

see Alan Flatman for prospectus & other details