Planning, Designing and Installing a Pre-Terminated Copper and Optical Cabling System for the Data Center

(1)

Planning, Designing and Installing

a Pre-Terminated Copper and

Optical Cabling System for the

Data Center

(2)

Presenters

• Charlie Bogolawski, RCDD, CDT Director of Technical Sales, Leviton – Danbury, CT

• Bo Conrad, RCDD

Director of Project Management, E2Optics – Ashburn, VA

(3)

House Cleaning items

• Please quiet all phones and loud conversations • Please hold all questions until Q+A section

(4)

Discussion on Planning

• Planning

– Definition of a Data Center – Roles and Responsibilities – Project Delivery Methods – The Architecture

– The Transport Technologies – Migration Strategy

(5)

Discussion on Design

• The Design

– Performance Standards and Codes – Pathways and Spaces

• Overhead • Under Floor – Media • Copper • Fiber – Patching Environment • Interfaces • Density

(6)

Discussion on Installation

• The Installation

– Timelines and Milestones

– Field and/or Factory Terminated

– False sense of security

– Performance Warranties and Guaranties

(7)

(8)

(9)

A Data Center is…

• A place that stores electronic information that is mission critical to my business.

• A place that provides telecommunications access to my electronic information without any delay or interruption.

• A place that protects my electronic information.

• A place that helps reduce my business cost and gets me closer to my customers.

(10)

What is being asked of a Data Center?

1. Improve application reliability 2. Align closer with business needs 3. Reduce complexity

4. Reduce cost

Get me to my volumes of information faster, better and reduce my cost. Anywhere and anytime I need it.

(11)

(12)

The Owner

• Principle stake holder, End User and Operator of Data Center

• Provide program for Data Center utilization • Issues Purchase Orders for The Work and

(13)

The Owner

• Owner’s Project Team should consist of not only IT personnel but also liaisons from every single

Business Unit (BU) within the Organization. Every BU needs to be a stake holder along with IT for

success of project.

• IT should be asking - What are the short term and what are the long term business requirements for each business unit. Know and communicate the requirements often.

(14)

The Owner’s analysis

• Size of facility – Build or Co-Lo

• Application performance needs • Service Level Agreements (SLAs) • Servers

• Storage

• Cabling Infrastructure

• Power • Cooling

(15)

The General Contractor

• Is issued and holds the Primary Contract to build / construct the Data Center and its’ systems for power, cooling, cabling, and

Service Provider connections for the Owner. Has a Team of in-house or sub-contracted

experts to help fulfill all details of the contractual obligations of the Contract.

(16)

The Low Voltage Contractor

• Constructs and/or assembles a Data Center cabling system that will support various

transport technologies as specified by electrical and/or optical performance

characteristics and specifications as outlined in the Construction Documents, also known as The Work. Contractors need to have their

Work deemed acceptable by Owner before payment of rendered services/products.

(17)

The Manufacturer

• Manufactures, as well as, assembles cabling systems and components that meet

performance requirements that are specified in construction documents. Ensures

tolerances and quality of finished goods are produced as expected and in a timely manner.

(18)

The Distributor

• Facilitates the logistics of materials to ensure specified products are delivered to the LV

Contractor.

• Provides credit for project builds, materials purchase / return and maintains inventory.

(19)

Project Delivery Methods

Advantages / Dis-advantages

(20)

Project Delivery Method Design-Bid-Build Owner General Contractor Architect / MEP / Consultant Sub-Contractor

(21)

Project Delivery Method Design-Build Owner General Contractor Architect / MEP / Consultant Sub-Contractor

(22)

Linear track toward construction

• Design-Bid-Build

– More time up front for design development and bid analysis

– Changes required to do The Work is slow • Design-Build

– Fast track build

– Changes can be done quickly – Need very strong management

(23)

(24)

TIA – 942 – A / Traditional

"This content, from the draft ANSI/TIA-942 Standard, Telecommunications Infrastructure Standard for Data Centers, is reproduced under written permission from Telecommunications Industry Association."

(25)

TIA – 942 – A / Fat-tree

"This content, from the ANSI/TIA-942 Standard, Telecommunications Infrastructure Standard for Data Centers, is reproduced under written permission from Telecommunications Industry Association."

(26)

TIA – 942 – A / Fat–tree w/port extenders

(27)

TIA – 942 – A / Fat–tree Pods

(28)

TIA – 942 – A / Full-mesh

(29)

TIA – 942 – A / Interconnected mesh

"This content, from the ANSI/TIA-942-1 Standard, Telecommunications Infrastructure Standard for Data Centers, is reproduced under written permission from Telecommunications Industry Association."

(30)

TIA – 942 – A / Centralized

"This content, from the draft ANSI/TIA-942-1 Standard, Telecommunications Infrastructure Standard for Data Centers, is reproduced under written permission from Telecommunications Industry Association."

(31)

TIA – 942 – A / Virtual switch

(32)

(33)

Current and Emerging Technologies over Single mode Application Nomimal Wavelength Channel Attenuation Channel Distance

Fiber Type TIA 492CAAA

OS1

Fiber Type TIA 492CAAB OS2 Ethernet 1000BASE-LX 1310 nm 4.5 dB 5000 meters (16405 feet) Supports Supports Ethernet 10GBASE-LX4 1310 nm 6.3 dB 10000 meters (32810 feet) Supports Supports Ethernet 10GBASE-L 1310 nm 6.2 dB 10000 meters (32810 feet) Supports Supports Fibre Channel 100-SM-LC-L (1062 Mbaud) 1310 nm 7.8 dB 10000 meters (32810 feet) Supports Supports Fibre Channel 200-SM-LC-L (2125 Mbaud) 1310 nm 7.8 dB 10000 meters (32810 feet) Supports Supports Fibre Channel 400-SM-LC-L (4250 Mbaud) 1310 nm 7.8 dB 10000 meters (32810 feet) Supports Supports Fibre Channel 1200-SM-LL-L (10512 Mbaud) 1310 nm 6.0 dB 10000 meters (32810 feet) Supports Supports

(34)

Current and Emerging Technologies over Single Mode continued Application Nomimal Wavelength Channel Attenuation Channel Distance

Fiber Type TIA 492CAAA

OS1

Fiber Type TIA 492CAAB OS2 FDDI SMF-PMD ANSI X3.184 1310 nm 10 dB 10000 meters (32810 feet) Supports Supports 40GBase-LR4 1264.5 to 1277.5 1284.5 to 1297.5 1304.5 to 1317.5 1324.5 to 1337.5 6.7 dB 10000 meters (32810 feet) Supports Supports 100GBase-LR4 1294.53 to 1296.59 1299.02 to 1301.09 1303.54 to 1305.63 1308.09 to 1310.19 6.3 dB 10000 meters (32810 feet) Supports Supports 100GBase-ER4 1294.53 to 1296.59 1299.02 to 1301.09 1303.54 to 1305.63 1308.09 to 1310.19 18 dB 40000 meters (131234 feet) Supports Supports 10x10MSA 1521 to 1525 1529 to 1533 1537 to 1541 1545 to 1549 1553 to 1557 1561 to 1565 1569 to 1573 1577 to 1581 1585 to 1589 1593 to 1597 2.6 dB 5 dB 2000 meters (6562 feet) 10000 meters (32810 feet) Supports Supports Supports Supports

40GBase-FR 1550 serial ? 2000 meters

(6562 feet)

(35)

Current and Emerging Technologies over Multimode Application Nomimal Wavelength Channel Attenuation Channel Distance Fiber Type TIA 492AA AA OM1 Fiber Type TIA 492AA AB OM2 Fiber Type TIA 492AA AC OM3 Ethernet 10 / 100 BASE-SX 850 nm 4.0 dB 300 meters (984 feet)

Supports Supports Supports

Ethernet 100BASE-FX 1300 nm 11.0 dB OM1 6.0 dB OM2 6.0 dB OM3 2000 meters (6560 feet)

Supports Supports Supports

Ethernet 1000BASE-SX 850 nm 2.6 dB OM1 3.6 dB OM2 4.5 dB OM3 varies Supports up to 275m (900 feet) Supports up to 500m (1804 feet) Supports up to 800m (2625 feet) Ethernet 1000BASE-LX 1300 nm 2.3 dB 550 meters (1804 feet)

Supports Supports Supports Ethernet 10GBASE-S 850 nm 2.4 dB OM1 2.3 dB OM2 2.6 dB OM3 varies Supports up to 33m (108 feet) Supports up to 82m (269 feet) Supports up to 300m (984 feet) Ethernet 10GBASE-LRM 1300 nm 1.9 dB 220 meters (720 feet)

Supports Supports Supports Ethernet 10GBASE-LX4 1300 nm 2.5 dB OM1 2.0 dB OM2 2.0 dB OM3 300 meters (984 feet) Supports Supports Supports

(36)

Current and Emerging Technologies over Multimode continued Application Nomimal Wavelength Channel Attenuation Channel Distance Fiber Type TIA 492AA AA OM1 Fiber Type TIA 492AA AB OM2 Fiber Type TIA 492AA AC OM3 Fibre Channel 100-MX-SN-I (1062 Mbaud) 850 nm 3.0 dB OM1 3.9 dB OM2 4.6 dB OM3 varies Supports up to 300m (984 feet) Supports up to 500m (1640 feet) Supports up to 860m (2822 feet) Fibre Channel 200-MX-SN-I (2125 Mbaud) 850 nm 2.1 dB OM1 2.6 dB OM2 3.3 dB OM3 varies Supports up to 150m (492 feet) Supports up to 300m (984 feet) Supports up to 500m (1640 feet) Fibre Channel 400-MX-SN-I (4250 Mbaud) 850 nm 1.8 dB OM1 2.1 dB OM2 2.5 dB OM3 varies Supports up to 70m (230 feet) Supports up to 150m (492 feet) Supports up to 270m (886 feet) Fibre Channel 1200-MX-SN-I (10512 Mbaud) 850 nm 2.4 dB OM1 2.2 dB OM2 2.6 dB OM3 varies Supports up to 33m (108 feet) Supports up to 82m (269 feet) Supports up to 300m (984 feet) 16G Fibre Channel 1600-MX-SN (10512 Mbaud) 850 nm 1.6 dB OM2 1.9 dB OM3

varies Not supported Supports up to 35m (115 feet) Supports up to 100m (328 feet) FDDI PMD ANSI X3.166 1300 nm 11.0 dB OM1 6.0 dB OM2 6.0 dB OM3 2000 meters (6560 feet)

(37)

Current and Emerging Technologies over Multimode continued

Application Nomimal

Wavelength _AttenuationChannel _DistanceChannel Fiber Type _TIA

492AA AA OM1 Fiber Type TIA 492AA AB OM2 Fiber Type TIA 492AA AC OM3 Fiber Type TIA 492AA AD OM4 40GBASE-SR4 850 nm 1.9 dB OM3 1.5 dB OM4

Varies No support No support Supports up to 100m (328 feet) Supports up to 150m (492 feet) 100GBASE-SR10 850 nm 1.9 dB OM3 1.5 dB OM4

Varies No support No support Supports up to 100m (328 feet)

Supports up to 150m (492 feet)

(38)

Current and Emerging Technologies over Copper

Application Media Distance

IEEE 802.3i 10BASE-T

Category 3 twisted pair or better

100m (328 feet) IEEE 802.3u

100BASE-T

Category 5 twisted pair or better

100m (328 feet) IEEE 802.3ab

1000BASE-T

Category 5e twisted pair or better 100m (328 feet) IEEE 802.3ak 10GBASE-CX4 Twinaxial 15m (49 feet) IEEE 802.3an 10GBASE-T

Category 6A twisted pair or better

100m (328 feet) SFF-8431

SFP+

Fiber Channel / Ethernet

Twinaxial 7m (23 feet)

IEEE 802.3ba 40GBASE-CR4 100GBASE-CR10

(39)

(40)

Market Drivers - Multi-Gigabit

Adoption Rates

(41)

Market Drivers – Network Port

Forecast

• By 2015, 100G will account for 25% of network ports

(42)

40/100 Gigabit Architecture

(43)

100G Line-Side Equipment Available • Brocade MLX Series

• Juniper T1600 Series • Cisco CRS-3 Series

(44)

40G Client-Side Equipment Available

• Extreme Networks Summit TOR & Black Diamond Core Switch

• Cisco Nexus 3064 TOR Switch

(45)

40G Client-Side Equipment Available

• Force 10 S4810 TOR & E-Series Core Switch

• Arista 7050 TOR Switch

(46)

Extreme Networks 40G

Architecture

(47)

(48)

Equipment Interfaces

• 40G – CFP – QSFP • 100G – CFP – CXP CXP Transceiver _{CFP Transceiver}

(49)

Migration strategies need to account for:

• Performance of existing 10G networks • Channel Insertion Loss Budget

• Channel Distance

• Connector Insertion Loss • System Polarity

• Reduced fiber strand loss

• Design flexibility for equipment architecture – Four 40 Gig Ports per Edge switch

(50)

(51)

Discussion on Design

• The Design

– Performance Standards and Codes – Topology

– Pathways and Spaces

• Overhead • Under Floor

• Determining cable length

– Media

• Copper • Fiber

(52)

(53)

Codes

• _{International Fire Code NFPA 70: National Electrical Code,}

2011

• _{NFPA 70E: Standard for Electrical Safety in the Workplace,}

2012

• _{NFPA 72: National Fire Alarm and Signaling Code, 2010}

• _{NFPA 101: Life Safety Code, 2012}

• _{NFPA 232: Standard for the Protection of Records, 2012}

• _{NFPA 75: Standard for the Protection of Information}

Technology Equipment, 2009

• _{NFPA 76: Standard for the Fire Protection of}

(54)

Standards

• TIA Common, Premise, & Component Standards • TIA 942 – A

• BICSI 002 – Data Center Design and Best Practices • NECA / BICSI 607 – Grounding and Bonding Standard • BICSI 009 – Building Information Modeling

• BICSI ITSIMM – 6th_Edition

• BICSI TDMM – 12th_Edition

• IEEE P802.3ba • EN 50600-2-2

(55)

ASHRAE References

• ASHRAE 62.1 Ventilation for Acceptable Indoor Air Quality 2007 • ASHRAE Best Practices for Datacom Facility Energy Efficiency 2009 • ASHRAE Design Considerations for Data and Communication

Equipment Centers 2009

• ASHRAE Gaseous and Particulate Contamination Guidelines for Data Centers 2009

• ASHRAE Structural and Vibration Guidelines for Datacom Equipment Centers 2008

• ASHRAE Thermal Guidelines for Data Processing Environments 2009

(56)

Understand the components of the

performance standard you are trying to attain

Common Standards Premise Standards Component Standards

ANSI/TIA-568-C.0

Generic Telecommunications Cabling for Customer Premises

TIA-569-B

Commercial Building Standard for Telecommunications

Pathways and Spaces

ANSI-TIA-606-A

Administration Standard for Commercial Telecommunications Infrastructure

ANSI-TIA-607-B

Telecommunications Grounding (Earthing) and Bonding for Customer Premises

ANSI-TIA-568-C.1 Commercial Building Telecommunications Cabling Standard ANSI-TIA-570-B Residential Telecommunications Infrastructure Standard ANSI-TIA-758-A

Customer-Owned Outside Plant Telecommunications Infrastructure Standard

ANSI-TIA-942

Telecommunications Infrastructure Standard for Data Centers

ANSI-TIA-1005

Telecommunications Infrastructure Standard for Industrial Premises

ANSI-TIA-1179

Infrastructure Standard for Healthcare Facilities

ANSI-TIA-568-C.2

Balanced Twisted-Pair Telecommunications Cabling

and Components Standard

Optical Fiber Cabling Components Standard

Broadband Coaxial

ANSI-TIA-862

Building Automation Systems Cabling Standard for Commercial Buildings

(57)

IEEE determines and establishes

transmission protocols/methods and component performance for network transport systems to support

communication reliably.

TIA determines component, link and channel performance. They also specify testing methods to validate

performance.

BICSI develops standards with a mix of best practices

(58)

(59)

Topology – Key items

• Flexibility • Scalability • Modularity • Manageability

(60)

TIA – 942 Topology

(61)

(62)

(63)

(64)

(65)

3/26/2014 P65

Using fiber out to server rows

1U 48 Channel panel

48 strand 48 strand Port 1 - 24 Port 1 - 24

(66)

(67)

(68)

Conveyance Systems

• Allow maximum flexibility

• Allow for Growth / Contraction • Minimal disruption

– Mechanical Systems

– Power Distribution Systems

– Maintenance, Adds and Changes

• Consider ALL Code, Standard, Ratings, Technology Performance

• Accessibility

(69)

Conveyance Design Considerations

• TIA-942 mandates minimum ceiling height of 8.5ft AFF to allow for 7 foot racks and cabinets

– Also mandates minimum clearance of 18” of

clearance above pathways (for building systems) • Potential limitation

• If frequent changes expected consider using trunk cables to overhead (“zero-U”) patch panels

• Design around life safety, mechanical systems – What will the impacts be to air flow

(70)

(71)

(72)

(73)

Additional Considerations

• Are there any Seismic Restrictions / Requirements

• Can overhead be supported structurally

– Does a floor mounted raised pathway support system work

• Multi-Tier Conveyance Systems – Media segregation

– Tennant asset separation

(74)

Overhead or Underfloor

Pro Con

Overhead •Lower Construction Cost •Easier MAC

•Easier to Trace cabling

•Separates from high voltage pathways (if under floor)

•Must accommodate cabinet height variations

•Not all ceilings can support

•Design around other building systems •Requires ladders etc. for MAC work Raised Floor •Presents “clean” appearance

•More secure pathways (limits access and tampering)

•Greater path flexibility

•Potential airflow disruption •Limited room for cabling

•Higher Construction Cost (when paired with flooring)

(75)

Pathway systems

Pro Con

Ladder Rack •Greater structural support •Higher material cost

•Ease of tiering

•Lower installation cost

•Limited feature set

Cable Tray •Lower material cost

•Vestal elevation changes

•Higher installation cost •Difficult to tier

Optical Trough •Resists environmental variables •Better protects fiber installations

•High installation cost •Limited growth capacity

(76)

(77)

Considerations

• Equipment Costs • Cable Fill • Power Consumption • Scalability / Flexibility • Life Cycle

(78)

Copper

• What is the Application • What is the life cycle

• Environmental conditions

• What if it could be scaled with the DC • IS COPPER DEAD?

(79)

The Categories

TIA classification of cabling & components: • Category 1 is specified up to 1 MHz.

• Category 2 is specified up to 4 MHz.

• Category 5/5e is specified up to 100 MHz.

• Category 6 is specified up to 250 MHz. • Category 6A is specified up to 500 MHz.

• Category 7 – not specified

• Category 8 – to be developed up to 2 GHz possibly.

IEC/ISO classifications of cabling & components: • Class A is specified up to 100 kHz.

• Class B / Category 1 is specified up to 1 MHz.

• Class C / Category 3 is specified up to 16 MHz.

• Class D / Category 5 is specified up to 100 MHz.

• Class E / Category 6 is specified up to 250 MHz.

• Class EA / Category 6A is specified up to 500 MHz.

• Class F / Category “7” is specified up to 600 MHz.

• Class FA / Category 7A is specified up to 1000 MHz. (under development)

(80)

Fiber

• What is the application • What is the life cycle • Density

• Cost Analysis – SM vs. MM • Polarities

(81)

(82)

(83)

Total Loss = 4.8dB

(84)

Loss Budgets – Multiple hops

(85)

(86)

ANSI/TIA-568C.0 - 2009 page 25

(87)

ANSI/TIA-568C.0 - 2009 page 26

(88)

ANSI/TIA-568C.0 - 2009 page 27

(89)

ANSI/TIA-568C.0 - 2009 page 28

(90)

ANSI/TIA-568C.0 - 2009 page 29

(91)