Data Centre Reliability and Redundancy. What do they mean and are you are getting what you need?

(1)

Data Centre Reliability

and Redundancy

What do they mean and are you

are getting what you need?

(2)

What's the message?

• Why consider Reliability and Redundancy?

• How do you know what Reliability and/or

Redundancy is required for your Data Centre?

• How do you communicate to the design

engineer?

• How does the design engineer communicate to

you?

• What are the typical issues and what can

typically go wrong?

• How to consider Reliability and Redundancy?

• Summary.

(3)

Why?

• Why do we need to consider Reliability

and Redundancy?

1. Equipment or system Reliability can not be

guaranteed.

2. All equipment and systems require some

form of maintenance, repair and/or

replacement at some time.

3. Data centre users require certainty of

operation.

(4)

Reliability vs Redundancy

Reliability

Redundancy

None

100%

None

(5)

How?

• How do you know what Reliability and/or

Redundancy is required for your Data

Centre?

1. Understand the intended Data Centre use.

2. Understand stakeholder expectations.

3. Understand the reliability of the equipment

proposed.

4. Understand the Reliability of the available

energy and water supplies.

(6)

How?

• How do you communicate to the design

engineer?

• How does the design

engineer communicate

to you?

Agree on a common

language!

(7)

What are the typical issues?

• Very complicated operational and servicing

inter-relationships.

• Poor briefing / wrong language used.

• Inadequate budget to meet brief.

• Downtime for repairs or maintenance.

• Cheaper equipment may be less reliable and

therefore require greater redundancy.

• Key stakeholders on ‘different pages’.

• Testing and Commissioning.

• Who will operate and maintain the systems?

• Understanding the various classifications, 999,

(8)

What can typically go wrong?

• Design does not meet the project

requirements.

(9)

• Single Point of Failure not properly

analysed.

(10)

• Partial or Multiple equipment failures not

considered properly.

(11)

(12)

• Equipment is not tested or commissioned

properly.

(13)

What else can go wrong?

• Complex systems difficult for technicians

to understand and maintain.

(14)

• Routine and/or monitored equipment

(15)

• End user compromises design through

poor change management.

(16)

• End user doesn’t understand design

principles.

(17)

(18)

How to consider Reliability and

Redundancy?

• Mean Time To Repair (MTTR)

• Mean Time To Failure (MTTF)

• Mean Time Between Failure (MTBF)

• Availability (MTBF / MTBF + MTTR)

• MIL-STD-756B

• Single Point Of Failure (SPOF)

• Failure Mode Analysis (FMA)

• Steady State Operation (SSO)

• TIA 942 Tier system

(19)

What is the relationship between

MTBF and Reliability?

• Reliability theory is a statistical approach to predicting

equipment failures.

• It can become very difficult to interpret in relation to Data

Centre infrastructure.

• Basic Theory is;

• MTBF = Mean Time Between Failure

• This is actually defined as the inverse of the failure rate

over a given time period.

• That is…failure rate (say failures per hour)

λ = 1/MTBF

• This then allows Reliability over a particular time period

to be estimated using an exponential equation of

(20)

MTBF Misconceptions

• Often thought to mean a certain

guaranteed time between failures.

• THIS IS NOT CORRECT!

• It is a calculated statistical average

assuming a constant failure rate.

• MTBF does NOT account for Infant

Mortality (Burn in) that occurs during

factory batch testing and specific project

commissioning activities.

(21)

More MTBF Misconceptions

• MTBF is NOT service life.

• Service life is determined by specific component

usage requirements and is addressed by

monitored and/or routine maintenance or

replacement regimes.

• An example of this is UPS battery replacement

and generator bearing replacements.

• It is important to note that in some cases MTBF

can exceed the service life of a component.

(22)

LED light fitting example

• MTBF

= 200,000 hours

• Duration of service (t) = 8,760 hours (1yr)

• Number of units

= 1000

• Reliability

= 95.62%

• Failure rate

= 4.38%

(23)

UPS System example

• MTBF

= 100,000 hours

• Duration of service (t) = 43,800 hours

(5y)

• Number of units

= 2

• Reliability

= 65%

• Failure rate

= 35%

(24)

Result

• It is therefore important to understand that

it is almost certain that one UPS will fail

over a 5 year period.

• Based on this understanding there must

be an expectation that any particular item

of critical plant will fail unexpectedly at

(25)

What does Reliability mean?

• Different things to different people!

• ICT vs Power vs Cooling.

• Short term power failure can result in long

term ICT failure.

• Short term cooling failure may not be critical.

• Can be defined as determining the risk of

(26)

Redundancy requirements

• Need to respond to equipment and / or

system failures.

• Depends on the specific Data Centre

requirements.

• Depends on the equipment installed.

• Depends on the tolerable ICT downtime

permissible.

• Depends on the specific system design

characteristics.

(27)

Communicating Redundancy

• We need a common language that

communicates redundancy

• There are very few recognized standards

that can be used

• One that has been useful is ANCI/TIA/942

• This is an American Standard and

therefore needs to be used carefully in

Australia.

(28)

Overview of Data Centre Critical

Systems

• ANSI/TIA 942 has an appendix that

addresses various aspects of redundancy

• It expands on the Uptime Institute Tier

System.

• The appendices cover components of a

Data Centre that affect reliability and

redundancy including Architectural,

Structural and fire systems in addition to

Mechanical and Electrical.

(29)

TIA 942 – Architectural (1)

Uptime Tiering

Reference

Site Selection

Proximity to flood hazard as

mapped on the federal flood

hazard boundry or flood

insurance map

No Requirement

Not in 50 year Flood

hazard area

Not within 100 year flood hazard

area and greaterthan 91m (300ft)

from 50 year flood hazard area

Greater than 91m (300ft) from

100 year flood hazard

Proximity to costal or inland

waterways

No Requirement

greater than 91m (300ft)

Greater than 0.8km (1/2 mile)

Proximity to major highway

trafic arteries

No Requirement

greater than 91m (300ft)

Greater than 0.8km (1/2 mile)

Proximity to major airports

No Requirement

Greater than 1.6km (1 mile) and

less than 48km (30 miles)

Greater than 8km/5 miles and

less than 48km/30 miles

Proximity to major

metropolitan area

No Requirement

less than 48km (30 miles)

less than 16km (10 miles)

Separate visitor and

employee parking areas

No Requirement

yes (physically separated with

separate enteries)

yes (physically separated by

fence or wall with separate

enteries)

Separate from loading docks

No Requirement

yes (physically separated with

separate enteries)

yes (physically separated by

fence or wall with separate

enteries)

Proximity of visitor parking to

data centre perimiter building

walls

No Requirement

9.1m (30 ft) minimum separation

with physical barrier to prevent

vehicle from driving closer

18.3m (60 ft) minimum

separation with physical

barrier to prevent vehicle

from driving closer

Multi-tenant occupancy

No Restriction

Allowed if occupancies

non- hazardous

Allowed if tenants are Data

Cenntres or Telecommunications

companies

Allowed if tenants are Data

Cenntres or

Telecommunications

Parking

Tier 4

Architectural

(30)

TIA 942 – Architectural (2)

Building Construction

Type of construction

No Restriction

Type IIA,IIIA or VA

Type 1A or 1B

Exterior bearing walls

Code requirement

1 Hour Minimum

4 Hours Miinimum

Interior bearing walls

Code requirement

1 Hour Minimum

2 Hours Minimum

Exterior nonbearing walls

Code requirement

1 Hour Minimum

4 Hours Miinimum

Structural frame

Code requirement

1 Hour Minimum

2 Hours Minimum

Interior non-computer room

walls

Code requirement

1 Hour Minimum

Interior computer room walls Code requirement

Code requirement

1 Hour Minimum

2 Hours Minimum

Shaft enclosures

Code requirement

1 Hour Minimum

2 Hours Minimum

Floors and Floor-ceilings

Code requirement

1 Hour Minimum

2 Hours Minimum

Roofs and roof-ceilings

Code requirement

1 Hour Minimum

2 Hours Minimum

Compliance with NFPA 75

No Requirement

Yes

Vapour barriers for computer

room

No Requirement

Yes for walls no

requirements for ceilings

Yes

building entrances with

security checkpoints

No Requirement

Yes (primary building entrance

manned)

Yes (primary building

entrance manned)

Access Floor Panel

construction (when provided)

No Requirement

Computer grade all steel

Computer grade all steel or or

computer grade steel with

concrete filled

Understructure (when access

floor is provided)

No Requirement

Bolted stringer

Bolted stringer with 1.2m x

1.2m (4 ft x 4 ft) basket weave

patten

Fire Resistive Requirements

(31)

TIA 942 – Architectural (3)

Ceiling construction

No Requirement

If provided - suspended with

clean room class 10M-100M

perforated tiles

Suspended with clean room

class 100 non perforated vinyl

coated gypsum board tiles

Ceiling Height (above raised

access floor if provided)

2.6m (8.5 ft) minimum

2.7m (9.0 ft) minimum

3.0m (10 ft) minimum (not less

than 460mm (18 in) above tallest

piece of equipment)

3.0m (10 ft) minimum (not less

than 600mm (24 in) above

tallest piece of equipment)

Class

No Restrictions

Class A

Type

No Restrictions

redundant with

Non-combustible deck (no

mechanically attached systems)

Double redundant with

concrete deck (no

mechanically attached

systems)

Wind Uplift resistance

Minimum code

requirements

FM I-90

FM I-90 minimum

FM I-120 minimum

Roof slope

Minimum code

requirements

Minimum code

requirements

1:48 (1/4 in per foot) minimum

1:24 (1/2 in per foot) minimum

Fire Rating

Minimum code

requirements

Minimum code

requirements

Min Code Req, not less than 0.75

hour at Computer room

Min Code req, not less than

1.5 hours at Computer Room

Door Size

Minimum code

requirements

Minimum code

requirements

Min Code Req, not less than 1m

wide and 2.13m High

Min Code Req, not less than

1.2m wide and 2.13m High

Windows on perimiter of

computer room

Allowed with minimum

Code required fire

rating

Allowed with minimum

Code required fire rating

interior windows allowed with

minimum 1 hour fire rating no

exterior windows allowed

interior windows allowed

with minimum 1 hour fire

rating no exterior windows

allowed

Ceilings with in Computer areas (when provided)

Roofing

(32)

TIA 942 - Architectural (4)

Physically Separate from

other areas of Data Centre

No Requirement

Yes

Fire separation from other

areas of Data Centre

Minimum code

requirements

Minimum code

requirements

Minimum code requirements (not

less than 1 Hour)

Minimum code requirements

(not less than 2 Hour)

Security Counter

No Requirement

Yes

Yes (physically separated)

Single person interlock to

prevent pass back

No Requirement

Yes

Physically Separate from

other areas of Data Centre

No Requirement

Yes

Fire separation from other

areas of Data Centre

Minimum code

requirements

Minimum code

requirements

Minimum code requirements (not

less than 1 Hour)

Minimum code requirements

(not less than 2 Hour)

Security Office

No Requirement

Yes

Physically Separate from

other areas of Data Centre

No Requirement

Yes

Fire separation from other

areas of Data Centre

Min Code Req

Not less than 1 Hour

Not Less than 2 Hours

180 deg peepholes on

Security equipment and

rooms

No Requirement

Yes

Dedicated and hardened

security equipment and

monitoring rooms

No Requirement

Yes

Yes, with 16mm (5/8) plywood

walls and solid core door

Yes, with 16mm (5/8) plywood

walls and solid core door

Operations Centre physically

Separate from other areas of

Data Centre

No Requirement

Yes

Fire separation from other

areas of Data Centre

No Requirement

1 Hour

2 Hours

Proximity to computer room

No Requirement

Indirectly accessable (maximum

of 1 adjoining room)

Directly accessable

Adminstrative Offices

Operations Centre

Entry Lobby

(33)

TIA 942 - Architectural (5)

Proximity to computer room

and support areas

No Requirement No Requirement If immediately adjacent, provided with leak prevention barrier

Not immediately adjacent and provided with leak prevention barrier

Fire separation from computer room and support areas

Minimum code requirements

Minimum code requirements (not less than 1 Hour)

Minimum code requirements (not less than 2 Hour)

Aisle widths for maintenance, repair, or equipment removal

No Requirement No Requirement Minimum code requirements (not less than 1m (3 ft) clear)

Minimum code requirements (not less than 1.2m (4 ft) clear) Proximity to computer room No Requirement No Requirement Immediately adjacent Immediately adjacent

Fire separation from other areas of Data Centre

Required Exit Corridors Fire separation from computer room and support areas

Width Min Code Req Min Code Req Minimum code requirements and

not less than 1.2m (4 ft) clear

Minimum code requirements and not less than 1.5m (5 ft) clear

Shipping and receiving area physically Separate from

No shipping and receiving area provided

no Yes Yes

Fire separation from other areas of Data Centre

Min Code Requirement if shipping and

receiving area present

Not less than 1 Hour Not Less than 2 Hours

Physical protection of walls exposed to lifting equipment traffic

No Requirement No Requirement Yes (minimum 19mm (3/4 in) plywood wainscot)

Yes (steel bollards or similar protection)

Number of Loading docks No Requirement 1 per 2500sqm of computer room

1 per 2500sqm of computer room (2 Minimum)

1 per 2500sqm of computer room (2 Minimum)

Proximity to computer room and support areas

No Requirement No Requirement If within Data Centre Building - min 2 Hour separation from all other areas

Separate building or exterior weatherproof enclosures with Code required building separation

Proximity to publicly accessable areas

No Requirement No Requirement 9m (30 ft) or greater separation 19m (60 ft) or greater separation

Generator and Fuel Storage areas Restrooms and Break room areas

UPS and Battery Rooms

(34)

TIA 942 – Security (1)

Uptime Tiering

Reference

Security

Tier 1

Tier 2

Tier 3

Tier 4

System CPU UPS capacity

No requirement

Building

Building + 8 hour

Battery minimum

Field Panels UPS capacity

No requirement

Building + 4 hourr Battery

minimum

Building + 8 hour Battery

minimum

Building + 24 hour

Battery minimum

Field Device UPS capacity

No requirement

Building + 4 hourr Battery

minimum

Building + 8 hour Battery

minimum

Building + 24 hour

Battery minimum

Dedicated Security staff per

shift

No security staffing provided

1 per 3000 sqm (1 Minimum)

1 per 2000 sqm (2 Minimum)

1 per 2000 sqm (3

Minimum)

No of 8 hour security shifts

No requirement

1 (minimum 5 days per week)

2 (7 days per week)

3 (7 days per week)

Generators

Industrial grade lock

Intrusion Detection

Card Access

UPS, Telephone & MEP

Rooms

Industrial grade lock

Intrusion Detection

Card Access

Fiber Vaults

Industrial grade lock

Intrusion Detection

Card Access

Emergency Exit Doors

Industrial grade lock

Monitor

Delay egress per code

Accessible exterior

windows/openings

No Monitoring

Intrusion Detection (with offsite

monitoring during shifts when no

security staff is present)

Intrusion Detection (with offsite

monitoring during shifts when

no security staff is present)

Intrusion Detection

Security Operations Centre

No requirement

Card Access

Network Operations Centre

No requirement

Card Access

Security Equipment Rooms

No requirement

Intrusion Detection

Card Access

(35)

TIA 942 – Security (2)

Doors into computer rooms

Industrial grade lock

Intrusion Detection

Card or biometric access for

ingress

Card or biometric

access for ingress and

egress

Perimiter Building doors

No requirement

Intrusion Detection

Card access if main entrance;

intrusion detection all others

Card access if all

entrance

Main door onto computer

room floor

Industrial grade lock

Card Access

Single person Interlock to

prevent passback

Single person interlock

to prevent passback,

with biometric

Security counter in lobby

No requirement

Level 3 minimum

Security counter in shipping

and receiving

No requirement

Level 3 minimum

CCTV Monitoring

Building Perimiter and

parking

No requirement

Yes

Generators

No requirement

Yes

Access Controlled doors

No requirement

Yes

Computer room floors

No requirement

Yes

UPS, Telephone & MEP

Rooms

No requirement

Yes

CCTV

CCTV recording of all

activity on all cameras

No requirement

Yes - Digital

Recording rate (Frames/sec)

No requirement

20 frames/sec min

(36)

TIA 942 - Structural

Uptime Tiering Reference

Structural Tier 1 Tier 2 Tier 3 Tier 4

Any seismic zone acceptable, but may require special support mechanisms

No Restriction No Restriction No Restriction No restriction

Facility designed to seismic zone requirements No Restriction No Restriction No Restriction In seismic zone 0, 1, 2 to Zone 3 requirements, in siesmic zone 3 & 4 to

Zone 4 requirements Site specific response spectra - degree of local

seismic accelerators

No No With operation status after 10% in

50 year event

With operation status after 5% in 100 year event

Importance factor I=1 I=1.5 I=1.5 I=1.5

Telcommunications equipment racks/cabinets anchored at base or supported top and bottom

No Base only Fully braced Fully braced

Deflection limitations on telcommunications equipment within acceptable limits

No No Yes Yes

Bracing of electrical conduits and cable trays Per code Per code with importance factor Per code with importance factor Per code with importance factor Bracing of mechanical system major duct runs Per code Per code with importance factor Per code with importance factor Per code with importance factor Floor loading capacity superimposed live load 7.2kPa (150lbf/sqsf) 8.4kPa (175lbf/sqft) 12kPa (250lbf/sqsf) 12kPa (250lbf/sqsf)

Floor hanging capacity for ancillary loads suspended from below

1.2kPa (25lbf/sqsf) 1.2kPa (25lbf/sqsf) 2.4kPa (50lbf/sqsf) 2.4kPa (50lbf/sqsf)

Concrete slab thickness at ground 127mm 127mm 127mm 127mm

Concrete topping over flutes for elevated floors affects size of anchor which can be installed

102mm 102mm 102mm 102

Building LFRS (Shearwall/braced

frame/moment frame) indicates displacement of structure

Steel/Conc MF Conc.Shearwall/Steel BF Conc.Shearwall/Steel BF Conc.Shearwall/Steel BF

Building Energy Dissipation - Passive Dampers/base isolation (energy absorption)

None None Passive dampers Passive dampers/Base isolation

Battery/UPS floor vs building composition. Concrete floors more difficult to upgrade for intense loads. Steel framing with metal deck and fill much more easily upgraded

PT Concrete CIP Mild Concrete Steel Deck & Fill Steel Deck & fill

Steel deck and Fill/ Pre tension concrete/ CIP Mild - PT slabs much moore difficult to install anchors

(37)

TIA 942 – Electrical systems (1)

Uptime Tiering

Reference

Electrical Tier 1 Tier 2 Tier 3 Tier 4

Ststem allowance concurrent maintenance

No Down to but not including UPS Output Panelboards

Down to but not including Power Distribution Units

Throughout Distribution Systems

Single points of failure One or more single points of failure fo rthe distribution systems

serving electrical equipment or mechanical

systems

One or more single points of failure fo rthe distribution systems serving electrical equipment or mechanical

systems

No more than single points of failure for the distribution systems serving electrical equipment or mechanical

systems

No single points of failure fo rthe distribution systems serving electrical equipment

or mechanical systems

Power Syatem Analysis Short Circuit Study Coordination Study Arc

Falsh Analysis

Short Circuit Study Coordination Study Arc Falsh

Analysis

Short Circuit Study Coordination Study Arc Falsh Analysis Load Flow

Study

Short Circuit Study Coordination Study Arc Falsh Analysis Load Flow

Study Computer and

Telecommunications Equipment Power Cords

Single cord feed with 100% capacity

Single cord feed with 100% capacity

Redundant cord feed with 100% capacity on remaining

cord or cords

Redundant cord feed with 100% capacity on remaining

cord or cords

Utility entrance Single feed Single feed N+1 Redundant Feed 2N Redundant Feed from

different utility substations Main Utility Switchboard

Service Shared Dedicated Dedicated Dedicated

Construction Panel Board with bolt on circuit breakers

Switchboard with stationary circuit breakers

Switchboard with drawout circuit breakers

Surge Suppression Optional Optional Yes Yes

Generators correctly sized for UPS

Yes Yes Yes Yes

Utility

Uninterruptible Power Supply System General

(38)

TIA 942 – Electrical systems (2)

Redundancy N N N+1 2N

Topology Single or Parallel

Modules

Single or Parallel Modules with Static Bypass

Parallel Redundant modules, Distributed Redundant Modules or Block Redundant

Systems

Parallel Redundant modules, Distributed Redundant Modules or Block

Redundant Systems

Automatic Bypass None Non dedicated maintenance

bypass feeder to UPS output switchboard

Dedicated maintenance bypass feeder to UPS output

switchboard

Dedicated maintenance bypass feeder to UPS output

switchboard Maintenance Bypass

arrangement

Bypass from same source Bypass from same source Bypass from same source Bypass from reserve UPS system

Output Power Distribution Panelboard incorporating standard thermal magnetic trip breakers

Panelboard incorporating standard thermal magnetic trip

breakers

Panelboard incorporating circuit breakers that are coordinated under overload

and ground fault current condition

Panelboard incorporating circuit breakers that are

coordinated under all overcurrent conditions Battery String Common string for

multiple modules

Dedicated string for each module

Battery Type 5 year valve regulated lead acid

10 year valve regulated lead acid or flood type

15 year valve regulated lead acid or flood type

20 year lead acid flood type Battery Minimum Backup

Time

5 minutes 7 minutes 10 minutes 15 minutes

Battery Monitoring System Optional Optional String level by UPS system Centralised automated system to check each cell for, voltage and impedance

or resistance

Flywheel Allowed Allowed Allowed Allowed

Power Distribution Unit

Transformer Optional K-rated or harmonic

cancelling

K-rated or harmonic cancelling, Front accessable

for maintenance

K-rated or harmonic cancelling low inrush, Front accessable for maintenance

Overcurrent Device NA Fuse Circuit breaker Circuit breaker

Maintenance Bypass NA Manual non Guided Manual Guided Manual Guided

Output NA Single circuit breaker Dual circuit breaker Dual circuit breaker

(39)

TIA 942 – Electrical systems (3)

Grounding

Lighting Protection System

Based on Risk analysis

Yes

Service entrance grounds

and generator grounds fully

comply to NEC or local

codes

Yes

Data Centre grounding

infrastructure in Computer

room

Optional

Yes

Installation

Do not install if not

required by Authorities

having juristriction

Do not install if not required by

Authorities having juristriction

Do not install if not required

by Authorities having

juristriction

Do not install if not required

by Authorities having

juristriction

Test Mode

Optional

Yes

Alarm

Optional

Yes

Abort Switch

Optional

Yes

Monitored Points

None

Utility, UPS, Generator

Utility, Main Transformer,

UPS, Generator, Automatic

Static Switch, PDU,

Automatic Transfer Switches

Utility, Main Transformer,

UPS, Generator, Automatic

Static Switch, PDU,

Automatic Transfer

Switch,es, Surge Protection

Devce, Critical Load Branch

Circuits

Notification Method

NA

Control Room console

Control Room console,

Pager, email, Text message

Control Room console,

Pager, email, Text message

Computer Room Emergency Power Off (EPO) System

(40)

TIA 942 – Electrical systems (4)

Separate room

No

Yes

Individual battery banks

isolated from each other

No

Yes

Shatterproof viewing glass

in Battery room door

No

Yes

Generator Sizing

Sized for UPS System

only

Sized for UPS & mechanical

System

Sized for total building load

with N+1 redundancy

Sized for total building load

with 2N redundancy

Generators on single bus

Yes

No

Installation

None

Provision for portable

Permenant

Equipment Tested

NA

Generator

Generator, UPS

Auto Shutdown

NA

None

Automatic upon failure of

utility

Automatic upon failure of

utility

Factory Acceptance Testing

Optional

Yes

Site Circuit Breaker Testing

Optional

Contact resistance test for all

critical circuit breakers in

critical and essential paths,

225 Amps and higher

Primary injection and

Contact resistance test for

all critical circuit breakers in

critical and essential paths,

225 Amps and higher

Commissioning

Optional

Component level, system level

Component level, system

level, Integrated system

Component level, system

level, Integrated system

Testing

Battery Room

Standby Generating Systems

(41)

TIA 942 - Maintenance

Uptime Tiering Reference

Maintenance Staff

On site day shift Only.

On call at other times

On site day shift Only.

On call at other times

On site 24 hrs M-F, on-call

weekends

Onsite 24 x 7

Preventative Maintenance

None

Limited preventative

maintenance program

Comprehensive preventative

maintenance program

Facility Training Programs

None

Comprehensive training

program

Comprehensive program including

manual operation procedures if it is

necessary to bypass control system

Tier 4

(42)

TIA 942 – Mechanical systems (1)

Uptime Tiering Reference

General

Routing of water or drain

piping not associated with the

Data Centre Equipment in data

centre spaces

Permitted but not

Not Permitted

Not permitted

Positive pressure in comuter

room and associated spaces

relative to outdoors and non

data centre spaces

No requirement

Yes

Floor drains in computer room

for condensate drain water,

humidifers flush water and

sprinkler discharge water

Yes

Mechanical systems on

Standby generator

No requirement

Yes

Water Cooled Systems

Indoor terminal Air

Conditioning units

No Redundant units

One redundant AC unit per

critical area

Qty of A/C units sufficent to

mainain critical area during loss

of one source of electrical power

Qty of A/C units sufficent to

mainain critical area during loss

of one source of electrical power

Humidity control for Computer

rooms

No requirement

Humidification provided

Electrical Service to

Mechanical Equipment

Single path of Electrical

power to A/C equipment

Single path of Electrical

power to A/C equipment

Multiple paths of electrical power

to AC equipment. Connected in

checkerboard fashion for cooling

redundancy

Multiple pathsof electrical power

to AC equipment. Connected in

checkerboard fashion for cooling

redundancy

Mechanical

(43)

TIA 942 – Mechanical systems (2)

Heat Rejection Dry Coolers (w here applicable)

No redundant dry coolers One redundant unit per system

Qty of units sufficent to provide minimal temporary capacity to critical area during loss of one

source of electrical pow er

Qty of units sufficent to mainain critical area during loss of one

source of electrical pow er Closed- circuit fluid coolers

(w here applicable)

No redundant fluid coolers One redundant unit per system

source of electrical pow er Circulating pumps No redundant condenser

w ater pumps

One redundant unit per system

source of electrical pow er Piping system Single path condenser

w ater system

Single path condenser w ater system

Headed parallel piped condenser w ater system

Dual path condenser w ater system

Chilled Water System Indoor Terminal Air Conditioning units

No Redundant AC units One redundant AC unit per critical area

Qty of A/C units sufficent to mainain critical area during loss of one source of electrical pow er Humidity Control for Computer

rooms

No requirement Humidification provided Humidification provided Humidification provided Electrical Service to

Mechanical Equipment

Single path of Electrical pow er to A/C equipment

Multiple paths of electrical pow er to AC equipment

Heat Rejection

Chilled Water Piping System Single path chilled w ater piping system

Single path chilled w ater piping system

Dual path ladder drop chilled w ater system w ith isolation valves

Dual path chilled w ater system

Chilled Water Pumps No Redundant units One redundant unit per system

source of electrical pow er Air-cooled Chillers No Redundant units One redundant unit per

system

Qty of units sufficent to maintain critical area during loss of one

source of electrical pow er Water Cooled Chillers No Redundant units One redundant unit per

system

source of electrical pow er Cooling Tow ers No Redundant units One redundant unit per

system

source of electrical pow er Condenser Water Pumps No Redundant units One redundant unit per

system

source of electrical pow er Condenser Water Piping

System

Single path condenser w ater system

Headed parallel piped condenser w ater system

Dual path condenser w ater system

(44)

TIA 942 – Mechanical systems (3)

Air-Cooled Systems

Indoor Terminal Air

Conditioning units/Outdoor

Condensors

No Redundant AC units

One redundant AC unit per

critical area

Qty of AC units sufficent to

provide minimal temporary

capacity to critical area during

loss of one source of electrical

power

Qty of A/C units sufficent to

mainain critical area during loss

of one source of electrical power

Electrical Service to

Mechanical Equipment

Single path of Electrical

power to A/C equipment

Single path of Electrical

power to A/C equipment

Multiple paths of electrical power

to AC equipment

Multiple paths of electrical power

to AC equipment

Humidity control for Computer

rooms

No requirement

Humidification provided

HVAC Control System

Control system failure will

interrupt cooling to critical

area

Control system failure will

not interrupt cooling to

critical area

Control system failure will not

interrupt cooling to critical area

Control system failure will not

interrupt cooling to critical area

Power source to HVAC control

system

Single path of Electrical

power to HVAC control

system

Redundant, UPS electrical

power to BMS control

Redundant, UPS electrical power

to BMS control

Redundant, UPS electrical power

to BMS control

Plumbing (for Water-cooled

heat rejection)

Make-up Water

Single water supply with no

on-site storage

Dual sources or one source

+ on-site storage

Dual sources or one source +

on-site storage

Dual sources or one source +

on-site storage

Points of connection to

condenser water system

(45)

TIA 942 – Fuel Oil

Uptime Tiering Reference

Bulk Storage tanks

Single storage tank

Single storage tanks

Multiple storage tanks

Storage tanks, pumps and

piping

Single pump and/or

supply pipe

Multiple pumps and

multiple supply pipes

Multiple pumps and

multiple supply pipes

Multiple pumps and

multiple supply pipes

Generator fuel capacity at full

load

8 hrs

24 hrs

72 hrs

96 hrs

Tier 4

(46)

TIA 942 – Fire Suppression

Uptime Tiering Reference

Fire Detection System

No

Yes

Fire Suppression System

When required

Pre-action (when

required)

Pre-Action (when

required)

Pre-Action (when

required)

Gaseous suppression system

No

Clean agents listed in

NFPA

Clean agents listed in

NFPA

Early warning smoke detection

system

No

Yes

Water leak detection system

No

Yes

Tier 4

(47)

TIA 942 - Items beyond the designers

control

THIS IS A LIST OF ITEMS BEYOND THE CONTROL OF THE DESIGNER

ITEM

Tier 1

Tier 2

Tier 3

Tier 4

SECURITY

Dedicated Security staff

per shift

No security staffing

provided

1 per 3000 sqm (1

Minimum)

1 per 2000 sqm (2

Minimum)

1 per 2000 sqm (3 Minimum)

No of 8 hour security

shifts

No requirement

1 (minimum 5 days

per week)

2 (7 days per week)

3 (7 days per week)

MAINTENANCE

Maintenance Staff

On site day shift Only.

On call at other times

On site day shift Only.

On call at other times

On site 24 hrs M-F,

on-call weekends

Onsite 24 x 7

Preventative

Maintenance

None

Limited

preventative

maintenance

program

Comprehensive preventative

maintenance program

Facility Training

Programs

None

Comprehensive

training program

Comprehensive program

including manual operation

procedures if it is necessary

(48)

TIA 942 - Additional items beyond designers

control

These items are also beyond the control of the designer as they are determined by site selection rather than data centre design

ITEM Tier 1 Tier 2 Tier 3 Tier 4

Site Selection

Proximity to flood hazard as mapped on the federal flood hazard boundry or flood insurance map

No Requirement Not in 50 year Flood hazard area

Not w ithin 100 year flood hazard area and greaterthan 91m (300ft) from 50 year flood hazard area

Greater than 91m (300ft) from 100 year flood hazard

Proximity to costal or inland w aterw ays

No Requirement No

Requirement

greater than 91m (300ft) Greater than 0.8km (1/2 mile)

Proximity to major highw ay trafic arteries

Requirement

greater than 91m (300ft) Greater than 0.8km (1/2 mile)

Proximity to major airports No Requirement No

Requirement

Greater than 1.6km (1 mile) and less than 48km (30 miles)

Greater than 8km/5 miles and less than 48km/30 miles Proximity to major metropolitan

area

Requirement

less than 48km (30 miles) less than 16km (10 miles) STRUCTURAL

Telcommunications equipment racks/cabinets anchored at base

or supported top and bottom

No Base only Fully braced Fully braced

Floor loading capacity superimposed live load

7.2kPa (150lbf/sqsf)

8.4kPa (175lbf/sqft)

12kPa (250lbf/sqsf) 12kPa (250lbf/sqsf)

Concrete slab thickness at ground 127mm 127mm 127mm 127mm

Concrete topping over flutes for elevated floors affects size of anchor w hich can be installed

102mm 102mm 102mm 102

Building LFRS (Shearw all/braced frame/moment frame) indicates

displacement of structure

Steel/Conc MF Conc.Shearw all /Steel BF

Conc.Shearw all/Steel BF Conc.Shearw all/Steel BF Building Energy Dissipation -

Passive Dampers/base isolation (energy absorption)

None None Passive dampers Passive dampers/Base isolation

ELECTRICAL

Utility entrance Single feed Single feed N+1 Redundant Feed 2N Redundant Feed from different utility substations

Factory Acceptance Testing Optional Optional Yes Yes

Site Circuit Breaker Testing Optional Optional Contact resistance test for all critical circuit breakers in critical and essential paths, 225

Amps and higher

Primary injection and Contact resistance test for all critical circuit breakers in critical and essential paths, 225 Amps and higher

(49)

TIA 942 - Design parameters that are

affected by capital cost

ITEM

Tier 1

Tier 2

Tier 3

Tier 4

ELECTRICAL

Battery Type

5 year valve regulated

lead acid

10 year valve regulated

lead acid or flood type

15 year valve regulated lead

acid or flood type

20 year lead acid flood type

Factory Acceptance

Testing

Optional

Yes

Site Circuit Breaker

Testing

Optional

Contact resistance test for all

critical circuit breakers in

critical and essential paths,

225 Amps and higher

Primary injection and Contact

resistance test for all critical

circuit breakers in critical and

essential paths, 225 Amps and

higher

FUEL SYSTEM

Bulk Storage tanks

Single storage tank

Single storage tanks

Multiple storage tanks

Storage tanks, pumps

and piping

Single pump and/or supply

pipe

Multiple pumps and

multiple supply pipes

Multiple pumps and multiple

supply pipes

Multiple pumps and multiple

supply pipes

Generator fuel capacity

at full load

8 hrs

24 hrs

72 hrs

96 hrs

FIRE

Fire Suppression

System

When required

Pre-action (when

required)

Pre-Action (when required)

Gaseous suppression

system

No

Clean agents listed in NFPA

(50)

TIA 942 - Non Australian references

Building Construction

Type of construction No Restriction No Restriction Type IIA,IIIA or VA Type 1A or 1B

roofing Class No Restrictions Class A Class A Class A

Fire Resistive Requirements

Exterior bearing w alls Code requirement Code requirement 1 Hour Minimum 4 Hours Miinimum Interior bearing w alls Code requirement Code requirement 1 Hour Minimum 2 Hours Minimum Exterior nonbearing w alls Code requirement Code requirement 1 Hour Minimum 4 Hours Miinimum Structural frame Code requirement Code requirement 1 Hour Minimum 2 Hours Minimum Interior non-computer room

w alls

Code requirement Code requirement 1 Hour Minimum 1 Hour Minimum Interior computer room w alls Code requirement Code requirement 1 Hour Minimum 2 Hours Minimum Shaft enclosures Code requirement Code requirement 1 Hour Minimum 2 Hours Minimum Floors and Floor-ceilings Code requirement Code requirement 1 Hour Minimum 2 Hours Minimum Roofs and roof-ceilings Code requirement Code requirement 1 Hour Minimum 2 Hours Minimum

Compliance w ith NFPA 75 No Requirement Yes Yes Yes

Roofing STRUCTURE Building LFRS (Shearw all/braced frame/moment frame) indicates displacement of structure Steel/Conc MF Conc.Shearw all/St eel BF Conc.Shearw all/Steel BF Conc.Shearw all/Steel BF

Building Energy Dissipation - Passive Dampers/base isolation (energy absorption)

None None Passive dampers Passive dampers/Base isolation

ELECTRICAL

Pow er Distribution Unit - Transformer Optional K-rated or harmonic cancelling K-rated or harmonic cancelling, Front accessable for maintenance K-rated or harmonic cancelling low inrush, Front accessable for maintenance

UPS Output Pow er Distribution

Panelboard incorporating standard thermal magnetic trip breakers Panelboard incorporating standard thermal magnetic trip breakers Panelboard incorporating circuit

breakers that are coordinated under overload and ground fault current condition

Panelboard

incorporating circuit breakers that are coordinated under all overcurrent conditions These are examples w here TIA942 calls up standards that are not familiar to Australian designers. In some cases it is not possible to easily interpret these requirements

(51)

TIA 942 - Common design deficiencies (1)

THE FOLLOWING LIST OF ITEMS FROM TIA REPRESENT AREAS WHERE DESIGNS ARE OFTEN DEFICIENT

Ststem allow ance concurrent maintenance

No Dow n to but not including UPS Output

Panelboards

Dow n to but not including Pow er Distribution Units

Throughout Distribution Systems

Single points of failure One or more single points of failure fo rthe distribution systems

serving electrical equipment or mechanical

One or more single points of failure fo rthe

distribution systems serving electrical

equipment or

No more than single points of failure for the

distribution systems serving electrical

equipment or

No single points of failure fo rthe distribution systems serving electrical

equipment or mechanical systems

Pow er Syatem Analysis Short Circuit Study Coordination Study Arc

Falsh Analysis

Short Circuit Study Coordination Study Arc

Falsh Analysis

Short Circuit Study Coordination Study

Arc Falsh Analysis Load Flow Study

Short Circuit Study Coordination Study Arc Falsh Analysis Load Flow Study

Construction Panel Board w ith bolt on circuit breakers

Sw itchboard w ith stationary circuit

breakers

Sw itchboard w ith draw out circuit

breakers

Sw itchboard w ith draw out circuit breakers

Grounding

Lighting Protection System Based on Risk analysis Based on Risk analysis Yes Yes Service entrance grounds

and generator grounds fully comply to NEC or local codes

Yes Yes Yes Yes

Data Centre grounding infrastructure in Computer room

Optional Optional Yes Yes

Computer Room Emergency Pow er Off (EPO) System

Installation

Do not install if not required by Authorities

having juristriction

Test Mode Optional Optional Yes Yes

Alarm Optional Optional Yes Yes

Abort Sw itch Optional Optional Yes Yes

Electrical Service to Mechanical Equipment

Single path of Electrical pow er to A/C equipment

Multiple paths of electrical pow er to AC equipment. Connected

in checkerboard fashion for cooling

redundancy

Multiple pathsof electrical pow er to AC equipment.

Connected in checkerboard fashion for