Case study; A sustainable Green Data Centre, fact or fiction?

(1)

Case study;

A sustainable Green Data Centre,

fact or fiction?

Brian O’Hora, BSc MBA MPM Networks & Infrastructure Manager

(2)

200 Pearse Street

• Single UPS, no external bypass • 20 low power racks

• Total IT/IS capacity 40kW • Multiple contracts

O’Reilly building

• 4 racks • No backup mains • No fire suppression • Split AC units • Multiple tenants • IT/IS capacity 10kW

• Other on campus hosting service providers and needs

• College Strategic Plan 2009/14 action item 6.10 make College a Green campus • College high level risk register

(3)

Project trigger

Procurement strategy

• Initially to include IT/IS

• Final excluding IT/IS

Project risk management

Contracting approach

• ICE standard contract

• D&B

Post project operational risk management approach

• D&B and maintenance for 4 years (alliance)

Procurement strategy

and risk management

(4)

Challenges identified

• Single shared sub station

• Over capacity in 200PS DC

• External routes

• Indoor physical spaces

• Outdoor physical spaces

• Planning permission?

• Occupied working building

• Disruptive works window

• Site establishment

• Internal/external site access

• H & S

Pre project feasibility,

proposal and initiation

(5)

Feb 2009 - May 2010 - Pre project feasibility, proposal and initiation

June 2010 - Project planning

Aug 31st_{to March 31}st₂₀₁₁_{– Procurement and selection phase, PQQ & ITT}

April 2011 - grant of planning permission pending appeal period & final grant May 13th_{2011 - final confirmation of funding availability and allocation}

July /Aug 2011 – Detailed Design phase - prime contactor instructed to proceed with design including 4th_{floor roof space}

Aug/Dec 2011 - Build phase, construction

Jan/March 2012 – Integration, testing , commissioning, training & familiarisation

March 16th₂₀₁₁_{– Substantial completion & handover}

(6)

• Green showcase – minimum EU CoC compliance across load range • Resilience and availability - Tier Level 3 desirable

•Scalability and modularity – 25kW to 250kW in 25kW increments and two phases •Phase 1 - 150kW IT load, all single upstream sized for phase 2 - 250kW IT load • Phase 1 - min x16 - 20kW capable racks, plus passive wiring rack and adjacent active network equipment rack

• Energy monitoring; live PUE and detailed monitoring to PDU outlet level • Supporting tools for IaaS; shared service & multi tenancy aligned, capacity management, fault management, performance management, accounting and security management

(7)

• Procurement restricted EU competition, PQQ and ITT and phases

• Q3 2010 PQQ, 17 responses to PQQ, 9 successful

• Q4 – 2010 ITT release and site visits scheduled Dec 2010 Jan 2011

• Q1 Jan 2011 Planning application lodged for generator enclosure

on roof and ITT closes, clarifications & selection starts Feb, preferred

bidder appointed end Q1

• Q2 2011 Contract negotiations, staged payments agreed, DCC

planning award favourable, detailed design commences, focus on

structural issue

• Q3 2011 Structural issue resolved, design completed, focus on

long lead time items and disruptive works, rooms cleared & services

relocated, site established

Procurement and

(8)

Roof challenges

• Enabling works

• Surface condition

• Leak avoidance

• Access

• Heavy equipments - road closure notice PS opposite fire station

(9)

• Capacity issue in main DC

- Additional 3 phase long autonomy UPS-B on general services

- Installation of UPS-B also complicated due to capacity issue

• Impacts related to single shared sub station with main DC

-

Full day substation down time required to connect new facility

-Failure of emergency backup system in main DC - reschedule

- Impact on cost and schedule

- Additional local and full functional testing requirements

- Multiple failures of contractors generator on second connection

attempt

(10)

•Formal witness testing and commissioning of all major systems

• Snagging

• Substantial completion agreement • Security system integration & testing CCTV, door access control, intruder alarm • Fire Alarm and VESDA system

• Familiarization training including (H & S fire detection gas suppression) • End to end Management System & network integration

Integration and

(11)

(12)

(13)

(14)

(15)

• Modular room - installed within the existing building to enclose the data centre, constructed of a powder coated metal skin internal & external and rockwool infill

• Raised floor, leak detection, no leak prevention

• x20 20kW IT racks plus one passive wiring plus adjacent network equipment rack, 25% uplift of rack space

• Overall power room power budget 150kW

• Dual 3 phase ePDUs per rack, all ePDU outputs monitored (≈ 1,000) for power and kW hours

• Link from room passive wiring rack to main communications room 120 pair SM and 120 pair OM3 MM and 12 pair SM and MM and 24 CAT6A to each rack

• Dual high efficiency UPS, 20min autonomy per UPS at full phase 2 load, 10 year batteries

• 700kVA generator, 32 hr tank at full phase 2 load, dump tank at ground level

• Rear door heat exchangers, 18oC water temp in, 22-24oC return water temp,

• Standalone Dry Air Cooler, partial free cooling below 20-22oC, 100% free cooling at 15oC and below

• Operation of rear door heat exchangers and DAC

(16)

Operation of rear door

heat exchangers

(17)

• Independently Verified load test results

• PUE below EU CoC over entire range and < 1.2 @ > 33% load, below 15oC OAT

• Broader definition of Green or sustainability includes carbon foot print of input power,

facility PUE and efficiency of network/compute/storage

• Highly integrated UCS network/compute and storage selected as IaaS platform to

populate facility, further encouraging virtualization and discouraging over provisioning

• Design driven by energy efficiency – e.g. RDHC & no leak prevention

• Design options selection driven by energy efficiency - colour or racks, lighting

• Strategic Plan action item 6.10 make College a Green campus aligned, a Green showcase

and shared services aligned, less control over input power sustainability

• A community shared services low latency HA facility offers opportunity to extend energy

sustainability upstream to include low carbon foot print electrical supply

A sustainable

Green Data Centre?

PUE measured (OAT 12-13.7 degrees)

IT load 25kW 50kW 75kW 100kW 125kW 150kW

PUE 1.36 1.17 1.15 1.16 1.13 1.18

(18)

Thank you

Brian O’Hora, BSc MBA MPM Networks & Infrastructure Manager