Great Lakes Data Room Case Study

Great Lakes Data Room

Case Study

Problem:

During a warm summer period in 2008, Great Lakes experienced a network outage due to a switch failure in the network enclosure. After an equipment audit and a site survey, it was

determined that the switch had overheated and the room itself had exceeded its designed HVAC capacity.

After discovering the causes related to the failure, it was determined that the entire infrastructure and cooling methodology should be redesigned.

Datacenter concerns typically trend in a cyclical fashion.

Driven by increased demand from software requirements, increased computer capacity is needed which creates a demand for increased power capacity. As these two demands are met the increased load in the datacenter adds additional heat which, in turn creates a need to add cooling capacity, or make existing cooling technologies more efficient.

The Room:

• 250 sq. ft. with single Mitsubishi Mr. Slim cooling unit rated at 34,600 BTU or 10.1 kW

• 2 enclosures with plexi- glass front doors and solid rear doors, vented top for thermal exhaust

Using two LAN furniture systems to house the servers and workstations in addition to the switch racks, the total BTU of equipment load was calculated at 39,600 BTU/12kW. This capacity, in addition to lack of enclosure cooling design (plexi-glass front doors) was creating thermal issues in the switch, raising the intake temperatures of other equipment in the room. In order to maintain a stable operating temperature, the set-point of the CRAC unit was 64 degrees F. IT plans for new server and networking hardware increased the equipment load to 44,300/13kW.

The new rack design uses perforated front doors to allow for proper equipment ventilation as well as cabling best

practices to maximize airflow.

The use of brush

grommet along the side rails and filler panels in the unused RMU’s to restrict

by-pass air as well as short circuits while still providing cable pass-through from front to rear of the enclosure.

After careful design

consideration, it was decided that another identical CRAC unit was required to achieve effective cooling and to add redundancy. This provided 69,000/20.2 kW BTU of cooling as well as “failover” in the event of a single unit loss. When engineering a cooling solution for this room, there were several design considerations which required consistency based on the room layout.

This is a view of the previous rack configuration used to house the IT infrastructure (i.e., patch panels, switch gear, telephony, firewall and routers) used to run the network. The remainder of the equipment (servers, workstations etc.) were installed on LAN furniture. This design was inefficient as it relied on ambient air cooling from the room, instead of using a rack configuration to direct air to the intake and segregating the exhaust air reducing short circuits resulting in excess

cooling and poor equipment reliability.

This switch failed due to overheating as a result of insufficient airflow. The primary cause of the airflow restriction was the use of acrylic front doors with limited ventilation.

Room Layout

• The room has a slab floor construction. The enclosures intake side face away from the CRAC units conditioned air flow.

• The design of each CRAC unit is such that it sits at the top of the wall and has an intake parallel to the wall, while supplying cold air from the base of the unit.

• This provided a unique challenge to the room as the unit would essentially be providing cold air in the standard “hot-aisle”.

• To handle this load while maintaining N+1 CRAC unit redundancy the set point of the CRAC units would have to be raised, by providing more BTU of capacity at a lower delta.

Given these design constraints Great Lakes chose to use it’s existing proven cooling technologies along with a custom solution designed to improve the efficiency of the CRAC unit based on its design.

New Design:

Four Enclosures (described in order from left to right) First and Second Enclosure Configurations

GL780ES-3042 • Contour Mesh front door Split Solid rear door • Brush Grommet Kit

Top cable trough • Levelers and Casters • Filler Panels First Enclosure (Telephony)

Second Enclosure (Networking)

Third and Fourth Enclosure Configurations GL780ES-2442

Adjustable Air Manager w/ active cooling Custom chimney w/ active cooling

Filler Panels Top Cable Trough Levelers and Casters Third Enclosure (IT)

Great Lakes adjustable air manager (AM- ES) directs air passively on a raised floor or through active cooling by utilizing fans on a slab floor as shown with casters and levelers.

Two 6″ air manager fans created 600 CFM @ 70W draw on 120 VAC Custom chimney directs air at 85^o F and uses a

acrylic baffle to assist exhaust direction toward the CRAC unit intake, preventing it from mixing with the conditioned air and maximizing the efficiency of the unit.

Two 6″chimney fans created 600 CFM

@ 70W draw on 120 VAC

Cross section view of the Great Lakes close coupled air cooling solution. This demonstrates how the conditioned air is used more effectively by directing the air in front of the equipment intakes and reducing the amount of room white-space being cooled.

Air Manager

With Air Manager (and optional chimney) Hot Exhaust Return

Ducted Exhaust Chimney (Eliminates Recirculation)

Hot Aisle Eliminated

Solid Rear Door 60^oF - 65^oF

at equipment intake

(60^oF - 65^oF) Conditioned Air

Aisle Space Temperature

Estimate*

65^o - 72^o F Mesh

Front Door

*Actual temperature will be based on actual data center

Temperature 105^o 99.7^o 94.4^o 89.1^o 83.8^o 78.4^o 73.1^o 67.8^o 62.5^o

An example of CFD (computational fluid dynamic modeling) analysis showing airflow intake and exhaust rates tested at 10 kW+ equipment load.

CFD Model of the room shows airflow and temperatures in the room during operation. The model is used to plan for additional capacity and to improve airflow performance.

Intake & Exhaust Temperatures Comparison

w/Air Manager and

Chimney (fans on) Temperature Decrease

Enclosure Number 1 2 3 2 3 2 3

Eq Intake @ Bottom 68 68 70 66 68 -2 -2

Eq Intake @ Top 76

81 83 69 69 -12 -14

Equipment Exhaust Top Rear 74 85 85 86 76 -9 -10

Δt F (Exhaust-Intake Avg.) +1.75 +10.5 +9.5 +8.5 +8.5

Cold aisle temperature 72^o (Recommended Range of 65-80 ASHRAE TC 9.9 specification) Measured at 4′ above the floor 2′ from the front enclosure door.

AVG#1

#2 #3

Data was acquired using an anemometer to obtain airflow speeds as well as air temperature.

Additionally Upsite Technologies temperature strips were installed at two points (air-manager, and top RMU) and in the upper RMU on back of the enclosures.

The Conclusion:

The new design has

exceeded the performance expectations projected by the design team and has achieved a higher degree of efficiency, capacity and redundancy. The active air manager in use with the brush grommet kit and filler panels has created a consistent intake temperature inside the IT and Data enclosures from bottom to top as well as a lower delta of temperature rise from equipment intake to exhaust. This allowed us to raise the set point of the CRAC units to 70 degrees F while still keeping server intake temperatures low. With this new rack configuration we were able to see an 11 to 13 degree decrease in intake temperatures.

ETSI Associate Member

Corporate Headquarters P.O. Box 551 • Edinboro, PA 16412 1-866-TRY-GLCC • (1-866-879-4522) Phone: 814.734.7303• Fax: 814.734.3907

Western Distribution

4750 Joule Street • Reno, NV 89502 Phone: 775.829.9913 • Fax: 775.829.9926

®

REGISTERED ISO 9001 : 2008 ISO 9000 : 2008