3. Data Center Overview
3.1. Standards for Data Centers
In earlier times, a room in which one or more servers, a telephone system and active network components were located was often called a computer room.However, because of the development of cloudcomputing,virtualization technologies, and an increased tendency to use outsourcing, “computer room” requirements became more complex. The one room was replaced by a room structure, whose individual rooms are assigned defined functions.
The data center is now divided up into an entrance area, the actual computer room and a work area for administrators, right through to separate rooms for UPS batteries, emergency power generators and cooling. In addition, attention must be given to matters like access control, video monitoring and alarm systems. An emphasis on active and passive components requires an improved infrastructure that includes such equipment as a cooling system and power supply, which in turn affect installation and construction as well as the data cabling structure. The point-to-point connections that were used previously are being repla- ced by a structured cabling system, which allows for rapid restoration in case of a fault, effortless system expansion and easy admi- nistration.
The data center layout, hierarchical structure and individual zones and their functions are described in detail in sections 3.2 and 3.3 below.
3.1.1. Overview of Relevant Standards
Increasing demands on datacenters have led standardization bodies to take this development into account and to take a close look as this topic. Cabling standards issued from international and national committees describe the structure and characteristics of a cabling system almost identically, but differ in terms of content. There are three big organizations in the world which are concerned with the standardization of data centers.
The ISO/IEC (InternationalOrganizationforStandardization/InternationalElectrotechnicalCommission) develops international standards, the CENELC(Comité EuropéendeNormalisationÉlectrotechnique)European standards andthe ANSI(AmericanNationalStandardsInstitute) American standards.There are other organizations for the area of data centers, though these play a subordinate role to the ones above.
The standardization bodies also remain in contact with one another in an attempt to avoid differing interpretations on the same topic, but this is not always successful. To achieve harmonization in Europe, CENELEC and ISO exchange information with national committees (SEV/SNV, VDE/DIN, ÖVE, etc.). Data center cabling is covered by the following standards:
• ISO/IEC 24764 • EN 50173-5 • TIA-942
All three standards focus on cabling, but describe the requirements for a data center in different ways. New standards were developed for this area so as not to change the general (generic) cabling structure all too greatly. The central focus of these standards is to cover the structure and performance of data center cabling. The goal was to move away from the general point-to-point connections normally used in data centers and construct a structure that is flexible, scalable, clear and permits changes, troubleshooting and expansion.
Where do the differences lie between the different standards for a data center?
A “cabling planner” designing a generic building cabling system must consider space requirements, the connection for potential equalization and other factors, but in a data center the planner must also consider things like low space availability, air conditioning, high power consumption, redundancy and failure safety including access restriction.
The type of the data center and the future prospect for transmission protocols and data rates have an effect on the development of standards. These requirements and the interfaces to other standards are examined differently, depending upon the standard. The most important points in the different standards are classified in the following table.
Criteria ISO/IEC 24764 EN 50173-5 TIA-942
Structure
Cabling performance
Redundancy
Grounding/potential equalization IEC 60364-1 EN 50310 2
Tier classification
Cable routing IEC 14763-2 1 EN 50174-2 /A1 .2
Ceilings and double floors IEC 14763-2 1 EN 50174-2 /A1 6
Floor load 2
Space requirements (ceiling height, door width) IEC 14763-2 1 EN 50174-2 /A1 3
Power supply/UPS
Fire protection/safety EN 50174-2 /A1 4 4
Cooling
Lighting
Administration/labeling IEC 14763-1 5 EN 50174-2 /A1 5 1
Temperature/humidity
1
not data center-specific, 2 refers to TIA-607, 3 only door widths and ceiling height, 4 refers to local standards,
5
refers to complexity level, 6 refers to TIA-569
On first glance, we see that TIA-942 devotes much more attention to the subject of data centers than EN 50173-5 or ISO/IEC 24764. Many important terms and their descriptions can be found in additional documents in ISO/IEC and EN, and are kept at a very general level. Some IEC documents were created a few years ago and are therefore not at the current state of technology. There are also differences in terminology in the central areas of this cabling standard.
ISO terminology is used in this section since that is the international worldwide organization. ANSI terms are used for items for which the ISO has no corresponding terminology.
3.1.2. ISO/IEC 24764
The configuration and structure of an installation per ISO/IEC 24764 must be carried out in accordance with this standard and/or ISO 11801. Testing methods for copper cabling must follow IEC 61935-1 and for optical fiber cabling the IEC14763-3 standard must be followed.In addition, a quality plan and installation guidelines in accor- dance with IEC 14763-2 must be observed for any compliant installation.
Since IEC 14763-2 is an older document, ISO/IEC 18010 must be consulted for cable routing issues. This stan- dard not only contains descriptions for cable routing, but also rudimentary instructions concerning failure safety. Nevertheless, neither IEC 14763-2 nor ISO/IEC 18010 is explicitly designed for data center requirements. The cabling system is arranged in a tree structure that starts out from a central distributor.
Point-to-point connections must be avoided wherever possible. Exceptions are allowed as long as active devices are located very closely to one another or cannot communicate over the structured cabling system.Not included in this structure are cabling in a local distributor per ISO 11801 and cabling at the interface to external networks.
The data center layout, its individual zones and their functions are described in detail in section 3.2 below.
Evolution of the basic computer room into a data center involves increased layout requirements. As mentioned at the start, it has become necessary to separate the different areas of the data center into rooms. It must be clearly indicated that the network structure in the data center is separate from building cabling. In addition, the connection to the internal network must be established over a separate distributor that is spatially detached from the data center. The interface to the external network (ENI) is created either within or outside of the data center. All other functional elements must exist in the data center permanently and be always accessible.
Cabling components, patch cords and connection cables as well as distributors for the building’s own network are not components of the standard. As the table below shows, ISO 11801 has terminology that is different from that of ISO/IEC 24764.
Different distributor functions may be combined, depending upon the size of the data center. However, at least one main distributor must exist.
Redundancy is not explicitly prescribed, but is referred to underfailuresafety.Thestandard gives information on options for redundant connections, cable paths and distributors. Like the consolidation point, the local distributionpoint(LDP)islocated in either the ceiling or in the raised floor.Patching is not per- mitted, since the connection must be wired through.
ISO/IEC 24764 refers to ISO 11801 regarding performance. The properties described in detail in ISO 11801 apply for all transmission links or channels and components. The difference is that for the data center minimum require- ments for main and area distributor cabling apply.
A cabling class that satisfies the applications listed in ISO 11801 is permitted for the network access connection.
ISO/IEC 24764 ISO 11801
External network interface (ENI) Campus distributor (CD)
Network access cable Primary cable
Main distributor (MD) Building distributor (BD)
Main distributor cable Secondary cable
Zone distributor (ZD) Floor distributor (FD) Area distributor cable Tertiary/horizontal cable Local distribution point (LDP) Consolidation point (CP) Local distribution point cable Consolidation point cable Equipment outlet (EO) Telecommunications outlet (TO)
ENI: External Network Interface MD: Main Distributor ZD: Zone Distributor LDP: Local Distribution Point EO: Equipment Outlet EQP: Equipment
Cabling Infrastructure:
• Copper cables: Category 6A, Category 7, Category 7A
• Copper transmission link or channel: Class EA, Class F,Class FA • Multi-mode optical fiber cables: OM3, OM4
• Single-mode optical fiber cables: OS1, OS2 Connectors for Copper Cabling:
• IEC 60603-7-51 or IEC 60603-7-41 for Category 6A for shielded and unshielded respectively • IEC 60603-7-7/71 for Category 7and 7A
• IEC 61076-3-104 for Category 7and 7A Connectors for Optical Fibers:
• LC duplex per IEC 61754-20 for GA and ENS
• MPO/MTP® multi-fiber connectors per IEC 61754-7 for 12 or 24 fibers
It is important that the appropriate polarity be observed in multi-fiber connections so the transmitter and receiver can be correctly connected. ISO/IEC 24764 references ISO 14763-2 with regard to this. The polarity of multi-fiber connections is not described, due to the age of the document.
3.1.3. EN 50173-5
This document is essentially the same as ISO/IEC 24764, but has a few minor differences. The last changes were established through an amendment. Therefore EN 50173-5 as well as EN 50173-5/A1 are necessary for the complete overview. This also applies for all other standards in the EN 50173 series. EN 50173-5 also uses the hierarchical structure per EN 50173-1 & -2.
A standard-compliant installation is achieved through both application of the EN 50173 series and application of the EN 50174 series and its amendments. In particular, an entire section about data centers has been added to EN 50174-2/A1. The following topics are covered in this section:
• Design proposals • Cable routing
• Separation of power and data cables • Raised floors and ceilings
• Use of pre-assembled cables • Entrance room
• Room requirements
As opposed to ISO 11801, this standard uses optical fiber classes other OF-300, OF-500 und OF-2000; possible classes are OF-100 to OF-10000. Cabling performance is determined by EN 50173-1 and EN 50173-2. However, minimum requirements for data center cabling were also established in the standard. A cabling class that satisfies the applications listed in ISO 11801 is also permitted for the network access connection in EN 50173-5.
Cabling Infrastructure:
• Copper cable: Category 6A, Category 7, Category 7A • Copper transmission path: Class EA, Class F, Class FA • Multi-mode optical fiber cable: OM3, OM4
• Single-mode optical fiber cable: OS1, OS2 Plug Connectors for Copper Cabling:
• IEC 60603-7-51 or IEC 60603-7-41 for Category 6A for shielded and unshielded respectively • IEC 60603-7-7/71 for category 7 and 7A
• IEC 61076-3-104 for category 7 and 7A Plug Connectors for Optical Fibers:
• LC duplex per IEC 61754-20 for GA and ENS
• MPO/MTP® multi-fiber connectors per IEC 61754-7 for 12 or 24 fibers
Though different options are listed for the use of multi-fiber connections, polarity is not covered in any of these standards. Detailed information on this appears in sections 3.10.1 and 3.10.2.
3.1.4. TIA-942
Without a doubt, the American standard is currently the most comprehensive body of standards for data centers. Not only is cabling described, but also the entire data center environment including the geographical location, as well as a discussion of seismic activities. Among other things, the data center is divided up into so-called “tiers”, or different classes of availability (see section 2.2.1).The data center must satisfy specific requirements for each class in order to guarantee availability.
What differences from ISO/IEC exist, apart from cabling performance?
• The interface to the external network may not be in the computer room with TIA-942.
• The maximum length of horizontal cabling may not exceed 300 meters, even with glass fibers.
• No minimum requirements are provided for cabling.
• The limits for Category 6A(≠6A)are less strict than those of ISO/IEC24764orEN50173-5.
The smallest common denominator of the data center standards listed is cabling. TIA-942 references the EIA/TIA- 568 series in the area of performance. TIA-942 itself is divided into three sections; TIA-942, TIA-942-1 and TIA- 942-2. All data center details are covered in the main standard. The use of coaxial cables is covered in TIA-942-1, and the latest version of the Cat6A (copper) performance class, laser-optimized fibers (OM3 and higher), changed temperature and ambient humidity as well as other changes in content are introduced in TIA-942-2. Cabling is defined as follows in TIA.
Cabling Infrastructure:
• Copper cable: Category 3 – 6A
• Coppertransmissionpath:Category3–Category6A(Category6Aisrecommended,TIAdoesn’tuseclasses)
• Multi-mode optical fiber cable: OM1, OM2, OM3 (OM3 recommended)
• Single-mode optical fiber cable: 9/125 µm
• Coaxial cables: 75 ohm Connectors for Copper Cabling:
• ANSI/TIA/EIA-568-B.2 Plug Connectors for Optical Fibers:
• SC duplex (568 SC) per TIA 604-3
• MPO/MTP® multi-fiber connector per TIA 604-5 (only 12-fiber)
• Other duplex connectors like LC may be used.
A new version of TIA-942(TIA-942A)was in the planning stages at the time this document was published.Since this change includes important areas, a preview of the new version is meaningful.
• Combining TIA-942, TIA-942-1 and TIA-942-2 into one document
• Harmonization with TIA-568.C0, TIA-568.C2, TIA-568.C3
• Grounding is removed and integrated with TIA-607-B
• Administration is integrated with TIA-606-B
• Distribution cabinets and separation of data and power cables is integrated with TIA-569-C
• The limit restriction for glass fiber paths in horizontal cabling is rescinded.
• Category 3 and Category 5 are no longer allowed for horizontal cabling.
• The glass fiber for backbone cabling is OM3 and OM4. OM1 and OM2 are no longer allowed.
• LC and MPO plugs are a mandatory requirement for the data center.
• Adaptation of ISO/IEC 24764 terminology Combining Bodies of Standards
None of the bodies of standards is complete. A body will either omit an examination of the overall data center, or its permitted performance class will not allow for scaling to higher data rates. It is therefore recommended that all relevant standards be consulted when planning a data center. The best standard should be used, depending upon specific requirements. One example of this would be to use the cabling performance capability from the standard which provides the highest performance requirement for cabling components. In the case of other properties, other standards should be used if they handle these properties in a better way.
Recommendation:
A specification of the standards used must therefore be listed in the customer requirement specification in order to avoid ambiguous requirements!