3. Data Center Overview
3.10. Implementations and Analysis
3.10.6. EMC Behavior in Shielded and Unshielded Cabling Systems
The motivation to once again investigate the topic of the EMC behavior of shielded and unshielded cabling, which has been examined a number of times already, is the introduction of the new application 10GBase-T.
The use of ever higher order of modulation may well reduce the necessary bandwidth, but it makes the new proto- cols more and more susceptible to external interference. Here it is worth noting, that the separation between symbols with 10GBase-T is approximately 100x smaller than with 1000Base-T (see Figure below).
Comparison of the signal strength of various Ethernet protocols at the receiver
The figure shows the relative levels of signal strength at the receiver for various protocols after the signal has been transmitted through 100 m of Class EA cabling. As a general principle, the higher the signal’s frequency or bandwidth, the larger the attenuation through the cabling is. For reasons concerning EMC, the level of the output signal is not increased above +/- 1V. Of this 2V range, depending on the frequency and attenuation in the cabling, an ever smaller part reaches the receiver. In addition, the voltage differences from one symbol to the next get increasingly smaller as a result of higher modulations. While in the past the separation between symbols at the receiver has decreased by roughly a factor of three for each step from 10M to 100M to 1G, in the final step in development from 1G to 10G it has decreased by a factor of 100.
If for visualization some noise is added, it becomes obvious that at faster data-transmission rates, the sensitivity to interference increases tremendously. As a result, the sensitivity to interference in the EMC area gets increasingly higher as the data-transmission rate increases.
During 2008, a group of cable suppliers, including R&M, came together to compare the EMC behavior of various cabling types in a neutral environment. To ensure the independence and objectivity of the results, a 3rd party test laboratory was commissioned with the investigation, specifically the “Gesellschaft für Hochfrequenz Messtechnik GHMT” (“Organization for High Frequency Testing”) located in the German town of Bexbach. The goal of the study was to answer the following questions concerning the selection of the cabling system:
• Which parameters must be estimated in order to draw meaningful conclusions about the EMC behavior of shielded and unshielded cabling systems?
• Which special measures are necessary when using shielded or unshielded cabling in order to ensure operation that conforms to legal requirements and standards?
• How does the EMC behavior of shielded and unshielded cabling compare during the operation of 1G and 10GBase-T?
The basic idea was not to pit shielded and unshielded systems against each other, but rather to clearly indicate the basic conditions that must be adhered to in order to ensure problem-free operation of 10GBase-T that is in conformance with legal requirements. By doing so, assistance can be offered to planners and end customers so they can avoid confusion and uncertainty when selecting a cabling system.
The results presented are exclusively those from the independent EMC study carried out by GHMT; the inter- pretation of the results and conclusions drawn are, however, based on analyses by R&M.
Equipment under test and test set-up
The examination of six cabling systems was planned: • 1x unshielded Cat 6
• 2x unshielded Cat 6A
• 3x shielded Cat 6A with shielded twisted pairs (foil) and different degrees of coverage of the braided shield (U-FTP without braided shield, S-STP “light” with moderate braided shield, S-STP with good braided shield).
The Figure below shows the results from the antecedent measurements according to ISO/IEC 11801 (2008-04). The values given are the margin vs. the Class EA limits, with the exception of coupling attenuation, which is given as an absolute value.
Cable parameters in accordance with ISO/IEC 11801 (2008-04)
Surprisingly, the transmission parameters (IL, NEXT, PS NEXT, TCL and RL) of the Cat 6A UTP systems are at a similar level as those from the shielded systems. Except with the older Cat 6 system, the values are more or less comparable to each other. Noticeable is that the PS ANEXT requirements are barely reached, if at all, by the unshielded systems. As soon as shielding is present, the ANEXT values are improved 30 to 40 dB so much that they no longer present a problem.
With the “EMC” parameter “coupling attenuation” the difference between the systems becomes very clear. The Cat 6 system deviates widely from the necessary values, while the newer UTP systems meet the requirements. Here too, there is a clear difference between the shielded and unshielded cabling systems. In international stan- dardization, TCL is used as the EMC parameter in unshielded systems instead of the coupling attenuation that is used with shielded systems. The comparison of values for TCL and coupling attenuation of Systems 0 – 2, however, puts this practice into question. That is because the relatively small difference in TCL between System 0 and 1 or 2 makes a huge difference in the coupling attenuation (Pass for TCL, Fail for coupling attenuation).
Basedonthesepoorresults,noEMCmeasurementswerecarriedoutwithSystem0fortheremainderofthestudy.
As is common with all EMC investigations, cabling systems cannot be examined for EMC aspects on their own, but only in connection with the active components as a total system. The following active components were used:
• Switch: Extreme Networks; Summit X450a-24t (Slot: XGM2-2bt) • Server: IBM; X3550 (Intel 10gigabit AT server adapter)
Both servers, which created data traffic of 10 Gbit/s, were connected to a switch through the cabling. The figure shows a diagram of the test set-up.
Functional diagram of the test set-up
In order to be able to test the EMC properties from all sides, all data-transmission components for the test were installed on a turntable (d = 2.5 m) in an anechoic EMC test chamber. An anechoic test chamber is a room that prevents electric or acoustic reflections by means of absorbing layers on the walls, and it shields the experiment from external influences. A 19” rack containing the switches and the servers was placed in the middle of the turntable, with an open 19” rack with patch panels on both sides. The panel and active components were connected with 2 m patch cords. Each end of the tested 90 m installed cables was terminated to one of the RJ45 panels. To achieve a high degree of reproducibility and to have a well-known test set-up, the cables were fixed to awoodencablerackthatwassuggestedasatestset-upbyCENELECTC46XWG3.Therackwaslocatedbehind the 19” racks.
Basically, this is a system set-up that simulates the situation in a data center.
Radiated power in accordance with EN 55022
This test is mandatory in the EU’s EMC directive and thus has the force of law. The operation of equipment and systems that do not pass this test is forbidden in the EU.
The test is designed to recognize radiated interference from the tested system that could impair the operation of receiving equipment such as broadcast radios, TVs and telecommunication equipment.
Thetestsystem’sreceivingantennaiserectedatadistanceof3mfromtheequipmentundertest,andtheradiated power is logged in both horizontal and vertical polarization. There are two limits: Class A (for office environments) with somewhat less stringent requirements, and Class B (for residental areas) with correspondingly tougher re- quirements.
Emission measurements for a typical shielded and unshielded system Limits: Red: Class B Violet: Class A
Asanexamplethefigureabovecomparestheemission measurementsofSystem2andSystem5fora10GBase-T transmission. It is obvious that the unshielded system exceeds the limits for Class B several times while the shiel- ded system provides better protection, especially in the upper frequency range, and thus meets the requirements. Here it makes no difference whether the shielded cabling was grounded on one side or both.
Because 10GBase-T uses the frequency range up to more than 400 MHz for data transmissions, it can be assumed that for unshielded systems it would not be possible to improve radiation enough even if filtering tech- niques were applied. The limits for Class A, in contrast, are met by both cabling types.
Other measurements show that for 1000Base-T, both shielded and unshielded cabling can comply with the limits for Class B.
From these results it can be concluded that, at least in the EU, unshielded cabling systems in residential environ- ments should not be operated with 10GBase-T. The responsibility for EMC compliance lies with the system operator. In an office environment, and that also means in a data center, it is also permissible to operate an un- shielded cabling system with 10GBase-T.
Immunity from external interferences
The EMC immunity tests are based on the EN 61000-4-X series of standards and were carried out following the E1/E2/E3 conditions for different electromagnetic environments in accordance with the MICE table of EN 50173-1 (see next figure).
In these tests the continuous transmission of data between the servers was monitored with a protocol analyzer, and the transmission system was then subjected to defined stress conditions. It was reported from what stress level onward the transmission rate was impaired.
Stress conditions for Classes E1 to E3 in accordance with the MICE table from EN 50173-1
Immunity against high-frequency radio waves
The test outlined in EN 61000-4-3 serves to examine the immunity of the equipment under test (EUT) against radiated electromagnetic fields in the frequency range from 80 MHz to 2.0 GHz. This test simulates the influence of interferences such as radio broadcast and TV transmitters, mobile phones, wireless networks and others.
The transmitting antenna was set up at a distance of 3 m from the EUT, which was then irradiated from all four sides. The field strength, measured at the location of the equipment under test, was selected in accordance with the MICE table (Figure, “Radiated high frequency”).
Results in accordance with EN61000-4-3 (left) / Operational test of a mobile phone (right)
All shielded cabling variants are well suited for 10GBase-T operation in offices and light industrial areas. For harsher industrial areas, an additional overall braided shield (S-FTP design) is necessary. Single- or double-sided grounding has no influence on the immunity against external radiation. A good unshielded cabling is suitable for 1000Base-T in offices and areas used for light industry. For the use of 10GBase-T with unshielded cabling, how- ever, additional protective measures are necessary; e.g. metallic raceway systems and/or additional separation away from the sources of interference.
An additionally performed operational test confirmed the high sensitivity of the unshielded systems to wireless communications equipment in the 2 m to 70 cm band when operating with 10GBase-T. If a personal radio or mobile phone was operated at a distance of 3 m (see Figure above), with unshielded cabling there was an interruption in the data transmission, while all shielded systems experienced no impairment of the transmission.
Immunity against interference due to power cables
A test in accordance with EN 61000-4-4 was carried out in order to check the immunity of the equipment under test (EUT) against repeated, fast transients as produced by the switching of inductive loads (motors), by relay contact chatter and by electronic ballasts for fluorescent lamps. In order to achieve reproducible coupling between the power cable and the EUT, a standardized capacitive coupling clamp was used in this test. The disturbances overed voltage peaks of 260 - 4000 V with a wave shape of 5/50 ns and a separation of 0.2 ms. The voltage levels in accordance with the MICE table were applied (see Figure, “Fast transient (burst)”).
Results in accordance with EN 61000-4-4 (left) / Results operational test “mesh cable tray” (right)
The figure left shows the results of this test. All shielded systems allow the use of 10GBase-T in all environmental conditions (E1, E2, E3). At the same time, higher quality shielding provided better immunity against fast transients. High-quality unshielded cabling allows the use of 1000Base-T in an office environment. For an industrial environ- ment and for 10GBase-T, shielded cabling is necessary. Unshielded cabling, if it is to support 10GBase-T, needs additional protective measures such as the careful separation of the data cables from the power cables.
A double-sided ground improves the immunity of shielded cabling against external fast transients above the minimum requirements of the standard. If the shield is not continuous, its effectiveness with 10GBase-T is negated, and the protection is then the same as with unshielded cabling. At lower frequencies (such as used in 1000Base-T), a certain protective effect of the non-continuous shield is apparent, especially with double-sided grounding.
An operational test with fluorescent lamps that were located 0.5 m from the data cable showed that the test conditions in the test based on the standard were entirely realistic. The interference that arises when a fluorescent lamp is switched on influenced the 10GBase-T data transmission in the same way as during the test according to the standard. Not only the lamp itself, but also the power cable to it, created interference. It must therefore be ensured that there is sufficient separation of both from the data cabling.
In order to compare the standard test using the coupling clamp to an actual installation situation, the “mesh cable tray” experiment was also carried out. The data cables and a power cable were laid in a mesh cable tray with a total length of 30 m and with a constant separation from 0 to 50 cm. The interference signal according to the stan- dard was then applied to the power cable. The figure shows the results of these measurements. A comparison of the results with those of the test according to the standard from the figure shows that the standardized test simulates a separation of the cables of approximately 1 - 2 cm. In order to guarantee the operation of 10GBase-T, a separation between the data and power cables of at least 30 cm must be maintained with an unshielded system. The shielded cabling met the requirements even without any separation between the cables.
According to EN 50174-2, a separation of only 2 cm is defined for the unshielded system in this configuration, which however is insufficient for 10GBase-T. In other words, with unshielded cables, when using 10GBase-T, a far greater separation must be maintained than is defined in the standard.
A further test in accordance with EN 61000-4-6 was carried out to check the system’s immunity against conducted RF interference in the range from 150 kHz to 80 MHz on power cables located nearby. Power cables can act as antennas for high-frequency interference coming from external sources (such as shortwave and VHF transmitters) or also intentionally subjected to a powerline signal. In this test, too, the previously mentioned coupling clamp was used. The stress levels were chosen in accordance with the MICE table (see Figure, “Conducted high frequency”).
The results corresponded to the well-known pattern that the shielded cabling meets all requirements for 10GBase- T. Unshielded cabling meets the requirements for offices and areas with light industry for 1000Base-T, however for 10GBase-T transmissions additional protective measures such as increased separation of the data cabling from power cabling is necessary.
Immunity from magnetic fields arising from power cables
This test in accordance with EN61000-4-8 checks the ability of a system to function in the presence of strong magnetic fields at 50 Hz. These magnetic fields can be generated by power lines (cable or busbars) or power- distribution equipment (transformers, distribution panels). The stress levels were selected in accordance with the MICE table (see Figure, “Magnetic fields”). All cabling fulfills the highest environmental class (E3) with both 1000Base-T and 10GBase-T. No difference was determined between the susceptibility of shielded or unshielded cabling. No heightened susceptibility of the shielded cabling based on ground loops can be observed.
Immunity against electrostatic discharge
This test in accordance with EN 61000-4-2 checks the immunity of a system against electrostatic discharge. This phenomenon, which we all know from everyday life when a spark jumps from our finger to a conductive surface, can be generated in a reproducible manner with a test rig with metallic test fingers. Environmental and climactic conditions – such as low humidity, insulated plastic floors and clothing made of synthetic fibers – can promote electrostatic charging.
The test points were selected to simulate normal contact of the cabling during operation and maintenance. At each test point, 10 sparks per polarity were generated with a separation of more than 1 second. The test levels used were set in accordance with the MICE table (see Figure, “Electrostatic discharge – contact / air”).
The shielded cabling systems did not show sensitivity to electrostatic discharges; no faults arose. With unshielded cable, the active devices reacted in a very sensitive way to discharges as soon as these could impact on the signal conductors.
The good performance of shielded cabling can be attributed to the fact that the shield can serve as a bypass path for the flashover and in this way no energy could make its way inside the cable. For 10GBase-T operation with unshielded cabling, additional measures must ensure that no electrostatic discharges can take place. Suitable protective measures are well known in electronics manufacturing and can include grounding stations, ESD wrist straps, antistatic floors, etc.
Summary
Ithasbeenshownthattheintroductionof10GBase-Tinfacthasaconsiderableimpactontheselectionofcabling. The increased sensitivity of 10GBase-T transmissions compared to 1000Base-T was clearly evident with unshiel- ded cabling in terms of immunity against external interference.
In order to guarantee the operation of 10GBase-T, it is not sufficient to pay attention to the cabling alone, rather the environmental conditions must also be considered and the cabling components must be properly selected. Coupling attenuation can serve as a qualitative comparative parameter for the EMC behavior of cabling.
Summing up, this investigation has shown that shielded cabling for 10GBase-T can be used without any problems in every environmental class. The following applies: The better the quality of the shielding, the smaller the emis- sions and the better the immunity of the cabling against external interference.
Unshielded cabling, in contrast, is suited for use outside residential areas only and in conjunction with additional preventive measures for the use of 10GBase-T. Within the EU, this cabling shall be used only outside residential areas, in dedicated work areas (like offices, data centers, etc.).
In choosing between shielded or unshielded cabling for 10GBase-T, the influences and applications of additional protective measures and operational limitations must be taken into consideration.
Recommendations for the operation of 10GBase-T
In industrial environments (Classes E2 and E3), shielded cabling should be used. In harsher industrial environ- ments (E3), an S-FTP shield design with braided overall shield is necessary and, if possible, a double-sided grounding should be applied to the cabling.
In residential areas, unshielded cabling should not be used. In office areas and data centers with unshielded cabling, the above-mentioned additional protective measures should be applied.