Previous Issue: New
SAER-6114
22 May 2007
Process Automation (I-Field/SCADA/Remote Sites)
Communication Network Architecture
Table of Contents
1 Executive Summary... 4
Part I: Wireline Network Architecture 1 Introduction... 7
2 Background... 7
3 Existing SCADA Network Design... 10
3.1 SCADA Over Dedicated Transmission... 10
3.2 SCADA Over Dedicated SDH Network……... 12
3.3 SCADA Over IT WAN Packet Network... 15
3.4 SCADA Over Radio Network………...…... 17
4 Industry Trends... 17
4.1 Purdue Enterprise Reference Architecture (PERA)... 17
4.2 Instrumentation, Systems and Automation Society... 18
4.3 Industry Trends Summary... 19
5 Future I-Field/SCADA/Remotes Communication Network Architecture... 21
5.1 Existing Performance & Reliability Issues... 21
5.2 Analysis & Results... 22
5.3 Remote Process Automation Application Network Architecture... 24
5.3.1 Future Process Automation (SCADA, I-Field, Remote sites) Network Design Architecture………... 25
5.3.2 Next Steps... 32
Part II: Process Automation Wireless Access 1 Background... 33
2 Introduction... 33
3 Wireless for Process Networks... 33
4 Industrial Wireless vs. Non-Industrial Wireless... 34
5 Industrial Wireless Classifications... 35
5.1 Wireless Personal Area Networks (WPAN)... 36
5.2 Wireless Local Area Networks (WLAN)... 38
Table of Contents (Cont'd)
6 Wireless for Process Network Conclusion... 45
7 Next Steps... 45
Part III: Network Timing & Synchronization Architecture 1 Synchronization Architecture... 46
2 Next Steps... 48
APPENDICES... 49
Appendix 1: Design Performance Criteria Analysis... 50
Appendix 2: Bandwidth Calculation for Wireline Network... 53
Appendix 3: Wireless Technology Comparison... 56
Appendix 4: Wireless Technology Classifications... 57
Appendix 5: ISM Frequency Band... 58
Appendix 6: IEEE 802.11 WLAN Standards Model... 59
Appendix 7: Support Structure………..… 60
1 Executive Summary
This report summarizes the findings of a detailed assessment of existing Saudi Aramco implementations, industry trends, and recommends network design architecture for data acquisition, control, and monitoring for remote Process Automation applications; which includes I-Field and SCADA. This report also covers other non-real time process control applications such as Cathodic monitoring, Vibration monitoring, etc. The network infrastructure covered in this document, includes Wireline Networking, Wireless Networking, and Network Synchronization.
Recommendations outlined in this document will be mapped to the different Process Automation standards and procedures (e.g., SAES-Z-010, SAES-Z-001, 23-SAMS-020, draft SAES-Z-004, etc...). In the interim, this report will serve as a guide for any future SCADA, I-Field, and remote site network connectivity until all related standards are updated with these recommendations. These recommendations are based on benchmarking of international organization standards, industry trends, existing Saudi Aramco different implementations, and lessons learned. The outcome of this effort shall provide higher level of end-to-end network reliability, flexibility, and optimal design configurations.
In summary, the recommendations call for the following:
I. Wireline Network Architecture: The remote Process Automation applications
network architecture shall be based on either one of the following configurations:
1. Industrial Ethernet using dedicated fiber in a self healing ring topology for grass root projects such as Khurias, Manifa, etc.
2. For existing implementations, utilize Synchronous Digital Hierarchy (SDH) dedicated bandwidth based on self healing ring architecture. These two network toloplogies implemenation shall be based on the following general guidelines:
• A standalone fiber optics cable shall be utilized to connect remote sites (i.e., well sites) to a centralized Process Automation Network (PAN). Alternatively, Radio links, or Broadband wireless network solution could be used to connect dispersed remote sites where fiber optics cable installation proves economically not feasible.
• Utilize point-to-point ring system for scattered sites and/or where a true closed loop physical ring topology is economically not justifiable. • A PAN comprising of multiple scattered (PANs), shall interface with
the Corporate Network via a centralized firewall(s). These scattered PANs shall be connected together to form a unified PAN utilizing the corporate SDH transmission infrastructure with a dedicated bandwidth of 10 Mbps. Alternatively, dark fiber can be utilized if available.
• SCADA over IP/ATM is a unique implementation for East/West Pipelines. It was recommended to PMT to move SCADA traffic from ATM and have it directly transported over a dedicated SDH transport network to fulfill the physical separation requirements as per SAES-Z-010. This would be part of the ongoing BI (BI-10-00179) that will replace the existing Microwave system to SDH.
Any future implementation similar to the East/West Pipelines such as Water Injection Plants, scattered GOSPs will be based on either one of the two design options stated above. This is mainly because remote GOSPs or WIPs operation is tightly coupled with centralized process control systems.
Detailed network analysis and recommended guidelines for the Wireline Network Architecture are included in Part I, section 5.3 of this report.
II. Wireless Network Architecture: Wireless network solution provides cost effective
and efficient connectivity for different Process Automation systems. The following are the recommended wireless connectivity solution mapped to their associated applications.
1. Zigbee (in conjunction with IEEE 802.15.4): wireless solution considered as a
potential emerging short distance wireless technology for instrumentations networking, subsystems networking of network latency tolerant applications. 2. Wireless LAN (based on standards IEEE 802.11a, b, and g technologies):
considered a potential mature and proven wireless technologies for mid range (100’s meters up to few kilometers) applications.
3. Wireless Wide Areas Network (e.g., GSM phone based service such as Short
Messaging Service, General Packet Radio Systems): can be utilized to provide efficient and cost effective process monitoring solution for semi-real time non-critical applications.
4. Broadband Fixed Wireless connectivity (WiMAX or Propriety solution)
• Provides a cost effective and flexible point-to-point connection
• WiMax requires frequency approval from local authority Communications and Information Technology Commission (CITC) and should be requested through a 3rd party local service provider
Hence, the strategy is to use propriety Broadband Fixed Wireless Solution to avoid licensing cost and dependence on 3P
rd
P
party service providers.
Detailed network analysis, benchmarking, and recommended next steps for Wireless Network Architecture for Process Automation are included in Part II of this report.
III. Network Synchronizations: Network synchronizations will be based on Global
Positioning System (GPS) coupled with Network Timing Protocol (NTP) server within a plant to provide precise time stamping for systems and network infrastructure. To minimize cost, existing plant’s GPS/NTP servers will be extended to its remote sites. Detailed analysis and recommended next steps for the Network Synchronizations are included in Part III of this report.
P&CSD will utilize and benchmark ongoing technology trials in both Wireless and Wireline networks to develop Saudi Aramco engineering deployment guidelines such as material specifications, standards, and best practices.
As part of the overall recommendation, future Capital Projects (i.e., Budget Items) for Process Automation networks will mainly be referenced under the Z standards index (Process Control). Telecommunication T-index will be sub referenced for overlap areas.
Part I: Wireline Network Architecture 1 Introduction
This part of the report provides recommended network architecture for SCADA systems, I-Field, and other remote site Process Automation applications. The recommended architecture is based on a comprehensive network evaluation for existing Process Automation applications that include SCADA systems, ongoing I-Field initiative, and other remote sites Process Automation requirements.
This evaluation includes network topologies, protocols, reliability and recommended network configuration options. Selected Saudi Aramco SCADA and I-Field implementations have been used as a baseline in this assessment. Further, industry trends and international organization bodies (e.g., ISA, API, etc.) have been used to provide an optimal recommendation that would meet existing and future Process Automation applications requirements.
2 Background
In April 2006, an assessment to existing Saudi Aramco Standards and Material specifications revealed the need for comprehensive detailed requirements for Process Automation Network (PAN) to address SCADA and other remote site Process Automation applications’ communication network. In addition, different network implementations types in Saudi Aramco have emerged over time based on different site topologies and the available technologies at that time. These can be described as follows:
1. Dedicated Transmission network was used for some of these implementations; while others were based on transmitting Process Automation traffic utilizing shared transmission infrastructure.
2. These different network implementation either using self healing rings topology, or point to point redundant links, or single threaded links.
3. Propriety protocols were used as a defacto of the applications. However, there are some implementations that are based on open standard communication protocols. These different configurations have caused a non streamlined network design implementations and created a gap in Process Automation Standards. Here is a summary of what the existing standards, as of April 2006, are calling for a long with high level analysis of these standard requirements.
● SAES-Z-010:
○ Scope: This standard establishes the requirements for design, installation, configuration and commissioning of Process Automation Networks (PANs). “Geographically spread Remote Terminal Units (RTU's) are not in the scope of
this standard”
○ This standard calls for:
5.2 “The network design shall provide physical and logical separation
between PAN and Corporate Network below firewall. Logical separation, at minimum, is mandatory for network connections above firewall”
○ Issue: The standards scope does not address remote areas (RTU’s) connectivity. ● SAES-J-902:
○ Scope: This Standard establishes the design criteria for the installation of power and wiring systems for electrical instrumentation.
○ Standard calls for:
12.3.2 “Data Links: Data links, including fiber optic cables, shall be specified
and installed per system manufacturers' recommendations. When redundant data links are provided, the primary cable shall follow a different route from the back up cable. Primary and backup data link cables shall preferentially enter cabinets or consoles from opposite sides. Data link cables shall not be routed in the same conduit, duct, or tray with other instrument cables.”
○ Issue: The scope does not address SCADA or other remote sites’ Process Automation application related traffic. It is focused on the instrumentation. ● SAES-Z-001:
○ Scope: This Standard prescribes the minimum mandatory requirements and guidelines governing the engineering, design and installation of Process Control Systems (PCS) in Saudi Aramco plants (i.e., Oil/Gas Plants, Refineries, and
○ This standard calls for:
14.3 “Interface to Corporate Wide Area Networks (CWAN): The control
system communication to Corporate Wide Area Network and other non-control computer systems shall be designed to ensure that no failure, no request for information, or network loading problem will impact the performance or availability of the PCS. Use of standard software and hardware protocols for interfaces, such as TCP/IP and OPC, are preferred.”
○ Issue: The standards calls for generic requirements (e.g., no failure, no loading problem will impact the performance, etc…). The standards does not address communication medium (e.g., physical separation vs. logical separation).
● 23-SAMSS-010:
○ Scope: all DCS equipment and associated software required to monitor and control a process plant.
○ This standard calls for:
11.2 “Communication at the control network level shall have redundant or
fault tolerant paths. Communications from the controller to the I/O subsystem shall have redundant paths.”
11.3 “DCS internal communication shall be designed such that no single
failure will degrade the performance of the system. This requirement applies to all communication between DCS modules, including communication between controllers to their I/O modules.”
○ Issue: The standards scope does not address SCADA or remote areas (RTU’s). Further, the standard calls for fault tolerant network but does not address the communication in detail.
● 23-SAMSS-020:
○ Scope: This specification defines the minimum mandatory requirements for Supervisory Control and Data Acquisition (SCADA) systems for oil & gas and utility applications.
○ This SAMSS calls for:
13.1.4 “The system LAN shall be fault tolerant utilizing a network
configuration that prevents a single point of failure.”
○ Issue: The standard calls for fault tolerant network but does not address the communication in detail.
3 Existing Remote Network Connectivity Design
There are three (3) different existing remote network connectivity designs in Saudi Aramco. These network design topologies have evolved over the past many years. These topologies are:
1. The earliest is SCADA network based on dedicated point-to-point digital circuits over digital transmission network and then evolved to be over Synchronous Digital Hierarchy (SDH) transmission network. OSPAS and PDD are the primary users for this network topology at this time.
2. SCADA using dedicated bandwidth on shared, transmission systems such as Synchronous Digital Hierarchy (SDH). This is typically implemented within oil and gas fields (e.g., Qatif, Haradh, etc.).
3. The third network design option is based on carrying SCADA traffic over IP then over ATM and then over SDH. This is limited to East /West Pipelines network design implementation and Shaybah’s inter GOSPs connectivity.
Detailed design overview for these different existing options is outlined in the subsequent sections.
3.1 SCADA Over Dedicated Transmission
The earliest SCADA network implementation was based on dedicated point-to-point digital circuits over Digital transmission transport network or Synchronous Digital Hierarchy (SDH) networks. This implementation is based on using
Figure 3.1: Dedicated Point-to-Point Circuits
Typically, a RTU is either connected by a digital circuit (2 wire or 4 Wire E/M) or a radio link from remote sites to the nearest transport network node Digital Access Cross Connect System that is part of the Saudi Aramco SDH network. The majority of the dedicated lines operate at low speed, i.e., 1200 to 9600 bauds. Further, many of these dedicated lines utilize twisted telephone line pairs party line features; which can have up to six (6) RTUs connected.
Remote sites supported by leased lines circuits from Saudi Telecom Company (STC) are connected to the nearest Saudi Aramco SDH point of presence to STC (i.e., Jizan Bulk Plant, SSSP, etc...). The circuit is then backhauled to the SCADA terminal server at the control room.
The implementation of this design configuration is widely used in support of Oil Supply Planning & Scheduling Department (OSPAS), Power Distribution Department (PDD), and some of the pipelines sites.
OSPAS Center 500P PS1PS2 Omn iSwi t c h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1
2
32 UTP 10/100 Mbps -Ethernet ESX32 UTP 10/100 Mbps -Ethernet ESX
Fiber Optics MUX/Modems connecting digital Point to Point Circuits
UHF Radio Fiber Optics Mux RTU 5 0 0 P PS1 PS2 O m n i S w i t c h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1 2 32 UTP 10/100 Mb ps -Ethernet ESX 32 UTP 10/100 Mb ps -Ethernet ESX 5 0 0 P PS1 PS2 O m n i S w i t c h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1 2 32 UTP 10/100 Mb ps -Ethernet ESX 32 UTP 10/100 Mb ps -Ethernet ESX Terminal Server RTU IT SDH Transport Network
Fiber Optics MUX/Modems connecting digital Point to Point Circuits
RTU
Fast Ethernet
Ethernet Switch
OSPAS alone has over 400 point-to-point circuits companywide covering key oil and electric distribution systems offshore and onshore utilizing IT SDH transport network. Some of these connections are backhauled to Dhahran Core areas providing a centralized SCADA management system. The primary user of this architecture is OSPAS, where they monitor and control oil and gas flow from different wells and GOSPs. Also, electric power flow from SCECO to Saudi Aramco substations and distribution lines within Saudi Aramco facilities are managed by PDD through this design configuration.
Others are based on a distributed SCADA architecture where the remote sites are connected to a local control room site within the same serving area. An example of this is offshore to onshore SCADA connections.
3.2 SCADA Over Dedicated SDH Network
It must be noted in this document, the term Optical Transport Network (OTN) is used interchangeably with SDH as OTN is a vendor product solution deployed in several Saudi Aramco plants.
There are two approaches for using Synchronous Digital Hierarchy (SDH) transport network in carrying SCADA traffic. The first is based on circuit emulation for a point- to-point serial connection using the inherent multiplexing hierarchy of SDH systems. This approach is a proven concept that has been used in the industry for real time and non real time applications.
Figure 3.2.1 depicts the network topology for this design configuration. Ring network topology is the normal approach. For large fields with many scattered oil wells, multiple rings are implemented to provide robust network; while serving dispersed Remote Terminal Units.
Figure 3.2.1: Dedicated Bandwidth over SDH Transmission Network
The configuration is typically deployed in a self healing SDH ring with a recovery time of close to 50 milliseconds. This is a powerful capability as long as the SCADA application’s communication time delay is compatible with distance between the RTU and the SCADA Master Sever on the two sides of the ring.
Oil & Gas Fields/Remote GOSP
Ethernet Switch
CCR
SCADA PAN Backbone Transmission Ring using SDH nodes
Operator Control Console
Fast Ethernet
ENG & OPR Console Ethernet Switch 500P PS1 PS2 Omn i S wi tc h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1
2
32 UTP 10/100 Mbps -Ethernet ESX32 UTP 10/100 Mbps -Ethernet ESX
SCADA Server WHESD/ESP/PDHM, etc.. PAN Access Transmission Ring Using SDH Nodes 500P PS1 PS2 Omn i Swi t ch 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1
2
32 UTP 10/100 Mbps -Ethernet ESX32 UTP 10/100 Mbps -Ethernet ESX
Radio Firewall Plant Network RTU Enterprise Office User Work
Station Enterprise IT Wide Area
Network Office User Enterprise PI Server (s)
GOSP: Eng &
OPR Console/ Data Historian
The second Synchronous Digital Hierarchy (SDH) transmission implementation approach is based on establishing a dedicated bandwidth on the SDH system, providing logical connection on shared SDH transport system. This eliminates the need for serial point to point connection emulation on the SDH and provides higher flexibility in allocating dedicated bandwidth based on actual bandwidth requirements. This option requires the introduction of Ethernet switches at the Remote Terminal Unit (RTU) site.
Figure 3.2.2 depicts a conceptual network design using SDH ring based on dedicated bandwidth allocating.
ESP MOV RTU=409Kbps/ 8M Buffer Ethernet Access Switch Ethernet Aggregator Switch SDH SDH Ethernet Switch Server PDHM CMS=2.4Kbps/0 CP=2.4Kbps/ 2M Buffer MPFM=38.4kbps/ 4M Buffer SWC= 19.4Kbps/Buffer = 4 Mbps Phone Line=64kbps/ 0 Buffer
Load= Number of Oil Well Work Areas Interface Capacity Max= 10Mbps to 1 GE
Single Mode Fiber Distant 2KM to >10 KM
Transmission Network SDH or OTN Ring
Effective Bandwidth Capacity = Dedicated Bandwidth Capacity Provisioned on the SDH Ring
Ranges from 0 to 100 Mbps
Oil Well Work
SDH SDH
CCR Room
SDH Access Node / Ethernet Aggregator
Access traffic, SCADA and others, is aggregated to each serving SDH transport node and then logically mapped to the SDH node hosting the SCADA servers at the control room. The term OTN (Optical Transport Network) is used interchangeably with SDH as OTN is a vendor product solution for SDH deployed in several Saudi Aramco plants. There are over 600 OTN nodes serving well heads, RTU’s and others in different Saudi Aramco facilities (e.g., Qatif, Haradh, etc.).
In this example, the peak load for each RTU is 516Kbps. This is the composite traffic resident at the oil well work area automation systems: RTU, MPFM, SWC, CMS, CP, Voice, etc. All of these services are connected to an Ethernet switch; an Access switch. Each access switch with a Giga Ethernet interface is connected to another Ethernet switch at the SDH node; an Aggregator switch. The Aggregator switch is then connected to the SDH network using Fast Ethernet connection.
A dedicated bandwidth channel is provisioned from the SDH node serving the Aggregator switch to the SDH node located at the CCR room. Therefore, the effective bandwidth allocation from the Aggregator switch to the CCR room is the dedicated bandwidth provisioned between the SDH node severing multiple oil well working areas and the SDH node at the CCR room. Careful considerations shall be taken when designing such a network from a bandwidth and redundancy perspectives. Sections 5.3 will outline engineering guidelines when design this type of network topology.
3.3 SCADA Over IT WAN Packet Network
The IT Wide Area Network (WAN) uses Asynchronous Transport Mode (ATM) cell switching technology as a backbone for Enterprise applications. There are some existing scenarios where the ATM backbone is used to support Process Automation application (i.e., SCADA) for linking remote sites to a central site. This is design was implemented for East/West Pipelines and Shaybah’s inter-GOSPs connectivity.
The SCADA traffic is interfaced via IP over Ethernet to IP over ATM WAN node in the local site (e.g., Pump station). The traffic is then routed to the Central SCADA server at the Control Room (CR) (e.g., Yanbu Crude Oil Terminal –YCOT). The process of carting the SCADA from the local site to the central site is supported by the IT ATM network based on best effort utilizing Local Area Network (LAN) ATM emulation capability. Both local SCADA and central sites are considered virtually on the same LAN using shared infrastructure.
Each pump station is managed locally by its local SCADA Operation group. However, the YCOT SCADA centers can control the sectional valves in between pump stations and provide 2P
nd
P
level control support for the local SCADA operation support groups.
Figure 3.3 depicts a conceptual network design using IP over ATM network.
Figure 3.3: SCADA over IP using ATM Transport Network
Fiber Optics with SDH Nodes Pump Station #1 500P PS1PS2 Omn i Swi t c h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1
2
32 UTP 10/100 Mbps -Ethernet ESX32 UTP 10/100 Mbps -Ethernet ESX
Ethernet Switch with ATM Uplink Operator Control Console 500P PS1 PS2 Omn i Swi t c h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1
2
32 UTP 10/100 Mbps -Ethernet ESX32 UTP 10/100 Mbps -Ethernet ESX
S D SY ST EM S SER ETHNEXTSELECT RESET TXCRXL PWR A S X - 1 0 0 0 BD BD B D BD AC AC AC AC S D SY ST EM S SER ETHNEXTSELECT RESET TXCRXL PWR A SX - 10 0 0 BD BD BD BD AC AC AC AC IT ATM Backbone Network ATM Switches Router IP Router Network IT SDH Transmission Network Local Pump Station
SCADA Network Ethernet Switch Digital Transmission Fiber Optics Ethernet Switch with ATM Uplink Pump Stations #2 thru 11
YCOT-Yanbu PI Servers
Digital Transmission
3.4 SCADA Over Radio Network
SCADA over Radio based on UHF band, 460MHZ, is used to establish point to point network connectivity between the RTU’s and the SCADA host. This technology approach was implemented in Zuluf, Marjan, and Safaniya offshore, Haradh III and several other remote sites.
Radio RTU’s are depicted into the previous figures 3.2.2 and 3.3. One of the key disadvantages of Radio is the shortage of available frequencies in the VHF/UHF (Very High Frequency / Ultra High Frequency) bands. Besides, the disadvantage of the Radio frequency is that the transmission distant is limited to a line of sight path between antennas. Radio is subject to atmospheric attenuation, distortion, and interference.
4 Industry Trends
There are two major organizations working on guidelines for plant and enterprise interconnectivity: Purdue Enterprise Reference Architecture (PERA) and the Instrumentation, Systems and Automation Society (ISA). Both organizations strive to provide a fundamental basis for good design and operational procedures; though their approaches are from different perspectives.
4.1 Purdue Enterprise Reference Architecture (PERA)
The PERA group looks at system / plant / infrastructure from a cradle to grave approach ensuring that the full life cycle is identified and implemented in the most efficient manner. PERA model defines the need to separate critical plant functions such as plant communications and control away from the enterprise. In addition to this logical and physical separation, PERA provides a conceptual (design philosophy) means of interconnecting the enterprise and plant communications at locations conducive operational requirements while maintaining security. The Physical System Architectures are best designed using certain rules. These rules are based on assessment of the 4R's of system design, specifically: Response; Reliability; Repairability; and Resolution.
Each industry is likely to have different functional requirements, as well as different 4R's, which will result in different system architectures. Hence, it is left to the user of this model to define their network outlook based on the above criteria.
4.2 Instrumentation, Systems and Automation (ISA) Society
The ISA’s assessment is based on many industry experts that have long contribution to the success of ISA standards development activities. ISA has an extensive history of collaboration with the following organizations for securing control systems communications and forming best practices guidelines: US National Laboratories, the NIST Process Control Security Requirements forum, The Chemical industry Data Exchange, the North American Electric reliability Council and various others in order to stay ahead of the growing threats.
The two main ISA standards of relevance to this assessment are: ● ISA IS-95 Enterprise-Control System Integration
● ISA IS-99 Security Technologies for Manufacturing and Control Systems ISA research generated numerous documents and guidelines discussing the benefits and concerns with the legacy designs. Five major themes were stressed in most of the material found during the research and they are as follows:
1. Process Control System Communication Network Security:
Historically process control systems (PCS) have been essentially proprietary and isolated. Over the past decade and with the rapid advances in Information Technology, these systems have become more open and powerful. DCS systems are increasingly based on standard technologies and communications protocols. Most DCS systems in Saudi Aramco are now connected to the local area network (LAN) either directly or via data historian or advanced process control applications. Hence, process control system communication network security becomes vital to overall business operation. This network security theme is inline with the current Saudi Aramco practices. The practice in Saudi Aramco is to install a Firewall between the Corporate and the Process Automation network; refer to SAES-Z-010 for more details.
2. Network and Communication System Attacks: Network and
communication system shall adopt a plan, design and implement a system that ensures a high level of separation or implement extensive firewall/ DMZ setups to prevent an authorized access to the control network. This theme is also inline with Saudi Aramco; refer to SAES-Z-010 for more details.
3. Cyber Security: System and network attack analysis has shown that the
current threats are coming from outside hackers trying to break into systems, just for the sake of doing it or for malicious intent. Numerous incidents reports show electrical power plants shutting down or experience temporary loss of control. There have been cases of nuclear power generation stations also experiencing such issues. This over emphasize the need for designing secure systems and keep them updated to minimize the security venerability. This theme is also inline with Saudi Aramco; refer to SAES-Z-010 for more details.
4. Control System Operation and Availability: The primary requirement
for the plant control system is to stay operational and in control. Systems must be designed and implement with that point in mind. If a shared communications system is employed, all facility and cost savings will vanish if a control system / plant communications becomes compromised either by a failure due to poor design or exposed vulnerability due to lack of isolation / security.
5. Communication Infrastructure Weaknesses: Primary concerns fall into
two main categories; communication system redundancy (hardware, fiber, copper, etc.), and communication systems operational control. As with any critical communication connection, all possible single points of failure must be eliminated by providing infrastructure redundancy and or alternate paths to ensure operational status. The other point of concern is control/ownership of communication infrastructure that your critical plant controls links are connected through.
4.3 Industry Trends Summary
Here are the key recommendations of the industry trends. These recommendations were evaluated as part of our final analysis and recommended network architecture for remote Process Automation applications.
● The plant control system with its dedicated, isolated, private control LAN / communications provides the best way to ensure security and control. This is the practice within Saudi Aramco as outlined in SAES-Z-010.
● In today’s demanding business environment, it has become common to utilize a tightly controlled firewall / multi-zone DMZ secured communication connection that allows the plant control system to operate securely while providing a safe means of sharing required plant information with the enterprise proponents. This is the practice within Saudi Aramco as outlined in SAES-Z-010.
● Shared communications infrastructure is used to transport the remote plant control information and mainline enterprise (voice, data and video) communications as long as designs provide congestion free, accurate, priority QoS based delivery of plant control data in a consistent manner. Utilize separate communications systems if the facilities and funds are available or tightly control and secure a shared communications system to handle both plant process controls and enterprise requirements. This will be part of our recommendation since fiber optics and communication infrastructure is invested in the new fields (i.e.,Khurais, Khursaniyah, etc.). ● Plan to eliminate or minimize all single points of failure and if at all possible
procure, install and control your own communication infrastructure. This will be part of our recommendation since fiber optics and communication infrastructure is invested in the new fields (i.e.,Khurias, Khursaniyah, etc.).
5 Future I-Field/SCADA/Remote Sites Network Architecture
An evaluation criterion was developed based on different performance issues outlined in section 5.1. The criteria a long with the impact was mapped and tabulated in section 5.2. A recommendation is provided based on this assessment in section 5.3.
5.1 Existing SCADA Performance & Reliability Issues
The table below summarizes the different performance issues for the SCADA systems in different areas within Saudi Aramco. The most common performance issues are enumerated below:
Network Design
Performance issues
Dedicated Lines Low speed/Daisy chained RTUs. Failure of one RTU will impact
subsequent RTU’s Primary:
IP/ATM/SDH Backup: IP/ATM/Radio
Performance issues when failover from SDH to the back up radio links. Also, delay is noticeable when other bandwidth demanding applications (e.g., Video streaming) are pushed through the network on the E/W link Route
Dedicated Bandwidth/OTN
System freeze due to sever failover from primary to secondary. This issue is attributed to the point to point connection switch over at the Terminal server located in the control room.
Self healing ring is not working when an intermediate transport node fails. This is attributed to the node technology specs where the node uses a shared CPU for both primary and backup links.
Dedicated lines over SDH
Synchronization and network timing issue in the transport communication network.
Fiber cuts due to lack of coordination between plant project groups and plant operation. Also, fiber cables wiring are environmentally prone to attacks by rats.
Daisy chained Design
A failure of one RTU impact the performance of some RTU in the daisy chain link.
A failure of one of the intermediate instruments impacts the subsequent instruments that are daisy chained on the same link.
Self healing transport network ring architecture
A node failure does not properly trigger the self healing ring capability. So, all nodes that is below the failed node become out of service.
System Servers: Primary to Backup
Intermittent access failure between the application servers (Primary and back up) and the Network Attached Storage (NAS).
In addition, frequent failures of the primary and the back up servers. Intermittent failure in the switching between the Primary and the back up server.
5.2 Analysis & Results
There are different design options that can be used to support Remote Process Automation Application traffic. These are:
1. Process Automation traffic over SDH Transport network using dedicated bandwidth or serial point-to-point circuit emulation utilizing shared infrastructure. This network design can be based on allocating a dedicated bandwidth for each specified service.
For example, a dedicated bandwidth for SCADA, a second for CCTV, etc. The second option is where all of the services are subscribed and queued to one dedicated bandwidth.
2. Process Automation traffic over IP/ATM based on using ATM best effort LAN emulation capability.
3. Process Automation traffic has a dedicated fiber optics strand where SCADA will have its own SDH/OTN network.
4. Process Automation traffic utilizing dedicated Giga Ethernet over dedicated fiber optics infrastructure (dedicated strands).
Five (5) performance criteria were used in a comparative analysis format for the stated above design options. These are: Traffic impacts, Security vulnerabilities, Cost, Support & Management operation, and System availability. Each category for each network design was qualitatively mapped to either a Yes (for yes there is a negative impact) or a No (there are no negative impacts). The degree of impact was not reflected as this is table was used to qualitatively screen the most positive design option. Table 5.2 outlines the results.
SCADA over SDH using Shared Infrastructure SCADA Over ATM Shared Infrastruct-ure Dedicated over Dedicated Fiber Strands SCADA over GE Ethernet Dedicated Strands Network Design Impact Criteria Option # 1 Dedicated Bandwidth using Shared Infrastructure Option #2 Subscription over shared Infrastructure Option #3 Option# 4 Single SDH Node Option # 5 Dual SDH Nodes Option # 6 Traffic
Impacts No (1) Yes (7) Yes (13) No (19) No (25) No (31)
Security Yes (2) Yes (8) Yes (14) No (20) No (26) No (32)
Cost No (3) Yes (9) No (15) Yes (21) Yes ++ (27) No(33)
Support &
Management Yes (4) Yes (10) Yes (16) Yes (22)
No
(28) No (34)
System
Availability Yes (5) Yes (11) Yes (17) No (23) No (29) No (35)
Results (# of No’s)
2 No's (6) 0 No's (12) 1 No (18) 3 No's (24) 4 No's, but very costly
(30)
5 No's (36)
+ = costly ++ = Very Costly No: Signify no impacts; positive.
Yes: Signify an impact and a concern; negative. (#): Note number with details in Appendix 1.
Table 5.2: Comparative Analysis- Refer to Appendix 1 for details
The following is a summary of the alternatives that would provide sufficient reliability at an optimal cost. These are:
1. Dedicated bandwidth, option #4; using dedicated fiber optics stands based on single node SDH ring architecture.
2. Industrial Ethernet over dedicated fiber optics strands, option #6. This option eliminates the need for an SDH network layer thus improve cost and reduce network traffic encapsulations delays. This option is being trial tested by Saudi Aramco Project Management at this time.
3. Option #1 will be suitable for remote process automation application where dedicated infrastructure does not exist (e.g., East/West Pipelines, Water Injection Plants); since the company has already invested in a self healing SDH network.
Implementing this solution for East/West pipelines for example, will substantiate the existing ATM performance issues and position the implementation of an efficient secure solution: implementing one (1) firewall at the Yanbu YCOT CCR Room rather than implementing a Firewall at each Pump station.
The following options are not recommended due to either manageability, availability, security concerns or they are at a higher cost. These are:
1. Option 3 (Utilizing ATM over SDH) is currently used in East/West Pipelines SCADA Network and will be replaced by option #1; upon BI-10-00179 implementation.
2. Option 2 (Subscription over shared) requires higher level of switching intelligence to support necessary quality of service settings. The complexity in managing and associated cost does not warrant this option. 3. Option 5 (Dual SDH nodes) provides higher level of redundancy but
economically unjustifiable.
5.3 Remote Process Automation Application Network Architecture
The different network architectures outlined in this document and associated analysis has lead to two acceptable network technology and topology designs for future Process Automation networks. These are:
1. Industrial Ethernet using dedicated dark fiber in a self healing ring topology.
2. For existing implementations, utilize dedicated bandwidth based on Synchronous Digital Hierarchy (SDH) transport network self healing ring architecture supported by a dedicated fiber optics strands.
These two network toloplogies implemenation shall be based on the following general guidelines:
• A standalone fiber optics cable shall be utilized to connect remote sites (i.e., well sites) to a centralized Process Automation Network (PAN). Alternatively, Radio links, or Broadband wireless network solutions
• Self healing ring: is a ring topology based on a uni/bidirectional links between a set of stations providing optimum protection against fiber cable cut and equipment failure.
• Utilize point to point ring system for scattered sites and/or where a true closed loop physical ring topology is economically not justifiable. • A PAN comprising of multiple scattered (PANs), shall interface with
the Corporate Network via a centralized firewall(s). These scattered PANs shall be connected together to form a unified PAN utilizing the corporate SDH transmission infrastructure with a dedicated bandwidth of 10 Mbps. Alternatively, dark fiber can be utilized if available.
• SCADA over IP/ATM is a unique implementation for East/West Pipelines. It was recommended to PMT to move SCADA traffic from ATM and have it directly transported over a dedicated SDH transport network to fulfill the physical separation requirements as per SAES-Z-010. This would be part of the ongoing BI (BI-10-00179) that will replace the existing Microwave system to SDH.
Any future implementation similar to the East/West Pipelines such as Water Injection Plants, scattered GOSPs will be based on either one of the two design options stated above. This is mainly because remote GOSPs or WIPs operation is tightly coupled with centralized process control systems.
Detail design requirements for these two wireleine design options are reflected in section 5.3.1. These two design options will provide the required physical and logical separation between Plant Automation Networks (PAN) and other traffic. The outcomes of each recommendation will be mapped to the appropriate Process Automation Standard to be included as part of the next revision. In the interim this report will serve as the guide for any future SCADA, I-Field, and remote site network connectivity until all the standards are updated with these recommendations.
5.3.1 Future Process Automation (SCADA, I-Field, Remote sites) Network Design Architecture
The architecture for I-Field, SCADA application, and Remote sites Process Automation Application communication network shall comply with the following guidelines:
1. The remote Process Automation applications network architecture shall be based on either one of the following configurations:
i. Industrial Ethernet using dedicated dark fiber in a self healing ring topology for grass root projects such ,Khurias, Manifa, etc,
Figure 5.3.1.1 depicts the network topology.
ii. For existing implementations, utilize Synchronous Digital Hierarchy (SDH) dedicated bandwidth based on self healing ring architecture.
Figure 5.3.1.2 depicts the network topology.
iii. Radio links, or Broadband wireless network solution could be used to connect dispersed remote sites where fiber optics cable prove economically not feasible.
iv. Number of nodes in the Ring topology shall be carefully calculated from a time delay, bandwidth utilization, and ring switch over perspective so that application performance is not hindered.
v. Self healing ring topology is based on optical ring with physical route diversity that provides protection against fiber cable cut and equipment failures.
Also, the optical ring can be a linear system without a physical routed diversity. This system would protect against equipment failure for sites that are limited/scattered or where a true physically ring topology is not economically justifiable. vi. Industrial Ethernet will be used as a migration path for the
existing Optical Transport Node (OTN). Expansion for existing OTN fields’ networks are recommended to be supported by Industrial Ethernet; thus eliminating the need for communication shelters and the associated cooling requirements.
2. The network design shall provide physical and logical separation between Plant Automation Networks and Corporate Network. However, logical separation utilizing the company’s provided transmission network (i.e., dedicated SDH bandwidth) is permitted for scattered and remote sites with a minimum SDH bandwidth of 10 Mbps.
3. Real-time control communications must be supported on plant owned and dedicated infrastructure.
Supervisory control and remote engineering communications can utilize the company’s provided transmission network (i.e., dedicated SDH bandwidth, or dark fiber). However, a prior approval from P&CSD manager for SCADA applications that are directly linked to the Emergency Shutdown (ESD) systems.
4. Monitoring plants applications from the corporate network shall be allowed via only proxy servers.
5. Logical bandwidth separation between each remote site and the Computer Control Room shall be achieved by provisioning dedicated bandwidth (i.e., trunk). The following shall be considered when supporting this network topology model:
i. Logical separation (i.e., dedicated SDH link) must be utilized to separate supervisory control and data acquisition traffic from other traffic such as voice, CCTV, etc.
Refer to Appendix B for Bandwidth Calculation of this report. 6. Industrial Ethernet over dark fiber network design ring topology
depicted in figure 5.3.1.1, is the direction for future remote Process Automation application network deployment as stated previosuly. This should eliminate the need for an SDH network layer; thus significantly minimize cost and network traffic encapsulations delays.
This network design shall adhere to the following:
i. Hardened industrial type communication equipment shall be used for all outdoor Process Automation networks.
Moreover, these switches can interface directly to Ethernet enabled RTUs, PLCs, instruments, and services stations. ii. Industrial Ethernet Access switches with 1 Gbps uplinks shall
be installed at the remote sites in a ring network topology connecting to the backbone Giga Ethernet switches.
iii. Industrial Ethernet Access switches shall have VLAN capabilities to support traffic segregation.
iv. Industrial Ethernet Access interface ports at the remote site shall be physically and logically interoperable with the Remote Terminal Unit and I-Field Surface Units (IFSU; i.e., PDHM, MPFM, etc) instrumentations. This includes but not limited to port auto-sensing, Duplex/simplex, cable pin-out automatic configuration, etc.
v. Ethernet backbone switches shall be redundant and provide minimum speed of 1 Gbps utilizing ring network topology architecture.
vi. Ethernet backbone switches shall be layer 3 non blocking switches; (carrier class core switches).
vii. Ethernet backbone switches shall perform the routing functionality for whole network.
viii. The uplink switch-over for for both access and backbone Ethernet switches shall not exceed applications maximum time out parameters.
ix. All TCP/IP addresses shall be obtained from Saudi Aramco network management group.
x. Provide integrated Domain Name Service (DNS) and Windows Intergated Name Service (WINS) to address the name IP address resolution.
xi. Tight integration (e.g., using OPC Tunneling) between
Network Management System and the Process Automation application (e.g., SCADA) is recommended for the ease of overall PAN network manageability.
Figure 5.3.1.1: Future Network - Giga Ethernet over Dark Fiber; Eliminating the Need for SDH/OTN
Remote Sites Ring Size Dependent on # of RTU’s 10/100/1 GGE Giga Ethernet Switches Giga Ethernet Switches 10/100/10G 10 10 10 10 Redundant GE Ethernet Switches Remote Radio RTU=1, thru n MIS User CCR Operator Control Console Ethernet
ENG & OPR Console PI Server
(etc..) Firewall
IT Wide Area Network
Host Radio
RTU=1, thru n
Oil & Gas Fields /Remote GOSP Remote GOSP IT Wide Area Network Enterprise PI Server Enterprise PI User
Figure 5.3.1.2: Future Networks-Dedicated Bandwidth over SDH Transmission Network Using Dedicated Fiber Strands
Oil & Gas Fields Ethernet
Switch
CCR
Fiber Optics Backbone Transmission Ring
Operator Control Console
Giga Ethernet
ENG & OPR Console Ethernet Switch 500P PS1 PS2 Om n i S wit c h 1 2 3 4 5 6 7 8 9 VACANT SLOT VACANT SLOT VACANT SLOT VACANT SLOT MPX OMNI SWITCH 9WX
2 PORTS OC-3 SMF -ATM ASX-155 RFS
1
2
32 UTP 10/100 Mbps -Ethernet ESX32 UTP 10/100 Mbps -Ethernet ESX
Data Acquestion Hostorian Server WHESD/ESP/PDHM, etc.. Fiber optics Transmission Access Ring REMOTE GOSP Wireless Firewall IT Wide Area Network Process Automation Network Enterprise Office User Enterprise PI Server MIS Work Station Enterprise PI Server
8. Data acquisition frequency (e.g., polling cycle, deadband) shall be identeifed by the proponnet in the Funcational Specification Dcoment for each installation.
Comments: For I-Fields pressure transient analysis application data acquisition frequency rate shall be set to 1 second. If the communication protocol is based on report by exception, then the deadband shall be set to 0.01 for the Permanent Down Hole
System Pressure and Temperature readings.
9. Redundant RTU and network interface requirements (Single vs. Dual) to the Process Automation servers will be determined at the early design stage for each application. For example, high pressure gas wells may require complete end-to-end system, RTU, and network redundancy.
10. Each RTU shall have its own dedicated network connectivity. A failure of one RTU shall not impact the data acquisition from other RTUs. It shall also be equipped with local storage capacity that can support local data acquisition for 96 hours.
11. The design criteria for the installation of electrical instrumentations are covered in SAES-J-902.
For I-Field Surface Units (IFSU; i.e., PDHM, MPFM, etc) network connectivity shall adhere to the following guidelines:
i. Communication interface between IFSU and the RTU/Ethernet switch shall be interoperable. Ethernet is preferred.
ii. The connectivity design between IFSU and the RTU/Etherenet switch shall be based on a dedicated point-to-point connection. Daisy chained IFSUs wiring design is not permitted to minimize failure impacts to the overall business operation.
iii. IFSUs shall be able to support standard Ethernet and/ or Serial interfaces. Ethernet is preferred.
iv. IFSU shall be equipped with local storage capacity that can support local data acquisition for 96 hours.
12. Process Automation application servers shall be deployed as follow: i. SCADA and other critical process applications such as I-Field
computer system hardware (e.g., servers) shall adhere to 23-SAMSS-20.
ii. Non mission critical applications (e.g., Cathodic monitoring, Power System Automation, etc.) shall be supported by single servers.
Comments: It is recommended at the design stage for new projects to minimize the number of servers at the Process Automation control room by combining applications on the same
set of servers; if feasible.
15. A Business Continuty Plan (BCP) shall be established to remedy possible network design deficiencies if any; but may help reduce the impacts and improve the response time upon a network failure.
5.3.2 Next Steps
Following are the next steps to be completed as part of Phase II of this report.
1. P&CSD will evaluate different industrial Ethernet vendor solutions, complete performance testing and test procedures.
2. The outcome of this effort will be used to develop the required material specifications for Industrial Ethernet that would accommodate Saudi Aramco Process Automation requirements for I-Field, SCADA, and Remote Site network connectivity.
Part II: Process Automation Wireless Access 1 Background
This wireless Process Automation outlook was carried out as part of plant networks optimization efforts. The plan is to enable and promote efficient and cost effective and proven industrial wireless technologies for various process related applications such as I-Field, SCADA and other process remote monitoring. Also, this effort would help to streamline the different ongoing Process Automation wireless trials into uniform implementation for future projects.
In this effort, we emphasized on the importance of considering and complying with the requirements of critical and time-sensitive process information applications as well as considering wireless reliability, security and interoperability. Moreover, we avoided discussing proprietary wireless solution since it hinders capability, interoperability and flexibility.
2 Introduction
Wireless technology is becoming increasingly popular solution for a wide range of applications. After evaluating the technology, many users are convinced of its reliability, satisfied with its performance and are ready to use it for different applications in a large-scale and more complex wireless networks.
Wireless LANs (WLANs) differ from regular “WIRED” LANs in the means they use to transport data. WLANs use air instead of a network wired cable as a medium for data transport. During the last few years, WLANs have many applications in warehousing, manufacturing, airports, hotels and health-care.
Recently, wireless technology is becoming among the most significant cutting-edge technologies as a viable network connectivity option for Process Automation networks (PAN) as well as instrumentations networking. Different Wireless technologies are benchmarked in this assessment and were mapped to the suitable Process Automation application (s).
3 Wireless for Process Networks
In nearly all industrial plants, communication links carry vital information between field devices (instruments, sensors, actuators, etc.) and the process control center. From periodic updates to ongoing process and manufacturing management, reliable and secure data flow is critical operations.
Control message and status information transferred in industrial settings—actuator position, temperature, or liquid levels, for example—is carried in short “bursts” which require relatively little bandwidth and connection speed. At the other extreme, large file transmission, such as activity logs from a production run, requires moving a lot of data very efficiently.
Whatever the specifics of the data being moved, all industrial communications share common critical requirements: Secure, Reliable and Timely data delivery.
The following applications have been identified as potential industrial applications for wireless access:
1. Exploration and Producing operation (Wireless-I-Field, data connectivity) 2. Remote Data Acquisition (temperature, pressure, flow, level)
3. Energy management & sub-metering
4. Process monitoring and control (Condition based, Power, Vibration, wellhead,…etc.)
5. Waste water management
4 Industrial Wireless vs. Non-Industrial Wireless
The Process Automation applications are considered mission critical applications and conventional wireless solution won’t be able to serve these applications in an active environment. Hence, industrial wireless is considered to be the alternate solution that would ensure security, reliably and real time delivery for the Process Automation data. The Industrial Wireless solution should meet the required process networks reliability, availability and security as shown in Table 4.1.
No Industrial requirements for Industrial Wireless solution
Office / consumer requirements for non-industrial wireless
1 Reservation of bandwidth for selected critical clients (e.g. PLC), Uguaranteed accessU to radio channel
Radio channel is a shared medium; everybody can access it (best efforts)
2 The communication cycles are predictable for UdeterministicU data traffic
The radio traffic is prioritized, but cannot be predicted
3 Redundant radio communication on two separate frequency bands for increased availability (URedundancyU)
Usually one radio channel
4 Metal casing, vibration-resistant connectors and cables Low-cost plastic casing
5 IP 65 degree of UprotectionU, +60°C and moisture-resistant for harsh industrial and outdoor use For home and office use
6 UIndustrial approvalsU: ATEX (EX area), Area Classification, EMC, UL, FM
No demand for increased requirements
(CE, shock protection) 7 UFlexibleU use through Batteries, 18 - 57 VDC, 100 – 240 VAC, Power-over-Ethernet Voltage supply 100 VAC – 240 VAC, PoE
8 Ethernet, antennae cables, casing Uresistant to chemicalsU, halogen-free, silicon-free, flame-retardant No demand for approvals with increased requirements
Table 4.1: Industrial Wireless vs. Non-Industrial Wireless
5 Industrial Wireless Classifications
Industrial wireless technologies can be classified into four main categories based on their coverage area as shown in Figure 5.1. These are:
1. Wireless Personal Area Network (WPAN) – few meters
2. Wireless Local Area Network (WLAN) – few 100’s meters up to few Km 3. Wireless Metropolitan Area Networks (WMAN) – few Km
Figure 5.0: Industrial Wireless Classifications Bandwidth vs. Distance
5.1 Wireless Personal Area Networks (WPAN)
The Wireless Personal Area Network (WPAN) technology uses a short-range radio link that has been optimized for power conscious, battery operated, small size, lightweight devices.
There are two dominant WPAN technologies namely Bluetooth (IEEE 802.15) and Zigbee technologies.
The UBluetooth Technology operates in the 2.4 GHz band and used in several
computing devices such as notebook computers, phones, PDAs to exchange information. Bluetooth is not considered as potential industrial wireless solution for Process Automation due to the excessive latency and limited networking capabilities due to synchronization handshaking process.
U
ZigbeeU Technology addresses the unique needs of low-cost, low-power,
wireless instrumentation/sensor networks for remote monitoring & control and
WLAN 802.11g ZIGBEE 802.15.4 BlueTooth 802.15.1/1a UWB (802.15.3a?) WLAN 802.11b 2.5G 3G 10m 100m 2km 1Mb/s 50Mb/s 10km 20km+ 100Mb/s WPAN WLAN WMAN WWAN WLAN 802.11a M WiMAX 802.16e 4G MWBA 802.20 FWBA 802.16 3.5G 5km 2 2000088//99 2 2000066 22001100 2 2000066 2 2000055 WLAN 802.11n 2 2000066 BlueTooth 2.0 802.15.4a 2 2000055 2 2000044 2 2000055 2 2000033 1 1999999 1 1999999
Bandwidth
Distance
The general specs of Zigbee along with IEEE 802.15.4 solution could be summarized as follows:
● Dual PHY (2.4GHz ISM and 868/915 MHz)
● Data rates of 250 kbps (@2.4 GHz), 40 kbps (@ 915 MHz), and 20 kbps (@868 MHz)
● Optimized for low duty-cycle applications (<0.1%) ● CSMA-CA channel access
● Yields high throughput and low latency for low duty cycle devices like sensors, instruments and controls
● Low power (battery life multi-month to years) ● Multiple topologies: star, peer-to-peer, mesh ● Addressing space of up to:
○ 18.45Exp+16 devices (64 bit IEEE address) ○ 65,535 networks
● Optional guaranteed time slot for applications requiring low latency ● Fully hand-shaked protocol for transfer reliability
● Range: 50m typical (5-100m based on environment)
In conclusion, the Zigbee solution (in conjunction with IEEE 802.15.4) is considered as a potential emerging short distance wireless technology (WPAN) for instrumentations networking for periodic, intermittent and slow latency applications. It must be noted that Zigbee has the major limitation with coverage area; distance coverage in the 10’s of meters. The overall Zigbee model is shown in Figure 5.1.1 and Zigbee Network Architecture is shown in Figure 5.1.2.
Figure 5.1.1: ZigBee Network Model
Figure 5.1.2: ZigBee Network Architecture
5.2 Wireless Local Area Networks (WLAN)
On the other hand, IEEE 802.11(a,b,g) is a mature and proven wireless technology that already implemented in various applications. Currently, there are three main IEEE 802.11 WLAN standards each operate with different characteristics such as modulation type, data rate, frequency band, and transmit power (Table 5.2).
Table 5.2: IEEE 802.11 Standards
It should be noticed that IEEE 802.11b and g are backward compatible wireless technologies that operate in the 2.4GHz band (ISM) while IEEE 802.11a operates in the 5GHz band. The operating frequency, the data transfer rate, the transmission technology and compatibility of the present IEEE standardized WLANs is illustrated in Figure 5.1.
Figure 5.1: IEEE 802.11 Standards Compatibilities
There are three main open standard frequency bands known as Industrial, Scientific and Medical (ISM) bands as shown in Figure 5. The 900 MHz band is becoming overcrowded due to various consumer products that operate at the same band. It does offer longer range (for the same gain antennas) than the 2.4 GHz band and 5GHz, but it has limitations on the maximum size of antennas that limits its overall range and the maximum data rate that is reliably obtained is under 1Mb, due to the limited frequency range.
At 2.4 GHz, the lower power transmitter allows very high gain antennas, which allows long distance communication (up 15km). The frequency range is also much wider than 900 MHz, allowing higher data rate (54Mbps) with a reliable range.
The 5 GHz band offers more bandwidth, allowing higher data rates; however, the nature of the higher frequency limits range. Typical range for 5 GHz band is much lower than the other ISM bands.
Figure 5.2: WLAN Frequency Band Characteristic
For critical industrial applications, Access Point (AP) should support polling mechanism to guarantee application channel access moreover, redundant wireless connection in the 2.4 GHz and 5 GHz bands which is known as Tri-Mode Dual-Band solution. Both signals (2.4 GHz and 5 GHz) are transmitted simultaneously to achieve extremely high level of radio channel reliability and to guarantee interference-free wireless connection (Figure 5.3).
Figure 5.3: WLAN Radio Redundancy
900 MHz band 2.4 GHz band 5 GHz band
PROs
CONs
Greater range than 2.4 GHz band ( for in- building LANs)
Global market IEEE 802.11 Higher data rates (10+ Mbps)
Global market IEEE 802.11 Higher data rates (20+Mbps)
Less range than 900 MHz Maximum data rate 1 Mbps Limited bandwidth Crowded band
Much less Range than 900 MHz or 2.4 GHz Higher cost RF components Large antenna required
The IEEE 802.11a,b,g wireless technologies are considered a potential wireless solution for mid range (100’s of meters up to few Km) applications. Remote monitoring such as wellhead monitoring, motors monitoring, vibration monitoring interconnectivity are candidate applications for IEEE 802.11a/b/g WLAN solution as shown in Figure 5.4.
Figure 5.4: WLAN Industrial Applications
5.3 Broadband Wireless Connectivity
There are several wireless solutions falls under broadband wireless connectivity categories. However, WiMAX technology is the most promising last mile wireless connectivity solution. WiMAX is a worldwide wireless standard that addresses interoperability across IEEE 802.16 standard-based solution. WiMAX technology offers greater range and bandwidth than the WLAN family of standards and provides a wireless alternative to wired backhaul and last mile access (Figure 5.3.1).
Generally, WiMAX technology can reach 1 ~ 50km+ coverage radius and FinFan
Machine Tool
Central Control Room
Roll Pump
Access Point
Figure 5.3.1: Fixed WiMAX Deployment Models
WiMAX solution would require frequency approval from local authority (CITC) and should be requested through a 3P
rd
P
party local service provider. This may hinder WiMAX utilization for Process Automation applications.
5.4 Wireless Wide Area Networks (WWAN)
Cellular Network (GSM) is considered WWAN solutions. GSM network could be utilized to provide cost effective connectivity for various semi-real time, non critical applications. P&CSD had successfully deployed WWAN solution using GSM network (Short Message System/GPRS) to monitor the performance and status of the water treatment plant instruments, sensors and equipment (pumps, MOVs, Flow, Voltage, Temp,..) as shown in Figure 5.4.1. Customer was able to remotely monitor water treatment plant process and takes immediate corrective action as needed as well as eliminated the daily dispatch of the technician to the remote site.
Figure 5.4.1: GSM Solution for Process Monitoring (SMS/GPRS)
This solution could be used for non critical applications that are covered with GSM network. Similar solutions could be developed using TETRA solution in the future.
Technology Summary: Hhttp://eway.aramco.com.sa/cops/coredocs_cop/Wireless
Remote Monitoring and Control using SMS Technology for Water Treatment Plant _PCD-13_03-T.PDF
Technology Presentation:
H
http://eway.aramco.com.sa/cops/coredocs_cop/Wireless Remote Monitoring and Control using SMS_Final.PDFH
Finally, Satellite communication (LEO, MEO, GEO/VSAT) is also considered as WWAN connectivity solution. This design option can be utilized in a point to point or point to multipoint connection. However, this design option can be costly. So, the concept of utilizing WLAN as an aggregator at the remote sites and then use Satellite communication to uplink the WLAN for the different remote sites back to a Centralized location.
Satellite communication typically is susceptible to latency, security and running cost that should be considered when selected in any future Saudi Aramco projects.
RIYADH REFINERY Remote Site (STP)
P Prreessssuurree T Teemmpp A Ammppss MTU MTU PPuummpp GSM Wireless Network