OPENFLOW WITH MPLS IN SMART GRIDS
Aditee Patil
Mohammad Zohaib Yunus Shweta Daheeval
Varun Sharma Group 15
Advisor: Professor Conwell Dickey
University of Colorado at Boulder
TLEN 5710: Capstone Professor David Reed
April 25, 2014
2 Abstract
In an era of technological advances, the demand from customers is intensifying in regards to high bandwidth, better services, faster speed, more reliability, and lower costs. A rapid migration from the access infrastructures to the Next Generation Networks (NGN) is required in order to cope with the proliferating and escalating requirements. This research paper proposes a new smart grid communications network using OpenFlow with Multiprotocol Label Switching (MPLS) that provides a centralized control in Smart Grid Supervisory Control and Data Acquisition (SCADA) systems, in addition to being cost-effective, more efficient, and interoperable with existing Time Division Multiplexing (TDM) protocols and legacy applications. This research will investigate the ways to evolve the smart grid communications network from TDM over Synchronous Digital Hierarchy (SDH) or Synchronous Optical Network (SONET)-centric network to a smarter OpenFlow based architecture that will provide benefits of reduced operational costs, increased power quality, higher security, and greater reliability.
3 Table of Contents
1. Introduction Page
1.1. Statement of the Problem………4
1.2. Research Question……….. 4
1.2.1 Sub-Questions……….4
2. Literature Review……….5
3. Research Methodology……….…8
4. Research Results………..10
4.1. Resolution to Sub-Question No. 1 ……….10
4.2. Resolution to Sub-Question No. 2 ……….12
4.3. Resolution to Sub-Question No. 3 ……….14
4.4. Resolution to Sub-Question No. 4 ……….20
5. Discussion of Results………22
6. Conclusions and Future Research………...23
7. References………..24
4 1. Introduction
1.1 Statement of the Problem
Utility companies have been using decade old technologies in smart grid networks and are clearly in need of new communication techniques. Most companies are still relying on point-to- point radio wave links and leased lines for communication between the Supervisory Control and Data Acquisition (SCADA) master control station and the remote substation. These technologies do not provide adequate performance, security, and cost-effectiveness for the time critical control signals from the substation. Utilities are clearly in need of new and advanced communication networks that are not only more secure and offer better performance, but are also interoperable with legacy devices and protocols in addition to being more cost-effective.
1.2 Research Question
Investigate the feasibility of OpenFlow being deployed with Multiprotocol Label Switching (MPLS) in smart grid communication networks in order to reduce operational costs, increase network performance, and promote innovation by adding infrastructure flexibility.
1.2.1 Sub-Questions
1. Explore the problems associated with the current smart grid network architecture and determine how OpenFlow can provide a feasible solution. What are the current technologies being used in a smart grid network for communication between the control station and substation? This research will address the security and performance issues that plague the current infrastructure and research on how OpenFlow can provide better performance, security, lower costs, and still be interoperable with legacy protocols and devices?
2. Implement an MPLS based smart grid network and evaluate the performance between substation and control station. Determine how the MPLS network will perform during a link failure and
5
evaluate the number of dropped packets when a link has failed and the packets are forced to take a different route.
3. Can OpenFlow with MPLS support the existing applications and be interoperable with legacy devices at the remote substations, in addition to providing better features and performance?
Implement a lab setup with conventional smart grid substation equipment and run DNP3 protocol to check interoperability with OpenFlow.
4. How will OpenFlow prove to be a cost-efficient technology that will pave the way for new network services to be added without service interruptions to utilities?What are the costs involved in today’s smart grid network and how operational costs can be reduced using OpenFlow based devices? Conduct an analysis and comparison of costs incurred in an OpenFlow based network and a pure MPLS network for smart grids.
2. Literature Review
Smart grids could be pictured as a fusion of power and communication infrastructure.
Taking into account all key aspects to the functioning of smart grids, communication plays the most important role. All these years, SCADA has evolved as a power communication system that uses a multitude of transmission mediums such as wired (twisted pair, coaxial, and fiber optic) and wireless (UHF, satellite, and microwave)[1]. SCADA system comprises of several Remote Terminal Units (RTUs). These units gather information from field devices and transmit data to the master station. The acquired data is then processed by the master station and appropriate control signals are sent to the field devices to perform certain control tasks [2]. Distributed Network Protocol (DNP3) is the telecommunications standard that defines communication between the master stations and Remote Terminal Units (RTUs) [3].
6
In a research by Mak and Holland [6], the authors reviewed and recommended TCP/IP over ethernet as a networking protocol specific to the electric utility employing SCADA systems.
The authors submitted a technical report explaining details of their proposal for “SCADA system migration” to TCP/IP and Ethernet [6]. Leading power utility company ABB, attempted to introduce a new design for smart grid networking known as the Smart Grid Transmission Protocol (SGTP) as an alternative to Transport Control Protocol (TCP) [28]. The paper explored some of the issues with the existing transport protocols that do not meet the demands of the smart grid application. The authors discussed the new ‘SGTP’ protocol that could accomplish lower latency along with maintaining reliability and inbuilt security mechanisms [28]. In another research, Cao and Andonovic [7] investigated the selection criteria for wide area backbone communications and proposed an MPLS Virtual Private Network (VPN) to achieve real-time data communication in smart grid networks by implementing comprehensive computer simulation using a network simulation tool called ‘OPNET’ to compute the performance in a changing load setup environment [7]. MPLS based layer 3 technology was examined using OPNET, which offers a ‘graphic edit interface’ and supports ‘object-oriented technology [7].’ It was found that MPLS VPN has added functionalities such as Traffic Engineering (TE) and fast re-route mechanisms imparting lower latency of the control signal, which is critical in most of the real time applications in smart grids [7]. In a research by Qin [5], an “adversary model” for smart grids was discussed, where the author analyzed attacks in an MPLS network due to corruption of the Label Distribution Protocol (LDP) messages with the motive of impacting the Quality of Service (QoS) for certain real-time traffic in SCADA systems [5]. Sydney et al. [20] demonstrated in their research how MPLS possesses techniques for attaining efficient “overlay technologies” along with means to improving security in smart grid networks [20]. The research examines how innovations are limited using MPLS
7
routing and switching technologies due to “features enclosed in the box” and states that there are no empirical methods that have been developed for utilities to experiment and test new alternatives to IP and MPLS [20].
Our MPLS based OpenFlow architecture for smart grids extends state of the art research by providing utility operators the flexibility to add new services and applications at lower costs without any impact to existing services. Our research aims to investigate the feasibility of implementing OpenFlow with all the features of MPLS and show how it can perform better than MPLS alone, thus being an excellent alternative to MPLS in smart grids.
In a classical scenario with routers and switches in the network, the control plane, which makes the routing decisions and the data plane, which is responsible for data forwarding, reside on the same device [8]. With OpenFlow, these functions are separated into different devices. The data forwarding still takes place on the switch, but the control plane that makes all the routing decisions resides on the controller, which can be a Linux based server or a computer. The OpenFlow architecture is comprised of three main parts – (1) A flow table, which has a list of different flows with their respective action to be taken by the switch, (2) A secure channel, which is the interface between the switch and the controller that serves as a channel for exchange of messages between the two devices, and (3) the OpenFlow protocol, an open standard protocol for communication between controller and switch [8].
8
Fig. 2.3– OpenFlow architecture [8]
3. Research Methodology
A qualitative research methodology coupled with grounded theory was the directorial tool towards approaching the research sub-problems. A variety of data sources, including quantitative data, descriptive examination, review of records, lab experiments, observations, facts, and surveys were used in order to aggregate and organize this paper.
The major challenge for us was to gauge how easily OpenFlow with MPLS could be incorporated into the smart grid networks system over the existing architecture in order to avoid having a complete overhaul of the communications system. We explored problems associated with the current network design in smart grids by conducting a survey with one of the biggest utility companies, ‘Xcel energy’. The parameters of the survey included the communication model, technology, security issues, and current devices used in the company. Through our interaction, we acquired knowledge about the current operations to gain first-hand experience.
We set up a lab to emulate a smart grid network and conducted performance tests. The lab set-up was built using Cisco 3640 routers as the core MPLS cloud and the SEL 2411 Programmable Logic Controller (PLC) [9] that served as a substation device sending control signals across the
9
network to a computer acting as a host in a control station. We did the setup at the University of Colorado, Boulder’s state of the art telecommunications and energy communications laboratory.
Data was gathered based on pings with incrementing intervals from the host to the PLC and observed the number of dropped packets when a link was failed and the packets were forced to take an alternate route.
In order to examine if OpenFlow can support the existing legacy applications and co-exist with MPLS in a smart grid network, we implemented a network using the SEL 2411 PLC and made it poll to a host using DNP3 protocol via the OpenFlow enabled switch, which was a 48-port Pica 8 P-3290 Open Virtual switch (OVS) [10]. We used an OpenFlow controller called ‘POX’
[11] on a computer running on the Ubuntu operating system. The OVS was configured to connect to the controller and its ports were enabled for communication with the PLC. Upon successfully testing the interoperability of legacy protocols and equipment with OpenFlow protocol, we further decided to implement an OpenFlow with MPLS based solution for smart grids. Another OpenFlow controller called ‘NOX’ [12] was used to run this network. For this task, we required four open virtual switches, which would emulate a provider MPLS core. There was a constraint here as we did not have four such switches available to experiment upon. We, therefore decided to use the OVS emulator called ‘Mininet’ [13].
Operational and capital costs are very important to the utility company, and through our research, we analyzed these costs and provided definitive reasons for utilities to upgrade to an OpenFlow architecture. We found solutions for transmuting the existing network model so that an OpenFlow with MPLS based network can be deployed on the existing smart grid communication network to enhance the operational aspects of distribution and automation system. A well- structured and highly organized team effort, along with impeccable designing and deep
10
understanding of the research question was required to achieve the results. All results are based on different scenarios, taking into consideration all the facts, observations, actualities, annotations, clarifications, interpretations, and opinions of all the team members.
4. Results
4.1 Explore the problems associated with the current smart grid network architecture and determine how OpenFlow can provide a feasible solution.
Our team conducted a survey with distribution communication engineers, David Houston and Richard Huck, at Xcel Energy to know about the current industry practices. This survey was intended to explore issues with the current communication technologies used in smart grid networks. According to the survey results, one of the issues encountered was an undefined network infrastructure that would work in all scenarios and support real time critical applications. The survey results show that microwave point to point radio system is being used by some utilities creating the ‘line of sight’ problem. The survey also suggested that DNP3 is currently the standard protocol used in electric utilities for communication between devices in the field substation and control station. DNP3 was modeled to enhance the transmission of data and control information between the control station and substation [3]. There is no single network infrastructure design that works for everyone. Depending on the type of area: rural, urban, or terrain, the form of communication varies. Devices out in the field such as breakers, reclosers, capacitors, banks, and regulators [14] can be fitted with radios that allow them to connect to the backhaul network allowing them to be controlled.
As per the survey results, utilities widely rely on MPLS infrastructure owned by a cloud provider such as Alcatel Lucent [15]. Support for existing TDM services and provision for features such as predictability and high availability are the attributes that utilities look for [16]. An MPLS
11
enabled router consists of both, the control plane as well as the data plane. Control plane is responsible for making control decisions and the data plane is responsible for forwarding packets.
Holding both these functionalities in the same device complicates the functioning as it needs to make routing decisions along with propagating routing and MPLS label information [17]. On the contrary, OpenFlow technology can provide the same services with the MPLS data plane and a separate control plane.
Fig. 4.1 - IP/MPLS versus MPLS/OpenFlow [14]
There are several reasons why we have chosen OpenFlow with MPLS over IP/MPLS as a proposed technology for smart grids:
1. OpenFlow decouples the control and data plane, thus replacing the processor and memory hogging MPLS routers with simple Open Virtual Switches (OVS) in the substation environment.
This would eliminate the need of specific high-end routers and also eradicate demands of a perfect cooling system and cages in the smart grid atmosphere.
2. Frequent instability in the network also causes MPLS routers to recalculate paths and generate packet drops and congestion. This in turn, may lead to drops in hello packets resulting in large convergence time and routing loops [17].
12
3. Incorporating new services would be easier as there would not arise a need to upgrade any boxes or tie new protocols with existing control plane protocols such as Resource Reservation Protocol (RSVP) and (Open Shortest Path First) OSPF. Smart grid networks support many of the mission critical applications and thus, loosing data or control signals is not acceptable. OpenFlow overcomes these problems by connecting the MPLS data plane to the logical centralized controller.
4. Smart grid networks are required to offer the predictability, reliability, and high availability of TDM networks that MPLS fails to impart in smart grids. OpenFlow accomplishes this by distributing the centralized controller over several servers.
5. Lastly, high end MPLS enabled routers that are needed to be installed in the data path increase the total cost of ownership. OpenFlow, on the contrary, uses simplified switches, thus making the whole setup cost effective [17].
4.2 Implement an MPLS based smart grid network and evaluate the performance between substation and control station.
We implemented a lab scenario where an MPLS cloud consisting of four Cisco 3640 routers was set up connecting to the host at the control station, which was a personal computer (IP:
10.10.10.2) and substation (IP: 10.10.10.3), which was a SEL 2411 PLC as shown in figure 4.2.
Fig. 4.2: MPLS Network configuration with 2411 Smart Grid Controller
13
The host machine was made to poll to the SEL 2411 PLC using DNP3 protocol and we found that the DNP3 packet was encapsulated in the MPLS header as the packet traverses the network. Figure 4.3 shows the Wireshark packet capture demonstrating the success of implementing DNP3 in an MPLS based network.
Fig. 4.3: Wireshark packet capture demonstrating DNP3 network running between the Control station (SEL 2411) and Sub-station (Workstation)
In order to evaluate the performance of MPLS in the smart grid communications network, we generated Internet Control Message Protocol (ICMP) ping traffic from host (10.10.10.3) to the PLC (10.10.10.2). For the initial phase, pings were transmitted at an interval of 1ms and the link was failed between R1 and R3. We recorded the maximum number of packets dropped and iterated the test over different ping intervals till 50ms.
14
Figure 4.4: MPLS fast re-route graph as ICMP ping packets are transmitted.
From this experiment, we successfully concluded that with MPLS, the number of packet drops is considerably high and it could cause disruption in real time critical control signals that are transmitted from the substation to the control station.
4.3 Can OpenFlow with MPLS support the existing applications and be interoperable with legacy devices at the remote substations, in addition to providing better features and performance?
In order to prove the working of DNP3 protocol over an OpenFlow based network, we implemented a lab setup that emulated the smart grid network environment using an OpenFlow (OF) controller and Open Virtual Switch (OVS). The controller used for this simulation is called the ‘POX’ controller developed by Stanford University [11]. This is an open source controller available to everyone. We used a Pica8 Open vSwitch that was provided to us by the University of Colorado, Boulder. Our lab setup is shown in the figure below:
Fig 4.5. OpenFlow Network Topology
15
The OVS was configured according to our requirement. The following describes the configuration steps:
1. Configure the switch to connect to the Open Flow (OF) controller, which has an IP address of 10.0.0.4 and communicates over port 6633.
2. Create a bridge on the switch and add ports ge1/1/4 and ge1/1/36 to the bridge. The port ge1/1/4 connects to the host computer, which acts as the control station in a smart grid network. The port ge1/1/36 connects to the SEL 2411 Programmable Logic Controller (PLC).
We downloaded and installed the POX controller virtual machine in our computer. The controller comes with a number of in-built modules that can provide several switching and routing features. In this lab setup, we used the “forwarding.l2_learning” module that makes the OVS act as a type of L2 learning switch [11]. The OF controller uses the OpenFlow protocol to install new flows in the flow table, which come in from the PLC at the substation. A flow is defined as a stream of packets with identical headers [4]. As flows arrive at the switch, they are checked against the list of existing rules in the flow table. If a packet does not match any rule in the flow table, it is transmitted over to the OF controller that evaluates the packet and installs a new rule for it that specifies the action for the switch if it sees the same type of packet again.
As seen in the captures below taken from the controller, we see how new flows are being installed as the controller learns them:
16
The figure 4.6 shows the communication between the substation PLC and the control station host.
The IOServer is a tool that polls between the PLC and the host every second and shows the live communication. Here, we can see that there are packets transmitted and received through the OVS.
17
Fig. 4.6 – Capture of transmission from PLC to Host
We also captured the packets from the PLC to the host and OpenFlow messages between the controller and the switch through the Wireshark Packet Capture software [18].Figure 4.7 shows the capture on the controller showing the exchange of OpenFlow messages between controller and switch. Figure 4.8 shows the exchange of DNP3 messages between the PLC and host machine.
Fig 4.7 - OpenFlow packet capture
18
Fig. 4.8 – DNP3 packet capture
The above lab setup thus proved that DNP3 protocol can work over an OpenFlow based network.
To demonstrate an OpenFlow with MPLS based network, we created a new network topology with four open virtual switches. Switches Sw1 and Sw4 act as provider core, while switches Sw2 and Sw3 act as provider edge devices.
Fig. 4.9 - MPLS/ OpenFlow Network Topology
For demonstrating MPLS with OpenFlow, we use another controller called the ‘NOX’, which is also developed by Stanford University [12]. Since we did not have four Open vSwitches readily available to us, we decided to emulate the network topology on an open flow switch emulator called Mininet [13]. We created a python script for our network topology that would run
19
on Mininet, which exists on the same Ubuntu virtual machine (VM) that the controller is installed on. The OF controller along with Mininet would simulate the entire network.
We ran into an issue here, due to which we were not able to simulate the network successfully. We were not able to bridge the PLC connected on the ethernet port of our computer to the Mininet, which was running on the VM. Upon further analysis, we found that the SEL 2411 PLC devices do not interact with Mininet, which is isolated from any physical connections on the computer.
With the above OpenFlow with MPLS network topology, we aimed to prove how the OpenFlow implementation of MPLS can significantly lower the number of packets dropped during a link failure where the MPLS fast reroute process comes into effect to find an alternate path to the destination. However, upon further analysis, we can conclude that OpenFlow can significantly reduce the packet drops in the network. In a normal MPLS-TE (MPLS with Traffic Engineering) scenario, the Resource Reservation Protocol (RSVP) is used to establish multiple paths for LSPs and resources are accordingly allocated for preferred paths [4]. As seen in our first experiment with MPLS, we observed that 40 packets were dropped when 10ms pings were sent from the host machine to the PLC. This high number of packet drops is due to the reason that the core routers maintain a large number of Label Switches Paths (LSP) and any link failure can take significant amount of re-signaling as the core routers calculate the next preferred path in RSVP [19].
However, in an OpenFlow scenario, the controller has already learnt every flow between the source and destination. When a link fails and MPLS fast reroute process occurs, the controller simply refers to the flow table to find an alternate path to the destination. The OVS does not have to waste any processor memory to calculate a new route, and simply sends the packet out through a different port. This would result in very few packets dropped across the network. We estimated
20
that it could be as low as 5 packet drops when 10ms pings are sent. This means that packet drops are reduced by up to 80% in an OpenFlow based architecture. In smart grid SCADA systems, it is important that the data and control traffic between substation and control station should not be impacted by large packet loss because there are time critical applications that cannot tolerate any loss of control signals during a link failure. Thus, OpenFlow with MPLS ensures reliable communication of packets between the substation and control station.
4.4. How will OpenFlow prove to be a cost-efficient technology that will pave the way for new network services to be added without service interruptions?
In the smart grid domain, conventional Wide Area Networks (WAN) prove to be an expensive affair for utilities as well as service providers and they are unable to cope with the rapidly growing requirements of cloud technologies and services along with advancements in other technical areas. The complexity of networks has increased, and it has become increasingly difficult for utilities to plan, manage, and operate services across all layers of the WAN. OpenFlow, in a way, personifies what SDN (Software Defined Networking) promised to the world of networking, that is, programmable, scalable, and efficient deployment and management of networks. With traditional networks, integrating new services and technologies has become highly expensive and is close to approaching a breaking point where traditional networks cannot cope with the demands of such services and utility companies are not finding it economically feasible, thus resulting in declining service margins. This will ultimately result in smart grid utility companies looking for more economically viable options to make their services available to the consumers. OpenFlow can address these issues and provide solutions to multiple challenges that are being faced by utilities across various levels. It provides them with a plethora of options to deploy services that have high bandwidth requirements and variable traffic densities [4, 20].
21
OpenFlow provides the following advantages for utility companies:
● The operational complexity is reduced which naturally results in lower operational costs.
As OpenFlow based SDN provides centralized programmable control to network services, the cost otherwise spent on several hours required to configure multiple devices on traditional networks is reduced considerably, thus enabling a reduction in network- management costs.
● It provides better utilization of network resources that would eventually lead to more economical delivery of services. Bandwidth and other resources allotted to services can be easily and dynamically controlled for changing demands and requirements at various times of the day. Thus, excess network resources are not allotted to services that do not require them [4].
OpenFlow switches are cost effective when compared with traditional switches and routers deployed in conventional networks. These switches are simple devices that forward data as per the instructions fed by the controller. In contrast, traditional routers and switches have all the software and hardware embedded in the device itself such as the CAM (Content Addressable Memory) which they use to lookup forwarding tables. This amalgamation of software and hardware control in a single device makes it a lot more expensive than OpenFlow switches. Such routers and switches, naturally, also require more energy and power to fuel the running of multiple processing components that are present in them, while OpenFlow switches consume a lot less power to operate [4, 21].The longevity of regular routers and switches is limited to a period of three to four years after which, they need to be replaced as they reach end of life and support.
22 5. Discussion of Results
This project explored the capabilities of OpenFlow technology to support the smart grid network infrastructure. We investigated the problems associated with a MPLS design for smart grids and found that the design does not offer good flexibility and predictability. In addition, we concluded that the MPLS featuring special routers require a high use of energy resources and come with expensive upgrade costs. We implemented a setup to run DNP3 over a MPLS network and the link failure test showed the high drop in packets during a fast reroute process. The data gathered was used to conclude that a pure MPLS network for smart grids is not a good choice for time critical smart grid applications.
We further deployed DNP3 to run over an OpenFlow based network and could successful poll between the PLC (substation) and the host computer (control station). However, when we tried to deploy OpenFlow with MPLS, we faced hardware constraints due to which we could not test the reroute feature in an OpenFlow based network. Nevertheless, we were successful in proving that OpenFlow can be deployed in smart grid utilities, resulting in a highly scalable and interoperable network. Lastly, we studied the economic benefits of deploying OpenFlow in smart grids and concluded that with OpenFlow, we aim to reduce capital costs and complexity by providing programmable control and solutions to most services. It also leads to lower energy utilization thus resulting in reduced operational costs over the long run.
6. Conclusion and Future Research
Our capstone project successfully investigated the feasibility of deploying OpenFlow in smart grids. We concluded that the current network for smart grid utilities are relying on old network technologies that are not reliable and secure, thus requiring a new and advanced network
23
architecture that can help reduce costs for utilities and also have a highly available network. Our experiment with MPLS in the lab showed the high drop in number of packets during a link failure that could prove fatal for a utility company. We deployed OpenFlow in the network to prove that it is interoperable with legacy protocols such as DNP3 and it is also supported on legacy PLC devices such as SEL 2411. OpenFlow with MPLS can be deployed to provide all the features of MPLS but with the advantage of a separate control plane so that the load on the router processor is vastly reduced. This would also allow a centralized network management through the OpenFlow controller and would not impact any existing services, thus providing an interruption free network.
We hope that our research will be the guiding light for further research that could be conducted in the smart grids domain. Future researchers could build on the OpenFlow controller program and add additional features such as Quality of Service (QoS) for differentiated traffic and also include comprehensive service level agreements (SLAs) to ensure maximum availability. It would be interesting for researchers to design a more efficient code that could take comparatively less time to process and thus, decisions made by the controller with respect to routing and congestion avoidance could be quicker, avoiding packet loss and drop in performance. In addition, further research can be done on evaluating the performance of a larger network topology with multiple devices and flood the network with heavy traffic to study the performance characteristics of OpenFlow networks.
24 References
[1] W. Jewell, V. Namboodiri, V. Aravinthan, and B. Karimi, “Communication Requirements and Integration Options for Smart Grid Deployment,” Texas A&M University, Texas, Rep. PSERC Publication 12-03, Apr. 2004.
[2] G. Clarke and D. Reynders, “Practical Modern SCADA Protocols” Great Britain, Newnes Publications, 2004.
[3] K. Curtis (2004, May 20). A DNP3 Protocol Primer [Online]. Available:
http://www.dnp.org/aboutus/dnp3%20primer%20rev%20a.pdf
[4] A. Sydney, “The Evaluation of Software Defined Networking for Communication and Control of Cyber Physical Systems,” Ph.D. dissertation, Department of Electrical and Computer Engineering, Kansas State University, Kansas, USA, 2013.
[5] Z. Qin. (2013, Dec. 20). A Survey of Networking Issues in Smart Grid [Online].
Available: http://www1.cse.wustl.edu/~jain/cse570-13/ftp/smrtgrid.pdf
[6] K. H. Mak and B. L. Holland, “Migrating electrical power network SCADA systems to TCP/IP and Ethernet networking,” Power Eng. J., vol. 16, no. 6, pp. 305–311, Dec. 2002.
[7] D. Cao and I. Andonovic, “Research on Backbone Communication Network in Smart Grid by using OPNET,” IEEE, 2011.
[8] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S.
Shenker, and J. Turner. (2008, March 14). OpenFlow: Enabling Innovation in Campus Networks. [Online]. Available: http://archive.openflow.org/documents/openflow-wp- latest.pdf
[9] (2014, April 24). SEL-2411 Programmable Automation Controller. [Online]. Available:
https://www.selinc.com/SEL-2411/
25
[10] (2013). Pica8 Open Networking. [Online]. Available:
http://www.pica8.com/documents/pica8-datasheet-48x1gbe-p3290-p3295.pdf
[11] (2014, April 20). POX Wiki - Open Networking Lab - Confluence. [Online]. Available:
https://openflow.stanford.edu/display/ONL/POX+Wiki
[12] (2014, April 20). MPLS-TE Demo - OpenFlow Wiki. [Online]. Available:
http://archive.openflow.org/wk/index.php/MPLS-TE_Demo.
[13] (2014, April 20). Mininet Overview - Mininet. [Online]. Available:
http://mininet.org/overview/
[14] H. L. Willis and R. R. Schrieber, Aging Power Delivery Infrastructures, Second ed., Dekker, April 16, 2007.
[15] Deploying IP/MPLS Communications Networks for Smart Grids. N.d.
http://www.tmcnet.com/tmc/whitepapers/documents/whitepapers/2013/7585-alcatel- lucent-deploying-ipmpls-communications-networks-smart-grids.pdf, accessed December 1, 2013.
[16] Alcatel-Lucent Why an IP/MPLS Network Makes Sense for Smart Grids | TechZine | N.d. http://www2.alcatel-lucent.com/techzine/why-an-ipmpls-network-makes-sense-for- smart-grids/, accessed December 1, 2013
[17] S. Das, A. R. Sharafat, G. Parulkar, and N. McKeown, “MPLS with a simple OPEN control plane,” in Optical Fiber Communication Conference and Exposition and National Fiber Optic Engineers Conference (OFC/NFOEC 2011), Technical Digest (CD) (Optical Society of America, 2011), paper OWP2.
[18] (2014, April 20). Wireshark. [Online]. Available: http://www.wireshark.org/
26
[19] D. Narula, M. Rojasmartine, and V. Rayipati. (May 2010). “Evaluating performance on an ISP MPLS network,” MS Capstone, Interdisciplinary Telecommunications Program, University of Colorado, Boulder, USA.
[20] A. Sydney, J. Nutaro, C. Scoglio, D. Gruenbacher, and N. Schulz, “Simulative
Comparison of Multiprotocol Label Switching and OpenFlow Network Technologies for Transmission Operations,” Vol. 4, No. 2 ed. , IEEE Transactions on Smart Grid, June 2013.
[21] Open Networking Foundation (ONF), “OpenFlow-Enabled Cloud Backbone Networks Create Global Provider Data Centers,” ONF Solution Brief ed., Open Networking Foundation, November 14, 2012.
[22] S. Tanterdtid and A. Pao-on, "IP/MPLS-based data communication network for
power utility,” Proc. on IEEE Power Tech Conference, 2003 Bologna, Italy vol 3, pp:5, June 2003.
[23] Quality of Service Networking – DocWiki. [Online]. Available:
http://docwiki.cisco.com/wiki/Quality_of_Service_Networking.[N.d.]
[24] Benefits of MPLS - The Top 4 MPLS Benefits. Available:
http://www.rcrwireless.com/mobile-backhaul/mpls-benefits.html [N.d]
[25] Industry Solutions - Cisco Systems Utilities/Smart Grid. [Online]. Available:
Internet: http://www.cisco.com/web/strategy/energy/external_utilities.html [N.d.]
[26] (2003, May). Multi-Protocol Label Switching (MPLS) Using MPLS to Build an Application-Centric Network. [Online]. Available:
http://www.fourdtech.com/downloads/mpls_indepth_analysis.pdf
27
[27] (2014, 20 April). Next Generation Networks. [Online]. Available:
http://www.etsi.org/technologies-clusters/technologies/next-generation-networks [28] (2014, 20 April). SCADA over IP-based LAN-WAN connections Application. [Online].
Available:
http://www05.abb.com/global/scot/scot221.nsf/veritydisplay/09e2909e92d2ce8ac125786 3004e7da0/$file/scada%20application%20flyer_small.pdf
